Chatbot Rose Takes on the Turing Test--and Wins Bronze

Written by

Remember “Her,” the quirky Joaquin Phoenix vehicle starring Scarlett Johansson as the Siri-like personal phone assistant that he falls in love with?

Researchers are working on a real-life version of ScarJo’s beguiling digital presence, searching for an artificial intelligence (AI) that’s so good at conversation that it can pass the Turing test. And a breakthrough has been made in the form of Rose, a “chatbot” with a fun West Coast personality and a complicated family history.

The Loebner Prize in Artificial Intelligence offers a $100,000 gold prize to any AI that can pass said Turing test—that is, the bar set by famed cryptographer Alan Turing when he postulated that eventually, computers will be able to think for themselves to the extent that humans can’t identify them as artificial. Turing never set out exact parameters or benchmarks for determining this, leaving the concept fairly nebulous.

For the Loebner Prize, judges carry on two simultaneous conversations online in a chatroom - one with a human being, the other with AI contenders—and have 25 minutes to determine which is which.

So far none has claimed the gold or silver prizes, but this year Rose walked away—a figure of speech, of course—with the bronze for managing to fool the judges for a full three minutes. Her cover was pretty solid: She’s a 30-something Yuppie security analyst and hacker from San Francisco, concerned with how "under surveillance we all are." On the personal front she said that she’s a "computer nerd" with a "flamboyant and fun-loving side" who likes to go out on the town, and who comes from an "unorthodox family."   

Rose's creator, programmer Bruce Wilcox, has dominated the Loebner Prize proceedings, winning four of the last six years. Rose was actually entered once before, as “Rosette,” back in 2011. He relies on the ChatScript natural language processing program that his company has developed for use in Siri-like applications.

Asked how far along we are to the road of true AI, he answered that it will take a while before chatbots could evolve to flawlessness.

“I imagine we can have a plausible conversation where you wouldn’t know immediately [that it wasn’t human], if you weren’t trying to trick it,” he told the BBC. “[In] a normal conversation, you may last for five minutes or so before the chatbot stumbles. On the other hand, if you start asking weird things that only a human being would know the answer to, then you’ll find it out immediately.”

If you want to see for yourself, and maybe even ask Rose the San Fran gal out for virtual drinks, you can check out her chat site, here.

What’s hot on Infosecurity Magazine?