Rollo Carpenter Interview - Loebner 2006 Winner

Rollo Carpenter Interview - Loebner 2006 Winner

Background

The 2006 Loebner Prize competition was held on Sunday, 17 September 2006 at Torrington Theatre, University College, London. The competition sees judges assess which machine is the most human, otherwise known as a Turing Test - named after Alan Turing the British mathematician.

The Bots

Rollo and his chatbot 'Joan' chatted their way to the top to gain the bronze award, followed in second place by 'Ultra Hal' the popular chatbot program by Robert Medeksa. In third and fourth were Noah Duncan with 'Cletus' and R. Churchill & M-C Jenkins with 'John' respectively. No one has yet convinced the judges they are chatting to a human being rather than a bot - and so the gold award has never been claimed.

The Winner

This is the second year Rollo has won with a 'Jabberwacky' chatbot developed by his company Icogno. These bots are constantly learning online, running on windows servers with Dual Xeon's. Joan's character is that of a 26 year old writer and development of this persona was helped by the english writer Ariadne Tampion. The bot boasts an incredible 5 million lines availabe for it's use and Rollo predicts that the full Turing test will be passed by the year 2016.

Happily

The award was deemed news worthy and after seeing Rollo interviewed on the UK's Channel 4 news program, I decided it would be a good idea to send him a few questions and our congratualtions...

How long did the Loebner winning bot take to develop ?

It's been more than 15 years since the learning AI started to grow, and went on the web as jabberwacky in 1997. In 2003 it became a full-time activity, and is now starting to become properly commercial.

Did you think at the outset you had a good chance of winning again ?

I did, yet at the same time the subjectivity of the test is such that with just 4 judges, you never can really tell. That's one reason why winning it for the second year is pleasing, suggesting more strongly that a lead over other technologies is emerging.

How well did you think Joan performed in the competition personally, ie was Joan able to show full potential or are there more tricks up your sleeve ?

Joan certainly didn't perform as well as she can do some of the time on the web, but nothing is certain in this game, and being tested by Turing Testers is quite a different experience for an AI that has learnt in the environment of online chat.

I generally avoid "tricks" as much as possible, leaving the general AI a lot of freedom to get up to whatever it likes. However, the event did involve some, including virtual typing with "typos" and the opportunity for the AI to interrupt the person even while they're talking. It did do that, and then would delete what it was saying if they carried on typing after a pause, but I'm not really sure it helped.

The IRC-style interface, pure text, in which anyone could say anything any time instead of Bot-Judge-Bot-Judge did make for an interestingly new challenge, but I'm not convinced that it has a lot to do with the Turing Test.

You were interviewed on UK television's Channel 4 News last week, how did that feel ?

It felt pretty good! I'm pretty sure Jon Snow expected both Joan and I to be a lot harder work than it proved. At the outset he jokingly suggested that I didn't look sufficiently boffinlike! I barely had time to get it set up, and it was all over in a flash. There was BBC News the week before and ABC Nightline aired in the US the same night as C4 News, too - with a piece that was all populist and "robots taking over the world!"

In the light of that interview and press coverage generally do you feel optomistic about the future of AI technology as a viable product ?

I am optimistic by nature anyway, but yes, I believe that the impact of AI is only just starting to be felt, and in many ways - not just chat, obviously - will be quite transformative for economics, science and society.

Is there any advice you would like to pass on to budding bot makers out there ?

Context, context and context!

Thank you Rollo for finding the time to answer my questions, I really enjoyed reading your replies. Best wishes and success for the future and congratulations from everyone here to you and your company, and of course Joan!