Turing test claim gets rough reception on Twitter

The University of Reading breathlessly announced this weekend that a "supercomputer" called Eugene Goostman had "passed for the very first time" the "iconic Turing Test."

University of ReadingAn historic milestone in artificial intelligence set by Alan Turing - the father of modern computer science - has been achieved at an event organised by the University of Reading. The 65 year-old iconic Turing Test was passed for the very first time by supercomputer Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

News media, including the CBC, carried the story and the skepticism surrounding it. But on Twitter, programmers and tech journalists almost immediately began to question the claim on a number of fronts.

Eugene Goostman isn't a "supercomputer," but a computer program called a "chatbot," meant to emulate a person typing into an instant messaging service, they pointed out.

As for the claim that it "passed" the "Turing Test" "for the very first time," well, they found each part of that claim questionable.

The Turing test is based on a question and answer game, proposed by renowned British mathematician and codebreaker Alan Turing, to distinguish humans from computers.

Turing predicted in a 1950 paper that within 50 years, computers would play the game so well that an "average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning."

You can't run one test with judges you picked and claim you passed the Turing Test.Chris Dixon

Also, this isn't the first time programmers have claimed to have passed the Turing test.

2011 claim of a bot to have passed the Turing test: http://t.co/gsZruZl2Iv Fooled nearly 60% of participants.Mike Masnick

And there's even dispute over whether the test as the researchers set it up was really the "iconic Turing Test." The judges were told that "Eugene" was a 13-year-old boy from Odessa, Ukraine, and that English wasn't his first language.

So, right away, the bar was lowered.

Reading between the lines of the Io9 story, I'm calling interesting BS on the Turing Test news (AI is my day job) http://t.co/iBbQwzIYr4Martin Robbins

It's interesting because they used a trick of human psychology to win votes (fair play), but it tells us nothing much about AI.Martin Robbins

It's also only about 4% more than the same Chatterbot scored on the Turing Test in 2012, so it's not exactly a revolution this week.Martin Robbins

There's even argument over whether Turing's prediction -- that a computer would be able to convince 30 per cent of interrogators that it is human -- was actually meant as criterion for a test.

Philosophy blogger Brian Leiter asked David Chalmers, a cognitive scientist at Australian National University and New York University, about the claims that Eugene Goostman had passed the Turing Test.

David Chalmers on whether that computer passed the Turing Test. http://t.co/p32ivvzwM5 http://t.co/GFoGOwhvyOPaul Kelleher

Perhaps the most damning evidence against the claims that this chatbot could be mistaken for a person came from actual chats with Eugene itself. (You can chat with the program yourself at princetonai.com. )

Eugene answered much the way you'd expect a computer program to answer: "I can't make a choice right now. I should think it out later. And I forgot to ask you where you are from..."

Time editor Doug Aamoth conducted an "interview" with Eugene, in which the bot asks that same question three times in five minutes: "And I forgot to ask you where you are from..."

Interview with Eugene Goostman, the Fake Kid Who Passed the Turing TestChatbot Eugene Goostman supposedly passed the legendary Turing Test on Sunday, tricking 33% of a panel of judges into believing he was a real boy during the course of a five-minute chat conversation.

Other journalists posted screenshots of their own conversations with Eugene, with similar results.

The whole of a chat with Eugene. This is the sort of thing the Turing Test judges decided on: http://t.co/NIKKMy3yHs http://t.co/oHdp2WAMcQAlex Hern

No, "Eugene Goostman", is no where close to passing the Turing Test. http://t.co/OJHQWjcxuVJohn Graham-Cumming

So, was Eugene passing the Turing test a hoax?

Turing test story not really a hoax, but certainly hyperbolic and misleading.John Allen Paulos

To encourage thoughtful and respectful conversations, first and last names will appear with each submission to CBC/Radio-Canada's online communities (except in children and youth-oriented communities). Pseudonyms will no longer be permitted.

By submitting a comment, you accept that CBC has the right to reproduce and publish that comment in whole or in part, in any manner CBC chooses. Please note that CBC does not endorse the opinions expressed in comments. Comments on this story are moderated according to our Submission Guidelines. Comments are welcome while open. We reserve the right to close comments at any time.

Note: The CBC does not necessarily endorse any of the views posted. By submitting your comments, you acknowledge that CBC has the right to reproduce, broadcast and publicize those comments or any part thereof in any manner whatsoever. Please note that comments are moderated and published according to our submission guidelines.