This copy is for your personal non-commercial use only. To order presentation-ready copies of Toronto Star content for distribution to colleagues, clients or customers, or inquire about permissions/licensing, please go to: www.TorontoStarReprints.com

Was the Turing Test passed? Not everyone thinks so

The University of Reading in Britain announced last weekend that the Turing Test, the long-touted standard for artificial intelligence, was passed for the “very first time” — but not everyone agrees.

The university published a press release on June 8 saying that the 65-year-old Turing Test was passed by a computer program named Eugene Goostman during the ‘Turing Test 2014’ contest held in London last weekend.

Yet Professor Graeme Hirst, a researcher and lecturer in computer science at the University of Toronto, disagrees.

“The Turing Test is supposed to be a test of whether a computer is genuinely intelligent in the same way a human being is,” Hirst told the Star. “That’s simply not what we have here.”

The Turing Test was developed by Alan Turing, a London scientist viewed by many as one of the founders of artificial intelligence.

Article Continued Below

Turing predicted that AI would evolve to the point that computers would fully understand human language and become capable of passing themselves off as people in onscreen conversations.

The winning computer Eugene Goostman, designed by Russian and Ukrainian programmers, convinced the human contest judges 33 per cent of the time that it was actually a 13-year-old boy. Eugene was one of five computers competing in the “Turing Test 2014” contest.

Hirst argues that Eugene did not actually pass the Turing Test for several reasons.

Contest rules state that if a computer passed as a human more than 30 per cent of the time, it would pass the test. Turing’s original paper specified the computer would have to pass at a rate “better than chance,” or 50 per cent of the time, says Hirst.

In Turing’s 1950 paper, ‘Computing Machinery and Intelligence’, he wrote that, “I believe that in about fifty years time it will be possible, to program computers … to make them play the imitation game so well that an average interrogator will not have more than a 70 per cent chance of making the right identification after five minutes of questioning.”

Hirst argues that the University of Reading contest “misinterpreted” this statement to arrive at the 30 per cent baseline used in the contest.

Kevin Warwick, a visiting professor of cybernetics at the University of Reading, was one of the “Turing Test 2014” organizers. He maintains that the Turing Test was indeed passed last weekend.

Article Continued Below

“What we did was the Turing Test – that’s it,” Warwick told the Star in an email. “This was scrutinized by independent observers and a machine, and Eugene achieved 33 per cent success. Turing stated 30 per cent as the pass rate. This is all fact.”

Warwick adds that critics questioning how good the test is should know that ‘we are all skeptical in the same way…but what we did was the Turing Test’.

Hirst is not alone in dismissing reports of the milestone. While some initial reports repeated the university’s announcement unchallenged, technology experts worldwide quickly weighed in to discredit the claim. “That Computer Actually Got an F on the Turing Test,” reads a headline on a BBC story.

The development of Eugene is good work that advances the field, says Hirst, but, “is really a program that can carry on clever chats. There’s no way it would survive an examination of the kind of intelligence that Turing had in mind.”

Hirst added that a computer would need to be able to answer questions about feelings, literature, and metaphors, such as: “In Pride and Prejudice, which scene of the novel did you think was the saddest and why?”

“With these questions, a computer couldn’t just look through a book for the word sad or for sad ideas,” said Hirst. “To answer these it would need a real understanding of human emotion.”

Questions that are trivial to the point of being stupid are also important to ask, says Hirst, such as ‘Can the Prime Minister can jump over a bus?’

“For people it’s all so obvious, but it isn’t for a computer,” said Hirst. “For a real Turing Test we would put together thousands of questions like that. And if the computer could answer them as well as a human - then we might be starting to talk about passing the Turing Test.”

Hirst believes that the Turing Test will eventually be passed, but is hesitant to say when — or the implications a computer that can pass as a human would have on the world.

“It will be a very interesting world in the next 50 years,” Hirst said. “Within my career, the changes I’ve seen in computing are just astonishing. But artificial intelligence — or passing the Turing Test — isn’t one of them.”

More from The Star & Partners

LOADING

Copyright owned or licensed by Toronto Star Newspapers Limited. All rights reserved. Republication or distribution of this content is expressly prohibited without the prior written consent of Toronto Star Newspapers Limited and/or its licensors. To order copies of Toronto Star articles, please go to: www.TorontoStarReprints.com