A Weekly Lament about the Public Understanding of Science

Tag Archives: Computers

I was going to save this for next week, but it is getting a lot of play. So here’s a bonus post for this week. Lucky you.

In a recent Gizmodo post that has singularity fans all atwitter (literally), it was reported that a program calling itself Eugene Goostman, written by Russian programmers, had beaten the Turing Test.

Spoiler Alert: No, it didn’t. But that didn’t stop the onslaught of “gee whiz!” reporting and spreading of the word. Even the Gizmodo article breathlessly claimed

It’s still an obviously exciting breakthrough, though, one that has critics already raising red flags about its implications. “Having a computer that can trick a human into thinking that someone, or even something, is a person we trust is a wake-up call to cyber crime…”

and

Are there serious concerns about what this means for online security in the future? Sure. But today they’ll have to take a back seat to the understanding that we’ve entered a new era of computing. One that’s alive with possibilities, or at least convincingly enough so.

Umm…no.

And, on Tuesday, June 10, 2014, The Colbert Report had a lengthy (and hilarious) segment on the victory of machine over man. But it seems like Stephen Colbert is one of the few people in the media that understand the insignificance of this story.

The Turing Test was proposed by the late, great Alan Turing in the middle of the last century in order to give some structure to the maddeningly vague question, “Could machines ever think?”

Turing was inspired by a party game of the era, the so-called “Imitation Game,” in which an interrogator asks questions of two contestants, a man and a woman. The contestants are in a different location, and the interrogator tries to determine from their answers (preferably written) who is who. The contestants try to fool the interrogator and win if they do. It’s really kind of like texting or chatting with someone these days and trying to figure out whether you are dealing with a pervert. Or like getting on online customer-service “representative” that is really just a bot.

I don’t know about you, but I have never been fooled by those bots yet. I always ask them to write a sonnet, and they just can’t do it (see below). Citing Stephen Colbert once more, these online bots are about as convincing as this.

Turing decided that a similar game could be used to test whether a machine, in his case, a digital computer, was thinking at the same level as a person. An interrogator asks questions of two respondents and, if the interrogator cannot do better than guessing at which is the person after she is through with her questioning, the machine has passed the test and should be considered to have been thinking in as important a way as the other respondent was. After all, all we have to go on when we decide another humanoid is thinking is their external behavior, so getting fooled is no different being confronted with a thinking person.

In his seminal article, Computing Machinery and Intelligence, Turing gives the following hypothetical question-and-answer exchange as an example to skeptics who think the test would obviously be conclusive in every case:

Q: Please write me a sonnet on the subject of the Forth Bridge. A : Count me out on this one. I never could write poetry.

Q: Add 34957 to 70764. A: (Pause about 30 seconds and then give as answer) 105621.

Q: Do you play chess? A: Yes.

Q: I have K at my K1, and no other pieces. You have only K at K6 and R at R1. It is your move. What do you play? A: (After a pause of 15 seconds) R-R8 mate.

Later, during his exhaustive attempt to answer objections to the very idea that machines could think, he gives us this Q&A script:

Interrogator: In the first line of your sonnet which reads “Shall I compare thee to a summer’s day,” would not “a spring day” do as well or better?

Witness: It wouldn’t scan.

Interrogator: How about “a winter’s day,” That would scan all right.

Witness: Yes, but nobody wants to be compared to a winter’s day.

Interrogator: Would you say Mr. Pickwick reminded you of Christmas?

Witness: In a way.

Interrogator: Yet Christmas is a winter’s day, and I do not think Mr. Pickwick would mind the comparison.

Witness: I don’t think you’re serious. By a winter’s day one means a typical winter’s day, rather than a special one like Christmas.

Specifically, Turing is replying to a skeptic who has suggested that the most a machine could do is learn things in “parrot fashion.” After all, we could record a series of answers on a tape machine and then turn on the appropriate clip at the appropriate time and mimic or “parrot” intelligence.

But after the above exchange, Turing reasonably writes

I do not know whether [the skeptic] would regard the machine as “merely artificially signaling” these answers, but if the answers were as satisfactory and sustained as in the above passage I do not think he would describe it as “an easy contrivance.” This phrase is, I think, intended to cover such devices as the inclusion in the machine of a record of someone reading a sonnet, with appropriate switching to turn it on from time to time.

Clearly, the “test” that Eugene Goostman passed was not a test like the one envisioned by Turing. Eugene “portrayed” “himself” as a 13-year-old boy for whom English was a second language. And with that caveat, “he” “fooled” one out of three judges in a 5-minute online interview. I hope it is clear that this is a pale imitation of the imitation game Turing proposed in his paper. The limiting of the scope to questions that an ESL teenager could be expected to answer is just one of the problems. Calling this test “The Turing Test” is another—several organizations sponsor such things for their own reasons, and several bots have “passed” them in similar ways to the Goostman bot. Add to that the time limit and the fact that we have no idea what the questions (and answers) were and this just isn’t interesting news. For anyone. Watson’s performance on Jeopardy! was more exciting.

This was a contrived and artificial experiment, and Turing’s genius deserves better than this intellectual equivalent of a hot dog eating contest.

Turing believed that it was likely that his test would be passed by a digital computer eventually

…I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.

Our computers (and programs) aren’t there yet, and in a future post I will wade into these controversial waters. We’re certainly closer than Turing was to achieving this milestone, but Eugene Gootsman is surely not evidence of our progress.

Until next time, I’ll leave you with a link to the upcoming movie about Alan Turing: The Imitation Game, starring Benedict Cumberbatch.

I think it’s a fair bet that the test being written about was, in part, created to promote the film.