February 21, 2011

Much of the conversation about last week’s supercomputer smackdown on Jeopardy! has revolved around Watson‘s prodigious buzzer advantage. In the second game, for example, Watson answered 23 of the 30 clues on the board. I’m not really sure if I ever had a 23-answer round while I was on Jeopardy!, but if I did, it sure as hell wasn’t against Brad Rutter and Ken Jennings. Watson was very, very fast.

Some have called this an unfair advantage; I’m inclined to think of it as a fair one. (The headline writer at the Daily Newsdidn’t get the memo.) Obviously, computers have faster reflexes than humans do. Why should Watson have to handicap its greatest advantage? That’s the nature of the beast if you’re playing a computer. I compared it once to Deep Blue beating Kasparov: no one ever suggested that Deep Blue should limit itself to a three- to five-move lookahead, just because human brains are incapable of consistently doing better than that. To paraphrase Aaron Sorkin: let Watson be Watson.

But it’s certainly true that Watson needed that speed advantage to hang with top human players. Over on the message boards, IBM’s Chris Welty points out that a buzzer-free version of Jeopardy! would have been less computationally interesting to Big Blue, since it removes the computer’s need to rate its confidence in its own answers. (He doesn’t mention that a buzzer-free game would also have been suboptimal for IBM in that (a) it would have made for boring TV, and (b) Watson would have lost such a game essentially every time.)

I don’t like the suggestion that Watson’s speed should have been handicapped by, for example, introducing a random delay. And the much-bitched-about fact that Watson received the clues electronically is a red herring; there’s no advantage to be had there, because the Jeopardy! buzzers aren’t activated until Trebek finishes reading the question anyway.

But I can think of two better approaches to leveling the playing field buzzer-wise, should anyone want to try a Watson-like challenge in their basement or garage sometime.

Remove the Extra Human.Jeopardy! viewers know that there are four humans involved in each clue: the host, and the three players. But in fact, a fifth human is just as important: the anonymous staff member (I think it might still be researcher Ryan Haas) who manually activates the buzzer after Trebek finishes reading each clue. If you buzz in before Anonymous Human activates the buzzers, you get locked out for a fraction of a second and are pretty much doomed. When A.H. flips his switch, less than a syllable after Trebek gets done reading the clue, some lights around the game board turn on to let you know it’s time to buzz. In theory you’re supposed to wait for these lights; in practice, the really good contestants try to anticipate them. A.H. is very good and very consistent, but he is human, so there are tiny variations in his timing. These variations never affected Watson (who can pounce so fast it doesn’t need to anticipate) but could submarine its human opponents (who will never beat Watson if they don’t anticipate, and rarely even if they do). If the A.H. were replaced–say, by voice recognition software keying precisely off of Alex’s voice–then humans could try to master that now-fixed interval. We might even have a fighting chance against Watson.

Make Watson “Anticipate the Lights.” Alternately, you could have Watson anticipate as well, rather than pouncing. Get rid of the in-studio lights and the electronic “go” signal for the computer. Feed Watson the clue electronically, but also give it speech-recognition capabilities. It would be able to track Trebek’s voice as the end of the question approached, just like Brad and I were doing, and then try to time its buzz a split second after Trebek got done. Because of A.H.’s slight variability, it could buzz as optimally as possible but it would still be a little early sometimes and a little late at other times–just like a good human buzzer.

I figure either of these fixes would do the job: human superiority would be rightfully established on game shows, just like God intended. Alternately, I guess you could level the playing field like this.