Last night, IBM’s Watson computer bested human “Jeopardy” champions Ken Jennings and Brad Rutter, in the final game of a three-night series.

Here are some thoughts from Mr. Baker now that the Watson challenge is over. (Also: Yesterday, Speakeasy talked to Mr. Baker about the challenges of the publishing experiment that his book represented.)

It was the second night of the Man vs Machine Jeopardy match that the Sony nightmare appeared to be coming true. Throughout all of the negotiations between IBM and Jeopardy!, a division of Sony Pictures Entertainment, each side had been obsessed with its own disaster scenario. IBM, of course, worried that its computer, Watson, would lose and look foolish, embarrassing the company and the team. Jeopardy, meanwhile, worried that Watson would grow too smart and too fast, and steamroll the human competition. This would be demoralizing and, worse, bad entertainment.

It was this concern that led Jeopardy to press IBM, only nine months before the match, to fashion a finger for Watson, so that it could physically press the buzzer. Previously, Watson had been buzzing electronically. IBM’s chief scientist, David Ferrucci, initially objected to the finger. “They’re trying to graft human limitations onto the machine,” he told me one day over lunch. This would slow down Watson’s buzzing by perhaps 8 milliseconds. He worried that Jeopardy might continue to ask for adjustments in their hunt for a fair and exciting match. The downside from IBM’s point of view? First, it might cause Watson to lose. Second, IBM needed sets of uniform data to create the scientific case for Watson. Adjustments screwed up the data.

On Tuesday night, as Watson demolished Ken Jennings and Brad Rutter, it looked horrible for Sony. But things improved in Final Jeopardy, when Watson botched a clue about U.S. Cities. It didn’t have confidence in any of its answers. But the best it could come up with was Toronto. (My explanation of how Watson came to this questionable conclusion.)

Watson’s gaffe was great for Jeopardy. It showed the machine’s vulnerabilities, and it was funny. For IBM, of course, it spoiled what should have been a celebration of Watson’s smarts. I immediately started looking at the comments on Twitter. And it was clear that many people didn’t understand how Watson works. The impression seemed to be that Watson had lists of facts, and if Toronto was on one of those lists as a Canadian city, it could not be a U.S. city–hence the computer was dumb.

Well, on a number of clues, including this one, the computer is in fact dumb. (For some, that might be comforting.) But it comes to its decisions not through black and white analysis, but instead through a massive and complex analysis of evidence. Watson sees things in gray.

In the final game, the only chance for the humans was to land on Daily Doubles and multiply their scores. Watson beat them to the two Daily Doubles in the second round. And it was smart enough not to risk its commanding lead with big bets. After all, it has the best betting algorithms IBM could devise.

Now, with Jeopardy over, it’s time for Watson to find a real job. That’ll be its next drama–which will probably come with breakthroughs as well as gaffes.

Comments (3 of 3)

When Deep Blue became the world chess champion, the philosopher and cognitive-science researcher Douglas Hofstadter said that meant not that computers were now more intelligent than people but that we needed to refine the criteria for "intelligence" so that they're not met simply by being able to play chess. I suspect he'd say something similar about Watson. Winning Jeopardy doesn't make you intelligent; it makes you really good at searching a large database when given peculiarly-expressed search parameters. All Watson is, is an improvement on Google that understands natural language a bit better. It's not revolutionary but only evolutionary.

3:06 pm February 17, 2011

Steve Baker wrote :

RC, for almost everything we do with our minds, Watson comes up empty. But in the very small domain of digging through data and coming up with probable answers, it's great. So its has great potential as a tool, not as a human

2:20 pm February 17, 2011

RC wrote :

It can be Jeopardy champs, but it's probably no smarter than a 5 year old. Stick the thing in a kindergarten and see if it can figure what's going on. I doubt it can. It has a massive database, fancy algorithms, and super fast processors, but it's not superior to humans (for now). Hopefully when humans do finally develop a computer or android truly superior to us, let's hope we are smart enough to install an off switch on it (like Data's in Star Trek TNG).