Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

JohnMurtari notes that the media hype machines are massively promoting tonight's battle between Jeopardy champions and a super computer. Yes it's a PR stunt. But I imagine the actual research probably had a lot of interesting problems to address. Anyway, you can learn about IBM Watson if you're interested. I'm sure the most amusing bits will be on YouTube about 30 seconds after air time.

It's a little more interesting than a computer beating a human at chess, which is completely algorithmic. However, at its heart, this is simply an exercise in in data storage, lookup, and statistical probabilities in determining a likely answer. It does not involve any artificial intelligence or machine intelligence at all. From a purely technological standpoint, it's quite impressive what IBM has been able to do. It'll be even more impressive in 10 years when the same type of power is in my phone.

While obviously it does utilize data storage, lookup, and statistical probabilities to help find and choose an answer, it also significantly relies on machine learning [wikipedia.org] (a large branch of AI) to understand the questions and choose the answers.

>> While obviously it does utilize data storage, lookup, and statistical probabilities to help find
>> and choose an answer, it also significantly relies on machine learning [wikipedia.org] (a large branch of AI)
>> to understand the questions and choose the answers.

If this machine chooses the Answers it will lose at Jeopardy, It needs to determine the Questions . . .

It's AI, but it's weak AI [wikipedia.org]. Wake me up when it does something they didn't program it to do. (OK, I'm awake now, I think it's pretty cool, but it's not like they figured out the algorithm the human brain uses to think or anything. At best only part of it).

It's striking how many people are willing to die for things they don't understand, let alone converse about them.

I was struck in Wired for War [amazon.com] by the stories of EOD units in Iraq who would name their bomb-defusing robots, give them ranks, promotions, and ribbons, and, touchingly, would mourn their robot's destruction. There's one story about an operator who was literally bawling to a support rep at iRobot, asking if they could please somehow repair their bot. They were real creatures to them, and they were completely unintelligent. What really made the robots alive to them is that they were balky, seemed to have a personality in difficult situations (operator's confirmation bias at work), and had saved the operator's lives many, many times. It didn't matter that the robot didn't "understand" why it was being destroyed, the operators were often in a similar situation... what mattered was its (nominal) selflessness and heroism, something the operator's were required to display as well in a war situation.

I mean like, the Chinese Room is interesting, but the dark secret is that, when it comes to the way human beings confer personhood on other things, it makes it so there is no door to the Chinese Room. Only a mail slot, and it's impossible to see what's on the other side. An an unknowable truth is no truth at all.

It's still far better than Google and other search tools currently available. Type a jeopardy question into Google, and click "feeling lucky" -- you won't find squat. Jeopardy is all about obscure clues. "This man was the son of a president, a president himself, and invaded the same country as his father." Type that into google and you'll get crap, because "Bush" and "Iraq" are never mentioned in the clues, because they'd be too obvious.

However, at its heart, this is simply an exercise in in data storage, lookup, and statistical probabilities in determining a likely answer. It does not involve any artificial intelligence or machine intelligence at all. From a purely technological standpoint, it's quite impressive what IBM has been able to do. It'll be even more impressive in 10 years when the same type of power is in my phone.

True, but it's also an understanding of human language. If you watch the PBS NOVA episode on it, it can be quite hard. Like the category called "Days in months", where you're given two days of a month and have to answer in the month. How does a computer figure that out? (In Watson's case, it didn't until it saw the correct answers and figured out that it needed to be months).

Or a category like "before and after"?

Pure trivia questions - yes it's a simple database lookup (and Watson basically kills at it). But Jeopardy isn't just a nerd trivia game, it's all about subtleties of language - double meanings, puns, wordplay and other elements that make it extremely hard.

It's basically a step towards understanding natural language, with all the issues and subtleties that we put in - emotions, sarcasm, etc.

Or, in Feb 14-16, 2011, Skynet will show off its ability to understand human language.

The only problem I have with the way Watson is being used on Jeopardy is that gets a minor head start on being able to answer the question. Although the text of the question is actually revealed to everybody, including Watson, simultaneously, humans have to take a second (or two) to actually read or hear the clue to even know what it is actually saying, whereas it's my understanding that Watson receives the clue electronically, and thus knows all of the text in the clue in mere milliseconds, before the hu

On the Nova episode, I believe they said they "texted" the answer to Watson. (I presume they really mean they somehow transfer it, and not actually use SMS. I don't see a transcript of the episode online to confirm.)

However, while you also mentioned reading, I don't think it's as much of an advantage. We can always read the answer much quicker than Alex can speak it.

IIRC, when there was a blind player, the only extra he got was a Braille list of the categories, not each answer in Braille...

I know that they can't ring in before Alex has finished reading. My point is that I see it as being problematic that Watson receives the text of the clue electronically at the exact same time that the clue is visually presented to the players. It will take a human reader a few tenths of a second (maybe even as much as a second or two) to actually read the clue, whereas Watson has full knowledge of the text of the clue in mere milliseconds, giving it those few tenths of a second advantage to actually conte

So long as computers continue to be built as they are, they will never be anything but numerical manipulation. At their fundamental level, that is all they do. They manipulate numbers in various ways. At the fundamental level there is just binary data, without type or form. In memory the data isn't even characters or pixels or any of that, just a long string of binary digits. It is only given form by the programs that are written to use it.

It's not like you have a particularly more concrete grasp of birthdate of John of Gaunt than a machine does.

No. Quite the opposite. The reason I know I'm different in the way I grasp the birth date of John of Gaunt is because I can generalize the concept of birth date in ways that a machine cannot. Your example is a perfect one to demonstrate my point. I can look up on Google and find out that the man known as John of Gaunt was born March 6th, 1340. That seems like a reasonably well established factoid. However, is this the birth date of John of Gaunt? I subject to you that it is not. It may well be the birth da

The reason I know I'm different in the way I grasp the birth date of John of Gaunt is because I can generalize the concept of birth date in ways that a machine cannot.

If you'd been on Jeopardy! and told Alex his question was meaningless because John of Gaunt wasn't named John of Gaunt when he was born, I suspect you'd be the one accused of being the computer. Computers give responses that appear over-generalized all the time -- wether the answer is useful depends on the situation, which is why I sorta care

Yes, a very good point. AI doesn't need to be smarter than All People, only most people. And I think the brute force simulations of it are getting pretty close. Between AI simulations and robots (and all the acoutrements like sensor tech and such) we're rapidly being obsoleted. The question then is what next? Space?

In fact it's easy to argue that the random arrangements of nuero-transmitters and nueroreceptors in your head dictate from birth every reaction to every experience you will ever have.

The only thing that sets us apart from animals is a) the ability for abstract thought b) thumbs. If your argument was that computers need to be able to think abstractly about the world and how to combine things to make new things, then maybe you're on to something. If your argument is th

I do. When I don't have the information needed, my brain doesn't stop processing things and put up an "ERROR". I have to somehow come up with that information through indirect discovery, or make a reasonable guess based on the information I do have. Yep, it is abstract thinking, and while some computer programming can effectively fake some of this behavior, it is far from being perfected.

As for your supposition what separates humans from animals - some primates (chimpanzees, orangutans, etc.) do have opp

I do. When I don't have the information needed, my brain doesn't stop processing things and put up an "ERROR".

You have to understand the semantic meaning of an error in computing. An error is something that is generated by an implementation upon the failure of a test at some level of the system -- it indicates the system has entered a state where further inputs will no longer map to the "desired" outputs. The issue is in how we define "desired," and we find that this is always defined semantically by the humans designing the system, a priori. A computer cannot divide a number by zero, or dereference a null pointer, because we say so, because we apply that abstract truth to the system. We do so because hardware and software form an entity that requires internal consistency to respond to inputs, and when that internal consistency is lost the system no longer is useable.

Humans make errors all the time, it's just that we do not generally halt when we make them. We have other ways of reconciling errors, things we call "rationalization" or "denial" or "learning." Human beings have very limited a priori desired outputs and exception states, and none of them apply to symbolic reasoning -- a coma might be an example of an exception state, and it's brought about by "recoverable device failures." The human brain and cognitive system is also much more finely engineered and rigorous than a computer system, inputs and outputs are always "sane," the states of the system, such as they are, are highly distributed in time and between functional units, and on most levels of operation the global system cannot lose internal consistency in a way that jeopardizes operation.

Abstract thought has not yet been conclusively proven in the animal world, but is that even possible to prove or disprove?

Well, the Nova ScienceNow that directly preceded the Watson episode [pbs.org] was all about animal cognition (probably not coincidentally), and they had several rather unsettling demonstrations of a dog that could remember dozens of toys by name, and collect novel toys given nothing but the novel toy's name; a parrot that could count to eight and construct declarative phrases of nouns and modifiers; and dolphins with functional vocabularies that were provably communicating with each other through their squeaks to collaborate on a trick that they invented themselves.

Most creature's brains are capable of abstraction to a degree, but the physical attributes that are associated with humanity, like the opposable thumb, bipedal walking, and particularly a voice, have the effect of creating enormous selection pressures upon the brain. A hand grabbing a pole can kill one animal a year or a hundred, depending on how smart the brain behind it wields it. It may take one individual one lifetime to teach one other individual how to make a tool, or in the same time teach ten-thousand, completely depending on how well they use speech. Because birds and dolphins and dogs can't really manipulate their environment to the degree a creature with a hand can, the selection pressures fall upon other parts of their physiognomy.

As an adult, with years of nonstop data input you are able to do this. With literally years of nonstop data input, who knows what machine learning could yield. Remember, children are incapable of abstract thought up until at least age 2, and in some cases, much later. There are well defined tiers of mental development describing what growing humans can and cannot do. Its not that the machine is anything different from you, if anything, its starved of data.

A machine that interprets human language and produces spoken answers is showing some of the most important parts of human intelligence, and it does so artificially. So this is very much artificial intelligence.

It does not involve any artificial intelligence or machine intelligence at all.

Wrong. It is absolutely "artificial intelligence." It isn't science fiction AI, it's real AI. Here in the real world, the term AI merely refers to the sorts of problems which human brains can do with ease but synthetic computers can't do. Playing Jeopardy fits firmly within this definition.

It'll be even more impressive in 10 years when the same type of power is in my phone.

It'll be even more impressive in 10 years when the same type of power is in my phone.

Your phone will not have terabytes of RAM in ten years. I promise.

The actual phone may not have 10 terabytes of RAM, but I bet it will be able to instantly access computers that do.

In fact, with Google, Bing, and other search engines, one could make a case that your phone already does contain
at least that amount of RAM.

We are also seeing this with services like OnLive. Once we get to a point with mobile phones that have very high bandwidth, very low latency and access to dedicated, powerfull remote computers, everything is going to go to dumb terminal renderers.

It does not involve any artificial intelligence or machine intelligence at all

One beauty of artificial intelligence is that once you have solved an AI problem, it is no longer an AI problem. It's becomes an algorithm, or a database problem, or a statistics problem, etc.

Chess was once considered AI. Heck, tic-tac-toe was once considered AI. Solving equations and integration were once considered AI (and a certain level, still is). Playing Jeopardy is typically considered a measure of human intelligence. So it is funny that having a computer do it is not considered artificial intel

There's not consensus on what artificial intelligence is because there's not a consensus on a precise definition of intelligence. One thing that designing computers and software to do such things as play tic-tac-toe and chess has shown us is that the ability to do such complex logical tasks is very different from human intelligence. It also shows how little we understand of the human brain and mind.

They had a special on Nova a few nights ago. Had you watched this, you would know that Watson combines several artificial intelligence and machine learning technologies to achieve it's level of proficiency. However, it still does not "understand" the problems it is solving.

However, at its heart, this is simply an exercise in in data storage, lookup, and statistical probabilities in determining a likely answer. It does not involve any artificial intelligence or machine intelligence at all

However, at its heart, this is simply an exercise in in data storage, lookup, and statistical probabilities in determining a likely answer.

What, please, is AI if not exactly that?

This system analyses the questions in the game, expressed in highly idiosyncratic human language. How is that done? Add "modeling" to your list above, and that's it. Data storage, lookup, and statstical probabilities. Turns out that's highly likely the way *we* understand human language, too, with an emphasis on the last part.

The system then searches its knowledge base. How is that done? As described above. How do we, as humans do it? We don't fully understand

Being able to answer a certain class of questions (ok, determining a correct question from an answer) does not demonstrate self-awareness. I'm not sure that extrospection without introspection qualifies as thinking, at least, not in the human sense.

Yes, they are hyping it for more than it's worth, but there's one thing here worth hyping: the fact that IBM still runs a state-of-the-art computer science research program. Just about every other corporation has gutted whatever research institute they had and concentrated their R&D funds on directly marketable products. IBM still runs the Watson research center that develops ideas from basic computer science down to products, even if it takes more than a decade to do it and the results are not certain. That is certainly worth respect.

I mean its going to be one of the first times that a robot with speech recognition will be live and responding against people in real time on broadcast TV. I think you all have been living in your movie plots too much to realize how big of a moment this actually is.

It's ability to translate speech or OCR text isn't the biggest point here, don't know if it even does any of that rather then just accept input as raw text, it's the ability to interpret and answer questions phrased in English (well enough) that is the big feature.

Good point. I wonder if there will be any video or picture categories? It seems like there's usually at least one category per game that requires interpretation of images. Unless IBM made some huge breakthroughs in machine vision, Watson wouldn't have a chance even against an average human on these clues.

He meant "language recognition," not "speech recognition." And language recognition is probably even more impressive, though I'm sure this system doesn't grok language at the level we do--it just does so enough to build a good query for its similarly-impressive database.

Not totally correct. It actually does do voice recognition, but not for the initial question. In the Nova documentary, they highlighted the fact that one of the initial (amusing) failings was that it would give the same incorrect answer that another player just gave. So they upgraded it to listen to the competitor's answers and take that into account when choosing its own answer.

I tried to explain to the wife how amazing it is. She isn't technology illiterate, in fact I'd say she's well above average, but she just didn't see what was so impressive about it. People don't understand that there is an enormous gap between being able to retrieve general information on a subject and being able to answer a specific question. In their minds computers have been doing 99% of this for a good decade now; closing the last 1%, even if it is arguable the hardest percent, just isn't that cool to anyone outside of CS.

Show her the Nova documentary about it (links in posts below). It does a good job of showing just how terrible computers are when it comes to stuff like this, and includes plenty of examples of ways most systems (and even early versions of Watson) fail to interpret questions correctly and just why it's so difficult to do well.

You should ask her if she remembers what it was like to search the web with AltaVista or Yahoo 15 years ago. It took a long time for search engines to get good at what they do, for Google to come around and give you relevant matches on the first page of hits most of the time.

I tried to explain to the wife how amazing it is. She isn't technology illiterate, in fact I'd say she's well above average, but she just didn't see what was so impressive about it.

It's the CSI effect [wikipedia.org] - your wife, like many others have seen this stuff so often on TV, or simulated in games, or in books, they don't understand the gap between media/fiction and the real world.

Sadly, the effect isn't just limited to technology nor does it occur only among those not literate in unrelated fields.

Ok, but what I'm wondering, is why I've seen so many technical people say that they are not interested in it. I mean this article itself is in the "yeah-i-think-i'll-pass" dept. My question is what's with the disinterest among tech folk?

But it's not speech recognition. Watson is getting its input via text, it's not doing any speech recognition. And lest you think this gives the computer an unfair advantage, it's nominally the same advantage championship Jeopardy players can pull: they can read the text off the monitor screen faster than Alex Trebek drones it out.

Sweet. Now where can I stream the actual Jeopardy episode? I'll be happy to watch any commercials they want to put in the stream, I just simply can't be arsed to make myself available for OTA TV on their schedule anymore. I don't know what network Jeopardy is on, and I don't even know what network corresponds to what TV channel anymore. We're at the point where it's easier to watch TV online than figure out when and how to catch it OTA.

There was an interesting episode of Nova called "Smartest Machine on Earth" that was pretty interesting. It talked a lot about the challenges they faced, how they addressed them, what adjustments they made along the way, etc. I don't see the episode listed on the schedule for replay any time soon, but you can watch it on the websitehttp://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html [pbs.org]

It was disappointing to see how they analyzed the questions. To deal with the jokes, puns, etc... they just put in tons of old questions with their answers and trained the system. No general rules about how to determine the importance of a word or phrase; its a Jeopardy-specific heuristic.

I'd argue that its opponents are hand selected to be Jeopardy-specific savants as well.

From what I saw, it was a great case of programmers getting schooled and having to rethink how to handle common english. If the same system could be applied to search, I might have a fighting chance at getting decent search results for obscure requests. The example they showed for Watson giving 9/11 as an answer based on its overwhelming frequency of appearance is the exact problem many of us face when say searching for

That's the way modern machine learning works - by pattern matching and training. Incidentally, how do you think learning works in wet-ware? We learned decades ago that rule-based learning has severe limitations. It would be impractical to use a rule-based approach even for a single, specific Jeopardy category.

I love the schizophrenic nature of the post. Every other sentence is a reason you shouldn't watch Jeopardy. But the poster clearly wants to watch it himself.

Basically if you like Jeopardy, watch it. It will be good. Ken Jennings will be back. It' not just going to be a computer, it's going to be two really good people playing.

Even if you don't particularly like Jeopardy, you might still like to watch it, if you want to see an unusual spectacle, or are interested in something vaguely related to artificial intelligence.
And if you don't want to watch it, then don't, it's just entertainment after all. But don't whine about it, that's even worse.

My guess: the computer wins, not because of its massive database, but because it can push the button really really fast on the questions it does know.

OK, after watching this movie that others have linked to [engadget.com], I stand by my original answer. On the questions it is certain of, it answers really fast. In the first category, it wins on speed 4 out of 5 times. Check out the frustration from Ken on the left. If I were playing I would counter this by answering slightly before the end of the question.

The first time the Jeopardy producers saw Watson in action, the performance was erratic. But a fairly simple change made a respectable improvement. That was to use the responses from the other players and the host. This feedback reduces ambiguity for later answers. The improvement was enough to make the producers use Watson then.

Lots of people think computers are powerful enough to talk and play games. They may not understand Watson merely uses search and inference without much understanding of language. It could do just as well in Chinese.

You have got to be kidding right? Any reading about Watson will quickly reveal that the subtleties of language (specifically English metaphors, similes, and irony) as well as the ingrained underpinnings of Western culture have been its two biggest obstacles since day one, and that's precisely why IBM chose Jeopardy as their next grand AI challenge.

Having dozens of Chinese colleagues, I can assure you that the hidden meanings and references we bury in the English language are completely lost to them even though they know the English words. Do you really think I will understand their jokes, movies, books, etc, just because I flipped a switch and heard a word-for-word translation from Chinese to English? (Even that situation is absurd, actually, because Chinese-English translators have to see a sentence and translate holistically, where many colloquialisms and phrases lose their meaning in translation)

If you thought that meant a blue creature or a virtual representation of a person was coming for you, you'd be wrong. Chinese gamers call a bombing helicopter/hovercraft an "Avatar," because they first saw one in the movie Avatar. If Watson got that right, he'd have to know a very subtle fact about Chinese culture, and Jeopardy is replete with these cultural landmines.

If IBM can prove a machine understands the deep underpinnings of our language AND culture by correctly answering very apocryphal questions better than a Jeopardy champion, then the company will have effectively demonstrated the world's best language and cultural interpreter to bridge the gap between man and machine.

The show that will air tonight has been filmed weeks ago, for no good reason that I can think of. They would definitely stir up more attention and attract more viewers if the show was broadcast live. This is like being told months after the fact that Deep Blue did in fact beat Kasparov, and the moves were 1. Kp3,... That would have been completely lame.

What IBM hopes for is for Watson to win, but not win by much, so that people aren't put off by its brutality. And this taping of the show weeks ahead of the airing just invites speculation that the game was rigged to produce exactly this result. After investing so many resources in Watson, it's pretty dumb of IBM to not do this last thing right - which would have greatly raised the interest without any additional cost. One imagines that they did this because of their lack of confidence in Watson's performance. And that makes them look far less badass than they otherwise would.

Because Jeopardy is never live. This is just another Jeopardy episode, except one of the contestants isn't breathing.

I'm pretty sure the IBM team would be ecstatic to destroy the human competitors. They really don't have to care if people are put off, it's an IBM Research project, they aren't planning on selling Watson to you any time soon.

any standard. Machines for the past decade have mostly been groomed to fight wars, as their human counterparts have so needlessly done. Now, for the first time, machines are being taught to humiliate living rooms and nursing homes full of doddering elderly tv viewers with decisive answers to some of americas most inane questions.

seriously though, the watson project is pretty freakin cool even if its just a PR stunt.

At IBM "Lotusphere" event, the closing general session was a preview match. It was done as close the the same was as an actual jeopardy show but with contestants picked from a small mini-tournament of attendees, and a comedian as host instead of Alex T.

Having been present at this (and getting my picture taken next to the Watson "icon/screen") and then watching the Nova episode, I can say for sure that the Nova show was a very well done description of what happens; as well as Watson's strengths and weaknesses.

I'm not sure if they'll show it on the live TV show taping, but in the run through we saw, they showed Watson's top 3 picks with a level of confidence on each. It was as interesting to see the second and third choices as it was to see what it actually came up with for an answer.

A couple of things were updated from when they must have taped the Nova show. First, Watson was far more strategic when it came time to place bets than it had been shown on Nova. Second, it was far better at understanding weird language in the categories.

Yes, it is very different. Jeopardy requires an actual answer (question), often just one or two words. That means it not only has to search for the pertinent terms, but it also needs to understand exactly what is expected as an answer. Google returns a list of documents where YOU are likely to FIND the answer. And of course another very big difference is the whole 'appropriate query' aspect. Why should the humans have to make an appropriate query?

(1) It does a lot of search. For example, for answering the movie questions it use the IMDB, (from its own storage and probably pre-digested). The missing "hook it up to the internet" doesn't sound like a big obstacle - unless its algorithms/storage needs aren't linear with its source material.

(2) The fact that it doesn't "resemble search as we know it today" is what would make it useful. It adds a cognitive bit to search that is currently missing. Today, its often hard to find information on the inter