Now that IBM’s Watson has pounded the best human Jeopardy competitors into a fine slurry, let’s take stock. Our human proxies took their ass-kicking in good spirits, with Ken Jennings writing on his ‘Final Jeopardy’ card, “I for one welcome our new computer overlords.” (For the sake of adding a bit more inane trivia, the …

COMMENTS

I hate titles

The project has some interesting avenues that could follow later down the line, and I'm sure are the real reason the IBM researches persued this in the first place. For example, being able to feed a bunch of rules on financial transactions and then load up the raw data of a companies financial records, and have the model spit out any irregularities would be one possible use.

or worse...

easy...

"God... imagine if you unleashed this beast on the reg's comments section?"

We human types are ambiguous. We have nearly endless ways to say the same thing.

We possesses an over-abundance of phraseology for equal concepts. In other words, humans can articulate equivalent answers in a multiplicity of natural language structures, or to put it into words of one syllable; There is a long list of ways in which we are able to say one thing. We simply obfuscate concepts using the full variety of language for aesthetic purposes. Or to put it simply, an idea can have numerous equal expressions in written or verbal form. A collection of apparently different words and phrases can be evaluated to reach a singular interpretation.

@oddie

splitting hairs much?

so.. it doesn't "search" for answers but.. it makes a list of possible answers and.. weights them based on the most likely? Not like.. returning the most relevant search results first or anything? Or waiting search results based on what is popularly considered, by human searchers, the "correct answer" to their search term?

Do massive "off the shelf" clusters not actually come under the term "supercomputer"?

Just what are the exact definitions of "search" and "supercomputer" and would you stop being such a whiny little pedant.

Quite frankly it's all a load of b/s anyway, I'm quite capable of translating what I want into machine or search engine friendly terms and anyone who isn't probably doesn't have anything worthwhile to add to humanity and should just fuck off and die in a hole somewhere.

The trouble is there are too many people with an inflated sense of their self worth, too many people who think they're scientists but are just getting in the way, holding things back, lying on grant proposals, fudging the numbers just a little.

We're all too selfish and vane to accept that we aren't apex group members and will resort to dishonesty to keep up our delusions.

Watson's NLP abilities and the future possibilities.

Most important, Watson's natural language processing-NLP programming/"abilities?" is one of many break thru AI researchers are striving for. NLP is the holy grail of these endeavors. Quite simple, as massive amounts of computer data is accumulated there is no way a human can make any sense, or perhaps a better description might be how the data items are related to each other, thus the need for supercomputer(As a complete system, Watson is a super computer cluster.)

While the average IT department might purchase one IBM Power 750 server, my belief is that most IT departments would be hard press to justify the purchase proposal of ninety servers each with 250 gigabyte ram memory. Watson NLP abilities are now just being explored, image Watson as a front end interface for data mining to any, all? databases that have terabytes of information. Watson latest endeavor, are medical records examinations, in order to assist in clinical medical diagnosis. Imagine the possibilities, maybe one day Watson might be appointed to a Presidential cabinet position, this way all of the Intelligence Agencies could submit their reports to Watson, or better yet, submit the full stream of intelligence data, Watson would then analyze the data and make the appropriate recommendations to the President, no more ambiguous, bias (PDB) reports; Yes, I'm a big fan of the movie "Colossus"- The Forbin Project. ;-)

Owned by a dog

It is, indeed, interesting to consider technology as a new species, like dogs and cats, that, for a little pleasure and convenience in return, has us constantly worrying about its health and improvement, while our memory and intelligence goes down the tubes.

What if...

you put two or three of these things together but not linked - as stand alone - and get them to talk to each other? Could they group solve problems? and if you slighty bias each ones algorithms - they would come up with slightly different best answers and learn from these?

Even just playing jepordy against each other?

If it's possible with these - then that would be a really interesting experiment.

Maybe we don't want to go there for real fear of having to welcome our new overlords?

Of course Watson is about commercialization

Sure - showing the world what's happening in research labs is best done with glam and flair.

Watson wasn't just a well timed expo of incredible technology toppling meager humans. Nor is it the start of the end of humanity.

Technology companies will always paint the 'art of the possible' in the public mind because it gets folks excited. When the first caveman invented knife technology, you can imagine the excitement around the cave fire, unless you were the first test case of course.

IBM has always pushed the boundaries of material science and continued to drive innovation. Their patent records year on year speak to that. The fact that they have the largest dedicated math department in any private company means they are dedicated to solving problems.

Let's be pragmatic however - IBM's a company that is in business to make money. They have stock holders that want a good return on their investment. Clients of IBM enjoy a trust relationship and value assurances that what happens in the Research and Development Labs will make it's way into commercial applications that will give them an edge in their market.

Think of IBM's leading thought campaign with "Smarter Planet". It's not rocket science to associate the demo of Watson's incredible data crunching and accuracy that could very well lead to the next generation of medical diagnosis tooling, better decision support systems for critical infrastructure (emergency services, energy and water management systems, etc).

I for one am excited about the possibility of potential new applications of the technology developed during the creation of Watson.

Now to the next challenge

not super???

"Watson is made up of 90 4-socket IBM Power 750 systems with 360 8-core POWER7 processors running at 3.55GHz with 16 GB of memory per server. The systems are connected together via 10 GbE networking"

No, not at all a supercomputer. How could even a stupid carbon-based lifeform mistake a 2880-core distributed processing system for a supercomputer. I've got three on my desk right here. By next year, there will be one in your wristwatch. Super indeed. (snort).

I for one, welcome our commercial non-super-computing overlords

Not Super Computer?

By this reasoning then an Linux cluster is not a super computer either (which is what this is) since they are just commodity hardware. I am sure that thing well in the TFLOP range. Does any one have those figures.

that Watson was not a super computer, is a conclusion of one...

that has not the foggiest what a super computer is...

'Super Computers' be they one processor (none at this time) Six to ten un-hobble Play Station CPUs used to be many, and the Air Force is resurecting many), a thousand really old AMD cpus, A Cray, Which hobbles together many more AMD newer CPUs... Or any of the IBMs which are listed on the fastest Computers lists, with the Chinese 'super comuter, etc, etc, etc,

are all Super Computers, made of several commercially available (except possibly the Chinese one) cpus, computers, systems or glued together whizz-bangers.

NASA sent men into space with less computer power than the later 486 desk tops (which, were faster than the 'first' super computers.) Many desktops today, with multi-cpus and gawdawful amounts of memroy, terra-bytes of disk, and SSDs. are much faster than some of the 'Super Computers' of the 1990... and used to play desktop tournaments at things like HALO...

Nailing down a definition of 'Super Computer' has always been hard, since the speed changes are dramatic over time. Super Computers of yesteryers, are usedtowazis, compared to today.

With the potential of Graphen combined iwth other 'atom thick' elements on the very near horizon, super computers will be possible in garrish ear-rings of the next 'Hot Spy Momma' coming down the pike.

If Scientists were upset because Sony screwed up their Super Computers because Sony disabled 'Linux' in the Play Station....Your faux definition of 'commercially avaiable' and hacked together' have little meaning in the Definition of 'non-super computer'.

It has to do with how many instructions and threads are handled - - - ONLY.... and how fast they are handled.

Good article

I attended a presentation at the Watson Research Center last year about "Watson" - it was a fascinating presentation which managed to describe the complexity of the problem really well. You've almost managed to get that across in the article.

It's been quite frustrating reading some of the comments from people who seem to just think that the underlying problem is really simple if you've got a big enough data set. The key thing about it is that Watson had to do all this within the time that Ken and Brad had to hear the question and press the button. Jennings' previous winning streak was purely down to button pressing.

Your point about the number of ways in which the same thing can be said is what makes this so difficult a problem. IBM have done something really new here.

DON'T PANIC

AI

Standard AI skeptic response: "oh wow, just like chess, it turns out being good at jeopardy is not proof of intelligence".

Not that I think Watson *is* intelligent, but I have to say I'm dazzled that it's possible to fine-tune a machine learning system with a dataset that large. Normally, ML systems get just good enough to be a little interesting, and then they hit a local maximum that the scoring system is never able to improve on.

Good work guys, now pare it down a bit and put it on the internet. Being able to ask a computer "what's that giant rubber thing that's one sixth of something else", and get the answer "a ningi"? PRICELESS.

That's another way of saying...

How long will it take to evaluate '42'

the exact time it takes the philisophers and psychologists to figure out they are going to be redundant, plus the time it takes to hire the Vogon fleet and they clear the way for the bypass... minus a few seconds for the answer to be available, but, unable to be recaptured (they think... until book 5 of the trilogy)

Ohhh... black-and-white cathode-ray-tube scary...

"When THEY eventually make some wrong decisions or miscalculations the same as their designers and progammers did before they created"

Really! The old trope "computer connected to world, makes mistake, we all die" is so dusty that homeless people are using it to scrape dirt off their shoes. HAL's affirmation to be "foolproof and incapable of error" only generated sardonic grins nowadays.

You may have noticed that today:

a) People know that complex real-world algorithms make mistakes

and therefore

b) Keep the algorithms simple, specify them to death and go over them with a toothbrush (look up software assurance and reliability engineering)

and also

c) Underwrite major insurance contracts in case SHTF even so (which it will).

No racisism because..

You are not correct. current computers have no preferences, they currently (for the most part) perform their function in Binary or variations of same. They do not prefer it. it is what they are. (some few were decimal, but, aside from being much more accurate, they still worked in a binary representation of decimal...)

They are not particularly fond of the Human species because of the sci-fi books making them out to be killers, ogres, and sentient. (they are none of those things at this time).

Watson was just a 'chess playing computer' with a much higher degree of database programming. (Human dust and moisture does computers no good, either... to say the spilling of various fluids, on sundry parts of the system.... Did I mention the Super Computer Keyboard again, yet? preferably with super computer still attached)

Computers can be progammed by humans, to do good, evil, so-so and screw up. But, the computer is just following the coded instructions (for which they get blamed, even though it was the programmers fault... (I don't need a programmer for the keyboard... a sexy typist would be nice, but I am not begging....pullllleeeeeeze---ok.. i'm begging... sigh)

Until ... is the answer...

Until more people understand the question you are seemingly worried about, there will be very little understanding of the question.

Most of your belief and fear, at this time, is predicated on a ficticious image.

Computers can not, at this time, take over the world. It is just as unlikely that golems powered by large racks of abaci will.

At this time, they are still just tools, used to manipulate data. And too often deliberately being manipulated themselves to give wrong answers. (sic Global warming computer models were required to come up with the answer they wanted or the data was adjusted... every time at all locations controled by the cadre)

The question should be, about how it can be done, if it can be done. Understanding that it, eventully, will come to pass, that there will be sentient, self-replicting machines. The problem is understanding, you will probably not be around for the unveiling. Once the machine gets to a certain point, it will happen quicker, but, it is no where near there, now.

Will they become our masters? For now, computer masters are the least of your worries. We have people in power that are trying to do that, with no need of computers.

Don't ignore computer development. Watch it and be marveled. But, there is much more out there you should not ignore... believe me

Watson did make some real mistakes

On the second day the final question was to name a US City that had their main airport named after a war hero and the second after a battle in WW II. (As I recall)

Watson piked "Toronto". Toranto? Since when is Toronto a US City?

Maybe there is a city named Toronto in the US. But, I doubt it has two airports.

But, give the guy a break. He did pretty much snow the humans.

Watson is a wonderful acheivement. There is no question about that.

We really only need to begin to worry when the answers come up and we have no idea if it is right or wrong. Then we have a problem.

In the meantime there are many industries where Watson type capabilities can do wonders for us. Medical diagnosis being one critical one. Critical because lives are at risk. Hopefully a real doctor will get the answer and question it before anyone is injected.

If you have seen House on the Fox network you know how difficult it can be to diagnois an illness. And even on the super show to hero makes numerios mistakes before time runs out and the patient has to either die or be saved. Of course, those mistakes are based upon incomplete or even changing information. Just like the real world. Give Watson incomplete or incorrect information and he will fail as well.

you end up with Bing coping Google

Computers already copy what other computers do. And it screws up Bing.

Microsoft likes to think that look at previous search results improves it own. Google has shown that Bing graps stuff from Google. But, that only contaminates the search.

The last thing you want is a search manipulated or altered based upon who is asking the question or how similar questions have been answered before.

Witness the crap you have if your SQL select is adjusted in an unknown way based upon previous selects? You would never know what you got? And prior mistakes cause additional mistakes.

And you have the lack of intelligence on the behalf of the surfer adversely affecting the new search. When you do a search it is assumed that you do not know the answer ahead of time.

And do you really want a search result that is different depending upon who is asking the question? You get one result. I get a different one? How can that be answering the question asked?

Do you really want your Bing search to be different if you went to Google first? Or, didn't?

Hint: Even Watson knew to ignore prior answers given by the humans or even its own. Not part of the question.

With Bing you get a contaminated search. Sorry to dump on Microsoft but the question was what happens when two computers start chatting with each other. Microsoft has shown that the computer will stop listening to the human.

Point: missed.

Watson is addressing TWO problems, not just the single one you mention. Asking questions of a computer in a manner it can understand is, in some cases, something that can be taught to most people.

The next step is the computer has to look through information to determine the correct answer. If the information it has access to is all correct and structured then this isn't a (massive) problem. Most data ISN'T - and currently computers can't use that for very much - Watson also looks to solve this problem.

So, while you're on your crusade to weed out people who can't construct search terms properly would you also mind structuring and verifying the mountains of information we have kicking around too?

HAL Did not Fail - Those who have him his mission orders did

If you read the books (as opposed to just watching the movies) you would know that HAL's behavior was due to a bad set of mission orders not due to any error on his/its part. The mission orders made completion of the mission priority number 1. The orders were to continue the mission no matter what happened (ie: The cold sleeping scientists and the two crew were expendable and not vital to the mission). Also the crew was not briefed on the real mission and was only to be told upon arrival. Thus there was a conflict between HAL's following the mission orders and the need to supply accurate information to the crew. Add to this that no HAL series computer had ever been powered off once activated (so the concept of being able to be reactivated after being powered off) and the question of if they would (not just could) power him up upon arrival lead directly to his attempts to kill the crew and actually killing the scientists. This was made explicate in (I think 2010 although that sequence might have been the 2001). There was another HAL on Earth running a parallel mission and the cause was found by analyzing its actions.

No Toronto Airport in the US

What you are missing is that this was a Final Jeopardy Question. Thus the need to supply SOME answer. The fact that Watson KNEW the answer was wrong (and would have not triggered a Buzz in the standard rounds) was signaled by two things with the answer. First was the string of ?s after the Toronto. Second was the low bid (in the $900s) for the answer.

Please...

Either structure your equations in an easily -understandable way or use brackets.

So, do you mean:

194 = ( 4e6 / 14 ) * ( 4.2 / 3600 )

or:

194 = ( 4e6 / (24 * 4.2) ) / 3600 = 4e6 / ( 24 * 4.2 * 3600 )

or even:

194 = ( 4e(6/24) * 4.2 ) / 3600

I know I could figure it out by working through the calculation myself, but why should I?

Sorry, rant over. It just riles me when people don't make their equations or calculations clear, especially in programming (I know the orders of precedence are set in stone for a particular language, but a few brackets make the thing SO much easier to read!)

The point...

The point I was trying to make was not that the calculation was ambiguous from a programming point of view, as I know rules of precedence define this strictly (although these do vary at times between languages).

The point is just that it is one of my pet peeves, as it takes very little extra effort to add them, and saves a lot of effort in reading it. Also, you cannot be certain that a person knows the rules of precedence, or is following them the same as you.

Actually...

...Watson didn't know to ignore previous answers given by the humans, this lead to Watson giving the same incorrect answer as one of the human players in the first round. To fix this, Watson would also need to be equipped with real-time voice recognition and be able to infer context from what a person is saying, in order to know that the other player had given an answer, and that it was wrong, so needed to be excluded from its own answer set. This is another, much harder, AI problem.

point: found

Structuring your query correctly will find you the relevant information.

Seems to me the only purpose for Watson is to find the question to ridiculously obscure answers rather than to find the answers to questions worth asking, as the majority of questions worth asking have at some point been asked, typing a question as a search term will often find you the answer!

Nothing to do with playing chess

"Watson was just a 'chess playing computer' with a much higher degree of database programming."

This is wrong for any useful explanatory purpose, even if "higher degree of database programming" was a meaningful phrase.

Chess-playing systems employ a variety of algorithms: brute-force searching of the game space, heuristic pruning of moves that are unlikely to be useful, predictions of opponents' moves, libraries of strategies, etc.

None of those are much like the NLP (Natural Language Processing) tasks required of Watson. Chess is a large but well-understood and unambiguous problem. NLP is very large, open (new words and phrase structures can be created at any time), not well-understood, and ambiguous.

"Colossus"- The Forbin Project - The Movie

Anyone who just saw the movie (or read the book) has a very slanted view of Colossus and its actions. The have to read the 2 sequels The Fall of Colossus (1974) and Colossus and the Crab (1977) you would know that Earth is under attack by Mars and Colossus took over to defend Earth (as was its job).

Not Exactly

"The key thing about it is that Watson had to do all this within the time that Ken and Brad had to hear the question and press the button."

Almost. They saw the question as it was displayed and was being read out. Thus they had the same amount of time to come up with the answer as Watson. Watson was able to beat them to the buzz if 2 things occurred. First that it came up with an answer that ranked at least 90% on the "probably right" scale. Second that it took over a set amount of time for the question to be read (Watson had a built in delay before it was allowed to attempt to buzz-in if it had a 90% answer). If you were watching, you would see that it was not the first to buzz if the displayed ranking on the bottom of the screen did not have a 90% answer OR if the question was short (and thus was finished before the delay was over).

Collossus kicked ass

...and there were even two sequels to the original book. I vaguely remember reading the first sequel that begins with Colossus has been driving the earth bus for a few years. It's a police state, but Forbin is well taken care of since he's the 'Creator'. Some try to rebel against the machine and are killed. Colossus does testing to destruction on humans to develop metrics. Martians start talking to Forbin's wife over the radio. Colossus not happy with this. Forbin gets special logic bomb from said Martians and uses it to take Colossus down.

I didn't read the third and final chapter to the Colossus story, but, according to Amazon reviewers, it seems that the Martians want 50% of our oxygen. Since they're quite a ways ahead of us tech-wise, they can take it. Forbin stalls them and brings up Colossus to save the day..or something along those lines....

Read Super Crunchers

Be very wary of a commercial competitive society which can predict humands better than humans can. A lot of services rely on "on average" behavior and a lack of knowledge. Insurance being the typical example - if the computer can accurately predict who will get ill and who will not, why insure the ones which are going to get ill - they will lose the insurance company money.

Keep data private - much of society relies on it - more than most people think. That said - it's a good piece of tech - I just get worried they will get used for less benign uses.

"Off-the-shelf gear ‘gets’ humans"

We already have this with the imperial COTS-based drone armada based in Af/Pak.

In the same vein, wait until state gets ahold of those tasty algorithms. Just connect it to the various databases and you will be visited by blue-clad goons for aggravated terrorist pedophilia with intention to evade taxes at least once a month.

interesting, but ...

Informative article. I'd just quibble with the idea that it's not a supercomputer and it's not doing search. As you say, " We human types are ambiguous. We have nearly endless ways to say the same thing. Our statements and questions are unstructured, and must be interpreted through the context in which they’re made."

To the majority of English speakers, a system that can beat humans is "super." As wikipedia says, the definition of super is rather fluid. I believe your point is that Watson is not a demonstration of raw processing power, which is a defining supercomputer trait. But if I may speak as a member of the great unwashed masses, Watson demonstrated a power not previously seen so I think one could argue that it expands the definition of supercomputer.

As for search, Watson goes into a database and returns an answer. I can't fault journalists for considering that search. It's taken search to a new level.

Again, the article was informative. My point, if I have one, is that language is consensus of meaning. To those of us without engineering degrees, this was super and it was search.

Big deal?

I'm not impressed by what IBM did. For one thing they were able to modify their program at the end of each day. They had two weeks to download as much information as they could from the Internet, including the entire Encyclopedia Britannica. At best, they wrote, modified as needed, a very expensive search engine.

Per NOVA's special on this story, when IBM showed the executives of Jeopardy the new computer they had built, it failed miserably. They had to modify the program several times in order to beat normal contestants.

Once they got on the show they needed to modify it yet again in order to beat the two "best" of Jeopardy. And to top it all off, the questions on Jeopardy are standardized, meaning that they are designed within a specific parameter which is easy for IBM to design a program to operate within that specific parameter.

Humans are much more complex and the questions could have been made even more vague and tricky which would have required much more time and resources for IBM to compete.

I am not impressed and no, we aren't going to be over run by computer overlords.

That reminds me....

SkyNet. Human GIGO.

2 things:

"Watson is the best recent example of a machine crossing over the divide between human and machine-style thinking." -- Horrifically scary if true, but what I think you meant was "...best example of humans creating algorithms that bridge the divide between human and computer information processing."

“There already are humans kept as pets by machines - they're called ‘iPhone owners!’” - very cliquish and droll, but iPhones are nothing compared to our true overlords: television.

super is as super does

Dan Olds, there are computers with far less compute capability than this on the supercomputer lists. It shouldn't grate you that something many, if not most would classify as a supercomputer is called one. Moreoever, you can't honestly say the computer doesn't do searches, even qualifying it with how we conventionally mean it. You described exactly a search process and it's similar to how internet information is searched in major search engines. You can say that it doesn't just do searches.

What a silly rant about some blurry semantic lines that one reg columnist seems to think are built like the wall of china around his own little interpretations of things.

That said, I wholeheartedly agree that the general press does seem to convey the description of this accomplishment in a way that diminishes and oversimplifies it. But it does that with most everything and this article did little to countervail it.

I like the Watson name

Human muscle responce times

once the human mind decides when it has a probable answer, it can not push a button faster than about 250ms. It is not usually possible to predict when a sentence will end, introducing further lag time .. where if they text being given Watson had a "." to denote the end of the answer-statement, there would be no lag for it to start processing the totality of the clue-statement.

once Watson got to it's decision, I doubt the lag to push the button was but 10ms

Watson had an unfair advantage then, and should have had an artificial lag put in to approximate the human's lag in determining when the clue was finished and the time for a hand to respond to the human brain's signal to push the button

Not a real test

I dare it to correctly answer half of the questions posed by the missus. You know the type, "should I wear these red shoes with this dress or those vermilion ones?", "do these jeans make by butt look big?" and "do you think she is pretty?"

Should have been ...

Reality Imitating Art ?

In "Satan, His Psychotherapy and Cure by the Unfortunate Dr. Kassler, J.S.P.S." by Jeremy Leven

Dr. Leo Szlyck imagines a machine with synapse-like circuits. After building the machine, he debates over whether he should turn it on, believing it could be a weapon of immense power. Against his better judgement, he does, and he's greeted by the giant machine that calls itself Satan.