IBM Watson: Fake It till You Make It?

Like many AI researchers, I watched wide-eyed when IBM’s Watson beat human world champions at the difficult game of Jeopardy in 2011. It was a watershed event in AI history and it certainly provided a much-needed boost to knowledge-representation-based approaches to AI, against a backdrop of 20 years of superior advances in statistical approaches to AI.

Given my interest in research problems at the intersection of logic and probability in the design of intelligent agents, naturally I tried to read up on descriptions of Watson in journal papers but, to my slight frustration, what I could find (at least as of mid 2015) still sounded rather “brittle” to me and I didn’t really find any superior insight, algorithmic or otherwise, on how Watson is able to sidestep this Achilles heel in basically all existing knowledge-representation-based AI systems. I just assumed they do have the secret sauce – the Jeopardy feat is that impressive – but IBM is not quite ready to share that with the world yet.

Fast forward a few years and we have been hearing that Big Blue is placing a huge bet on Cognitive Computing to turn the company around, which has just announced its 17th consecutive quarters of declining revenue. I continue to hear of large, expensive Watson deals, mostly to build experimental systems at this stage, at Fortune 500 companies and large government departments around the region. All appears well, until you dig a little deeper…

Here are some of the things I have been hearing from the grapevine:

Many of the large Watson deals were done way above the pay-scale of the working level and decisions to do an expensive experiment were made without proper technical evaluation. That such things happen is of course not surprising — a lot of enterprise IT solution selling is about “fooling” a non-technical executive — but the thing that concerns me slightly is that this appears to be the only way Watson deals are getting inked at the moment.

I have yet to hear of a single successful Watson experiment. Companies are spending millions and millions on this technology but I have not heard, from a working-level person, of even a qualified success, let alone an unqualified one.

IBM appears to be pursuing a “Fake it till you make it” strategy with Watson. It is opening large Watson Competency Centres in major cities around the world and it’s actively blurring the definition of Watson by lumping many different technologies, both organically grown and newly acquired, under the Watson brand. Watson is now no longer a simple question-answering system but all things to all people.

All these are big red warning signs to me. It’s possible that IBM can fake it until it makes it but, in looking at how events are playing out, I’m reminded of this Buffett quote: “When a management with a reputation for brilliance tackles a business with a reputation for bad economics, it is the reputation of the business that remains intact.” Cognitive computing doesn’t have bad economics, but general AI as a scientific problem is incredibly hard — just ask the three generations of brilliant scientists that have predicted that AI can be solved in the next 20 years, starting in the 1960’s. While it makes perfect sense for IBM to invest heavily in a technology like Watson, it’s a completely different story to bank the company’s future on a technology like Watson.

Disclaimer: I have indirect financial interest in IBM through ownership of Berkshire Hathaway shares so, if anything, I do want IBM to do well.