About this Author

College chemistry, 1983

The 2002 Model

After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases.
To contact Derek email him directly: derekb.lowe@gmail.com
Twitter: Dereklowe

February 28, 2013

IBM's Watson Does Drug Discovery?

I saw this story this morning, about IBM looking for more markets for its Watson information-sifting system (the one that performed so publicly on "Jeopardy". And this caught my eye for sure:

John Baldoni, senior vice president for technology and science at GlaxoSmithKline, got in touch with I.B.M. shortly after watching Watson’s “Jeopardy” triumph. He was struck that Watson frequently had the right answer, he said, “but what really impressed me was that it so quickly sifted out so many wrong answers.”

That is a huge challenge in drug discovery, which amounts to making a high-stakes bet, over years of testing, on the success of a chemical compound. The failure rate is high. Improving the odds, Mr. Baldoni said, could have a huge payoff economically and medically.

Glaxo and I.B.M. researchers put Watson through a test run. They fed it all the literature on malaria, known anti-malarial drugs and other chemical compounds. Watson correctly identified known anti-malarial drugs, and suggested 15 other compounds as potential drugs to combat malaria. The two companies are now discussing other projects.

“It doesn’t just answer questions, it encourages you to think more widely,” said Catherine E. Peishoff, vice president for computational and structural chemistry at Glaxo. “It essentially says, ‘Look over here, think about this.’ That’s one of the exciting things about this technology.”

Now, without seeing some structures and naming some names, it's completely impossible to say how valuable the Watson suggestions were. But I would very much like to know on what basis these other compounds were suggested: structural similarity? Mechanisms in common? Mechanisms that are in the same pathway, but hadn't been specifically looked at for malaria? Something else entirely? Unfortunately, we're probably not going to be able to find out, unless GSK is forthcoming with more details.

Eventually, there's coing to be another, somewhat more disturbing answer to that "what basis?" question. As this Slate article says, we could well get to the point where such systems make discoveries or correlations that are correct, but beyond our ability to figure out. Watson is most certainly not there yet. I don't think anything is, or is really all that close. But that doesn't mean it won't happen.

For a look at what this might be like, see Ted Chiang's story "Catching Crumbs From the Table", which appeared first in Nature, and then in his collection Stories of Your Life and Others, which I highly recommend, as "The Evolution of Human Science".

Eventually, AIs will progress to proposing de novo novel compounds for evaluation, and some will be successful. I hope I'm around for the gnarly questions on inventorship under US law that will occur when that happens. "...beyond our ability to figure out." Heh-heh.

Derek's recommendation re Ted Chiang is strongly recommended!! His "The Merchant and the Alchemist's Gate" is a modern-day Arabian Nights tour de force.

Hmmm, teams of bright PhDs with tons of experience, and insight wade through a great deal of poorly validated (clinically) biological data and progress compounds to the clinic where for the most part they fail. What makes you think that Watson or any other expert system will do better? It is not the sifting of the data that is the issue, it is the quality of and validity of the data.

This is a crock. First of all, if those pansies at IBM were real men, they would have made that stupid Watson parse the damn questions like the two humans had to do. The technology is certainly there for a camera to read the board, do the ICR and feed the answers into Watson's search engine. But Noooooo. They got the answer fed from a text file straight into Watson's feeble brain. So while Jennings and that other sap are trying to read the first character in the answer, Watson was already scanning the databanks and slamming the button. I was embarrassed for science to watch that crap and hear it called competition.

This would probably work reasonably well for things drugs all have in common, such as finding compounds with low toxicity, good pharmacokinetics, etc. Predicting compounds with totally new modes of action would probably require much more detailed information on how cells function at the molecular level then is available. Maybe in a few decades...

The day a computer does anything more than tell the programmer the same thing they could have calculated themselves but just in a faster manner is the day I...hey wait...uh, Google what do you mean 'self aware'..I didn't ask...easy now...[EXTERMINATE !, EXTERMINATE !]

It may not be yet, but it is a legitimate matter of discussion for the future. I would say that it should only qualify as a "designer" for the purposes of patents if it also qualifies as a *person*, and is thus as much an employee of the company as any human. Of course, that is its own giant massive ball of wax. . .

Bet Watson could shift through the Sirtris data and suggest the company was worth somewhat less than the $750M GSK spent for them.... But then, so did the due diligence team at GSK. You have to actually base decisions on data to care.

We've " been there" ever since genetic algorithms were introduced: completely impenetrable code and decision making. The jump is when the questions getting correctly answered are ones we have no understandable way of solving, as opposed to being merely difficult.

I'm struck by the fact that this may soon create "obvious" solutions for the pharma industry, meaning no more patents - "It's obvious to ask Watson and then try what he suggests." After all, this is well within the capabilities of the iconic PHOSITA (Person Having Ordinary Skill In The Art)

agree with 16
We saw this with visual control of vial production. There was no way the first generation of machines could match human skill. But over thirty years, the makers have made enough progress that the job is now better done by machine.

My guess is that the 15 candidate drugs it suggested were, in fact, mentioned as candidates by papers in the dataset. That is, in fact, a pretty good way of finding candidate drugs (as long as no one else got there first), but it'd be a case of Watson acting like the fancy search engine that it is, and not like the AI its PR claims.

Not impressed. They fed the scientific literature, primarily chemical and biological abstracts, and came up with only 15 new anti-malarials? One can do the same with Scifinder and use Tanimoto similarity searches to do the same thing. In about 15 minutes and using a laptop.

I'm reminded of a story (perhaps apocryphal) of a university that wanted a filter for the mountains of applicants. They used Machine Learning to build a complex AI model that did as good a job sifting the candidates as did their human experts, in a fraction of the time.

Punch line: they got sued for sex discrimination. They said, hey, not possible, we're using AI! Belatedly looking into the complex model, they found to their chagrin logic something like:

Of course, that opens some very interesting questions: what if an objective, unbiased AI *does* turn up logic like that in its statistical assessments? Is it discrimination if it can be actuarially supported?

A total crock. Science by press release is awful. Drawing attention to this diminishes all the efforts of people doing real virtual screening with actual followup testing. Derek, I think you should try to highlight some of those.