Bayes's rule: A statistical thriller

The US applied Bayes's rule to the task of tracking Soviet submarines (Image: Sasha Mordovets/Getty)

FEW single ideas can boast such an impressive résumé. What else could have possibly cracked the Enigma code in the second world war, predicted John F. Kennedy's close-run win in the 1960 US presidential election, and landed a winning blow against the nefarious antics of the tobacco industry?

The secret to these successes is a humble little theorem called Bayes's rule that allows you to assess the probability of one event from observations of an associated event. On election night, for example, you might use an early outcome from one US state to help predict the winner in states that have not yet announced their result.

This simple rule has experienced a turbulent history since Thomas Bayes outlined his ideas in the mid-1700s. The sticking point has been the fact that you must first take a stab - sometimes not even a very well-informed one - at the relative probabilities of your hypotheses, before plugging in the relevant data to achieve a more accurate answer. Researchers trying to root out any form of subjectivity in their methods found this kind of guesswork outrageous, leading to the theorem being shunned over the next two centuries.

Sharon Bertsch McGrayne's The Theory That Would Not Die is the first popular science book to document the rocky story of Bayes's rule. At times, her tale has everything you would expect of a modern-day thriller. Espionage, nuclear warfare and cold war paranoia all feature as she tracks the theory's crucial role in Alan Turing's code-breaking during the second world war, and the US navy's later use of the technique to track Soviet submarines.

Eventually, the theorem wormed its way into medical journals, insurance forecasting and business decision-making, before its eventual triumph in the age of the internet, as it became essential for spam filters, translation software and much more besides. In these less dramatic periods, a host of colourful characters and their bitter rivalries carry the tale. McGrayne can capture these mavericks in a single sentence: we are told that Robert Osher Schlaifer, who helped introduced the theorem to business, was "hot as a pistol, sharp as a knife, clear as a bell, quick as a whip and as exhausting as a marathon runner".

Despite these colourful descriptions, the book is not always an easy read. Even with a mathematics degree, I struggled to visualise just how Bayes's rule solved many of these problems. And while McGrayne's writing is luminous when describing the drama of the cold war, her attempts to portray earlier periods in a modern context sometimes fall flat. We are told that a theological pamphlet was "a kind of blog", for example, and she feels the need to describe a philosophical thought experiment as a type of computer simulation.

These are minor gripes, however, and to have crafted a page-turner out of the history of statistics is an impressive feat. If only lectures at university had been this racy. Then again, perhaps we should expect nothing less from a theory once described by one of Google's representatives as "the crack cocaine of statistics... seductive, addictive and ultimately destructive".

Book InformationThe Theory That Would Not Dieby Sharon Bertsch McGrayne
Yale University Press
£18.99/$27.50

Bayes theoreme is one of the most fundamental ideas. It is most interesting to learn the history of its adoption and the differents ways in which is interpreted. Wikipedia has a decent presentation. I was not aware of its use in Turing's work. Iwill have to get the book.