Chance: Peace talks in the probability wars

Working out probabilities is just simple maths, right? Wrong – from drug trials to court cases, being Bayesian or frequentist can make all the difference

By Regina Nuzzo

Video: How probability can help control your destiny

WE ARE in a bar, and agree to toss a coin for the next round. Heads, I pay; tails, the drinks are on you. What are your chances of a free pint?

Most people – sober ones, at least – would agree&colon; evens.

Then I flip the coin and catch it, but hide in it the palm of my hand. What’s your probability of free beer now?

Broadly speaking, there are two answers&colon; (1) it is still 50 per cent, until you have reason to think otherwise; (2) assigning a probability to an event that has already happened is nonsense.

Which answer you incline towards reveals where you stand in a 250-year-old, sometimes strangely vicious debate on the nature of probability and statistics. It is the spat between frequentist and Bayesian statistics, and it is more than an esoteric problem. “The frequentist-Bayesian debate is the only scientific controversy that actually does affect everybody’s life,” says Larry Wasserman of Carnegie Mellon University in Pittsburgh, Pennsylvania. A drugs company testing a new drug can come to apparently very different conclusions according to which method it uses to analyse its results. A jury might reach a different decision after hearing evidence presented in frequentist and Bayesian terms. “It’s not just philosophy, and it’s not just mathematics. It really is concrete,” says Wasserman.

The two approaches have often seemed at loggerheads. But statisticians are slowly coming to a new appreciation&colon; in a world of messy, incomplete information, the best way might be to combine the two very different worlds of probability – or at least …

To continue reading this premium article, subscribe for unlimited access.