Why We Need Nate Silver So Badly Now

Almost everyone who cares about American politics cares about Nate Silver's FiveThirtyEight blog at The New York Times, so it's perfect that his new book has come out just in time for the final run-up to election day — indeed, as Obama and Romney jockey for minor poll bumps from the debates. Even during less kinetic moments, we live in a world generally surrounded by statistics, in which quantities stand in for insight. Sometimes these numbers are obviously bullshit, as in this case. Others are much more difficult to spot. But during election season, the explosion of distracting numbers is constant and disorienting. The Signal and the Noise is many things — an introduction to the Bayesian theory of probability, a meditation on luck and character, a commentary on poker's insights into life — but it's most important function is its most basic and absolutely necessary one right now: a guide to detecting and avoiding bullshit dressed up as data.

Advertisement - Continue Reading Below

The key to the book is its humanistic approach to statistics. At the core of this approach is the Bayesian theory, which is complicated but which Silver does an excellent job of explaining to non-geeks like myself. I cannot do justice to it here, but, to give an example, the Bayesian theory is capable of ascertaining that, if a woman randomly comes home and finds another woman's underwear in her dresser, there is a twenty-nine percent chance that her husband is cheating. This involves the calculation of the prior probability, thus introducing a subjective element into the statistical model.

Most Popular

Election forecasts from FiveThirtyEight.

It's an approach that has massive advantages. On the one hand, it does not contain the arrogance of so much analysis rooted in big data. Through the Bayesian method, you cannot predict anything with certainty. You can only establish the probability of an event. Silver also confronts the so-often abused distinction between objective and subjective information: "The word objective is sometimes taken to be synonymous with quantitative, but it isn't. Instead it means seeing beyond our personal biases and prejudices and toward the truth of a problem. Pure objectivity is desirable but unattainable in this world." This has the great advantage of being true, even though it's obviously disconcerting. How scientific can something be if objectivity is unattainable?

But Silver is no flake. While acknowledging the necessarily subjective nature of forecasting, he is first and foremost a stat-head, with a ruthless contempt for predictions that don't fit basic math. The most hilarious part of the book is his assault on The McLaughlin Group commentators, who, he proves mathematically, have no better capacity for prediction than flipping coins. Silver has spent more time than is strictly healthy trying to establish exactly how stupid those commentators are, but his contempt extends far beyond The McLaughlin Group. The most important thing Silver has to say is also the most depressing: The better something sounds on television, the more likely it is to be wrong. This is an insight we should all pay attention to. A long-term study of punditry actually established it; the less often an expert appeared on TV or on the radio, the more likely his or her predictions were correct. In other words, the more simplistic and therefore attractive an idea is, the more likely it is to be proven false. As a writer, and as a human being, I find this amazingly sad. You really can only be lucky or good.

The last month leading up to an American election is a time crammed with voices that know exactly what the future holds. The world is full of books that have all the answers, or offer a path to them. These know-it-alls are giving us what we want, something we can cling to in the face of the world's unpredictability. What is most refreshing about The Signal and the Noise is its humility. Sometimes we have to deal with not knowing, and we need somebody to tell us that.