Nate Silver’s weatherman problem

With his popular FiveThirtyEight blog at the New York Times, using a statistical model and poll averages to predict the outcome of the election — and resulting in a fairly steady prediction of Obama victory — he has gotten an alarming number of backs up. Of course he has. He combines two of their greatest fears: losing their jobs and being required to do math.

It is a bad season to be an entrail-reader.

It used to be easy to get a reputation as a great forecaster. Octopuses could do it. You could pick the flimsiest gossamer reasons for saying what you said. Your knee started acting up, or your cattle became restless, or your cat sneezed in that peculiar way he had, and then you Knew What the Outcome Would Be. Your intuition told you.

Gradually, you built a reputation.

It is not hard to do. Even fortune cookies are occasionally correct. Stopped clocks often come oddly close to the time. Most forms of folklore include some story of a Simple Peasant or a Lucky Tailor who happens to be in the right place at the right time, where he makes several bewildering statements about eggs and dead flies that everyone takes to be True and Wise Predictions for the Kingdom’s Benefit. It’s a tidy living, if you can arrange it.

After all, there are two aspects to any good weather prediction: substance and style. You have to like the guy who tells you it will rain tomorrow. You want a weather forecaster who is agreeable, who looks nice, waves his arms in an amiable fashion at the whirling storm models and speaks in a tone that does not grate upon the ear.

But only up to a point.

All those other factors do not matter if he does not know how to forecast the weather. Especially if you don't like what he has to say, it had better be right.

Given that, as a general rule, people with the personality of weather forecasters who lack the technical skill wind up in punditry, the rise of numbers has folks quailing.

The advent of statistics in punditry has generated more comparisons to Moneyball than you can shake a skewed pole at. Sabermetrics? Say what? Forget the beautiful arc of that guy’s spin and the width of his stance? Just study the numbers? But what will we talk about on cable all day long?

Predicting Political Outcomes has been for decades an industry of vague general statements, of flair and dash and razzle-dazzle and beads and feathers. So when Silver came along with a mathematical model for this sort of thing – and, worse, turned out to be more right than not — it started to be pretty touchy going. Especially when he predicts Mitt Romney’s chance of victory around 24 percent.

I imagine this is how it felt to be an entrail-reader during the advent of Doppler radar. Doppler’s not perfect, but it’s a darn sight better than scrying storms from the livers of swing-state birds.

Everyone is screaming for their livelihoods.

Now Silver’s model is giving comfort to some and pumping others full of concern.

But to say that his evaluation of the odds of a Romney victory is a conscious effort to give the election away is like saying that someone who predicts (to pick a raw example) that a hurricane will strike is rooting for the hurricane. At least no one has ever accused the weathermen of that.

And will the shouting stop after the election?

The trouble with the kind of probability Silver depends on is that it is hard to tell, in any given instance, if the model was right.

If Mitt Romney wins on election day, it doesn’t mean Silver’s model was wrong. After all, the model has been fluctuating between giving Romney a 25 percent and 40 percent chance of winning the election. That’s a pretty good chance! If you told me I had a 35 percent chance of winning a million dollars tomorrow, I’d be excited. And if I won the money, I wouldn’t turn around and tell you your information was wrong. I’d still have no evidence I’d ever had anything more than a 35 percent chance.

The mere fact of rolling a three tells you precious little about how many threes were on the die. At least one. But beyond that, it’s difficult to say.

Really the only way for the public to start to tell if Silver’s model worked or not would be to have this election a hundred or so times. Even then you might not know.

Comments our editors find particularly useful or relevant are displayed in Top Comments, as are comments by users with these badges: . Replies to those posts appear here, as well as posts by staff writers.

To pause and restart automatic updates, click "Live" or "Paused". If paused, you'll be notified of the number of additional comments that have come in.

Comments our editors find particularly useful or relevant are displayed in Top Comments, as are comments by users with these badges: . Replies to those posts appear here, as well as posts by staff writers.