Nate Silver and the Accuracy of Late Presidential Polls

In my blog post yesterday I explained why Nate Silver’s projection that President Obama has more than an 85% chance of winning the election tomorrow might be inaccurate. Yesterday Mr. Silver listed 60 state poll averages over the three presidential elections from 2000 to 2008 in his New York Times blog. The average state on his list had been the subject of between 6 and 7 late polls leading up to the election. The average prediction error, the difference between the actual election outcome and the average of state polls, was about one-quarter of a percentage point averaged across all states and all years. The margin of error, or 95% confidence interval, for the prediction error in a single state and election was plus or minus 6.38%. Averages of state polls are accurate, on average, but can be widely inaccurate from one state/election to the next.

Mitt Romney needs to win the bulk of swing states in order to reach 270 electoral votes. This is more difficult to do if unexpected election outcomes are statistically independent across states. For example, if nine swing states are toss-ups and the challenger needs to win at least seven of them to win the electoral college, the challenger will lose 91% of the time with statistical independence across states. If however, success in one state is correlated with success in another, the challenger’s likelihood of winning can be much different from 9%. Everyone, including Nate Silver, knows that election surprises are correlated across states. Nationwide differences in voter enthusiasm, turnout, late deciding voters and under-represented voters can all make prediction errors positively correlated across states. For Mitt Romney to win tomorrow the nationwide component of the prediction error in polls must break his way and be large enough for him to carry the bulk of the swing states.

Nate Silver believes his model has accurately captured the true correlation in prediction errors, or unexpected election outcomes, across states and polls in 2012. This is a very difficult task because: (1) we won’t know the 2012 prediction errors until the election is over, (2) polls differ in their methodology and possible biases and (3) correlations in prediction errors among states and polls in previous elections need not hold today. Mr. Silver is savvy enough to recognize that regardless of the election outcome his statement that Mitt Romney is more than a 6:1 longshot will be hard to refute. By late tomorrow, however, it will be easier to evaluate his prediction that the President will win 307 electoral votes.