One small weakness in the Meta-Analysis

August 10th, 2008, 4:56pm by Sam Wang

The Meta-Analysis made a correct 2004 Election Eve prediction, and gives good single-glance information as the race heats up. Currently, in August, it moves relatively slowly due to the sparseness of polling data. But by September this problem will fade as the race picks up.

As currently implemented, the Meta-Analysis uses recent polls to generate a high-precision estimate of the outcome of an election held today. It calculates the probability distribution of EV outcomes. By Election Eve, this snapshot should be a good predictor of the final outcome.

However, August is a slow polling month – despite the fact that August 2004 was when the Swift Boat Veterans for Truth attacks began. In the last seven days, Electoral-vote.com reported only 24 new polls, meaning that fewer than half of the states were updated. Compare that with the 2004 database from later in the season, which showed 60-100 polls per week starting in mid-September and 264 polls in the week immediately before to the election. Look for the response speed of the Meta-Analysis to improve quite a bit in September.

In Ohio, a key state, the three most recent polls ended 15, 20, and 22 days ago. This implies that it will take several weeks for us to see the impact of a major political event. For example, it took about two weeks to respond fully to Hillary’s withdrawal on June 7th (see history graph).

So where are things today, really? Using the last week of available polling data from Electoral-vote.com, the median EV estimate is Obama 294, McCain 244, with a +/-1-sigma band of Obama [279,310] and a 95% confidence interval of Obama [267,325]. In this calculation, the most recent Michigan poll is 3 days old. As of today the race is fairly close.

8 Comments so far ↓

Over the past two months or so polls show that Obama has had an approximately 10% advantage in electoral votes, and only about a 5% (or less) advantage in national polls. Could you comment on this difference? Is there a fundamental difference between the sampling in national and state-wide polls? Does the difference in sampling frequencies of these polls capture different dynamics?

Yeah, until Labor Day, most of this is speculation without a whole lot of data to back anyone up. Once we get there, I would expect your Meta- analysis to to take top billing… but until then, it’s going to be a lot of “creative” ways to analyze the same uninformative data just to confuse everybody.

Independent, there are two reasons. The biggest is that because of the winner-take-all mechanism at work in nearly all states, popular vote victories are amplified. Generally, in more competitive races, a 1% popular vote margin translates to about a 30-35 EV margin, or about 6%. In extreme cases this leads to very lopsided results. For example, when Reagan was re-elected in 1984, Mondale still won over 40% of the popular vote but only 13 electoral votes, or 2% out of the total.

The second reason is minor, and has to do with the fact that the Electoral College does not always elect the popular vote winner. The 2000 election comes to mind, when Gore won the most votes. Based on today’s polls, if all the Obama-McCain margins shifted by 3.28% the median would be a 269-269 tie. Yet those same polls suggest a popular vote lead of +3.53% for Obama. In other words, McCain has a tiny advantage equivalent to about 0.2% in popular votes.

J.W. – yes, it’s too bad about the slowness of polls. I was considering whether to add a trend compensator based on national polls. But it would add a whopping amount of uncertainty – equivalent to at least one popular percentage point, and it would only be useful in August.

I too am a returning spectator. Glad to see you up again.
I was wondering if you thought about and if can account for what some media outlet are calling the “lie” that white voters tell themselves when they respond to a poll. They say the will vote for Obama, but will they? That could change the election very rapidly…
Thanks!

Marc – I haven’t thought of a way to capture staleness that’s any better than you looking at Pollster.com. It varies: in VA the polls are fresh (2 days) but in OH not (at least 20 days). We are currently using the 3 most recent polls at Pollster.com.

Marco – The evidence suggests that the “Bradley/Wilder effect” occurs for exit polls but not for the type of polls reported here. At some point I’ll post on this in the context of the bias tool I developed in 2004, which allows such effects to be added to the assumptions.