The Aggregating of Polling Aggregators

The news site Vox has taken the new polling-focused approach to political coverage to a very meta place. They don’t aggregate the polls — instead they are now aggregating the other poll aggregators. From Vox:

I’m strongly hopeful that other websites follow suit by start to create their own average of other sites’ polling averages, because then I can one up the entire news world with a planned new feature which will be an aggregate of aggregates of poll aggregators. It will be the most averagy average of other averages ever.

This does highlight, though, that attempts to predict elections with an exact percentage number have gotten a bit out of hand. The reality is that that political polls from reputable pollsters tend to be extremely accurate. That is why we have polling and why campaigns spend a fair amount of money on them. Public opinion polling works.

If you do just a simple average of the polls of reputable pollsters you tend to get a slightly better result since you are drawing from a large sample size, but the improvement is marginal. The result is something that will get well over 90 percent of races correct. Many of these aggregators try to do ever better by adjusting for things like house bias or running thousands of simulations, but the net impact these added elements make is so small it is hard to tell if they actually are improvements or just making their models look more sophisticated.

At the moment most of these polling sites differ by only a few percentage points in any given race and the difference between them will likely get much smaller as we get closer to the election and see more polls in each race. While some models will probably be labeled as the “winner” because the day before the election they said there was a 53 percent chance of Democrats winning the closest race while other put it only at 48 percent, you can’t really know if one was closer because of skill or luck without a sample size of hundreds of extremely close races, which we will never have.

While a definitive sounding percentage seems to have really triggered something in the mind of readers, the small sample size of really close races means these numbers will never be truly tested to that level of accuracy. They are providing a very good ballpark number but a quick reading of all public polling would do that too.

The Aggregating of Polling Aggregators

The news site Vox has taken the new polling-focused approach to political coverage to a very meta place. They don’t aggregate the polls — instead they are now aggregating the other poll aggregators. From Vox:

I’m strongly hopeful that other websites follow suit by start to create their own average of other sites’ polling averages, because then I can one up the entire news world with a planned new feature which will be an aggregate of aggregates of poll aggregators. It will be the most averagy average of other averages ever.

This does highlight, though, that attempts to predict elections with an exact percentage number have gotten a bit out of hand. The reality is that that political polls from reputable pollsters tend to be extremely accurate. That is why we have polling and why campaigns spend a fair amount of money on them. Public opinion polling works.

If you do just a simple average of the polls of reputable pollsters you tend to get a slightly better result since you are drawing from a large sample size, but the improvement is marginal. The result is something that will get well over 90 percent of races correct. Many of these aggregators try to do ever better by adjusting for things like house bias or running thousands of simulations, but the net impact these added elements make is so small it is hard to tell if they actually are improvements or just making their models look more sophisticated.

At the moment most of these polling sites differ by only a few percentage points in any given race and the difference between them will likely get much smaller as we get closer to the election and see more polls in each race. While some models will probably be labeled as the “winner” because the day before the election they said there was a 53 percent chance of Democrats winning the closest race while other put it only at 48 percent, you can’t really know if one was closer because of skill or luck without a sample size of hundreds of extremely close races, which we will never have.

While a definitive sounding percentage seems to have really triggered something in the mind of readers, the small sample size of really close races means these numbers will never be truly tested to that level of accuracy. They are providing a very good ballpark number but a quick reading of all public polling would do that too.

Jane Hamsher

Jane is the founder of Firedoglake.com. Her work has also appeared on the Huffington Post, Alternet and The American Prospect. She’s the author of the best selling book Killer Instinct and has produced such films Natural Born Killers and Permanent Midnight. She lives in Washington DC.Subscribe in a reader