Posted!

Join the Conversation

Comments

Welcome to our new and improved comments, which are for subscribers only.
This is a test to see whether we can improve the experience for you.
You do not need a Facebook profile to participate.

You will need to register before adding a comment.
Typed comments will be lost if you are not logged in.

Please be polite.
It's OK to disagree with someone's ideas, but personal attacks, insults, threats, hate speech, advocating violence and other violations can result in a ban.
If you see comments in violation of our community guidelines, please report them.

Don't trust the polls. Trust their average

Yes, it’s time for a back-to-school refresher class on how to read the presidential polls, which at this point are good indications of what will happen in November.

First assignment: What to make of the CNN poll released this morning showing Donald Trump coming out on top, by two percentage points, over Hillary Clinton among likely voters.

You’ve heard it a million times from me and others (such as political scientist Matthew Dickinson and number-cruncher Harry Enten): Look at the polling averages, not individual surveys.

As Greg Sargent at the Plum Line explained it this morning:

Surprising poll results can either be outliers, or can reflect statistical noise or short term fluctuations. Fortunately, we have a remedy for this: The polling averages, which have massive samples that cover longer periods of time and help screen out the noise. Depending on who is doing the averaging, Clinton is up by three (The Upshot), four (Real Clear Politics), or five (Huffpollster).

Those are the main numbers needed to make sense of what’s happening. And they show what they’ve shown throughout the campaign: Clinton remains likely -- not certain, but likely -- to win.

Looking at individual states and the Electoral College implications right now is more likely to confuse than enlighten. Yes, first-rate projection sites such as Drew Linzer’s at Daily Kos and Nate Silver’sFiveThirtyEight use predictions for each state, but be careful when trying to interpret any individual state's poll. Again, polling averages for states tell us more than any specific survey, and even then these averages can be misleading because there are fewer polls (and sometimes very few quality ones) in many states.

It’s still the case that the national estimates tell us what we need to know, because the winner of the national vote will almost always win the electoral vote. If there is a popular vote vs. electoral vote split, it’s still too early to see whether one party has an electoral-vote advantage.

Remember: Man-bites-dog stories always get more coverage, so “surprising” results such as today’s CNN numbers will always get the attention. And probability theory tells us that we absolutely should expect goofy-looking statistical fluctuations from time to time.

Yes, surveys can sometimes spot something new that is happening with the electorate. But there’s just no way to know from a single poll whether such a thing is really happening.

Individual polls, even from quality pollsters, could be poorly done, introducing more error. Some Democrats on Twitter this morning are going after the CNN poll, just as Republicans have gone after surveys that are good for Clinton. Polling is hard, especially since the response rates are so small these days. It has always been as much art as science, and it involves all kinds of debatable choices.

But questioning the methodology is the wrong way to respond to a poll with unwelcome results. Going down that path – carefully scrutinizing every result one doesn’t like – is certain to lead to biased results.

The right way to deal with a (seemingly) weird poll is the same way one should deal with every poll: Toss it into pool with all the others, and look at the averages.