Thursday, August 6, 2009

Media Poll Coverage Fail

PPP got name dropped in the White House press conference today- the Washington Times asked a question about our Virginia 'birther' numbers and Robert Gibbs had a good response.

This got picked up by several major media outlets and the ABC News coverage of our poll had this amazing caveat:

"Phone surveys are not considered reliable by many respected pollsters."

Really? What respected pollsters did they talk to? Did George Gallup come back from the grave to tell them only door to door or mail interviews are reliable?

I jest. I'm sure they meant automated phone surveys are not considered reliable by many respected pollsters, but this is a pretty good example of how often the media has no clue what it's talking about when discussing polls. Leaving out the 'automated' part is an enormous contextual error.

No offense to the ABC people in particular- you're certainly not alone- but I think there should be a required J School course for all aspiring political journalists on how to truly understand polling, the different methodologies, and the different organizations. I'd put pretty high up on the list that they should learn to look up and analyze the track records of various organizations instead of judging them on hearsay and other subjective criteria.

For those of you who do make the effort to get it-and I'll cite Garren Shipley of the Northern Virginia Daily as an example, showing it has nothing to do with the size of your outfit- thank you.

2 comments:

Tom, I would love to know what your and your PPP colleagues think of Rasmussen's methodologies. Rasmussen consistently produces much more ideologically conservative-favorable polling results than ANYONE else, and they really seem to have jumped the shark this year with some of their numbers.

I know some of the issues...one-night samples of only 500 or so voters...a "likely voter" screen that begs, in an off-year outside the VA and NJ election polls, "likely to vote for WHAT???"...and what consistently appear to me to be major sampling deficiencies with their non-white subsamples (although I'm aware small subsamples necessarily provide volatile results...but that goes to having a sufficient sample size and therefore still is valid).

But I'm curious if you wouldn't mind weighing in on what Rasmussen does, to the best of your knowledge?