Wednesday, December 28, 2011

PPP, Selzer, Cell phones, and Iowa Polls

With the Iowa Republican caucus only 1 week away, pollsters will poll the state like nobody's business. Some polls will have some candidates higher than others, and some polls will ultimately be more accurate than others. So how do we know which polls we should trust?

Most long-time Iowa watchers will tell you to watch the well known Selzer and Co. Des Moines Register poll. The poll's most recent triumphs include correctly nailing an easy victory for both Barack Obama and Mike Huckabee in 2008. In fact, according to some Washington insiders, it's the only poll to watch.

I disagree. Selzer's polling deserves its praise, but nobody's perfect.

Just look back to the 2010 Iowa Republican gubernatorial primary. Midterm primaries are obviously different than presidential caucuses (though remember Iowa Republicans use a straw-poll procedure that is much closer to a primary than what most think of a caucus). Still, this was a very recent matchup involving mostly Republican voters in Iowa.

In this contest, Selzer's final poll had Terry Branstad winning by 28%, while automated and frequent Iowa pollster Public Policy Polling (PPP) had Branstad winning by 15%. Branstand's final margin was only 9.4%.

PPP was clearly the more accurate pollster in predicting the final margin.

Now, this isn't to say that PPP will end being more accurate in the 2012 Iowa Republican caucus. I am merely saying that no pollster is the one and only.

While there is such a thing as really bad pollsters (Zogby Interactive, I'm looking at you), we really cannot rank most pollsters from best to worst. Mark Blumenthal first demonstrated this point in 2010, Dick Bennett has continuously showed it, and my own work looking back at the 2010 midterm elections confirms Blumenthal and Bennett's finding.

What about the argument that PPP and other automated pollsters may be missing cell-phone only voters? There is some evidence that there is a difference in the political opinions of cell-phone only users vs. landline users, and Iowa has a relatively high percentage of adults only reachable by cell-phone.

The real question we should be asking though is how will this affect polls for the 2012 Republican Iowa Caucus.

In 2010, automated polls were on average as accurate as live interview polls. In 2011, we saw automated pollsters accurately predict difficult to poll House special elections in both New York's 9th and 26th districts. And let's not forget, PPP did better than live-interviewer Sezler and Co. in the 2010 Iowa Republican primary.

The evidence from recent history seems rather thin that automated polls will miss something that live-interviewers will capture just because automated polls are automated.

You can look at this data in a few ways. First, Ron Paul did better in this polling sample thanks to cell-phones being included*. Second, cell-phones did not greatly impact the poll's findings. In fact, the variation between candidates' percentages in landline only (as automated phones have) and overall voting population was statistically insignificant.

But let's take a look at some more data.

During the 11/27-12/6 window in which the Marist poll was taken, a lot of pollsters were in the field. The race seemed pretty static during this period with Newt Gingrich with a solid lead. We had a good mix of both live-interviewers who called cell-phones and automated pollsters who did not. Did a similar Ron Paul effect happen within this group as in the Marist poll?

Somewhat to my surprise, it did! On average live-interviewers who called cell-phones found Ron Paul's support to be at 17.2%, while automated polls found it to be at only 14%. This is the same exact 3% gap Marist found.

Here's the kicker stat though. PPP was the only automated pollster during this period that did not exhibit the Ron Paul cell-phone gap. At 18%, Ron Paul's PPP percentage of the vote was tied for the highest any pollster recorded Ron Paul at during this period.

In recent weeks, PPP has continued to show Ron Paul flying high. Their polls have shown Paul at a greater percentage than most other pollsters (e.g. automated pollsters Rasmussen and We Ask America).

I think PPP may have something. The lack of cell-phone voters seems to be affecting most automated samples, but PPP somehow seems immune.

When we begin to see more live-interview polls goes into the field later this week, it's probable most of them will find the same Ron Paul lead in Iowa that PPP has. The question of whether that lead holds and which pollster is ultimately the most accurate is still one to be answered, but I think we will have a pretty good idea of the final result going into the caucus.

*Something I know comes to no surprise to his supporters who CONSTANTLY argue this point.

******CORRECTION: CNN's state polls are landline, but do not include cell-phones. Only their national polls include cell-phone only respondents. The recalculated average for the third chart is 17.3% for Paul support among landline with cell-phone polls. In other words, the conclusion is the same.

About Me

I am a graduate of Hanover, NH. I graduated with adegree in government summa cum laude with a concentration in statistics and elections.
I have previously interned at the NBC Political Unit in Washington D.C. (Spring 2009) & Pollster.com (spring and summer 2010).
For questions pertaining to this blog or about me in general, you can reach me at HJEnten@gmail.com
Visit me on Twitter (currently inactive): twitter.com/ForecasterEnten
Email for a resume.