However, CNN released a second poll of registered voters yesterday, conducted by Fairleigh University from March 21-28 that indicates Obama is in essentially a dead heat with both Mike Huckabee (46%-46%) and Mitt Romney (Obama leads 44%-43%) in the 2012 race.

As Peter Baumann notes in an e-mail, these results seem surprising; it is more often the case that incumbents do worse against hypothetical generic opponents because survey respondents can fill in the blank with their ideal candidate. When matched up with specific candidates, however, voters are asked to choose between two real people, and incumbents often benefit. If Peter is right, what explains the results in these two surveys which show Obama doing better against the hypothetical Republican? The answer, I suspect, lies in the question wording of the respective surveys and in the smaller sample size of the Dickinson survey.

Pew begins its survey by asking respondents whether they have a favorable or unfavorable opinion of a number of people, beginning with President Obama. They also ask about former presidents Bush and Clinton and House Speaker John Boehner.

Although Obama’s job approval ratings are hovering in the mid-40’s, his personal favorability ratings have consistently been higher. In the Pew poll, 58% of respondents view Obama “very” or “mostly” favorably, compared to 27% who view Boehner this way. (Bush is viewed favorably by 42% and Clinton leads the pack with 67% favorability rating). After leading with the question about Obama’s favorability rating, and implicitly comparing it to that of Boehner and Bush (and Clinton), Pew then goes on to ask about the 2012 hypothetical matchup with a generic Republican. So respondents, in thinking about that generic “Republican”, do so in the context of being asked about two unpopular Republicans: Bush and Boehner.

Now look at how Fairleigh Dickinson begins its survey.

By beginning the survey this way, respondents are “primed” to answer the head-to-head matchups question primarily in terms of their views on whether the country is heading in the right direction, and on Obama’s job approval – not his favorability ratings compared to other Republicans. As you can see, a substantial percentage of Americans believe the country is going in the wrong direction, and a near majority disapprove of the job Obama is doing. It’s not surprising, given the way the question wording primes them, that respondents in the Fairleigh Dickinson poll see Obama doing less well in head-to-head matchups than Pew finds him doing against a generic Republican opponent.

There is a second difference between the polls as well. Although the Dickinson survey of 800 respondents has a margin of error of +/- 3.5%, which is quite similar to Pew’s margin of error (+/- 3%) this does not hold for all the survey questions in the Dickinson poll. In fact, the numbers for these subsamples of respondents comparing Obama to Romney and Huckabee are much smaller (see table above) meaning the margin of error for both groups is substantially larger than the poll’s overall margin of error. So, we need to view these results indicating that Obama is in a dead heat with both candidates with the requisite dose of salt.

My point here is not to say that one poll is more accurate than the other. It is to remind you that, as we enter the 2012 presidential campaign, you will be inundated with polling data, much of which will be spun by politicians and pundits according to their political persuasions and which will be hyped by news media outlets that views poll results as “hard” news. As consumers, you need to cut through the spin and hype to examine the details of the polls. Pay attention to both question wording and order, and to the margin of error associated with survey subsamples. All polls are not alike.

Addendum (2:32 p.m.) Three days ago Quinnipiac released a nationwide poll taken March 21-28 that received quite a bit of national play for its headline “Obama Gets Lowest Approval, Reelect Score Ever”. His approval/disapproval rating was 42/48%, and 50% of respondents said he did not deserve reelection. Of greater relevance to this post, however, is that in the hypothetical 2012 matchup between Obama and a generic Republican, the poll found that Obama got 36% of the vote compared to 37% against the unnamed Republican. In contrast to the Pew poll, however, the Quinnipiac poll used different question wording; the generic matchup question was the first one Quinnipiac asked respondents, so they were not primed to respond in terms of a specific Republican, as they were in the Pew poll. Again, it is a reminder that question order can influence polling results, even when identical questions are asked.

2 comments

Polling data is baaccck!! I’ve been skimming over these and remembering with (significant) glee the discussions of polling results and the numerous ways in which they are misleading, if Obama hovers around 47% approval give or take as a general job approval rating – how does that compare in significance to when he is directly polled against against a hypothetical Republican challenger?

Elsewhere in the prediction universe (it wasn’t a poll as far as I recall) someone was basically saying if “three things” happen/don’t happen before September 2012, Obama is in good shape for re-election. They included no major economic back-tracking, generally positive international engagement, and one other….which made me think back to the three predictors of a presidential (general election) outcome that we talked about in PSCI101. One was the rate of economic growth in the 3/4 quarter yes? guilty look – what were the other two?

Hi Tarsi – In addition to some measure of economic growth, election forecasters often include some measure for the distribution of partisanship (Republican, Democrat or independent) in the electorate and an indicator for how long the incumbent president’s party has held the White House. This might be what you have in mind. But there are variations on this – some models include a measure of presidential approval as well.