Thursday, February 22, 2007

Polling on the Iraq war inevitably become entangled with the politics of the war, with partisans seizing on results that support their preferences. This was vividly demonstrated this week by a Public Opinion Strategies (POS) poll conducted for The Moriah Group, a Chattanooga-based strategic communications and public affairs firm. (N=800, MOE=3.5%, conducted 2/5-7/07.) The POS press release is here. POS is one of the top Republican polling firms in the country.

The ironic bit is that the poll isn't far out of line with other polling that has asked somewhat similar questions, yet those previous polls have not touched off a flurry of debate about opinion on the war. Rather, the strongly worded interpretation of the results, both in the POS press release and in the New York Post story, has provoked a reaction out of line with the novelty of these finding.

A further irony is that the POS survey uses wording for some options that seems likely to draw opinion towards those alternatives rather than others, and yet the results are only modestly different from previous polling. Whatever bias may exist in the wording, it did not produce results dramatically different from others we have seen in the last three months.

How questions are phrased, including about Iraq, can certainly affect the results. For comparison then, it helps if we can look at questions that pose reasonably similar alternatives, even if worded a bit differently. Let's look at how the recent polling has approached the question of what to do in Iraq. Eight recent polls plus POS have asked questions that address the choice among "immediate withdrawal", "withdrawal by a certain date", or "maintain troops until Iraq stabilizes." Some of these also include an option to "increase troop levels".

The POS question is

Which one of the following statements regarding the US involvement in Iraq do you MOST agree with...

1) The US should immediately withdraw its troops from Iraq.

2) Whether Iraq is stable or not, the US should set and hold to a strict timetable for withdrawing troops.

3) While I don’t agree that the US should be in the war, our troops should stay there and do whatever it takes to restore order until the Iraqis can govern and provide security to their country.

4) The Iraq War is the front line in the battle against terrorism and our troops should stay there and do whatever it takes to restore order until the Iraqis can govern and provide security to their country.

(The numbering of options is mine, for clarity.) It is a reasonable criticism of this question that options 3 and 4 offer elements that might be expected to draw respondents. Option 3 invites an expression of opposition to the war while supporting continued troop presence, and option 4 links the war explicitly to the war on terror as a rationale for maintaining troops. At the least, we might think this wording would increase support for the third and fourth options.

Other polls have phrased things differently while getting at the same policy options. For example, Fox puts the question as

Thinking about the situation in Iraq, do you think the United States should:

1. pull out all troops immediately,

2. pull out all troops gradually over the next year,

3. pull out after Iraqi troops are capable of taking over or

4. send more troops?

Here option 1 is essentially the same as the POS option 1. Option 2 is worded differently but amounts to the same policy option of withdrawal over a fixed period of time. And options 3 and 4 together are an expression in support of continued presence until the Iraqi's are able to take over. (One can argue if more troops is the same as continued presence, but I'm willing to argue that in this context.)

Gallup has offered a similar set of four options:

Here are four different plans the US (United States) could follow in dealing with the war in Iraq. Which one do you prefer--

1. withdraw all troops from Iraq immediately,

2. withdraw all troops by January 2008--that is, in 12 months' time,

3. withdraw troops, but take as many years to do this as are needed to turn control over to the Iraqis, or

4. send more troops to Iraq?

Again there are some variations in wording, but a similar thrust.

The George Washington University Battleground poll, conducted jointly by the Republican Tarrance Group and Democratic Lake Research Partners uses this wording:

As you have probably heard, there has been a lot of debate over the past few months about what the United States should do about its troops that are currently stationed in Iraq. I am going to read you four proposals on this issue. Please tell me which one comes closest to your own....

1. The US should begin immediately withdrawing all troops from Iraq. This is now a job for Iraqi forces to handle.

2. The US should set a date certain, no more than one year from now, when all troops be withdrawn from Iraq. This process should begin with some troops coming home immediately.

3. The US should keep its forces in Iraq until our military leaders there confirm that the situation in Iraq is stable enough that extremist forces will not be able to seize control once US troops leave.

4. The US should temporarily increase the number of troops in Iraq to help stabilize the situation more quickly.

Finally the Pew Research Center asks two questions:

Do you think the US (United States) should keep military troops in Iraq until the situation has stabilized, or do you think the US should bring its troops home as soon as possible?

(If bring troops home as soon as possible): Should the US remove all troops from Iraq immediately, or should the withdrawal of troops be gradual over the next year or two?

Combining the Pew responses we get three categories for 1) immediately, 2) within a year or two, and 3) keep troops until stabilized.

The variations in question wording mean exact comparison is not possible, but the categories are reasonably similar in the policy options they offer.

For comparison, I combine the last two options of the POS survey. Both say we should continue a troop presence in Iraq until the country is stabilized though they offer different rationales for that presence. Similarly, I combine those options in other surveys that say we should remain until the situation is stable or we should send more troops. Again either option implies a continued presence. This gives three policy options that are comparable across all nine polls, regardless of wording:

1) Immediate withdrawal

2) Withdrawal within a specified time frame

3) Continued presence until Iraq is stabilized

The graph at the top shows the results of this. While there is variation across the polls it is modest compared to the stability of results. The immediate withdrawal option wins from 12-21% support, and averages 16.5%. Between 28% and 40% favor withdrawal by some deadline, with an average of 34.1% choosing this option. And those favoring continued presence in Iraq range from 42% to 53%, and average 45.5%.

The POS survey is a little higher (at 50%) for the combined options 3 and 4, but still only 4.5% above the average for this question. (The average is 44.9% without the POS poll.)

The POS result of 17% for immediate withdrawal is close to the 16.5% average, and POS's 32% for withdrawal by a deadline compares to an average of 34.1% preferring that option. (Removing POS from the averages results in 16.4% and 34.4%, not an appreciable change.)

So we return to the irony. Despite question wording that seems extreme compared to the other pollsters here, POS got at most a 5 point increase in support for keeping troops in Iraq until the country is stable. And the POS question produced very little difference from the average results for immediate or timed withdrawal. So critics who have jumped on the question wording bandwagon may be right about the wording, but they are substantially wrong about the effect.

One of the most annoying aspects of question wording effects is that sometimes they are large when you don't expect it, and sometimes they are small even when you are sure they should be large.

It is similarly ironic that those who delighted in the POS results were oblivious to the fact that there was not really any new news here. Apparently other pollsters from the "drive-by media" and elsewhere had been reporting substantially the same results for a while now.

Finally, the results of this question are: 49% withdraw immediately or by a deadline, and 50% stay until Iraq is stable. I'd say a 49-50 split isn't a strong indication of overwhelming support for either side. (There are other questions in the survey that address different issues-- some are more supportive of the war and some are less. Towards the high end is "I support finishing the job in Iraq, that is, keeping the troops there until the Iraqi government can maintain control and provide security for its people" 57% agree. But also: "Iraq will never become a stable democracy", 60% agree. Picking which results you like while ignoring the ones you don't may be good politics but it is bad polling analysis.)

If we phrase the question differently do we get different answers? Sometimes. I could illustrate that with other questions about Iraq easily. Opinion about the war is complex, with shifting coalitions of supporters and opponents depending on how questions slice the policy options. And sometimes choice of wording plays a substantial role in that. But not this time.

Tuesday, February 20, 2007

Last week I wrote about the new Fox poll on support for six presidential candidates. Fox asked if you would "definitely" vote for, "might" vote for, or "would not vote for under any circumstances" each of the candidates. The question is framed in general and offered to all respondents, so the frame is in a general, rather than primary, election. This speaks to the electability issue.

Today I follow up on that post with a look at how candidate support varies by party identification. Not surprisingly Republicans are more likely to say they would vote for Republicans and not for Democrats and vice versa for Democratic partisans. By looking within partisan camps the questions throw light on how partisans may view the desirability of alternative nominees.

The plot above puts the previous results in partisan perspective. While there is a strong partisan bias, not surprisingly, there are still surprisingly few respondents who say they would "definitely vote for" candidates of their own party.

On the Republican side, more Republican partisans say they would "definitely" vote for Giuliani than would "never" vote for him. But this is not so for either McCain or Gingrich (and by a wide margin for the latter.) Among Democrats, a plurality would "definitely" vote for Clinton rather than "never". In this Clinton has the largest margin among the six candidates. Obama's "definite" supports only barely exceed his "never" critics (among Democrats!), and Edwards falls well short of a plurality of "definite" support.

As we've seen in previous analyses, Clinton is the most polarizing of the six candidates. Republicans respond 79% "Never" and less than 5% "definitely", with only 15% saying they "might" vote for her. Among Democrats, only 17% say never while 32% are definite supporters, with 48% saying they might vote for her. While the potential support is large, the actual opposition is very strong. Independents fall between, 39% never, 19% definite and 40% might. This puts Clinton solidly in the lead among her partisans, but with very strong opposition among Republicans. No surprise there.

Neither Giuliani nor McCain draw the same strength of Democratic opposition that Clinton does from Republicans. To be sure between 55% and 60% of Democrats say they would never support either of these, but that leaves a substantial number of Dems who would at least consider one of these Republican contenders. Giuliani does relatively better among Independents, with 30% saying they would never support, and 12% definite.

McCain on the other hand faces more opposition from Independents than we might expect given his 2000 reputation for appealing to independents and Democrats. Among independents 42% said they would never vote for McCain, while only 7% said they definitely would. So a supposed strength of the McCain campaign looks in these data to be far from a source of confidence.

Former speaker Gingrich may yet emerge as a credible candidate, but his current polling does not support a belief that he would be a particularly strong candidate even among Republicans, and certainly not among independents or Democrats. As with all these early polls, much change can yet occur, so these results should not be interpreted as predictions of future outcomes. But they do show that Gingrich has much to accomplish if he is to win over supporters from both inside and outside his party.

The ratings of Barak Obama show relatively high "might" vote for responses, as we would expect from a relative newcomer to national politics. His very slight edge among Democrats leaves him with deficits among independents who remain persuadable, and Republicans who are as opposed to him as they are to Edwards but not so opposed as to Clinton.

Edwards, who has often been viewed in the press as an attractive candidate with positive images left over from his 2004 campaign does not come off so well in these data. Democrats are more opposed than supportive, and independents do not appear to be drawn to him at this point.

What about those who "might" vote for these candidates? How do they alter the positions?

The closer to the lower left corner the points are, the more respondents said they "might" vote for the candidates. In all six figures it is apparent that this represents a large segment of the population. These "persuadable" voters could certainly change the landscape as they make up their minds.

One way to look at this is to add the "mights" in with the "definite" supporters to see the MAXIMUM level of support for each candidate. The figure below does this.

With the potential supporters added in, all candidates move close to the diagonal line, which is the limit of possible support, given the level of "never" responses. This doesn't change the relative ordering of the candidates on the opposition (vertical) dimension, but it shows potential growth in support for each. For candidates such as Obama and Giuliani the potential upside is great. For Gingrich it is most limited, while Clinton can benefit a lot among Democrats, not so much among independents and little among Republicans. McCain's potential is also substantial, though perhaps not as much as Giuliani's. Edwards falls between Obama and Clinton in potential.

Of course, this is the MAXIMUM gain for each candidate, and since not all of those who say they "might" vote for a candidate will in fact, the truth lies somewhat short of this rosiest of outcomes.

The partisan camps clearly align with their party's candidates, but the lack of overwhelming support within any camp clearly shows that this race is not solidly locked up within parties, let alone in the general election. And, we've yet to hear from those in single digits who will not stay there forever, if past elections are any indication. Stay tunes.

Monday, February 19, 2007

More than any recent election, the 2008 presidential contest raises the issue of whether Americans are ready to vote for a ... (woman, black, Hispanic, Mormon) for president. In 1960 Kennedy's Catholicism was a key question. In 1984 Geraldine Ferraro gave us the possibility of a woman a "heart beat away", if not quite in the oval office. With at least four "group" members in the race this year, the question of what groups are acceptable and not is again before us.

The data are pretty clear that women and African-American's have gained acceptance, at least in the abstract. In data collected since 1956 by Gallup, and more recently by a variety of other pollsters, we can track changes over time to the question (as phrased by Gallup):

If your party nominated a generally well-qualified person for president who happened to be ______, would you vote for that person?

Most pollsters use a similarly worded question. (Interestingly Gallup used "well qualified MAN" rather than "person" until the 1990s, except of course when asking about a female candidate.) The graph above shows trends in these questions for all groups and all pollsters I could find since the 1950s, 144 questions in all.

The striking effect of Kennedy's campaign and election is clear. While about 70 percent of the public said they would vote for a Catholic from 1955-1959, this jumped over 10 points following Kennedy's election. (62% said they would vote for a Catholic in 1940, the earliest poll I could find.) Since then it has gradually risen to a plateau in the mid-90s, though still slightly below the number comfortable voting for a "Baptist", the long time favorite in this graph.

Support for Jewish candidates has remained a bit below that for Catholics, maintaining a small but reliable gap. Interestingly, Kennedy's Catholicism may have helped increase acceptance for Jews as a side effect of the nation's consideration of religion in 1960. The upturn in acceptance of Jews as candidates tracks with the rise of Catholic acceptability in the early 1960s.

Mormon's may or may not have benefited from that movement as well. I could locate no questions about Mormon candidates prior to 1967 so we can't know what the trend was in the 1950s or early 1960s. In 1967, however, acceptance of Mormon candidates ran about 7 points behind that of Jews, and closer to 15 points behind Catholics. In the time since, there have only been two repetitions of the Mormon question (that I could find), and these two indicate stability through 2000 and a decline in 2007. With only 3 polls, it is very dangerous to conclude that there has been a real decline, or even to claim a very good estimate of the acceptability of a Mormon candidate. But given the little evidence that is available, Mormons clearly face more resistance from the public than Catholics or Jews (not to mention Baptists.) Governor Mitt Romney's campaign gives us an excellent chance to see if this resistance is primarily because voters haven't thought about it, and once confronted with an attractive candidate, as with Kennedy, will revise their opinions, or if this represents more strongly held suspicion of Mormons that will continue to register in the polls.

The one religiously defined group that should probably not jump in the presidential race is atheists. The public remains strongly opposed to putting non-believers in the White House. Despite some growth in tolerance of atheists this group still is acceptable to only just over 45% of the public. Homosexuals are the only group less acceptable than atheists, but that appears to have changed by 1999 when homosexual candidates gained acceptance by 59% of the public. (Again, beware the small number of polls that have asked about homosexuals. This is not a very reliable estimate, based on only four polls.)

The two groups that have moved most dramatically are not religiously defined. Women and blacks have moved from 50% or less support in 1955 to currently high acceptance rates. Blacks have in fact moved up to match Catholics and perhaps even Baptists in professed willingness to vote for a member of the group. Women were on a similar trajectory until recently. Since 2000 there has been some modest decline in acceptance of women candidates. It is possible that this is due to voters thinking of Hilary Clinton specifically in response to this question recently, raising a partisan bias, or it could be a fluke in the most recent half dozen polls. I've not located polls asking about support for an Hispanic candidate so not trend or other data are available on that for comparison.

The overall rise across most of these groups in willingness to vote for candidates from the group reflects some important changes in society over the past 50 years. The relative stratification of groups, however, is also generally stable. The order of acceptability--- Baptist, Catholic, Jew, Mormon and atheist--- has not shifted even as level of acceptance has risen. Women, blacks and perhaps homosexuals have seen dramatic rises, with the first two groups changing order with other groups. Still, the stratification remains clear, if reduced in magnitude. And some groups such as atheists and homosexuals remain well short of widespread acceptance as potential presidential candidates.

It is also worth noting that this measure of potential vote does not mean that in the actual choice these characteristics are irrelevant to voting decisions. Presented with an actual woman, black, Mormon, Jew or Hispanic, each of these categories may still produce a reduction in the probability of voting support. At least if the opponent is a Baptist male. It seems likely than only the election of presidents from these categories can assure us that such categories have become irrelevant in fact.

The Romney campaign offers the best opportunity for a natural experiment to see how malleable these perceptions may be. Note that the acceptance of Catholics did not increase until after the 1960 election. The last time the Catholic question was asked in Gallup data was December of 1959, and not again until August 1961. We are left to wonder if Kennedy increased acceptance during his nomination race and/or during the general election or if this change occurred only after he was elected president. Romney will have to overcome the resistance to Mormon candidates, and with the huge increase in polling, we should be able to use his race as an opportunity to learn something about how rapidly such attitudes can change, or not. Let's hope pollsters will monitor this question for us. Interesting social science, in any case.

Thursday, February 15, 2007

Fox/Opinion Dynamics has a nice new poll out tonight. One striking result is that NONE of the six leading candidates enjoys having more strong supporters than strong opponents. Fox asked:

Now I am going to read a list of possible candidates for the next presidential race. For each one, please tell me whether you would definitely vote for that candidate for president, if you might vote for that candidate, or if under no conditions would you vote for that candidate.

The "don't know" rate for all these candidates is pretty low, 10% or less.

The result, plotted above, shows that all six have considerably more voters saying they would never vote for the candidate than saying they definitely would vote for them. In this figure being towards the upper left is bad news, lower right is good news. As in previous polls, former Speaker Newt Gingrich fares worst among these candidates, with very substantial numbers saying they would not vote for him under any circumstances. The rest of the candidates are somewhat bunched together.

The further a candidate is from the diagonal line (or closer to the lower left corner) the more people said they "might" vote for them. (Recall the DK rate is both low and pretty even across candidates, so this is mostly a true statement.) Senator Clinton is the candidate with the fewest "might" vote for responses, but even in her case 34% said they would consider her. Obama shows the most room for growth with 45% saying "might" vote for him. The others range from 39-44% saying might.

So these front runners have a long way to go, and must all face stiff strong opposition from significant chunks of the electorate. Or put differently, those persuadable voters will matter a lot.

Monday, February 12, 2007

(UPDATED 2/13/07: The new USAToday/Gallup Poll is in with a 37% approval rating, up from the 32% of the previous poll discussed here. This raises the trend estimate to 33.3%, still a new low point, but not as drastic as the 32.5% prior to the new poll. I've updated the title and added the updated Gallup plot above, but not changed the text below which is unaffected by this new result.)

President Bush's approval ratings have fallen to 32% in the four most recent polls from CBS, AP, Harris and Gallup. After readings of 38% from Fox, 37% from Time and 39% from Democracy Corps (which uses "likely voters" rather than adults) helped boost the President's trend estimate a bit recently, this string of new polls at 32% has caused the estimate to sink well below previous all time lows, to the current 32.5%. This is well below the previous low from May 2006 at 34.0%. (Rumor has it that new polling will be out in a few hours that may improve the President's standing, or not. Check the morning papers.) (NOTE: USAToday/Gallup did come in at 37% based on their 2/9-11/07 poll, raising the trend estimate to 33.3%.)

The figures I'm posting here can also be seen at our Pollster.com comparison of all presidential approval polling here. The figures show how each recent poll has varied around the trend estimate. CBS has been running below trend, so the current reading of 32% is a move UP for CBS, but it only brings that poll to just below my current estimate.

The Associated Press/Ipsos poll has generally tracked the trend pretty well, with six of the last eight polls within a point or so of the trend. Some earlier AP readings were consistently a bit below trend. This should remind us that polls can vary a bit over time in their relationship to the trend line.

Harris has generally run a bit below trend, so the current reading is closer to the trend than is usual. Harris uses a four point, Excellent, Good, Fair, Poor rating system, rather than the more commonly used dichotomous Approve/Disapprove measure. This difference in question wording may play a role in Harris' "house effect." (Search labels for "house effects" for more on that.)

Gallup does more approval polling than anyone and is on average just over a point above the trend. Some recent polling has reflected that, though three of the last seven Gallup readings have been right on trend.

The current downward trend in approval of the President reflects his inability to rally support for his Iraq troop increase decision along with increased criticism from a Democratic congress and more dissent from within his own party.

Given the randomness of polls, we should expect to see some new polling come in up to four or five points above the current trend, so as high as 36% or 37% approval. But if the current "true" approval rating is close to my estimate of 32.5%, we should also see some polling below 30% in the near future. So far there isn't much good news for the White House.

Thursday, February 01, 2007

Democrats gained an average of 3.4% and Republicans lost 3.0% in partisan identification between 2005 and 2006, according to a new Gallup estimate based on over 30,000 interviews conducted throughout 2006. Gallup aggregated polls throughout the year to create estimates of party identification at the state level, as they did in 2005 and previous years. Gallup's report of the results is here.

The plot above shows how uniform the shift in Democratic and Republican partisanship was across the states. The colors of the points reflect the Democratic minus Republican balance in 2005-- the darker the blue the more net-Democratic identifiers and the darker the red the more net-Republican identifiers. Light or pale points are closely balanced states, as of 2005. The size of the points is proportional to the size of the state.

There is no apparent pattern to the shifts in partisanship: regardless of partisanship or partisan balance in 2005, states shifted by about the same amount in 2006. Likewise, Republican losses shifted uniformly. The +3.4 percentage point shift for the Democrats, and the -3.0 point shift against the Republicans produced a net -6.4 point loss for the Republicans in the balance of partisanship. The lower left figure in the plot shows that independents shifted more or less randomly between 2005 and 2006.

These shifts could, in principle, represent a non-trivial gain for the Democratic party. Recall that after the 2004 election there was considerable talk in Republican circles of establishing an "enduring Republican majority", a goal that seemed within the party's grasp though certainly not assured. That hope is clearly out of reach at the moment.

Before Democrats go wild with joy, there remains a question about the electoral impact of these partisan shifts. Party identification is the strongest single predictor of vote choice at the individual level. But the shifts in partisanship in the Gallup data do not predict the shift in voting for the U.S. House in 2006.

The bottom right figure above shows that the Republican U.S. House vote shifted more or less uniformly across states as well. However, when we look at the relationship between party id shift and vote shift across states, there is no relationship at all, as seen in the figure below.

Controlling for both Democratic gains and Republican losses doesn't add to the relationship. So the conclusion here is that both partisanship and vote shifted against the Republican party in 2006, but the variation in shifts appears to have been essentially independent between partisanship and vote.

Democratic gains and Republican losses in partisanship may affect the 2008 prospects in the House (and other) elections. But in 2006, it appears both vote and partisanship responded to conditions in the country without a clear impact of changes in partisanship on changes in vote.

Fox News' new poll, completed 1/30-31/07, provides a new look at the favorable and unfavorable ratings of the leading presidential contenders. Rudy Giuliani continues to lead the pack in the balance of favorable to unfavorable ratings in the new Fox poll with 54% favorable to 24% unfavorable . John McCain enjoys a net-positive evaluation, though not as strongly positive as Giuliani, at 45%-29%. Among Democrats, Hillary Clinton strongly divides voters yet manages a 50%-44% net positive rating. Less well known John Edwards has an 8 point net positive (41%-33%) while Barack Obama has the best balance, 41%-20% but the largest number of voters unable to rate him (38%.)

Three political figures have net negative ratings in the Fox poll. Mitt Romney is the least well known among all the candidates and suffers a net negative rating of 11%-22%, but with 67% unable to rate. Newt Gingrich, who has adopted a wait-and-see approach to the presidential campaign, suffers a 22%-49% net negative, while Al Gore, who seems quite uninterested in the race, has a somewhat better but still net negative 39%-51%.

The extent to which partisans divide over the candidates is clear above. Hillary Clinton remains the most polarizing figure with strongly net negative views among Republicans, strongly positive views among Democrats, and a near balance yet net negative among independents. (The purple dot is independents, the black dot is the population as a whole, with the now conventional red for Republican and blue for Democrats.)

McCain and Giuliani do well among independents and even with Democrats. Gore and Gingrich in contrast both divide the electorate and manage net positives only among their own partisans. Edwards splits partisan camps, though not as strongly as Clinton and roughly balances the sides and among independents. Obama does quite well among independents, but is developing a Republican opposition. Romney remains a mystery to all partisan groups, which cluster and don't know him.

There has been some movement in the Fox ratings, but the most recent prior poll varies a great deal in how old it is. Clinton's most recent poll was just last October, and little seems to have changed. Several other candidates were measured in May 2006. But Edwards' last reading was in October 2004 (during his run for the Vice-Presidency) and Gingrich's dates back to 1998! So look at change with due caution.

Finally, the balance of not knowing the candidate versus knowing but not enough to rate them can be instructive. In the earlier CBS News poll, there was a clear differentiation between the better known and the lesser known candidates. The new Fox poll lacks almost all of the lesser known figures, so that pattern is less pronounced here than in the CBS data (here)Still, among these candidates, it is interesting the Obama now more closely resembles the better known candidates, while only Romney remains in the space occupied by the least known in the CBS poll.

Links

About Me

I am co-founder of Pollster.com and founder of PollsAndVotes.com.
I am also a professor of political science at the University of Wisconsin, where I teach statistical analysis of polls, public opinion and election results. Data visualization is central to my approach to analysis.
Email me!