Demystifying the Science and Art of Political Polling - By Mark Blumenthal

June 29, 2006

It's What We Call Spin?

Thanks to alert reader LS who tipped me off to an old story that I had missed, but that is still worthy of notice on MP. Nearly two months ago, Dick Leggitt, the campaign manager for Colorado Republican Senate candidate Marc Holtzman admitted under oath that he had made up poll numbers and given them to a Denver Post reporter:

Holtzman's campaign manager, Dick Leggitt, admitted Friday [4/28] that he lied to a Denver Post reporter in an e-mail by fabricating poll numbers that purportedly showed Holtzman's name recognition going from "10 percent to 70 percent and his favorables among GOP primary voters are now just slightly less than (U.S. Rep. Bob) Beauprez's (39 to 42)."

Leggitt also admitted he made up polling results indicating that support for ballot measures Referendums C and D was lagging.

"We didn't have any polling results," Leggitt said during the administrative court hearing. "It's what we in the election business call spin."

Leggitt resigned from the Holtzman campaign a week later, but not before admitting to a Denver Post Columnist that making up poll numbers "probably wasn't the smartest thing I ever did."

No argument there.

From my twenty years of experience as a partisan campaign pollster, I can tell you that while it happens rarely, Leggitt is not the first campaign spinmeister to creatively invent poll results. Pollsters hate this, as the imaginary numbers inevitably damage our credibility. A pollster that tolerates fabricated numbers will not be in business for long. That is why my company and many others include language in our contracts obliging clients to avoid misrepresentation of our findings and reserving the legal right to publicly correct any such misrepresentation, should it ever occur.

This episode also serves as a cautionary tale to those who follow politics and -- most of all -- to those who cover it. Here is a tip for reporters: If a campaign operative spins you about favorable internal poll numbers, try to get a confirmation direclty from the pollster. If they are willing to put their name (and reputation) behind the results, you can have much greater confidence that they are genuine. If the pollster has not put the numbers in a memo on their letterhead, or will not otherwise confirm that the numbers are real, they probably aren't.

June 28, 2006

Update: Withdrawal Timetable Polls

A quick update to yesterday's post: With the help of Prof. Charles Franklin, I bring you a chart showing the trend in five questions that have tracked attitudes on whether and when to withdraw U.S. troops from Iraq. As explained yesterday, these questions are far from identical: Some ask explicitly about a timeline for withdrawal, others ask more generally whether troops should be withdrawn "as soon as possible" or stay "as long as it takes" or "until the situation has stabilized." Nonetheless, as Franklin's chart illustrates, support for withdrawing troops sooner rather than later on these measures has tended to converge since mid-2005, with no sign of an overall trend.

Notice the way the two measures that showed the greatest support for leaving - the lines at the top of the graph - have trended downward since about October 2004, while the support for a withdrawal timeline as measured by ABC News and The Washington Post (the light blue line) has trended upward. The net result is that a roughly 20-point spread between the highest and lowest numbers in October 2005 has reduced to a roughly 10-point spread.

Thus my conclusion yesterday that as the debate on timelines and other proposals for withdrawal has intensified, real underlying attitudes have been hardening and are less susceptible to the variation in opinion poll question wording and context.

The other trend evident in the graphic is that support for withdrawing troops sooner rather than later has increased significantly since the 2004 elections (as measured by the two more generally worded questions from CBS News and the Pew Research Center). For example, in October 2004, Pew showed a 21 point net preference for keeping troops in Iraq "until the situation has stabilized" (57%) rather than bringing troops home "as soon as possible" (36%). On the most recent Pew survey, that preference narrowed to a net 5 points (50% to 45% -- see Q52).

So while attitudes on withdrawing troops from Iraq appears to be hardening, Americans are more closely divided on whether to withdraw than they were in 2004.

UPDATE II: The Washington Post's Chris Cillizza has more about questions on a potential troop withdrawal in his weekly "Parsing the Polls" feature.

UPDATE III (6/29): As noted by DemFromCt in a comment below and on his site TheNextHurrah, Gallup's Frank Newport has an analysis posted this morning that takes a close look at their questions on proposals to withdraw troops from Iraq:

A majority of Americans would like Congress to pass a resolution
addressing the withdrawal of U.S. troops from Iraq, but Americans are
split down the middle over whether that withdrawal should be based on a
dated timetable, or left open-ended. Meanwhile, a majority of Americans
continue to believe that U.S. involvement in Iraq was a mistake.

As with all Gallup.com analysis, Newport's report is free for today, but available only to subscribers after that.

June 27, 2006

Polling on a Timetable for Withdrawal

The national polls released over the last few weeks have included a flurry of new or updated questions on proposals to set a "timetable" for the withdrawal of U.S. troops from Iraq. Back in December 2005, I noted that while attitudes about the wisdom of the War and Bush's performance on Iraq were mostly consistent, questions about what we should do next in Iraq were highly variable. I argued at the time that wide variation indicated an underlying uncertainty about what the U.S. Iraq policy should be. Looking at the questions pollsters are now asking about prospective Iraq policy, I see fewer differences and far more consistency, a finding that may reflect a gradual hardening of opinion.

Today I explored some of the recent questions on the Polling Report, and despite some important variations in question language and wording, their results are reasonably consistent. Support for some sort of timetable or withdraw varies between 47% and 54%. Some of these differences are meaningful, but the range is still not much different than the 3% margin of error reported for most of these surveys:

ABC News/Washington Post- Some people say the Bush Administration should set a deadline for withdrawing U.S. military forces from Iraq in order to avoid further casualties. Others say knowing when the U.S. would pull out would only encourage the anti-government insurgents. Do you yourself think the United States should or should not set a deadline for withdrawing U.S. forces from Iraq? (June 22-25, 2006, n=1,000 adults)
47% Should
51% Should not
1% unsure

USA Today/Gallup Poll - Here are four different plans the U.S. could follow in dealing with the war in Iraq. Which ONE do you prefer? Withdraw all troops from Iraq immediately. Withdraw all troops by June 2007, that is, in 12 months' time. Withdraw troops, but take as many years to do this as are needed to turn control over to the Iraqis. OR, Send more troops to Iraq (June 23-25, 2006, n=1,000 adults).
17% Withdraw immediately
33% Withdraw by June 2007
41% Take as long as needed
8% Send more troops
1% No opinion
50% Total: Withdraw immediately or within 12 months
49% Total stay s long as needed or send more troops

CNN/ORC - Here are four different plans the U.S. could follow in dealing with the war in Iraq. Which ONE do you prefer? Withdraw all troops from Iraq immediately. Withdraw all troops by June 2007, that is, in 12 months' time. Withdraw troops, but take as many years to do this as are needed to turn control over to the Iraqis. OR, Send more troops to Iraq (June 8-11, 2006, n=1,031 adults)
18% Withdraw immediately
29% Withdraw by June 2007
42% Take as long as needed
6% Send more troops
1% No opinion
47% Total: Withdraw immediately or within 12 months
48% Total stay s long as needed or send more troops

Pew Research Center - Do you think the U.S. should or should not set a timetable for when troops will be withdrawn from Iraq?" (June 14-19, 2006, n=1,501 adults)
52% Should
42% Should not
2% Get out now (volunteered)
2% Unsure

MP reader JB emailed to ask about the apparent conflict between some of these results and another question also asked on the most recent USAToday/Gallup poll (June 23-25, 2006):

Which comes closer to your view? Congress should pass a resolution that outlines a plan for withdrawal of U.S. troops from Iraq (or) decisions about withdrawing U.S. troops from Iraq should be left to the president and his advisers.
57% Congress should outline a plan
39% Leave decision to president/advisers
4% No opinion

This is a different question: It focuses as much on who should make the decision as on the timing of withdrawal. It is also vague about that timing of a withdrawal and is arguably a bit double-barreled. Suppose you oppose withdrawing troops in the next year but also oppose leaving all decisions to the president and his advisors. How would you answer the question? Keep in mind that on the preceding question on the very same survey, 49% of Gallup's respondents said they wanted to send more troops or wanted to let withdrawal take "as many years as are needed." In other words, some of the 57% who said "congress should outline a plan" in this question also opposed withdrawing troops immediately or in the next 12 months.

But back the fundamental question: Two polling organizations - the Pew Research Center and CBS news - have also tracked two similar but more vaguely worded questions about how soon the U.S. should withdraw troops from Iraq:

CBS News - Should the United States troops stay in Iraq as long as it takes to make sure Iraq is a stable democracy, even if it takes a long time, or should U.S. troops leave Iraq as soon as possible, even if Iraq is not completely stable (6/10-11/06, n=659 adults)
48% Stay as long as it take
46% Leave ASAP

Pew Research Center - Do you think the U.S. should keep military troops in Iraq until the situation has stabilized, or do you think the U.S. should bring its troops home as soon as possible? (June 14-19, 2006, n=1,501 adults)
50% Keep troops
45% Bring home
5% Unsure

On these vaguer questions, support for bringing troops home sooner rather than later is slightly lower, but only by a few percentage points. There are many possible explanations for the small differences among these questions, but focusing too narrowly on these discrepancies misses the larger point: Americans are divided almost down the middle on how quickly to withdraw from Iraq, and their attitudes on this question may be hardening.

Consider the question asked by ABC News and the Washington Post (the first one cited above). It defined the argument for a timeline narrowly (proponents favor it "in order to avoid further casualties") and was the only question to offered an explicit counter-argument: "Others say knowing when the U.S. would pull out would only encourage the anti-government insurgents" Back in December, that counterargument helped convince 60% of the Post/ABC respondents to oppose a timetable for withdrawal of forces (38% supported it). Now 51% oppose and 47% support - a result much closer to that of other surveys.

Looking at the other questions that have tracked attitudes on withdrawal or setting timelines, I see no apparent trend since late 2005 (more on that tomorrow). Only ABC/Washington Post shows a sharp increase in support for withdrawal. My hunch: the results on these various polls are converging because attitudes are solidifying. Question wording is making less of a difference because the underlying attitudes on withdrawal are becoming more real.

UPDATE (6/28): I just posted an update that includes graph showing the convergence of opinion on these various measures.

June 23, 2006

The Battle of the Bulge Poll

Last Sunday, Bush press secretary Tony Snow speculated about what polls might have shown during World War II: "If somebody had taken a poll in the Battle of the Bulge, I dare say people would have said, 'Wow, my goodness, what are we doing here?.'" Yesterday, Josh Marshall posted results from polls done by Hadley Cantril at Princeton which showed "no downtick in public support for the war around the time of the Battle of the Bulge." This morning, with the help of Adam Berinsky, an associate professor of political science at MIT and a regular MP reader, the Washington Post's Al Kamen has a far more direct rebuttal:

In fact, there was a poll taken by Gallup from Dec. 31, 1944, to Jan. 4, 1945 -- three years into that war and right in the middle of the bloody Battle of the Bulge, where U.S. casualties were estimated between 70,000 and 80,000. It found that 73 percent of Americans would refuse to make peace with Adolf Hitler if he offered it and that 86 percent of Americans thought there was no chance that we would lose the war in Europe.

The question asked was: "If Hitler offered to make peace now and would give up all land he has conquered, should we try to work out a peace or should we go on fighting until the German army is completely defeated?...

Support for the war was bipartisan. About 78 percent of those voting for FDR in 1944 wanted to keep fighting until the German army was destroyed, Berinsky found, and 73 percent of those voting for the Republicans' Thomas Dewey felt the same.

The rest of the column has more details from Berinsky, who found the survey in the Roper Center Archives while researching a book on World War II.

Read it all.

UPDATE - Adam Berinsky sends along his own analysis exclusively for MP readers. Thank you Adam!:

"If somebody had taken a poll in the Battle of the Bulge, I dare say people would have said, 'Wow, my goodness, what are we doing here?'"

- Tony Snow, White House Press Secretary

Tony Snow might be surprised to learn that, in fact, somebody did take a poll during the Battle of the Bulge. In studying public opinion during World War II for my book manuscript, America at War: Public Opinion during Wartime, From World War II to Iraq, I uncovered some interesting data. From December 31, 1944 to January 4, 1945, the American Institute of Public Opinion, headed by George Gallup asked Americans several questions about the war. At the time, survey research was in its infancy, and modern polling techniques were not yet well established. Nonetheless, the results are illuminating, not just for what they tell us about World War II, but what they can tell us about opinion concerning the Iraq War.

In the 1945 poll, Gallup asked his respondents, "If Hitler offered to make peace now and would give up all land he has conquered, should we try to work out a peace or should we go on fighting until the German army is completely defeated?" Contrary to Snow's speculation, 73 percent of the public expressed support for the stated U.S. policy of unconditional surrender; the American people wanted to continue fighting until victory was complete.

Support for the war crossed party lines. Of those respondents who had voted to re-elect FDR in the 1944 election, 78 percent wished to continue fighting. Among those who voted for the Republican candidate, Thomas Dewey, 73 percent wanted to fight until the Germany army met complete defeat.

Though war support was slightly higher among President Roosevelt's supporters than his opponents, this gap pales in comparison to partisans' opinions on the war in Iraq. As political scientist Gary Jacobson effectively demonstrates in his recent book, A Divider, Not A Uniter: George W. Bush and the American People, the Iraq war has created a schism between citizens who identify with Democrats and those who identify with Republicans. At the beginning of 2006, almost 80 percent of Republicans supported the Iraq war. However, barely 20 percent of Democrats backed the war at that time. The chasm in opinion on the Iraq war has characterized opinion on the war since 2003 and continues to this day.

This partisan gap is the real reason the war in Iraq finds only middling support among the mass public. Republican support for Iraq, after all, is comparable to Republican support for the U.S. military action during the Battle of the Bulge. Democrats, on the other hand, viewed the two wars very differently.

The roots of this partisan divide can be found in the actions of politicians. From 1938 through the end of 1941, support among politicians of both parties for some form of U.S. involvement in World War II increased generally over time. However, the gap between FDR and his critics on the necessity and wisdom of U.S involvement in the Second World War remained large. But after U.S. entry in the war, FDR secured the support of his Republican opponents and both parties expressed a strong pro-war message. Conversely, even before it began, the war in Iraq has been strongly associated with President Bush and his Republican allies in Congress. Though Democratic politicians have not until recently expressed open opposition to the war effort, they have never joined en mass with their Republican counterparts in openly supporting the war.

Patterns of agreement and disagreement among partisan political actors play a critical role in shaping popular responses to war. As long as Republicans continue to support the President, support for the Iraq will continue to hover in the mid-forties - where, as Jacobson shows in his book, it has stayed since early 2004. But without the support of politicians from across the aisle, the American people as a whole will never support the Iraq war. Among both politicians and the mass public, the Iraq war is a Republican war.

June 21, 2006

Update II: Military Social Conservatives

It gets more interesting. Here is the promised update to the posts of the lasttwo days, based on some fascinating analysis sent by Jason Dempsey, an Army Infantry officer and Columbia PhD candidate currently completing a teaching tour at West Point. As part of his doctoral dissertation, Dempsey conducted a true random sample survey of the U.S. Army population in 2004. The results show that Peggy Noonan's comment that in her experience, "career military men" are rarely "social conservatives" reflects the reality of the Army population in some ways but not in others.

Dempsey's data confirm the findings I discussed yesterday showing that Army officers are far more conservative than the U.S. population, both in terms of their ideological self placement and their stands on specific "social" issues. If by "career military" Noonan meant officers, then as Dempsey puts it, "that population is probably more socially conservative than she has experienced."

On the other hand, Dempsey finds that Army enlisted personnel, including those he classifies as "careerists," have attitudes on political issues that largely resemble those of the larger U.S. population. He also makes an important observation about the demographics of the enlisted Army personnel:

Enlisted soldiers also tend to be younger, much more diverse in terms of race and ethnicity, and come from more modest socioeconomic backgrounds than officers. This correlates with more moderate or apolitical stances on larger political/social issues and less engagement in the political process.

So in that sense, Noonan has a point.

Dempsey drafted some analysis exclusively for MP readers that I have posted in full on the jump. I have also created a PDF
ve
of his analysis suitable for printing. Either way, it is well worth reading in full. Thank you, Major Dempsey, for sharing it.

PS: See also this recent New Yorker "Talk of the Town" item on Dempsey and filmmaker Eugene Jarecki.

PPS: A tip of the hat to GWU Professor John Sides, who originally alterted me to Dempsey's research.

Jason Dempsey's analysis:

Mark,

Having studied the social and political inclinations of members of the United States Army for some time I'm happy to comment on your reader's question.

First, you fairly accurately summed up the state of polls on the social and political attitudes of active members of the armed forces-they are few and far between. One you didn't mention was the study conducted by the Triangle Institute for Security Studies (TISS) in the late 1990's (http://www.poli.duke.edu/civmil/). The TISS project was unique in that it got access to military schools and did some fairly extensive surveys of select groups of up-and-coming senior leaders. It is worth looking at for an overview of the attitudes of higher-ranking officers across the various services.

In 2004 I was able to conduct a true random sample mail survey of the Army population focused primarily on the question of Hispanic integration that included questions on civic engagement, socialization, and attitudes towards social and political issues. Everyone in the Army was eligible to be included in the survey with the exception of three groups 1) Generals and Command Sergeants Major (there are too few to guarantee confidentiality); 2) Privates (PV1 & PV2), who are notoriously hard to reach as they are predominantly still within their first year of service and are either in some form of training, moving between training, or moving to their first unit; and 3) those soldiers who were currently in a combat zone. Due to the high turnover of the previous year excluding group three did not prevent us from surveying recent combat veterans. Fully 376 of our 1189 respondents, or 32% of our sample, were veterans of either Operation Iraqi Freedom (2003-2004) or Operation Enduring Freedom in Afghanistan (2001-2004); and 143 indicated that they had been involved in direct ground combat in the previous two years. Overall, the response rate for the survey was 45%. All reported data is weighted to reflect the composition of the Army (minus excluded ranks) as of February 2004, when the sample was drawn. Now, with the methodological disclaimers out of the way, on to your question...

First, I think that Noonan's comment may have drawn more interest than usual because it is counterintuitive. We tend to think of the military as an inherently 'conservative' institution. And what is fascinating about her quote is that the way she describes the 'career military man's' approach to such questions used to fit the definition of conservatism (an outlook that shunned government activism and approached such issues as "private and not subject to the movement of machines").

Indeed, Morris Janowitz (the father of military sociology) and Samuel Huntington (the father of civil-military relations theory) both described the conservative ethic, as practiced by the military in the 1950s and 1960s, as a distinctly nonpartisan means of political expression (See Janowitz, 1960/1971 p. 236, and Huntington, 1957). That definition of conservatism clearly no longer holds so we just need to make sure that we are all comparing apples to apples in our approach.

In the Citizenship & Service survey the question I asked about conservative-liberal self-placement was worded, "In terms of politics and political beliefs, where would you place yourself?" Respondents then had the option of placing themselves on the standard 7-point 'extremely liberal' to 'extremely conservative' scale.

On the whole, the Army looks remarkably like the civilian population on this question, as represented by the 2004 NES dataset (Note that this data is not directly comparable. NES offered a "or have you not thought much about this" option that a large number of respondents chose. The Army survey did not offer this option, and it might be assumed that a large number of the NES respondents who said they hadn't thought much about the issue-had they been not given this option-- would have chosen the moderate category):

However, the story gets a lot more complicated when you start breaking the Army up into key subgroups. Among officers a clear majority, 63%, self-identify on the conservative end of the spectrum compared to only 32% of the enlisted ranks. (The overall numbers look the way they do because officers only make up about 14% of the Army). This leads to the second question about Noonan's statement: does she consider a 'career military man' to be an officer, or anyone who commits to 20 years or more of uniformed service?

Leaving open all possibilities, I present the following in terms of the entire Army, officers, enlisted, and 'careerists' (defined as anyone who has spent 15 or more years in the Army OR states that they will 'definitely' or 'probably' stay in the Army until retirement). In the interest of brevity, I will leave out any discussion of differences among gender, racial and ethnic groups, but suffice to say that many of the same trends that one sees in the civilian population apply to the Army as well.

Among 'careerists', 21% self-identify as liberal while 40% identify as conservative. This follows from a generally older and more established cohort (again pointing to the difficulty of broadly defining 'conservatism').

As for Noonan's point, it would seem the Army data supports her anecdotal evidence, unless she was specifically referring to officers.

When it comes to the importance of religion in daily life, the Army data follows a similar pattern, with officers the most likely to say that religion provides 'quite a bit' or a 'great deal' of guidance. Overall people in the Army are significantly less likely than civilians to state that religion provides 'Quite a bit' or a 'Great deal' of guidance in day-to-day living. HOWEVER, the Army question is not directly comparable to the NES question because the NES question did not explicitly offer a 'not religious' or 'no guidance' option.

On a few 'hot-button' social issues I offer the following summary stats. I don't have good comparisons with the civilian population handy at the moment, but the questions were worded as follows:

"Please indicate your position on the following domestic issues:
a. Banning the death penalty
c. Allowing prayer in public schools
d. Placing more restrictions on gun ownership
e. Outlawing abortion entirely"

Respondents were then given five options, Strongly Favor, Favor, Strongly Oppose, Oppose, and Don't Know/No Opinion.

Making rough comparisons from what we know historically from surveys of the general public, soldiers don't appear to differ much on these issues (for example see Ben Page and Bob Shapiro's, The Rational Public for an overview).

We do, however, have some comparable civilian data from the Chicago Council on Foreign Relations (CCFR) 2004 survey on preferences towards government spending on various social programs. On these issues the Army is largely comparable to the civilian population, with officers again being the exception.

Overall, the Army tends to track civilian attitudes and ideological self-placement. Untold in this brief overview, however, are underlying stories of demographics in the Army. The enlisted ranks drive overall attitudes due to their numbers relative to the officer corps. Enlisted soldiers also tend to be younger, much more diverse in terms of race and ethnicity, and come from more modest socioeconomic backgrounds than officers. This correlates with more moderate or apolitical stances on larger political/social issues and less engagement in the political process.

However, I'd say that if by 'career military men' Noonan is referring to officers, then that population is probably more socially conservative than she has experienced. At least by reported role of religion and ideological self-placement, the Army officer corps appears to exceed the rest of the Army and the civilian population in its 'conservatism'. This would make sense as the Army officer corps, and particularly its senior ranks, is predominantly white and male and comes from a middle- to upper-middle-class background (officers are required to have a Bachelor's degree before commissioning). Officers also deviate from the rest of the Army in terms of attitudes towards hot-button social issues, and they do appear more frugal when it comes to spending on programs related to social welfare (although some of this may be due to the Army's retirement package, which might reduce an officer's concern over the state of health care or Social Security). A tighter definition of 'social conservatism' and more detailed multivariate analysis (as well as tests of statistical significance) are therefore clearly needed to take this question any further.

Anyway, hope this sheds some light on an otherwise under-studied area.

Anyone interested in more nuanced analyses of these and related questions is welcome to a paper presentation I will give at the American Political Science Association convention on September 1st. Our paper on Army racial and ethnic attitudes, as they relate to Hispanic integration (co-authored with Bob Shapiro) should also be coming out in an edited volume from the Tomas Rivera Policy Institute this fall. Additionally, we have several papers in the works on voting and partisanship (co-authored with Craig Cummings and Mat Krogulecki) that should get out this fall and winter.

Initial funding and support for this project was provided by the Tomas Rivera Policy Institute through the Institute for Social and Economic Research and Policy at Columbia University. Additional funding was provided by the Academic Research Division of the United States Military Academy, the Saltzman Institute for War and Peace Studies and a fellowship grant from the Eisenhower Foundation.

Jason Dempsey is an Army Infantry officer currently completing a teaching tour at West Point. This research is part of his doctoral dissertation in political science at Columbia University. The views expressed in this article are those of the author and do not reflect the official policy or position of the United States Military Academy, the Department of the Army, or the Department of Defense or U.S. Government.

June 20, 2006

Update: Military Social Conservatives

Let's hear it for MP's readers. Both in comments and via email, I have heard from political scientists pointing to much better sources of data on military attitudes than what I cited yesterday. These confirm that, contrary to Peggy Noonan's speculation, U.S. military officers are more Republican and conservative than the general population and their conservatism extends to social and domestic policy. The best known studies largely pertain to elite military officers. A more recent effort indicates that the larger pool of regular enlisted personnel may not be quite so Republican or conservative.

Two MP readers, political scientists Richard Eichenberg and Paul Gronke, both left comments on yesterday's post pointing to articles based on a series of surveys of "military elite" conducted by sampling individuals found in directories such as Who's Who or senior Pentagon officials whose names appeared in the Congressional Directory. These were conducted by the Foreign Policy Leadership Project (FPLP) under the direction of Professors Ole R. Holsti of Duke and James Rosenau of Princeton. These were reviewed in an article by Holsti in the journal International Security ("A Widening Gap between the U.S. Military and Civilian Society?: Some Evidence, 1976-96," available online to those with access to the JSTOR archive).

Another survey conducted in 1998 and 1999 by the Triangle Institute for Security Studies (TISS) replicated the FPLP sample design. According to Gronke, a report on that data appears in Feaver and Kohn, "Soldiers and Civilians" and also in Gronke's early version of his own paper from that volume. Here is the bottom line, according to Gronke's comment:

Put simply: the military elite (this was NOT a rank and file survey) are more socially conservative, more religious, and more Republican than the public at large, and even more so than civilian elites.

I'm not sure [which] career military folks Peggy Noonan has talked to about social issues--likely those who have advanced sufficiently to be sufficiently politically astute as to not advertise their views--but her claim is not supported by any of the research that I am familiar with.

Alert reader JS also passes along a more recent survey conducted by an assistant professor at West Point named Jason Dempsey. Unlike FPLP and TISS, Dempsey surveyed a representative sample of the entire U.S. Army, both officers and enlisted soldiers.

Dempsy has co-authored a paper on the study with Columbia Professor Robert Y. Shapiro that is not yet published but was presented at a March conference. That paper indicates that enlisted personnel are significantly less conservative and Republican than their officers. As the authors consider the paper "pre-publication," I am honoring their request not to quote from it, but I contacted Dempsy via email and he has promised to pass along his comments. I will update this post with those comments when I receive them.

June 19, 2006

No "Social Conservatives" in the Career Military?

A reader emailed to ask about a comment made in a column last week by the Wall Street Journal's Peggy Noonan (as noted by Mickey Kaus):

I've never met a career military man who was a conservative on social issues. I think they tend to see questions such as abortion and marriage as essentially uninteresting, private and not subject to the movement of machines. (Connected to this, I suspect Mr. Webb will benefit to some degree by the high number of military retirees in Virginia. They're always assumed to be hawks on Iraq. From personal experience I'd say a high percentage have been dubious about the war, many from the beginning.)

The reader wondered what survey research might have to say about the views of the career military on "social" issues. I did an initial search this morning and the answer is not as easy to find as one might think.

As regular readers might guess, representative surveys of the career military are rare, difficult to conduct and tend to focus mostly on military issues. The Military Times newspapers, for example, have conducted regular mail-in surveys of its readers (who are more likely to be officers and "career oriented" than the larger military population), but the questions they ask do not directly address Noonan's point. There are some hints, however, in their most recent survey:

Half of the Military Times readers identify as conservative or very conservative (50%), but just as many identify as moderate (33%), liberal (7%) or refuse to say (10%). Only 8% describe themselves as "very conservative." Surveys of all U.S. adults typically put the conservative percentage somewhere between 30% and 40%.

They are more Republican than other Americans, but not exclusively so: 56% of the Military Times readers considered themselves Republicans, 13% Democrats, 15% independent and 16% either identify with another party or refuse to say. The Republican percentage among all adults typically falls in the high 20s to low 30s.

The one "social issue" that the Military Times asks about is also a military issue, and here the Military Times respondents are more conservative than other Americans: 27% answer "yes" when asked if "openly homosexual people should be allowed to serve in the military" (59% say no and 14% do not answer). Compare that to the recent Pew Research Center survey that found 60% of Americans in favor of "allowing gays and lesbians to serve openly in the military."

The 2004 National Annenberg Election Survey (NAES) is another data source that comes close but does not directly address Noonan's point. In October 2004, NAES did a two-part release on results among active military personnel and their family members that included tabulations (see Table B) among 371 respondents that were either active duty military (n=177) or Guard/Reserve members (n=199):

Consistent with the Military Times survey, 47% identified as Republican, 15% as Democrats and 26% as independent

During 2004 they gave George Bush strong job approval (74% positive) and favorable ratings (77%), while rating John Kerry negatively (26% favorable, 57% unfavorable). By a three-to-one margin (69% to 22%) they said that Bush rather than Kerry "shares my values."

So the members of the military seem more likely to describe themselves as Republican and conservative than most Americans. They were certainly more comfortable with George Bush than John Kerry in 2004. So concluding that "career military men" are rarely if ever "conservative on social issues" seems like a bit of a stretch. On the other hand, the fact that only 9% of the Military Times readers described themselves as "very conservative" may support at least the gist of Noonan's observation.

To be fair, none of these data are exactly on point. I'll dig further...

UPDATE (6/20): Thanks in large part to very helpful comments by Richard Eichenberg and Paul Gronke below, I've posted more here.

June 15, 2006

Updates

My apologies for the slow rate of posting this week. I had a number of non-blog deadlines all fall within about week's time and they have taken their toll. Among other things, I have not been able to finish my final installment of the RFK Jr./Exit Polls series, but will get to it soon.

Meanwhile, two quick updates on news about polls:

Charles Franklin updates his Bush job approval rating charts to include new releases from NBC/Wall Street Journal and Fox News. The regression estimate based on all public polls now shows a clear though modest upward trend -- a roughly two percentage point increase since mid-May.

The RealClearPolitics blog has a noteworthy apology regarding an article they had run earlier this week by MP friend Tom Riehle of RT Strategies. Riehle's article cited survey results from a recent RT Strategies poll to argue that Democrats would be foolish to attack Wal-Mart during the 2006 elections. Riehle inexplicably failed to disclose that the data he cited came from survey questions paid for by a client, a pro-Wal-Mart group known as Working Families for Wal-Mart. A post that appeared earlier tonight on the RealClearPolitics blog apologized for the omission and noted that Riehle has "acknowledged and apologized for the mistake" as well.

Whatever one thinks of polls sponsored by partisans or interest groups, the issue of disclosure is quite simple. Anyone -- pollster or not -- who cites numbers from a poll sponsored by an interest group should disclose the nature of that sponsorship. A pollster who cites numbers paid for by a partisan client, absolutely positively needs to disclose both the sponsorship and their economic interest in it. This is not a close call.

RealClearPolitics deserves credit for posting a speedy and complete clarification and linking to it from a prominent spot on their front page.

June 13, 2006

The Latest from CBS & USAToday/Gallup

Both USAToday/Gallup and CBS News are out with new polls this week that provide slightly conflicting views of the impact of the death last week of Abu Musab al-Zarqawi on the fortunes of President Bush. CBS News (article, results) shows Bush's job approval rating a statistically insignificant two percentage points lower than their reading four weeks ago (down from 35% to 33%)). USAToday/Gallup (article, results, Gallup free-for-today-only analysis) shows the Bush rating moving two points in the other direction (up from 36% to 38%) in the last week, but up seven percentage points (from 31%) since early May.

Prof. Charles "Political Arithmetik" Franklin graphs it all, as usual. His most recent plot (shown below) compares the recent sharp increase shown on the Gallup surveys (the red line) to the overall trend line based on his regression model. The overall blue trend line trend, which takes into account all available public polls, now shows a slight upturn in the Bush approval rating for the first time since late December. Thus, Franklin concludes that while the upturn is real, the "seven point gain" on the Gallup Poll "is almost certainly an overstatement of the 'real' gains made by the President since his May 15 address on immigration, which marked a turning point for his recent approval ratings."

Both surveys devote questions to the issue of perceptions of the War and the death of al-Zarqawi. Both find evidence of slightly more optimism, although readers should consult the analyses for more details.

One trend that Franklin spotted receives relatively less attention. Both the Gallup and CBS polls have registered sharp increases in Bush's ratings on immigration since his immigration speech, although the CBS poll would indicate some slippage over the last four weeks.

June 12, 2006

The First Iowa Caucus Poll

Yesterday, the Des Moines Register released the first public poll asking likely participants about how they might vote in the 2008 Iowa Democratic Caucuses. Thirty percent (30%) of their sample expressed an early preference for John Edwards, 26% for Hillary Clinton, 12% for John Kerry and 10% for Iowa Governor Tom Vilsack. The other five potential candidates received support in the low single digits. This first appearance of a pre-Caucus poll allows MP to make a point he will repeat again and again: Identifying "likely caucus goers" is no easy task, and the methods used by the various pollsters will differ significantly.**

The Register article explained the basics of their methodology (see the "About the Poll" sidebar on the left side of the page), but left out a few useful details. So I contacted J.Ann Selzer, whose firm conducts the Iowa poll, and she kindly provided some missing details.

Two weeks ago, the Iowa poll interviewed 600 Iowa registered voters who were likely to vote in the June 6 Democratic primary. They started with a list of Iowa's registered voters, selected a random sample of those who had actually cast a ballot in the 2004 general election for President, and interviewed 600 by telephone who said they "would definitely vote in the upcoming Democratic primary." The results of that survey showed gubernatorial candidate Chet Culver leading with 36% of the vote to 28% for Mike Blouin and 21% for Ed Fallon. Culver won 39% of the vote to 34% for Blouin and 26% for Fallon.

Within the same survey, the pollsters identified a subgroup (n=399) that said they "would definitely or probably participate in the 2008 Democratic caucuses." The results from that sub-sample were the ones reported on Sunday and summarized above.

The big challenge for polling this contest, of course, is that turnout for the Democratic caucuses is typically a small percentage of eligible voters. Iowa had roughly 2.2 million voting eligible adults in 2004, of whom (as of last month) approximately 1.9 million are considered "active" registered voters by the Iowa Secretary of State. But only 124,331 participated in the 2004 Democratic Caucuses for President (according to the subscription only Hotline). That number amounts to roughly 6% of all registered voters, so selecting "likely caucus goers" is no easy task.

We know that the Register asked roughly two-thirds of the voters they considered likely to vote in the Democratic primary last Tuesday about their caucus preferences (399 divided by 600 = 66.5%). This percentage is more than reasonable if we believe the Register's selection of likely voters for last week's primary was on target. The 2004 caucus turnout (124K) amounts to 84% of the turnout in last week's Democratic primary (148,000 voters).

What an educated poll consumer would really like to know, however, is the fraction of the population of adults or registered voters represented by the poll. In this case, it would be helpful to know how many respondents werescreened out of the first poll because they said they were unlikely to vote in the gubernatorial primary. Unfortunately, Selzer politely declined to provide that information. [Update: Selzer wrote back today with the information I was seeking -- see update below]

Now, I do not mean to pick on the Iowa Poll. In this instance, they have provided more details about their methodology than most pollsters, and let's not forget, their final 2004 poll nailed the initial vote preference in the 2004 Caucuses.

However, this example does raise a personal pet peeve: The degree of screening in polls concerning low turnout primaries and caucuses can make a huge difference in the results, yet with a few exceptions, most public pollsters prefer to withhold the details. As we get closer and closer to campaign 2008, let's keep asking the pollsters: How tight is your screen? What percentage of registered voters or eligible adults does your sample represent? A little more transparency in this regard would be a very good thing.

**And yes, it is very, very early. The Iowa Caucuses are still nineteen months away. These initial soundings are interesting, but they tell us mostly about how which prospective candidates the respondents know and what they think of the candidates they know.

UPDATE & CLARIFICATION (6/13):

Ann Selzer's initial relucatance to share information about the incidence of their study may have been because my query was unclear, and -- as it turns out -- my attempt to clarify via email managed to bounce back last night. That's life on the Internet (damn you "MAILER DAEMON!").

At any rate, she sent at message this morning that their initial survey of likely voters for the June primary had an incidence of 20%. Thus, to get their sample of 600 likely primary voters, they screened from the initial sample of those who had cast a ballot in the general election for President in 2004 all but the 20% who said they were very or somewhat likely to vote in last week's primary. And only two-thirds of the respondents sample identified themselves as very or somewhat likely to vote in the 2008 caucuses. Do the math (1.5 million voters in 2004 x 20% x 66%), and the sample of 399 represented a universe of roughly 200,000 likely caucus goers.

That seems a reasonably tight screen to me, especially since the Caucuses are still 19 months away and trying to precisely identify the real Caucus-goer universe is a long way from an exact science.

The good news is that the most respected Iowa Caucus poll has disclosed their incidence. Thank you Ann Selzer and thank you Des Moines Register! Let's hope this is a sign of things to come from the others who poll in Iowa between now and January 2008.

"Professional pollster Mark Blumenthal started Mystery Pollster to provide better interpretation of polling results and methodology... offers much needed help to Political Wire readers" - Political Wire