Margin of Error

Shrinking The Margin of Error One Decimal Point at a Time

Saturday, January 30, 2010

Would a Quinn Victory Buck Historical Trends? Nah.

Incumbent Illinois Governor Pat Quinn has at best a 50/50 chance of winning the Democratic Gubernatorial Primary over State Comptroller Dan Hynes on Tuesday. Hynes has seen what can only be described as a meteoric rise through the polls thanks to criticisms of Quinn's handling of the state budget and early inmate release program. Responding to the news that Quinn may lose, NBC's First Read blog (I interned with them last spring) declared "a sitting governor just might lose his primary, which doesn't happen often". Are my old bosses right?

A cursory examination of elections that have taken place over the past 10 years shows they are right. Since 2000, a total of 118 gubernatorial elections have taken place, and only 3 incumbent governors have been defeated in a primary or party nominating convention: Bob Holden (D-MO), Frank Murkowski (R-AK), and Olene Walker (R-UT). So, only 2.5% of gubernatorial elections experienced an incumbent governor losing his/her primary or party nominating convention. This statistic is a little misleading because not all governors run for re-election. If we limit our examination to the 72 races where the incumbent governor ran for re-election, the number of incumbent governors defeated in a primary or party nominating convention rises to 4.2%. This percentage is also quite small.

But what happens when we limit our search to incumbent governors who were not elected to the position? Pat Quinn was elected lieutenant governor and rose to the rank of governor due to the removal of Governor Rod Blagojevich. Since 2000, a total of six of these "replacement" governors have run for re-election. Amazingly, three of these "replacement" governors (50%) lost their bid for re-election in either the general or primary election: Joe Kernan (R-IN), Scott McCallum (R-WI), and Olene Walker (R-UT). Their rate of defeat is 36.5% greater than governors who had been elected to the office.

One of these defeated "replacements", Walker, lost in 2004 because the Republican party nominating convention refused to place her name on the primary ballot. This rate of defeat in the primaries for "replacement" governors (16.7%) is significantly higher than the rate of defeat for elected governors (3.0%).

Thus, a Quinn defeat on Tuesday would not be a shocker. Even if Quinn is able to survive on Tuesday, historical trends indicate that he would face a much a tougher fight in the general election than his elected brethren. These numbers are not surprising considering that "replacement" governors often have lower name identification and do not have an established "base" in the electorate because they have not previously run visible statewide campaigns.

My old bosses may be right that incumbent governors rarely are defeated in a primary (or overall), but "replacement" governors are an entirely different species.

Thursday, January 28, 2010

We are NOT the Anti-Abortion Rights Generation

Reading today's Dartmouth Op-Ed by Peter Blair "Pro-Life Generation", you would get the sense that not only are Americans increasingly against abortion rights, but the youth of America are also against abortion rights. Although the article is mostly composed of "field" reports from a "pro-life" rally (the obvious place for "fair and balanced" statistical evidence), the article does make mention of one actual poll. Blair writes "Earlier this year, a Gallup opinion poll showed that, for the first time since Gallup started taking this poll, more Americans are pro-life (51 percent) than pro-choice (42 percent)". Blair is right that this Gallup poll was the first in which more adults said they were "pro-life" than "pro-choice".

Of course, many other polls taken over the past 15 years have also registered more support for the pro-life than the pro-choice position. For example, five Fox News polls taken over the last ten years have found more pro-life than pro-choice Americans, while ten polls during this same time frame found more Americans were pro-choice.

As this table, provided by PollingReport.com, shows us, the changes in opinion do not occur in any sort of chronological order. They are, for all intents and purposes, random fluctuations. Gallup polls also shows these seemingly random changes.

Why do Americans seem to be consistently changing their positions between being pro-choice and pro-life? The answer is that the question is vague. For instance, many people claim that they are personally pro-life or pro-choice, while they don't "have a right to impose [their] view on the rest of society." Rudolph Giuliani, 2008 Republican candidate for President, was personally pro-life and promised to appoint "strict constructionists" to the Supreme Court, but he favored maintaining the legalization of abortion rights. Giuliani and people with similar ideological positions would probably answer a survey question asking whether they were pro-life or pro-choice "pro-life" when thinking about how they view themselves. However, they would call their position “pro-choice” when thinking about their public views. Two people who hold the same position could answer differently depending on what they think the question is asking. Also, the same person could answer one way one time and a different way another time depending on their mental-frame at the time the question is asked.

In order to truly understand Americans views on abortion, we should ask more straight forward questions. We should ask whether Americans want Roe V. Wade to be overturned. Nate Silver using the PollingReport.com database found that Americans have consistently answered that they do NOT want Roe V. Wade to be overturned. In fact, more Americans today are in favor of the decision that upholds Americans' right to an abortion than at any time in the past 10 years.

We should also ask whether or not Americans believe that abortion should be legal. The ABC-Washington Post poll has been asking that question for the past 15 years, and they have always found that a majority of Americans want it to be always or mostly legal (vs. mostly or always illegal). Note that only one change in the position of Americans seems to be drastic, which should be expected considering that one out of every 20 poll is assumed to have a result that falls outside the margin of error. Besides this one poll, the changes from survey to survey seem to be within the margin of error and are not the wild swings seen in the pro-life/pro-choice polls.

As to the point that the youth of America are becoming increasingly against abortion, I point to two Quinnipiac polls (because most pollsters do not release results by age). One was taken in December 2004, and one was taken five years later in January 2010. The 2004 poll found 55% of those 18-29 years old favored abortion being always or mostly legal, while 43% wanted it to be mostly or always illegal. In 2010, 53% of those 18-34 years old (slightly different age break down, but very similar) favored abortion being always or mostly legal, while 44% wanted it to be mostly or always illegal.

In layman's terms, the youth of America's position on abortion has been constant over the past five years: they want it to be legal. Their views on the issue are not different from the population at large (see chart above).

Mr. Blair may be seeing one thing at his "pro-life" rallies, but the numbers tell a far different story.

Tuesday, January 19, 2010

Turnout, Smurnout... Brown Is Heavy Favorite to Win No Matter What

Today, January 19th, 2010, is a date that will likely be remembered in the annals of modern American political history. Massachusetts State Senator Scott Brown will almost certainly become the first Republican United States Senator from the Bay State since 1979. Many have argued that a Brown victory will be because a depressed Democratic turnout and an enthusiastic Republican base. No one can doubt that Republicans are "fired up and ready to go", but the claim that Coakley will lose because of low Democratic turnout is simply not true.

To support my belief, I have compiled the turnout broken down by party identification for every statewide election (with an exit poll) in Massachusetts since 1996. I've also gathered the results of the polls for the 2010 Massachusetts Special Election released since Friday. For the purposes of my analysis, I could only use the polls in which the conductors of the polls broke down the results by party. I chose to average these poll results, as demographic subgroups in polls usually suffer from high margins of error. Averaging helps to shrink (but not eliminate) the margin of error.

As you can see, Brown holds on average an astronomical 86 point lead among Republicans and 36 point among Independents, while only trailing among Democrats by 60 points. Compare these margins to 2008 Republican Presidential candidate John McCain's who lost Democrats by 74 points and Independents by 14 points, while only winning Republicans by 79 points in Massachusetts.

In other words, Brown has gained among people of all political persuasions and especially among Independents. Brown is not just winning because of low Democratic turnout, but would his hold lead even if Democrats tied their highest turnout in the past 14 years?

To answer that question, I simply took the partisan breakdowns since 1996 and multiplied them by the projected average margin for Brown by party identification.

The numbers should bring confidence to the Brown campaign. Even if Democrats replicated their best turnout efforts (2000), Brown is still projected to win by a little less than 2 points. If the polls are correct (a big if), it would take a historic Democratic turnout not seen in the last 14 years for Brown to lose.

Of course, it is quite unlikely that the electorate will be as Democratic as it was in 2000. More likely, the electorate will mirror 2006 (for reasons previously outlined). If we see an electorate close to 2006, we should expect Brown to win by a little less than 6%.

In summation, a Brown victory today will be due to him convincing voters that he is the better candidate, not depressed Democratic turnout. Brown should be considered a heavy favorite.

And if Democrat Martha Coakley wins? These pollsters better find a new means of employment.

Monday, January 18, 2010

Do Senate Polls in Blue State's really underestimate Dems

Yesterday, Nate Silver sought to provide statistical hope to the burning reck in Massachusetts, better known as the Martha Coakley campaign for United States Senate. The piece makes the point that the margin between Martha Coakley and Steve Brown could be overstating Brown's lead. Silver points out that polling in close (margins of 10 or less in the polls) Senate elections since 2000 in deeply blue states (as measured by the Cook Political Partisan Voter Index) has by an average of 3.4 points underestimated Democratic candidates' margin of victories. In deeply red states, on the other hand, polling has underestimated Republican margins by 1.9 points.

Though he used a Pollster.com average in 2008 and Real Clear Politics average in 2006 and 2004, Silver believed he could only use a simple average of all non-partisan polls conducted in the final two weeks compiled by PollingReport.com in 2002 and 2000. It would seem did not know that a Real Clear Politics average is also available for 2002 and 2000.

I was interested what, if any, effect substituting the Real Clear Politics averages in 2002 and 2000 would have on Silver's results. Therefore, I decided to create a new dataset modeled after Silver's, but using the Pollster.com data in 2008 and Real Clear Politics average* in 2006, 2004, 2002, and 2000.

Using these new rules, we get the following:

The underestimation of Democratic margins in blue states stays the same at 3.4 points. This result is not surprising, as the blue states part of the dataset is small, and only three results are available from 2000-2002. The underestimation of Republican margins in red states drops from 1.9 to 0.9 points. The overall average of underestimation drops from 2.3 to 1.5 points. In only 7 of 23 contests was the underestimation above 3 points. In no contest did the polling average incorrectly predict the winner due to underestimation of Democratic candidates in blue states or Republican candidates in red states. The only incorrect winner chosen was in the South Dakota (a red state) 2002 race when the polls predicted a victory by Republican John Thune.

The bottom line is that perhaps the polls are overestimating Brown's margin in Massachusetts. The limited data involving closely polled elections in blue states suggest that Coakley might do better than the polls suggest, but when you look at the larger dataset of red states, Coakley should not expect a bump.

-*In some cases (such as Alaska 2004), no Real Clear Politics "average" existed. I just averaged all the polls listed on Real Clear Politics (and in the case of Louisiana 2002) conducted in the final two weeks. I, unlike, Silver use internal polls in these cases... modeling myself after Pollster.com's inclusion of them.

Saturday, January 16, 2010

The Winning Scott Brown Party ID Coalition Seen in the Suffolk Poll

Politics is one of the best examples of the phrase "history repeats itself". As I illustrated last night, Public Policy Polling is betting on this phrase with concern to its turnout model for the Massachusetts Special Senatorial Election. But history is not just limited to turnout, people's voting habits are also relatively consistent. That is, the people inclined to vote for a Democratic candidate in one election are much more likely to vote for a Democratic candidate the next election than a person who voted for a Republican candidate. At the same time, swing voters (those who could vote for either party's candidate) tend to be consistently swing voters. With this idea in mind, we should expect that in order for Republican Scott Brown to win in Massachusetts, he will need to put to together a similar coalition as those past Republicans who won statewide.

Unfortunately for our analysis and Republicans, Massachusetts' Republicans rarely win statewide races and have not won a Senate election since 1972 (making a Brown win all the more unbelievable). The last time a Republican won a statewide election in Massachusetts and an exit poll is available was 1998. In that year, Republican Paul Cellucci defeated Democrat Scott Harshbarger 50.8% to 47.4%, nearly identical to the 50%-46% Republican Scott Brown holds in the latest Suffolk University poll. The question we should be asking is "does this make sense?" Do the internals (what percentage of the electorate certain groups of voters make up and how these groups of voters are voting) match up with what we would expect Brown would need to win in 2010?

The first statistic we need to look is whether Suffolk seems to be projecting what I outlined as the most likely turnout model. Note that Suffolk's poll asks party identification in a different way than exit polls, but I have previously pointed out that this should not affect the polls results. Suffolk's poll expects 39% of the electorate will be registered Democrats, 44% undeclared (roughly independent), and 15% Republican. This breakdown is almost identical to the turnout seen in 1998. I personally expect the turnout will be slightly more Democratic and Republican and slightly less Independent.

What about the vote among among these demographics? In 1998, Cellucci won the election with 23% of the Democratic vote, 61% of the Independent vote, and 85% of the Republican vote. What about Scott Brown? Brown's locked up 17% of the Democratic vote, 65% of the Independent vote, and 91% of the Republican vote. These coalitions are very close. The reasons for these slight differences can easily be explained by a number of factors including margin of error, which is anywhere from 6.5% for the independent subgroup to 11.3% for the Republican subgroup in the Suffolk poll. Another contributing factor is that the 2010 electorate is simply more polarized because Democrats love Obama and Republicans hate him, and this vote is largely seen as a referendum on a Democratic Presidential administration.

Finally I wondered what happens if we apply my expected turnout among Democrats, Independents, and Republicans to the expected votes among these groups in the Suffolk poll? Brown's lead drops, but he still is favored 49%-47%.

While I would agree that pinning down exactly which voters (Democrat vs. Republican, Liberal vs. Conservative, Young vs. Old) will turnout is no sure science, I would argue that we should actually have a very good idea. First, I would point out that we had in 2007 a special election in Massachusetts' 5th district (nearly identical ideologically and demographically to the state as a whole). In that election as in the upcoming Senate election, the Democrats had a female candidate who won a contested primary against a Republican who faced minimal competition in his primary. The election, like the Senate election, also took place at an odd time: October 16th.

Although no exit poll data exists for that special election, we have the benefit of two SurveyUSA polls taken a week and a month before the election. These polls correctly estimated the Democrat's share of the vote and slightly underestimated (by 3 and 4 points) the Republican's share of the vote. Thus, the polls were, in my opinion, quite accurate. As it turns out, the partisan breakdown of these polls are nearly identical to the partisan turnout seen during the 2006 midterm elections in Massachusetts.

The question we should be asking ourselves now is "does a 2010 electorate look like a 2006 electorate?" It is tempting to say no. After all, the nation saw an increase in Democratic voters in 2008 (including Massachusetts); however, a look at the 2009 off-year elections suggests that 2008 was an aberration.

Exit polls in the New Jersey (see graph) and Virginia gubernatorial elections show that the partisan makeup of the 2009 electorate was very close to the partisan makeup in 2006 in those states (actually slightly more Republican). Nationally, we see something very similar when comparing November 2006 and 2009 partisan breakdowns among likely voters. Considering Republican and Democratic party identification nationwide among likely voters is at the same levels as they were two months ago, it would make sense that if, in some mythic land, general elections took place in January, this general electorate in Massachusetts should be similar to 2006. But remember, the 2010 election is not a general election. It is a special election.

So where does all of this information leave us? We know that the partisan turnout for a special election (2007) with similar characteristics to the 2010 election in Massachusetts was very much like the electorate for the general election preceding it (2006). Thus, even a special election in 2010 should look very similar to the general election preceding it (2009), which in turn looks like the 2006 electorate. Ergo, the electorate for the 2010 special election should have a very similar to the electorate as the 2006 general election (and perhaps slightly more Republican).

Actually applying this information to the polls for the Massachusetts Special Senate election is a little difficult due to differences in question wording for party identification. The most recent polls (Public Policy Polling and Rasmussen) ask voters "if they are Republican, Democrat, or Independent", while exit polls (and SurveyUSA similarly) ask "do you usually think of yourself as a Republican, Democrat, or Independent". Luckily, this difference in question does not seem to impact the partisan breakdown (see difference between the final national tracking poll of Rasmussen and the national exit poll in 2008).

By this logic, the Public Policy Polling poll, whose partisan makeup looks very close to 2006, is closest to what the partisan makeup of the electorate would look like for this special election (although it potentially overestimates the amount of Democrats and underestimates the amount of Independents and Republicans), while Rasmussen clearly overestimates the amount of Republicans and underestimates the amount of Independents.

Therefore, I believe the Public Policy Polling poll, which has Republican Scott Brown in the lead, is closest to the truth. The Rasmussen poll, which shows Brown down 2, is probably shortchanging his percentage of the vote.

Does that mean Scott Brown will win? No, because the election is still very close and within the margin of error. But I dare say, he should actually be considered a favorite.

About Me

I am a graduate of Hanover, NH. I graduated with adegree in government summa cum laude with a concentration in statistics and elections.
I have previously interned at the NBC Political Unit in Washington D.C. (Spring 2009) & Pollster.com (spring and summer 2010).
For questions pertaining to this blog or about me in general, you can reach me at HJEnten@gmail.com
Visit me on Twitter (currently inactive): twitter.com/ForecasterEnten
Email for a resume.