Conditions through August showed a 2% lead on the generic Congressional ballot for Democrats. As of September 20th, in the wake of the Democratic convention, the lead has widened to 4.0 +/- 2.0%. Although it has yet to be appreciated by pundits, this could well translate to a November loss of the House of Representatives by Republicans. Based on the generic Congressional ballot, the probability of a Democratic takeover is 74% with a median 16-seat majority. Whichever party is in control, the seat margin is headed for being narrower than the current Congress. Like any probability in the 20-80% range, this is a knife-edge situation. This picture may change over the coming six weeks as more information, especially district-level polls, becomes available.

[Update: Based on later discussion (here and here), some other factors (i.e. incumbency) do not affect this prediction. Two remaining unknown parameters do: (1) the quantitative effect of redistricting measured in units of the generic Congressional preference poll, and (2) whether the generic poll is more likely to move toward or away from the Presidential winner's party in coming weeks. These factors are likely to be small...but this poll is very close at the moment. More on this later.]

As seen in recent articles in Politico and U.S. News, few pundits think the Democrats will re-take the House. However, analysis of a leading indicator suggests to me that transfer of control is a distinct possibility.

Predicting the House outcome is challenging. First, there is the basic problem that we have to estimate how far opinion will move between now and November. On top of that, there is uncertainty in knowing how the polling measurement – generic Congressional ballot preference – translates to a seat outcome.

Another approach would be to use district-by-district polls and ratings. An estimate like that can be seen from our data partner, Pollster.com. Their House outlook shows retained GOP control, and RealClearPolitics implies the same. However, many of those polls are weeks or months old. My estimate today suggests that in the coming weeks, we might look for district polls to move in the Democrats’ direction. This is also an opportunity for a detailed analytical approach, as taken elsewhere, to shine.

In 2012, the picture looks very different from 2010. Congressional voter sentiment before Labor Day is often movable (see 2008 and 2004 history). But we are now entering the high season, so some sense of the outcome is starting to emerge.

In summer polls leading up to the 2012 conventions, Republicans were behind Democrats by a median of 2%, a 9-point swing from 2010. Consequently, many seats won in that Republican wave are now at risk, as I pointed out in my previous House outlook. To put it another way, midterm election conditions (as in 2010) work against incumbent Presidents; the pendulum has now swung back.

In post-convention polls, Democrats got a big bounce that peaked at D+6% and now appears to be subsiding. It is a good thing for the Republicans that the election was not last week. The most recent polls (Sept. 7-17) indicate a median lead of D+4.0+/-2.0% (+/- estimated SEM, n=7 different pollsters). (Note that the HuffPost smoothing software uses a different algorithm.) This 11-point swing from 2010, if it were to hold, would lead to big Democratic gains in the House.

As I pointed out two weeks ago, the eventual national D-minus-R House popular vote share is strongly predictive of the corresponding margin of House seats. Here is a graph based on data from 2000-2010 House elections.

It shows that each 1.0% of popular-vote margin translates to a 6.0-seat advantage. This plot shows no long-term advantage* for either side: a nearly-tied popular vote would translate to a nearly-tied House (x-intercept = -0.3 +/- 1.1%). Individual data points deviate from the fit line by 7 to 17 seats in either direction.

However, there is a known advantage to incumbency, which I estimate as being worth an equivalent of 1.3 +/- 1.01.8% of popular vote. This is smaller than a recent estimate**. In other words, on average a national win of D+1.3% is required for the House to change hands.

Next, let us translate the generic Congressional ballot to estimate House seat outcomes. Applying the Wisdom-of-Pollster-Crowds principle that has served so well in Presidential and Senate races, we will use the poll median to approximate the actual popular preference.

The HuffPost graph above includes telephone polls (i.e. not robopoll or Internet), but that is strictly for purposes of clear display. Using all polls and median-based statistics to address issues of outlier data gives the median of D+4.0% that I gave. That translates to a narrow 16-seat Democratic majority in an election held today.

This would be an unusual outcome. It would involve a Democratic net gain of over 30 seats, much more than the typical gain for a re-elected president’s party. But 2010 was also an exceptional wave year for the Republicans. Again, think of the pendulum. In any event, this is what the numbers are currently telling us.

The principal caveat. The main issue with this analysis is that it does not use district-level data. In the coming weeks, those surveys will become more abundant. In 2008, district polls did a very good job of estimating the outcome – on Election Eve. Six weeks out, the generic ballot preference is the week-to-week indicator that is available.

Where things could go in the next seven weeks. Assume a +/-4% opinion shift between now and November, and this leads to a popular vote prediction of D+0% to D+8% (1 sigma). This gives a Democratic takeover probability of 74%, approximately three out of four.***

It should be noted that current conditions emphasize the post-convention bounce, which could be transient. Conversely, if the Democratic lead increases, that would take House control out of the knife-edge territory that I defined previously. For now, a smart use of campaign donations is to donate to the DCCC through this ActBlue page, or the GOP through Crossroads GPS.

>>>

Are you interested in a specific race? For tracking specific districts, watch the aggregators such as our data source (HuffPost/Pollster.com) and RealClearPolitics. Considering the current 11-point swing, any race that was won in 2010 by a Republican by less than 15% should be considered competitive, and worth getting involved if you are a Democrat or Republican currently on the sidelines. I myself find this surprising, but that’s what the arithmetic is saying.

>>>

NOTES

*We can ask whether redistricting should affect the x-intercept, i.e. give one side a structural advantage. Often the goal of redistricting is to protect specific incumbents. This is different from the goal of altering the total number of seats expected by either side. Indeed, maximizing one goal can work against the other by collecting partisan advantage into a few incumbents’ districts, at which point even a little swing can flip the outcome in the other districts. This is a complex subject, and to my knowledge it is not proved that gerrymandering leads to a global structural advantage.

**A four-factor model at FiveThirtyEight has led to a meme propagating through the political blogosphere that there is a 3-4 point structural advantage for the Republicans. However, that estimate is less precise than it appears.

When doing multiparameter regression, it is important to quantify uncertainty, as well as avoid fitting to noise. Here, two quality checks are to (i) make sure the the uncertainty does not blow up when all the extra factors are added, and (ii) make sure a large fraction of the 7-17-seat residual can be accounted for by each added factor. After playing around with the four-factor analysis, I have concluded that it is likely to fail these criteria. The consequence is low confidence in estimating the structural advantage. A simpler model using only this-year’s-national-vote and last-Congress’s-seats (i.e. two factors) applied to 1996-2010 data has less of this problem, and indicates a structural advantage to Republicans in 2012 of 1.3 +/- 1.01.8%. Note: when fitting to data going back to 1946, this becomes 0.6 +/- 1.0%. It is not possible to rule out the possibility that in the aggregate over the whole chamber, this parameter is zero.

***This is in marked contrast to the InTrade price of 0.17, which corresponds to an imputed market opinion-driven probability of 17%. As I have previously pointed out, InTrade market opinion is often quantitatively wrong. In this case it might even be getting the direction (i.e. <50% or >50%) wrong.

20 Comments so far ↓

Sam, I have been eagerly awaiting your updated analysis of the House. Really glad to see that it is so thorough!

I am astonished that you give the Democrats a 74 % chance of re-taking the House; this is far better than I expected.

Granted, a lot can happen between now and the 6th of November, but as you point out the tendency is becoming clear.

Question 1: How strong is the evidence that might suggest Mitt Romney is making the Republican brand toxic, and that this is manifesting itself in the House and Senate races?

Question 2: How likely is it that we will see a very different “poll” on November 6th, as a result of Republican-controlled purges of the voter rolls in many states?

I realize that some readers may consider the latter suggestion to be overly paranoid, but I understand there is plenty of evidence for such a strategy. The question, of course, is how comprehensive and effective it has been.

Let’s put it this way. I’ve given you probabilities for President (88%), Senate (88%), and House (74%). If these are well-estimated probabilities, there is an approximately 50-50 chance that I will be wrong about one of them. (There are additional issues because the probabilities are non-independent, but forget about that for a moment.)

In regard to voter ID/purge issues, I think that will be small. But it does deserve a quantitative look.

@Olav Grinde, for your convenience here is what Prof Wang wrote about Voter ID laws on the morning of July 14, beginning with his concluding paragraph in quotes: “Pollsters will eventually take the new requirements into account in their likely-voter screen. They can ask questions such as ‘Do you have a government-issued picture ID?’ At some level, it feels best to leave that to them.”

Here is his whole short essay:

Can voter ID laws affect the Presidential election? by Sam Wang

In a very close race, voter ID measures could affect the outcome. In the extreme scenario, it could flip the Presidential election. How?

About 10% of Americans currently lack government-issued ID. This is a Democratic-leaning population. The percentage is higher among African-Americans. States that have new voter requirements include Pennsylvania, Florida, Iowa, Virginia and Ohio. This is disconcerting considering the fact that there is basically no evidence for voter fraud being a significant factor in US elections.

About 9% of Pennsylvania’s ~9 million registered voters lack a government ID. Pennsylvania’s in-person requirements for voter ID and provisional voting are here. If all of those votes are lost and they break 2-1 Democratic, that would be a net gain of up to ~3% for Romney. The current margins for Pennsylvania are Obama +7%. My guess is that if this margin drops to +3% or below, then Republicans have some chance to flip the outcome there.

Can this be accounted for in the Meta-Analysis? In a limited manner, yes. One way to give you a sense of the maximum possible impact of voter-ID laws is the “+2% for Romney” tool in the right sidebar. Another view comes from using the Meta-Margin, whose accuracy is a few tenths of a percentage point. It’s at Obama +3.0%, right on the edge for a Republican win if they get voter-ID laws in all swing states and if they extract the maximum possible advantage (and if pollsters somehow failed to assess this).

Pollsters will eventually take the new requirements into account in their likely-voter screen. They can ask questions such as “Do you have a government-issued picture ID?” At some level, it feels best to leave that to them.

A lot of Tea Party darlings are suffering from foot-in-mouth disease (does it spread from the top?). Some of them have been redistricted to a less favorable neighborhood. Some have flubbed their re-election campaign by focusing on social issues that go against their district. A perfect storm is the Walsh-Duckworth race in the district next to me. Duckworth is ahead by 9 points and Walsh just keeps on giving.

@Olav Grinde: Making the brand toxic is not as important as making his supporters stay home. If you go to the RAND link, and look at ‘Intention to Vote’. Since Monday, there has been a steady erosion of Romney supporters and increase in Obama supporters. That is a trend to watch. The shift in the past week is about 2 or 3%, that will affect all close races

That might be. It is a good estimate prior to having detailed district-level information, which will become more abundant later (and which I used to good effect in 2008). An example of such a summary can be found at Pollster.com. The main issue with that is that the data are old, and do not take into account the current swing.

I’ve written at length about the missing-data problem, and how it provides an opportunity for analysis-intensive approaches to be useful. I can think of sites you can read for that.

Generic polls can give an impression of what the electorate thinks, but, as the saying goes, “all elections are local ” (paraphrasing here), so I am delighted by the news but I’d rather wait for district polls that are more complete and recent. Speaking of polls, I have not read anything on the Pew research Poll, neither here on on Mr. Silver website. Is there something I missed?
8% point is a lot!
Marco

(I’m having trouble posting comments. If this shows up several times, that’s why)

Hi! big fan!

I’d like to believe you here, but I feel like your dismissal of the powers of gerrymandering is a little glib? Can you point us to any of the literature on that?

I’m thinking that if the people who draw up districts are not the incumbents in those districts (and my understanding is that state legislatures do it for the house seats in their state), their incentives should be exactly to maximize their own party’s number of seats, not to protect incumbents.

That is a very reasonable criticism. I have not yet found a satisfactory analysis of the problem, or the right way to think about it. Qualitatively it is a concern, but what is the *quantitative* impact?

Dear Professor Wang, I can’t thank you enough for this informed look at important uncertainties. Perhaps I’ve simply missed it, but I’d love to be able to scroll back through the snapshot maps instead of merely seeing today. That goes double this week. Of course, the data may simply be gone, too, and I don’t wish you to feel bad. Perhaps I ought to have been keeping screenshots. I’m more than willing to begin if that could help. -Your sincere admirer.

This might be a great test case scenario for how robust polling aggregation is. That is if the polls stay where they are roughly and the pundits keep dismissing the idea that the Democrats can take back the House. It’s definitely something to keep an eye on.

This analysis gives me hope. I couldn’t even imagine that this could be possible after the damage they’ve caused to our country with their allegiant bow to Norquist and negligent disregard for the needs of the American people while they do nothing but insult their own office.
Plus I think that because of the continued advances in our high tech information age it’s just harder for people to get away with inconsistencies (well let’s face it, LIES) because info immediately or eventually gets out.
Thanks for all of your work on this. I’ll pass it on.

It used to be the case that state legislatures redistricted to protect incumbents but that has changed with the rising partisanship of US politics and the increasing single-party dominance of state legislatures and governorships.

The parties are now more intent on redistricting for partisan advantage, and they are often in a better position to do so following sweeping electoral victories at the state level (as the GOP achieved in 2010, on the eve of the 2011 redistricting). The electoral effects, however, are constrained since, to a degree, the parties’ redistricting plans offset each others’ gains and losses and by Voting Rights Act constraints on redistricting plans in many Red states.

As for the question: I understand your analysis includes many polls that do not reach the 30% or so of Americans who depend entirely on cell phones. I also understand that these voters lean Democratic. True, and if so, how significant is this for your prediction of a Dem takeover?

Many of those pollsters sample cell phones and landlines, which allows proper weighting to account for your concern. However, not all of them do that. Scroll back a few days for Ed Freeland’s discussion of cell phone sampling.

Given a 4% Democratic lead and a 2% margin of error in the House Generic Poll, the probability is 97.5% the Democrats will win the popular vote – assuming the election was held today. That is close to the probability that they will win the House.

Click on the tab “View All Un-Matched Predictions”. Note how thin (i.e., low # of contracts) & how wide (the range of price bids) this market is on the bid side. As of right now, there are less than 500 shares total bid over a probability range of 50% – 73% ! This indicates both relatively low conviction & little professional interest in the market.

By contrast, I’ve noticed in the past several weeks that the contract on Romney to win typically has total bids on the order of 5000 – 10000 shares, all within a +/- 1% probability range. This is a very tight spread for Intrade & suggests professional arbitrageurs have enough confidence in “continuous price changes” to get involved.

(Note that for simplicity I’m ignoring the “offer” side of the market.)