Sunday, February 21, 2010

Republican Blizzard on the Generic Ballot

It would take a major turn of events for the Democrats to hold onto the House of Representatives. There I said it. Why the confidence on Republicans gaining 40 seats and getting to the 218 seats needed to control the House?

One of the more interested studies involving the generic ballot was carried out by Joe Bafumi (my introduction to statistics professor and an overall good guy), Bob Erikson, and Chris Wlezien. They found that by "adjusting" (the words of Andrew Gelman who linked to the study) for certain conditions, you can predict the national House vote pretty accurately as far as 300 days out. The basic findings for midterm elections are that Democrats tend to do better on the generic ballot than they do in the actual election (most studies agree on this fact), and that the party out of the White House (Republicans this year) does better than the generic ballot indicates. Both of these discoveries are good signs for the Republican party, and the signs are even better for Republicans when you look at the numbers in detail.

The current Pollster.com (in full disclosure, I'll be an intern at Pollster.com in the spring and summer of 2010) trend estimate has the Republicans with a 1.7% lead. If you allocate undecideds based on how those who already registered a preference say they are going to vote [Republican % of vote / (Republican % of vote + Democratic % of vote)], it's a 2 point Republican advantage (see right). For being this far out in an election (254 days), this lead is the highest since 1946 (when the generic ballot was first implemented by Gallup). What about if we take out all the polls that some Democrats would say are biased against Democrats (Rasmussen and all Republican pollsters)? The Democrats regain a lead of 1%, which readjusted is a 1.2% lead. Folks that number is still the most favorable polling result for the Republicans with a Democratic President since 1946. Of course, many of the earlier surveys in Bafumi, Erickson, and Wlezien's data set are based off national adult samples from Gallup. So what does the latest Gallup survey of national adults (conducted earlier this month) predict? A 45%-45% tie (readjusted to a 50%-50% tie), which is is also the worst number ever for Democrats this far out with a Democrat in the White House.

How accurate are these estimates? Well, Bafumi et al.'s regression (which takes into account the generic ballot and party in the White House) predicted a 10% Democratic margin over the Republicans in the last midterm election (2006). According to data from the Federal Election Commission, the Democrats won the national House vote by 8.4%. Not a straight on prediction, but not bad either. And if anything, it overestimated the Democratic margin. Such an error should be expected with only 15 case studies (or elections), and a root-mean-squared error (a tool that measures errors in estimate and penalizes for larger errors) of 1.90%. So, these results can be trusted, but with the understanding that some error is involved. Of course with the Republicans in such a good position, the idea that this error takes away from the basic conclusion that Republicans are in a great position is not a wise one.

How will the generic ballot results from 2010 at this point translate into vote in the general election? Based off the Bafumi et al. regression (see page 6), we would expect Republicans to win the national vote by anywhere from 7.3% (all polls but Rasmussen and Republican pollsters) to 9.3% (all polls), which extends well beyond the root-mean-sqaured error. Thus, I have a hard time believing based off the polling that the Democrats will win the national party House vote.

So how will this national house vote translate into seats controlled by the Republican party in the 112th Congress? It's without a doubt a tricky question. One could imagine a scenario where the Democrats lose a majority of the national vote, but maintain a majority thanks to winning many districts by small margins and losing fewer by large margins. Yet, the evidence indicates that this is not a likely scenario. Jonathan Kastellec, Andrew Gelman, and Jamie Chandler did a nice job in a 2006 paper to show that since 1946 the margin between the percentage of the national vote and percentage seats gained by either party shrinks as you get closer to an even split of the national vote. Thus, one would expect that Republicans would win a larger number of seats relative to their popular vote as their lead in the popular vote rises.

One advantage the Democrats do have is that fewer Democratic members of Congress are retiring, which Kastellec et al. find help Democrats in their bid to maintain control of the House. That said, Kastellec et al. still predicted in a 2008 followup paper that Republicans would win about 53% of the House's seats (or about 231 seats) with 52% of the popular vote in 2008 (a much more favorable year for Democrats in many ways including retirements).

With current polling in conjunction with Bafumi et al.'s paper predicting a Republican national vote between 53.6% and 54.7%, the Republicans could easily gain 50-60 seats from their current 178. Gains of greater than 60 seats also look quite possible. Even in the best case scenario for the Democrats, it would seem that holding the House would be very, very difficult.

It looks like a red blizzard is going to sweep through Washington in November.

As a Democratic voter, I look forward in trembling to the rest of the Obama administration.

Here are the major decisions Obama has made:

1.Ignoring previous Republican crimes, misdemeanors and profligacy – e.g. tax cuts for the wealthy. 2. Supporting a stingy stimulus that was half of what was needed and was one-third tax breaks, not jobs. 3. Killing the only option that would have slowed the cost of health care & led to universal coverage. 4. Accelerating the Bush bailout, $ 4.3 Trillions in bailouts, guarantees and purchasing assets from the private sector at well above market value. 5. Escalating a meaningless and fruitless war. 6. Gutting real financial reform and substitute finger wagging and silly taxes and fees, while banking fees continue up, lending freezes and credit tightens. 7. Not helping people with bankruptcy and mortgages remediation – accelerating middle class decline. 8. Fiddling around and not passing a jobs bill.

Thanks for your informative article. For a stats neophyte, can you please explain how you extrapolate a Republican national vote of 53 or 54%, when the current (as of 23 Feb) pollster.com general House ballot shows a 43.6% to 43% Republican edge? In other words, how does a 1%current polling advantage become a 7 or 8% predicted advantage in the actual ballot?

The polls are somewhere between art and science. We don't know the exact fudge factor needed to become science. That said, the Democrates, including the president, seem bent on pushing their programme regardless of what the American public wants. The appearance of 'bipartisan' conferences still lead to the president saying to the Republican, and the public, do this my way.

The Republicans have the appearance of having actually read the health bill; the Democrats appear not to and insist on saying that change is good, without being able to explain why.

A good president will listen to public opinion; a great president will lead public opinion.

This president will lead the Democrats down the road to Perdition and sadly, not understand how that happened.

The sheer idiocy of the Democratic party establishment to not have a message to present the country over the last years is mind-boggling. No wonder they are getting creamed. The only people we ever hear from on the D side are the ones griping about how evil the PO is and how terrible the party's far left is to deal with. We only hear from the Liebermans, Nelsons and Baucus of the world, not the Feingolds, Leahys, Testers or Wydens. The former do nothing but gripe and complain, so that's what the populace assumes the Ds represent. Discord and dis-unity. Then we have Reid talking in his soft voice like he would jump 10 feet high at the sight of a freaking cricket.

Put that together with a R congress that has no desire to partake in governance, instead they'd rather see the economy worsen so they can run in and save the day in November. What a bunch of cowards. The whole lot of them make me ill. Obama's the only one with a lick of sense but no cojones to match, apparently.

"Thanks for your informative article. For a stats neophyte, can you please explain how you extrapolate a Republican national vote of 53 or 54%, when the current (as of 23 Feb) pollster.com general House ballot shows a 43.6% to 43% Republican edge? In other words, how does a 1%current polling advantage become a 7 or 8% predicted advantage in the actual ballot?"

I tried doing that, but here is the low down. First, you extrapolate what percentage of the two-party (Dems and Pubs) vote Republicans currently hold... As of your writing, it was 50.3 (43.6/(43.6+43)). You then take that number and plug it into a regression model (which I link to on the blog), and you are able to determine what percentage of the vote you would expect them to get in November. You can see that regression model in action by looking at the line with all the blues surrounding it on the graph.

What you basically have is that the generic ballot this far from an election almost always overestimates the percentage of the vote the party who holds the White House... In your case, the 49.7% the Dems would get of the House popular vote would actually be more like 46 (and an 8 point edge for the Pubs) on election day considering historical trends...

As a social scientist (MA in economics), I've seen no explanation for how Republicans go from a 0.6% advantage in February to a 10% advantage in November.

The only explanation has been based on "previous Democratic administrations". Well, that reduces it to Clinton, Carter, LBJ/Kennedy, and then you are going back over 50 years to the "Dewey beats Truman" days. This is such a small sample size, just 4 data points, that they are essentially useless.

A better explanation is that current economic conditions are dragging down the Democratic vote. As the economic has been improving, it seems unlikely for Republicans to gain ground on a trend going the wrong direction for them. Unemployment has fallen 0.3% since November, not risen, and we have 8 months left.

This is irresponsible political analysis and presenting it as more than speculation is junk journalism.

Sorry you feel that way Axel. I think the post is steeped in science... I don't know what else to say... Polling from past elections do a great job at predicting future ones... See NYC mayoral elections (http://www.pollster.com/blogs/election_night_2009_live_blog.php) where under the banner name HJE posted early on the evening pointed out that Republican incumbents always had their leads greatly exaggerated... That same election night chat showed the same thing with concern to Maine gay marriage vote (http://www.pollster.com/blogs/enten_revisiting_the_maine_mar.php)... And the same thing with Brown in Massachusetts where my prediction was more accurate than Pollster.com AND 538.com...

Oh and please don't make this about me being a Republican... cause I'm actually a registered Democrat in NYC... and was the get out the vote chair for the College Democrats effort in 2008...

Let me ask you a question? Can you point out why you disagree with this?

I think you demonstrate the correlation between the Popular Vote poll and the seats in Congress exists.

And I think you successfully show that regardless of polling company the D's have a bad lag in the Popular Vote from their break even point.

The quibble I have is how do we know that the conversion of popular vote to seats lost is linear? Is not the biggest swing the 54 seats from 1994? It would seem that is probably a base level of Democrat seats that are not going to be persuadable, at least not in one cycle. The would suggest that the Generic Ballot/Seats Lost correlation tapers off at some point. Your suggestion of 50-60 is beyond what has ever occurred so I wonder if that is the resistance level.

BTW- become a Republican now. You are likely to in later life anyway, so get it out of the way now.

I think the point was that this thing is just off the chart... Something could be lost in the linear... It's part of my reason of going with 50-60 instead of 60+... The forecast is inexact... I don't claim it to be perfect.

Some notes for you:The economy cant save the Democrats. The election is about big govt, just like 1994, 1978 and 1966 were. In each of those good years for the GOP, the economy was actually better than it had been two years previous. Between 1992 and 1994, the unemployment rate fell 2% points GDP gwoth was over 4% and 6 million new jobs were created in 1993-94, to no avail for the Dems.

As for 2010 unemployment. The census is responsible for the decline in unemployment and will continue to help reduce unemployment until March. Beginning in the summer, those 1.2m census workers will get laid off again, probably pushing the unemployment rate up.

The Nov election as far as the economy goes, isnt 8 months away, it is 6. The last employment report before the elections will be Sep, just as the last of the census workers is laid off.

As for falling unemployment, 4% GDP growth and unemployment falling from 7.8% to 7.5% didnt much help GHW Bush in 1992 did it??

Axel, for a social scientist with an MA in Economics, your comment is really weak. Harry explained TWICE how he came to his projections, and provided data. You provide a theory that runs counter to his, but no data to back it up. If you bothered to research the issue of job gains over the next eight months saving the Democrats, my guess is that you would be disappointed. Other studies have shown that when coming out of a recession you need robust job growth at least six months prior to election day for voters to reward the incumbent party, which would would be the first of April. Is anyone predicting robust job growth in the next five weeks? If you have data that contradicts that, great, but until then show Harry the respect his work deserves.

"Harry; what do you make of Rasmussen's substantial deviation from the norms of the other polls?

I'm also curious: where does '06 fit on the Barfumi Erikson Wlezien graph, and is it reasonable to dismiss '02 as an outlier?"

I think my boss in the fall Mark Blumenthal over at Pollster.com has had some great write-ups on it (http://www.pollster.com/blogs/why_is_rasmussen_so_different.php)... The basic point is that it's probably a combination of things... Likely voter model + automated methodology (that by its nature gets the most motivated of voters)...

What I will say is as I've argued here http://poughies.blogspot.com/2010/02/another-argument-against-ivr-polling.html that I trust the automated methodology (sorry Gary Langer)... Now of course, Rasmussen is clearly a bit of an outlier... Could it be because they will actually have it right in the end (as the likely voter model helps predict what the actual turnout will be in November)? Maybe. They could just be flat wrong.

I'm just a big believer of using averages... They help to account for outliers.... and more than that, I took out Rasmussen, and it still looks like a disaster...

As for what the generic polls look liked in 06, I've actually got an email to Joe Bafumi requesting the dataset they used....

I do however have a dataset that Charles Franklin emailed me... I don't want to give away too much because I believe some of this data will be used for something else (and I don't have the time to complete a true average from the entire 241-300 day period).... BUT, from my calculations, the polls conducted over the same two week period as we are in right now (using polls that included leaners if say Gallup had one that did and one that didn't), I come up with an estimate of Democratic party taking the two way popular vote with about 53.7% (they had an average of 55.5% on the ballot) of the vote.... Pretty good... no?

Can't find any flaws in your statistical logic...good analysis. Of course, so much can happen in individual campaigns and should Dems manage to pass anything big that they can manage to sell as a positive, things could change.

Hi, my name is Max, Dartmouth '08. My question is - does this model in any way control for regional polarization? Polls from many different outfits this year have shown an increasing disparity between America's geographical regions, particularly the South, which trends ever-more Republican as the rest of the country remains either split or Democratic-leaning. Is it possible that national congressional ballot polls differ from those in previous years because of this? Ie, could Republicans simply be winning already strongly Republican districts by even-greater margins while making relatively small inroads in the rest of the country?

The comments on unemployment are uninformed. The official unemployment numbers reported in the MSM account for those collecting unemployment benefits; once benefits are exhausted an individual is no longer counted as "unemployed," but moves to the category "persons not in the labor pool"--a cohort not counted by the "official numbers." In other words, if the rate of individuals exhausting benefits exceeds the rate of new claims, the unemployment rate drops without anyone actually getting a job. So when AlexDC states that unemployment fell 0.3% since Novemeber, he's really stating that "persons not in the labor force" grew by 843,000 (Nov-Dec 09). And not that the economy actually produced jobs.

"Hi, my name is Max, Dartmouth '08. My question is - does this model in any way control for regional polarization? Polls from many different outfits this year have shown an increasing disparity between America's geographical regions, particularly the South, which trends ever-more Republican as the rest of the country remains either split or Democratic-leaning. Is it possible that national congressional ballot polls differ from those in previous years because of this? Ie, could Republicans simply be winning already strongly Republican districts by even-greater margins while making relatively small inroads in the rest of the country?

Keep up the good work!"

Hi Max,

First off, always good to meet a fellow member of the Big Green.

Second, the model does not (to my knowledge) in anyway control for that factor. The question is that a problem?

I don't believe it is... Here's my reason. The so-called regional divide has been in effect for many years... especially since 1994 (it's part of the reason the Republicans did so well that year... seats that were in the south that were naturally Republican were held on by yellow dog (or now blue dog) Democrats, and they simply reverted to what we would have expected).

If the regional divide had an effect on the generic ballot in such a way as to throw off the model, one would expect that elections from 1994 onward would have shown some inaccuracy or incongruity with prior years...

I don't really see that in the chart (granted, it's only a few cases)... In fact, I see only 2002 being a true outlier (due to 9/11 one would think), and in that case the error benefited the GOP.

"the Democrats [could] maintain a majority thanks to winning many districts by small margins and losing fewer by large margins. Yet, the evidence indicates that this is not a likely scenario."

Indeed, the exact opposite is more likely due to majority minority districts that will vote 85-15 for the D. A nationwide tie is bad news for Ds, because that D support is overly concentrated in places like NY-16, TX-18, or CA-35.

BASICALLY, the ONLY way the Democrats have a chance in keeping the majority is if they either trash the current Healthcare Bill and start over and do it bipartisanly or trash the bill and move on to something else; high deficit, over spending and many more important things. But either way, they have to trash the current bill and use it for scrap paper if they want to survive the next election if not the next three to four elections. Because I'm forecasting another 15 year wait until demecrats hold the majority. And if the republicans are smart and CONTINUOUSLY do what they did during the Clinton administration, balance the budget and keep it balanced every year until the deficit is paid off and past that, they just might be the party that STAYS in office. If not, then we just might have what we just had for the past four years.

SO a message to all,"VOTE". If you have a message to send, VOTE. You can't complain if you don't VOTE.

About Me

I am a graduate of Hanover, NH. I graduated with adegree in government summa cum laude with a concentration in statistics and elections.
I have previously interned at the NBC Political Unit in Washington D.C. (Spring 2009) & Pollster.com (spring and summer 2010).
For questions pertaining to this blog or about me in general, you can reach me at HJEnten@gmail.com
Visit me on Twitter (currently inactive): twitter.com/ForecasterEnten
Email for a resume.