538 Still Rates Massachusetts as Toss-up

Please don’t be too enthralled with/scared by the specific numbers below. They’re based a number of assumptions which may not be valid. However, I agree with the characterization that the model comes to, which is that the Massachusetts special election should continue to be regarded as a toss-up.

Been a while since we’ve posted one of those charts on here, eh? What this reflects is simply our 2008 Senate model, which while being less sophisticated in a number of ways than our Presidential model, nevertheless managed to call all the races correctly. After allocating the undecided voters, the model very tentatively shows a 49.2-48.6 Coakley win, with 2.2 percent of the vote going to third-party candidates.

Now then: why might this projection be totally wrong?

First, this was a model developed for a regular November election, when the Massachusetts election is a special. I’ve often made the analogy between special elections and primaries. If I were to design a model for primaries, I would (i) probably place greater weight on the most recent polls; (ii) almost certainly account for the greater uncertainty in primary polling.

If we were to weight the most recent polls more than we do, this would tend to benefit Scott Brown. And perhaps we should be doing that, but …

It should be kept in mind that a lot of Brown’s support is pretty new, which would ordinarily imply that it is pretty soft. Yes, I know there’s a core of people — maybe a fairly large core — who are really, really excited about Scott Brown. But they only get to vote once apiece. And what the earlier polling established is that it’s almost certainly not 50 percent of the electorate — it might be 35 percent or 40 percent, but it’s not 50 percent. He still needs some swing voters to get him over the finish line. Some of those voters were probably tending toward Brown over the last 7-10 days, when he was winning virtually every news cycle. But the headlines in the last 72 hours — the ones that those voters will be thinking about as they head into the ballot booth — may be a bit more even-handed (especially given Obama’s visit, etc.), and a voter who has swing once is prone to swing again.

The other reason to be at least a little bit careful about fetishizing trendlines and looking at only the most recent polling is that it’s not just you guys who are looking at the trendlines — it’s also the pollsters themselves. And in cases like special elections where pollsters have some tough decisions to make about how to calibrate their turnout models, they may be looking at what the other pollsters are saying in order to determine what they deem to be a “reasonable” result that they want to attach their brand name to. Recall how just before Election Day 2008, all the national polls, which had shown Obama leads ranging from 2 to about 13 points, miraculously converged on a +6 or +7 number. This may be a smart strategy for the pollsters, but from our point of view, it potentially renders the newer polls less independent of one another than their older ones, which means they potentially have less informational value.

The second questionable assumption is including the potentially very dodgy Pajamas Media / CrossTarget poll that shows a huge lead for Scott Brown. There are reasons to be vary, vary, vary wary of this poll (most of which will hold even if it hits the result on the nose). But it doesn’t appear to fail the bright-line test that we’ve applied in the past, which is that “a poll is excluded if it was conducted by any current candidate for office, a registered campaign committee, a Political Action Committee, or a 527 group”. In the wake of the Strategic Vision scandal, I will probably put some new procedures in place before November, which puts pollsters into some sort of probationary box if they are poor about disclosure issues and if they haven’t polled enough to establish a rating or a house effect. But since those procedures haven’t been developed yet, the old policy holds for now.

The third questionable assumption — although it’s actually a “traditional” part of our model — is the inclusion of what is termed the “538 Regression”. In the past, this has been based on the polling in other states, regressed upon certain demographic characteristics, the idea being to have a hedge in case the polling errors in a state are correlated.

I have whipped together an ad-hoc version of this based on taking a simple average of all public polls over the past two months in each open-seat Senate contest (excluding Kansas which hasn’t had any), and regressing it on each state’s PVI. The regression figures that the Democrat ought to be favored by about 15 points in Massachusetts given the current political environment. Incidentally, 15 points is also about what you’d get if you’d taken Barack Obama’s roughly 25-point margin of victory in Massachusetts from 2008 and subtracted 10 points from it, which is about the amount of deterioration the Democrats have experienced in the generic Congressional ballot since that time.

Of course, as Democrats have found out to their dismay, Martha Coakley and Scott Brown aren’t generic candidates: Brown is a somewhat better than average candidate, and Coakley an inferior one. At the same time, I feel OK about including this hedge because of the somewhat robust-seeming effect that I documented earlier today, which is that the polls have tended to underestimate the performance of Democrats in very blue states, and Republicans in very red states, in close Senate elections in the past. Including the results from the regression model moves the numbers by a net of 1.8 points in Coakley’s direction, a magnitude comparable to the 2.3-point effect that I found earlier — although admittedly this may be a case of two wrongs making a right.

So, that’s how the numbers got to where they got to. It’s certainly tempting to take the Ockham’s Razor argument for Brown — “look at the trendlines, duuuude!” — which has become the conventional wisdom even if nobody is saying it. And it’s perhaps just as tempting to play the role of the contrarian, sort of buy the rumor and sell the news, and insist that Coakley will leg it out. But for the time being — and subject to change based on last-minute polling — I’m not comfortable with any characterization of this race other than too close to call.

Nate Silver is the founder and editor in chief of FiveThirtyEight. @natesilver538