The presidential race

A heavily weighted coin flip

IF THERE is one thing America’s right- and left-leaning media seem to agree on as election day looms, it is that Barack Obama and Mitt Romney are locked in an exceedingly close and unpredictable race for the presidency. “With Time as Tight as Race, Romney and Obama Zero In”, ran a recent headline in the New York Times; “Obama and Romney Deadlocked”, concurs the Wall Street Journal. Even the most esteemed news outlets from abroad—such as, say, The Economist—pronounce it “about as close as it could be”. If people who can barely agree on the colour of the sky can find common cause on this point, we can surely take it for granted and move along, right?

Strangely, what might be the least controversial notion in contemporary American politics is also one of the farthest from the truth. Yes, the national polls are roughly tied. But unlike every other civilised country with a presidential system of democracy, we don’t have a national popular vote. And the state polls—which collectively represent a much bigger sample of voters than the national ones, and enable us to project what matters, which is the electoral college—tell a very different tale.

There are indeed a number of states that are extremely close. Colorado, Virginia and New Hampshire could easily go either way, and Florida and Iowa are still highly competitive. However, none of these states are likely to have an impact on the outcome of the election.

There are 19 states, totalling 237 electoral votes, which are so safely Democratic that Mr Romney did not bother to contest any of them until the final weeks of the campaign: CA, OR, WA, NM, MN, IL, MI, PA, HI, DE, MD, DC, NJ, NY, VT, CT, RI, MA and ME. Beyond those, Mr Obama appears to have an unassailable lead in Nevada, and the presence of Paul Ryan on the Republican ticket does not seem to have moved the needle in Mr Romney’s favour in Wisconsin. Adding those two states to the president’s tally, all he needs is Ohio to reach the magic number of 270 electoral votes—and he has consistently led the polls there by over two percentage points. Even if Mr Romney wins all of Colorado, Virginia, New Hampshire, Florida and Iowa, he would still lose the election if he cannot break through Mr Obama’s Midwestern “firewall” of Wisconsin and Ohio. The campaign is being fought almost entirely on Mr Romney’s turf.

Mr Romney faces such an uphill battle in the electoral college that most quantitative calculations regard the race as anything but too close to call. Nate Silver of the New York Times’s FiveThirtyEight blog, the best-known of the forecasters, currently gives Mr Obama an 86% chance to win. Sam Wang of the Princeton Election Consortium puts him at an even more generous 98%.

Bettors broadly concur with this analysis. Although Intrade, the most widely cited prediction market, is fairly kind to Mr Romney and shows him with a 33% chance of victory, tight legal restrictions on deposits to the site have made it very difficult for Americans to wager there. That makes it very thinly traded and unreliable, since bets of just a few thousand dollars can move the market price substantially. The real money laid on the election goes to bookmakers, who uniformly see Mr Obama as an overwhelming favourite. Pinnacle Sports in Las Vegas shows him with a 77.5% likelihood of victory, and Ladbrokes in Britain has him at 81%.

So why do so many reporters continue to peddle the notion of a tight race? One hypothesis is simply a healthy scepticism regarding statistical models, particularly in the wake of the financial crisis. Both David Brooks of the New York Times and John Cassidy of the New Yorker, while openly admiring Mr Silver’s work, warn that complex calculations can lead to unjustified certainty.

That may be true, but there’s nothing particularly convoluted about these methods. The Occam’s Razor approach of a simple average of the most recent polls, listed at RealClearPolitics.com, leads to exactly the same conclusion: that Mr Obama has small but significant leads in enough states to win the election. Some critics have argued that the state polls are biased towards Mr Obama because they overestimate Democratic turnout, but the historical evidence suggests otherwise.

A second interpretation, put forth by Alec MacGillis of the New Republic, is that the media are simply responding to institutional incentives. No one will stay tuned in to a programme saying that the outcome is a foregone conclusion. Suspense sells, and given a choice between a story and no story, journalists will always choose the former, regardless of their political leanings.

This account has undeniable “Freakonomics”-style appeal. However, it relies on a misunderstanding of the competitive dynamics among reporters. Prestige media organisations strive perhaps above all to be counterintuitive, and attract readers or viewers by debunking conventional wisdom. Publications like Slate, the Atlantic or the New Republic are often mocked for their knee-jerk contrarianism. Do writers really think they are going to attract precious eyeballs by jumping on the “it’s-coming-down-to-the-wire” bandwagon? Quite the opposite—I’m betting that the “reality check: Obama’s ahead” premise of this post will garner far more traffic.

I think two entirely different factors underlie the misleading coverage. The first is a modest dose of innumeracy. Reporters lacking statistical training who see a two-point lead for a candidate are likely to think, “That doesn’t sound like much. After all, the margin of error is four points, so it’s a statistical tie.” Actually, it’s not—it just means that particular pollster didn’t find a big enough gap to achieve 95% certainty that one candidate is ahead. Once you start talking about dozens of polls taken by independent firms that all point in the same direction, the sample sizes go up and the margin of error shrinks. As Mr Silver recently noted, the consensus of state polls in recent years has been remarkably accurate: it has called 74 of 77 states right, and two of the three misses occurred when the leading candidate had an advantage of one percentage point or less. Mr Obama leads all of Nevada, Wisconsin and Ohio by 2.8 points or more in the RealClearPolitics average. Mr Obama may not be likely to win those states, and therefore the election, by a particularly large margin, but that does not mean he is at substantial risk of losing them.

The other explanation is the old “false balance” problem. Reporters have long relied on the crutch of simply quoting representatives of both sides of an issue to appear impartial to their readers, even if one is much further from the truth than the other. To correct this tendency, the media have increasingly deployed “fact-checking” squads who are willing to call a candidate a liar when they deem it appropriate. This, in turn, has prompted a backlash from politicians and critics who sniff bias in purportedly objective fact-checking.

While not yet chastened, the fact-checkers have done their best to stick to debunking the most easily identifiable fibs. Poll analysis does not meet this standard. The Democrats say Mr Obama’s ahead in the key states; the Republicans counter that the national polls are tied, and that the state polls suffer from unsubstantiated assumptions and methodological deficiencies. Who’s right? “Who knows?”, a reporter on a tight deadline is likely to conclude. “Let’s just cite both arguments and say that time will tell.”

Time will tell indeed. Mr Romney certainly has a fighting chance, and if he wins, it won’t discredit Mr Silver. The proper way to evaluate forecasters is to look at all of their predictions, and see if events they say should happen 80% of the time fail to occur in around 20% of cases. (It would cast doubt on Mr Wang’s approach, however, since he sees Mr Obama as a virtual lock for a second term.) For the baseball fans among you, Mr Romney is in roughly the same position as a team starting the bottom of the ninth inning trailing by one run; for the poker players, he’s all-in holding pocket kings facing an opponent with pocket aces. But that’s nobody’s definition of a toss-up. Following Mr Silver’s lead, if anyone would like to offer me a wager, the comment thread beckons.

Addendum: The last batch of national polls moved in Mr Obama’s favour. Mr Silver’s final estimate of his chance of re-election is 92%.

I don't think that is the bias of the media to drum up controversy that is the problem. It is that political pundits face an existential threat from these poll aggregators. If indeed all we need to do is look at the average of the swing state polls to determine the outcome of the election then there is no need to waste time listening to political pundits. This is why we saw such anger from people like Joe Scarborough who claimed that the only way Nate Silver could come up with a 70%+ chance for Obama to win the election is through allowing his partisan bias to show through. Personally, I thought that remark was way off the mark. His model may indeed fail because of a garbage in/garbage out problem but I don't think for a minute that he is allowing his political bias to guide his calculations. If anything it is a bias towards the reliability of mathematical models.

I agree with Aegis, but will step outside that to say Muslims, Mormons, Hindus, Atheists, and believers in the Flying Spaghetti Monster are all entitled to represent this country, and may be truly said to do so.

Though gruellingly laborious and mind-bogglingly complex, I have spent every waking hour post-convention season constructing and perfecting a polynomial regression forecast of the election, the scale of which has never before been attempted (don't mention it). Peers and colleagues alike have commented that the model is a marvel of modern statistical analysis, containing an unprecedented (once considered not humanly possible, but whatever) number of independent variables.

The painstakingly compiled quantitative calculations of my model, the most sophisticated of its kind, ever, anywhere, ever in the history of numbers, produce the following assertion:

DON'T SAY OBAMA HAS ALREADY WON ECONOMIST! People might be lazy and think their vote doesn't matter. How about this for counter-intuitive TE:-

Obama intentionally sand-bagged the first debate for the sole purpose of making his own supporters think Romney had a chance. Fear has an ability to make people cough up the cash. Even with said tactic the Romney camp allegedly has been spending tens of $millions more each week than Obama's - although who believes official figures when I know at least 3 easy ways to circumvent the whole thing?

Even as a non-US citizen I gave Obama money thinking he might actually need it. If the consensus amongst analysts with skin in the game is Obama has >75% chance of winning, they didn't come up with that in the last 2 days. I find it likely Obama's team have known this for a while and they have wisely suppressed expectations for fear of turning off voters and telling them they don't REALLY need THEIR money.

Most of the fundamental building blocks for the analysts' conclusions will have evolved over time. Who knows, maybe an analyst designed a perfect model 6 months ago and it delivered a unanimous decision for Obama. Recently this model has gained validity as many of the predicted variables behaved as expected. Now other people believe in the numbers and thus it becomes fact.

I understand why people are hesitant to believe in statistics and model-built conclusions in light of the subprime crisis. I would argue the US housing bubble was entirely predictable based on fiscal policy, monetary policy and avaricious private-sector originators. The whole thing was a fraud. The AAA ratings were only legitimate pre 2004, after which house prices clearly became a bubble under the Allen and Gale (1998,2000,2002) regime. Those at the top aren't idiots. Those analysts who created the CDOs are amongst the brightest people on Earth. Those REALLY making the decisions (ICE) knew exactly what the score was. They designed the system to specifically induce the market response. They knew when to hold it and when to fold it because they set the rules and controlled most information.

Skilled analysts like Kyle Bass (Hayman Capital Partners) saw through this charade long before it played out and made billions. Insider sharks like John Paulson (no blood relation to GS CEO/US Treasurer Paulson) artificially created their own markets (with Deutsche Bank primarily) to bet against what they already knew to be a sure thing. The RMBS/CMBS/CDS markets had survived round after round against the matador, one of those daggers were going to be their last. It turned out to be Lehman Brothers, which suited ICE and US government agency heads - Lehman didn't spend enough politically (lobbying/PACS/PR/specialist interest groups) compared to those firms who were bailed out.

Anyway most financial products are just fine and aren't controlled by a collective secret society in league with the devil to destroy the middle classes. The alleged reason for the existence of complex financial products is to ameliorate risk.

A simple example:- A hen farmer has invested his entire life-savings in his hens. Today his hen is worth $3, but what about next year? If bird-flu breaks out his livelihood is ruined. So the farmer signs call contracts for his hens in future years at roughly today’s price. If all his hens drastically depreciate in value, he still has the insurance of his call contract and thus security. The farmer needs good advice though because the market has reasonable expectations of future food inflationary trends and countless other informational monopolies compared to the non-finance professional.

Complex financial products are designed to provide security and reduce risk, not to systematically bias free markets. As always the problem is not the products themselves rather it's the controlling influence within the human element.

I think you papered over the "second" interpretation. Sensationalizing news sells and a neck-and-neck race is just the recipe for it. This happened in 2008 as well, albeit not so excessively. Obama won that race decisively. Maybe it won't be that wide a margin this time around but a 332-206 win or something along those lines looks like a reasonable result.

Given the convenient intolerance of American sports for any kind of drawn or tied result, not to mention the fact I don't have statistics to hand to check my analogy against, this is tricky, but I'll give it a shot:

It's the beginning of the fifth day, Team Romney need three hundred runs to win with eight wickets in hand. And there's a storm forecast for the evening. And Team Obama only need a draw to take the series.

The distinction between the two key types of error here is not quite clearly made: random error and systematic error. Random error occurs simply from statistical variations in the population being polled, and if it were the only substantial kind of error it would indeed mean a 98% chance of victory for Obama.

Systematic error occurs when some factors are incorrectly accounted for in the calculation. Two weeks ago, nobody had factored in the probability of a storm getting more coverage than the election and Christie publicly complimenting Obama. Indeed, it is not the pollsters' job to try to predict such events.

Instead, the polls focus on converting the portion of people who answer the polls (often around 5% of people called) into a good representation of the population as a whole. Generally, rural homes are more likely to answer, so they are given less weight per answer. The difficulty is deciding just how much weight to give to various sets of people. Polls' methods vary here, so at least some of them introduce systematic error.

The bettors need to judge which of the polls properly map their samples onto the whole population. Some prefer to trust the older and more established polls, which lean more to Romney than the overall average this election cycle. Others consider those polls inaccurate because they often use "robo-polling" methods and some do not call cell phones.

At any rate, it can be agreed that Romney's chances are much less than even after the arrival of Sandy. Bettors also need to account for the likelihood of game-changing events, but there is hardly any time left for that. So the 20% chance they are giving Romney is not because they worry extra democrats have answered the polls; it is just in case Gallup is performing better than 80% of the other polls. Of course, a one in five shot is very important, and media outlets that overlook that are placing a very hefty bet of prestige on Mr. Obama's victory.

"The outcome is as certain as the sunrise tomorrow. Or the effect of gravity."

It really isn't. That's what a 13% chance (538) or 2% chance (Princeton Election Consortium) mean. It's unlikely, perhaps very unlikely, but possible. The odds the sun rises tomorrow are much higher. (And if it doesn't, or gravity fails, people will have bigger problems than finding their polling places floating in the dark.)

Not that ironic. As you point out, the Democratic Party doesn't mind the portrayal of a horse race. Assuming the "liberal media" are biased towards a Democratic win, they would be of the same thinking as the Democratic Party, which is:

Even if they are favored to win, it is better to portray the election as a toss up, so as to GET OUT THE VOTE!

If too many voters think the election is a foregone conclusion, and thus don't bother to vote, they risk changing the outcome of the election.

In other words, Democratic pundits are saying, "If you are a Democrat, you had BETTER vote, because our guy is neck and neck with the opposition".

President Obama, first elected with a mandate to not be Bush, will now be elected with a mandate to not be Romney. We'll see how far that gets him. He's going to have trouble pulling together the votes to grant him a spare key to the White House restrooms.