When Does ‘Close’ Become Too-Close-to-Call?

The Wall Street Journal‘s Editorial Board, whose previous coverage of the Minnesota recount has tended to reflect a lack of command of the facts and circumstances of the case, is now back for another round, this time pressing for a re-vote:

Even after the recount and panel-findings, the 312-vote margin separating the two men equals about .01% of the 2.9 million votes cast. Even without any irregularities, this is as close to a “tie” as it gets. And there have been plenty of irregularities. By the end of the recount, the state was awash with evidence of duplicate ballot counting, newly discovered ballots, missing ballots, illegal voting, and wildly diverse standards as to which votes were counted. Any one of these issues was enough to throw the outcome into doubt. Combined, they created a taint more worthy of New Jersey than Minnesota. […]

Another solution is to hold a special Senate election. Minnesota law does not specifically provide for such a runoff. However, the U.S. Constitution’s 17th amendment does provide states with a roadmap for filling “vacancies,” which might be a legal starting point for a do-over. Even before the shifting standards of the contest trial, the St. Paul Pioneer Press looked at the ballot-counting evidence and called for a revote. It could be that this is where the court case is leading in any event.

Is a 312-vote margin “as close to a ‘tie’ as it gets?”. Well, of course not. A 311-vote margin would be closer to a tie than a 312-vote margin, and a 310-vote margin would be even closer. Plenty of elections have been closer than Minnesota’s, including the somewhat infamous New Hampshire Senate election of 1974, when the Senate actually did order a re-vote. In that case, different recounts yielded different conclusions, with the Democrat — who had lost the Election Night count — winning the first recount by 10 votes and the Republican winning the second recount by 2 votes. Two votes is a heck of a lot closer to a tie than 312.

The practical question is at what point there is significant doubt about which candidate in fact had more legal ballots cast on his behalf. If I spilled a jar filled with several thousand red and blue marbles onto the floor and asked you to count them, there would probably be some margin of error in your calculations. Maybe you’d group the marbles into piles of 20, but get tired at some point and build one pile that contained 19 marbles, or 21. Maybe you’d return from a lunch break to think you’d counted a certain pile of marbles when in fact you hadn’t — or maybe you’d wind up counting those marbles twice. Your estimates would undoubtedly be pretty good — probably very good — but they might not be exact. If there were a roughly (but not exactly) equal number of red and blue marbles and I wanted you to tell me which type was more abundant, you might occasionally get the answer wrong.

There is some point, in other words, at which an election is so close that it is subject to tabulation errors that might change its outcome. In such cases, I think the argument for a re-vote is reasonably strong.

However, the margin of separation between the candidates might need to be very small indeed before the risk of tabulation errors becomes sufficient to alter the outcome of an election. Suppose, for example, that you have an election in which 3 million votes were cast — about as many as were cast in Minnesota. Suppose furthermore that you know you’ve made some mistakes in counting — but you didn’t know which candidates you’d made them on behalf of.

If we assume that tabulation errors are unbiased — meaning they’re equally likely to have been made on behalf of either candidate — and independent — meaning that the probability of one ballot having been miscounted does not affect the probability of the next one being miscounted — then we can estimate the chance of a mis-called election based on a binomial distribution:

Given these assumptions, elections are actually fairly resistant to tabulation errors (even if such occurring at relatively high rates). This is because in most cases the errors will cancel out: you’ll miss some blue marbles but also some red marbles. For instance, given a 5% error rate — that would be 150,000 miscounted ballots (!) out of 3 million — a 900-vote lead should hold up about 99% of the time given these assumptions. At more plausible (i.e. lower) rates of error, leads in the low triple-digits, or even the double digits, are generally quite robust.

Indeed, while Election Night tabulations from machine methods are relatively error-prone (although the optical scan ballots that Minnesota uses are far better than alternatives like punch cards), the rates of tabulation errors following manual recounts, as has occurred in Minnesota, are presumably fairly low. I’m not sure exactly how low they are, because there have been very few cases where there have been multiple manual recounts in a given election — but in the New Hampshire Senate race of 1974, the one that did trigger a re-vote, there was a net swing of 12 votes between the first and second recounts.

On the other hand, the assumptions that I made above are probably optimistic. Machine counts, for example, are clearly not unbiased, because different types of voters are more likely to complete ballots that are misread by machines, and different types of voters are also more likely to have voted for particular candidates. Inexperienced voters, for example, are more likely to have filled out their ballots imperfectly in ways that might confuse a machine; in Minnesota inexperienced voters were also more likely to have voted for Al Franken.

Manual recounts may produce results that are closer to unbiased, but they probably aren’t perfectly so. Different people will be responsible for conducting the recount in different precincts and counties throughout the state, and they may also have different levels of competence at doing so. If the candidates have differing areas of strength and weakness across the different precincts, which of course they will, this could introduce bias.

Moreover, the risk of tabulation errors is probably not independent from one ballot to the next. Rather, the errors would tend to be clustered. A whole stack of ballots, for instance, might go missing (as happened in at least one instance in Minnesota), or might be counted twice.

Thus, the risk is miscalling the election would be higher than indicated under the idealized conditions presented above. One could presumably convene a panel of statistical experts to come up with some more robust estimates of the probability for a miscalled election, by stress-testing the ballot counts for these and other types of contingencies. My guess is that they would find that a 312-vote lead following a manual recount was nevertheless very, very safe, but I do not know for sure.

The Journal, however, is not calling for a re-vote in response to the risk of tabulation errors; rather, they are calling for one in response to what they presume to be legal errors. This strikes me as a much more dangerous precedent. Clearly, courts are not infallible, especially under the pressure of an extremely close election. But, to build in a fudge factor for errors in judgement by hte courts is another thing entirely, and would seem to undermine the rule of law. If a court finds a defendant guilty of a serious offense, it does not then say “you know what, we’re not really certain about this, so we’re only going to sentence the defendant to probation rather than life in prison”.

Moreover, attempting to apply this principle would be problematic. You would need to deal with a world full of counterfactuals — alternate rulings that the courts could presumably have made. You would then need some way to differentiate “reasonable” counterfactuals from unreasonable ones — for example, if the courts had determined that no ballots should be counted from Minneapolis, this would dramatically alter the outcome of the election, but no court would issue such a ruling. Finally, you would need to demonstrate that from among the set of reasonable counterfactuals, some subset of them would have altered the outcome of the election. This is far from apparent in Minnesota. The Coleman legal team never really attempted to argue for a particular standard of ballot-counting. Instead, their approach was ad-hoc — they presented a series of ballots to the elections panel and said “here, count these ones!”. Although there are various reasons why this might have been their approach — incompetence, for instance — another interpretation is that the Coleman team knew it was likely to lose under any and all reasonable and consistent standards for ballot counting.

The Coleman campaign needs more than a winning legal argument, in other words; it also needs a legal argument that would win it the election. Unless it can be demonstrated that such an argument exists, the Journal‘s case for a re-vote is moot.

Nate Silver is the founder and editor in chief of FiveThirtyEight. @natesilver538