Big Data Meets Big Government; It's a Moneyball World Now

If you haven’t read Michael Lewis’ Moneyball or seen the film on which it is based, I highly recommend picking up a copy or Netflixing the movie tonight. Why? Because this week’s behind-the-scenes glimpse of how the Obama campaign used big data to influence voters, raise vast sums of money, and achieve winning results proves one thing: It’s a Moneyball world now.

On Nov. 4, a group of senior campaign advisers agreed to describe their cutting-edge efforts with TIME on the condition that they not be named and that the information not be published until after the winner was declared. What they revealed as they pulled back the curtain was a massive data effort that helped Obama raise $1 billion, remade the process of targeting TV ads and created detailed models of swing-state voters that could be used to increase the effectiveness of everything from phone calls and door knocks to direct mailings and social media.

The article explains how the Obama team realized early on that one major weakness to their effectiveness was the existence of too many databases not sharing information between another, similar to the FBI and the CIA prior to 9/11. Once they merged all that data together into one massive system—a “megafile”—all sorts of interesting patterns and predictions could be made by modeling the psychology of everyday Americans across the country.

The new megafile didn’t just tell the campaign how to find voters and get their attention; it also allowed the number crunchers to run tests predicting which types of people would be persuaded by certain kinds of appeals. Call lists in field offices, for instance, didn’t just list names and numbers; they also ranked names in order of their persuadability, with the campaign’s most important priorities first. About 75% of the determining factors were basics like age, sex, race, neighborhood and voting record. Consumer data about voters helped round out the picture. "We could [predict] people who were going to give online. We could model people who were going to give through mail. We could model volunteers," said one of the senior advisers about the predictive profiles built by the data.

What does the above reveal? As Moneyball proved a game-changer with baseball, there is absolutely no doubt that big data has changed the game of politics...forever. In a largely homogenous world, where people think the same or buy the same stuff or go to the same events, this wouldn’t be very useful. Winning the hearts and minds of the American public would simply require a single and solid voice. But today, in our extremely heterogenous, multi-racial and largely diverse society, where people have numerous and often conflicting interests, persuasion is not about large-scale leadership, but finely-tuned precision. With millions of people to reach and trillions of data points to consider, expert advisers are now computer models, volunteers an army of algorithms sending tweets, emails, and Facebook posts to mathematically targeted voters. Thought the political process was bad before. It's now going to be gamed at a whole new level.

Perhaps more than anything, it was this portion of the TIME article that blew me away the most:

"We ran the election 66,000 times every night," said a senior official, describing the computer simulations the campaign ran to figure out Obama’s odds of winning each swing state. "And every morning we got the spit-out — here are your chances of winning these states. And that is how we allocated resources." [emphasis mine]

66,000 times every night! Within a month that’s almost 2 million simulated elections. Let’s think about that for a moment. As time goes on and the collective mindset of American voters is increasingly gathered, modeled, and simulated for predicting election outcomes, eventually the government will just pick those who'll win for us. That is, whoever best-fits the data.

Forget human instinct. Forget conventional wisdom. Forget the illusion of choice and a rigorous selection process for the best leader. "In fact sir"—future advisers will say—"just do and say whatever the models tell you. Don't bother thinking too hard about things or trying to lead the nation. According to the data, people don't really care about experience or leadership anymore. Remember, all your speeches have been mathematically calculated to achieve the highest level of emotional response and voter turnout. Stick to the script and just let the teleprompter do the talking for you."