Fast Chat: Can Nate Silver Predict the Election Again?

Whether it’s baseball stats or political polling results, Nate Silver has made a living sifting through scores of data and making predictions, most of which seem to come true (Silver gained considerable notoriety after correctly picking 49 out of 50 states in the 2008 presidential election). While writing his data-driven, election-focused FiveThirtyEight blog for The New York Times, Silver found the time to examine the phenomenon of modern-day prediction in his new bookThe Signal and the Noise, which is out Thursday (Sept. 27). He sat down with Adweek to explain why presidential elections are incredibly difficult to forecast and why you really don’t want him in your fantasy league.

Adweek: A central theme of the book is cutting through the noise to get closer to the truth. I'm curious then, how do you get your news?

I don't think you should limit what you read. I have 500 people I follow on Twitter. They're more data-driven but made up of both liberals and conservatives. If you're keeping yourself in the bubble and only looking at your own data or only watching the TV that fits your agenda then it gets boring. I think reading broadly, but skeptically. That's key. You have to try to avoid the temptation to jump on the bandwagon with political news, especially on Twitter where things go viral every couple of hours. In a campaign you don't have game-changing events every single day. Most of these small viral things burn out. So it’s kind of about waiting those things out but not blinding yourself to the news.

You interviewed hundreds of people for this book over a long period; have your feelings on the way that humans predict changed substantially?

The book was going to start out being more Freakonomics-y, then it got kind of darker as I was going through and looking at economic forecasting and the financial crisis. But there are these silver linings in the book as you go through it, and I think it makes things interesting. You end up seeing people who are working on models—not get-rich-quick schemes—who think slowly and take time to study, and they end up making huge progress in how we predict. In a way, the book has a "lowercase c" conservative moral to it, which says, Be humble, work hard and you could get your rewards. It's often the case, though, that people are latching on to flashy narratives, and think they can get to solutions easily. If we rely only on those narratives, it can get scary.

How do your findings in this book change how you work every day?

Well, it’s important to be mindful of how big your data set is. For example, you have 33 U.S. Senate races every other year. In a four-year span, you have 66 Senate races and only one presidential race. For the Senate, you can really mine the data and test variables and see impacts. But for the presidential race, there are so many variables that influence voting, and we only have a handful of cases to test them on. You have to find a good structure for this data and provide some backbone to the model because you can't tell too much based on the cases. Every four years in the presidential election, some new precedent is broken. It’s tricky, and you have to be humble about how much uncertainty there is in the long run. The presidential race requires a lot of humility. It’s a great exercise, but it'll never be as complete a data picture.