The Academy Awards are important in and around Hollywood for primarily three reasons: the nominations which bring
audiences into theaters as a movie marketing tool, the lucrative ‘For
Your Consideration’ ads they generate, and the global telecast
announcing winners so everybody can bask in their reflected glory. Now
ABC is trying to corner the market on all with one move. Not only does
the network broadcast the Oscars but its news division is guaranteeing data guru Nate Silver
a role. How much of a Hollywood game-changer will this become? Not much
of one judging from how little attention his movie awards
prognostication has garnered in the past. Twice before, in 2009 and
2011, he sought to predict the Academy Award winners in 6 major
categories based on a “mix of statistical factors”. His track record was
9 correct picks in 12 tries, for a 75% success rate. “Not bad, but also
not good enough to suggest that there is any magic formula for this,”
he wrote. For the 6 marquee categories he hadicapped in 2013, he was
correct only for sure-things and missed the 2 that were more complex
to predict. Meh.

I’ve been pondering this news scooped by Politico’s Mike Allen about all the inducements ESPN/ABC News gave the 35-year-old to leave The New York Times, including extensive
air time, a digital empire, and inclusion in the Oscars. A lot of
showbiz websites and blogs large and small, smart and smarmy, clued-in
and clueless, depend on their Oscar prognostication to drive traffic and
foot bills. But unless Silver allows for the myriad variables that go
into Academy Award noms and wins – insider stuff that Deadline knows
from covering movie awards season in-depth – he won’t become more
accurate.

For instance: Who’s popular, deserving, and appropriately humble
enough to get nominated? Which film’s director is considered a douchebag
whom nobody wants to win? What studio did a lousy job campaigning for
the Academy Awards? How badly is Harvey Weinstein badmouthing the
competition? I’ve always said that most Oscar voters are not just
geriatric and cranky but also jealous and vengeful. Whether Silver’s
statistical model can take into account those indiosyncracies and also
cover more Oscar categories than just 6 is yet to be seen. But I’ll bet
on Deadline’s own awards columnist Pete Hammond to beat Silver’s
prognostications in 2014.

Obviously, the annual Academy Awards process isn’t as big a deal
as U.S. national election campaigns. But interesting to note that
Silver’s FiveThirtyEight blog was driving 20% of all traffic to the NYT as
the last election electrified. That’s because in 2012 he correctly
predicted the winners of all 50 states, in 2008 the winners of 49 out of
50 states, and the winners of all 35 U.S. Senate races that year as
well. What ESPN/ABC offered was to return Silver to his
flagship FiveThirtyEight.com and put him on air at ESPN and ABC, and
develop verticals on a variety of new topics. And now he’ll work for the
TV home of the Oscars. Odds are certain that Silver’s blog now will
become one of the go-to places for Oscar dollars. But not for accuracy.

Can Silver truly become a trusted player in this showbiz space?
Maybe. But he’ll have to do a lot better. Of course, if he’s wrong his
first time out after being hyped way more than in the past, he’ll be
laughed out of the biz. First, he has to stop relying on all the other
film awards each year. They simply don’t matter. It might help if the
Academy Of Motion Picture Arts & Sciences hands Silver its list of
voters. Considering that AMPAS and ABC are joined at the
hip because their broadcast pact goes at least through 2020, that’s
doable. Whether or not the membership will resent having their privacy
violated or participating in any polling is an impending challenge.
Certainly the Academy over the years has discouraged voters from
cooperating with any prediction schemes.

So what methodology will Silver use? As best as I can understand it
(and, please remember that I’m mathematically challenged), it’s
a so-called ’educated and calculated estimation’ stemming from his
reliance on statistics and study of performance, economics, and
metrics. This guy first developed the Elo rating for Major League
baseball, a system that calculates the relative skill levels of
players. He then developed his PECOTA system for projecting performance
and careers and sold it. His FiveThirtyEight is a self-created political
polling aggregation website (which took its name from the number of
electors in the U.S. Electoral College) using a calculated model. He
needs to better adapt that to the Oscars instead of just relying on
other awards shows.

I’m a huge fan of Nate’s (for his political coverage – I’ve followed him since his early days at Daily Kos).

I don’t think he’s going to thrive as well in Oscar coverage. Unlike many of the other things he so accurately predicts, the Oscars are dominated by X factors that are difficult to calculate in a formula. He also doesn’t seem to have inside contacts that are needed by the best predictors to be correct at an elevated level.

He likely is making way more than $1 million a year from ABC, so they will want to push him (Oscar sites being the source of a lot of potential ad revenue), but my guess is that this won’t add to his reputation.

Of course, if he’s wrong his
first time out after being hyped way more than in the past, he’ll be
laughed out of the biz. First, he has to stop relying on all the other
film awards each year. They simply don’t matter.

All the pieces matter. — Lester Freamon on The Wire

Nate Silver knows how to properly weigh and correct each factor. This is how he was able to glean accurate values from many biased polls during elections. And Nate Silver will not be laughed out of the business because his methodology is solid. This article sounds like somebody is worried that an Oscar predictor is going to actually bring math to the table instead of hyperbole. It is about time.

Unlike many of the other things he so accurately predicts, the Oscars are dominated by X factors that are difficult to calculate in a formula. He also doesn’t seem to have inside contacts that are needed by the best predictors to be correct at an elevated level.

Given enough data he will be able to define the voting demographic. Then he will be able to predict the awards. For him, it is only a matter of data collection.

It’s not that easy. He has amazing talents, but his limited track record so far suggests he gets the easy ones.

This is different than election or sports predictions.

If he can do it, more power to him. But I follow all of these to some extent, and awards voting is just vastly different and affected by too many things that go beyond normal formulas and often include elements that those not with an ear to the ground just can’t sense.

Also, since there is no record – unlike political elections – of results of the voting beyond the winners (a hugely important level in doing analysis) and five rather than two main candidates, the analytical tools he’s masterered just aren’t available to him here.

His limited track record is probably due to a limited effort. But there are a lot of things he can do to quantify Oscar probabilities. He can look at the demographics and create chains of connections between voters. He can review interviews and the job histories of the voters. He can also correlate different awards and well as critical reviews and lists. We all do that here, but he can do it with much more rigor than we could ever hope to do. It won’t be easy, but the more data he gets and the more practice, the more accurate he will be. I see no reason why there can’t be a statistical model of the Oscars. It is just voting. And while he won’t have polls of the voters themselves, he can make proxy polls based on those other factors.

Edit: He could even model Gold Derby (or alternate critics listing) predictors (many of whom have that ear to the ground that you like). Predicting the Oscars is not impossible. Many people here do it very well. Even weighting just that data, he would probably be able to make some very accurate predictions.

Edit 2: Actually, the weighted average method might be an interesting thing to do here. It would take a bit of work, but you could go through the lists for the past couple of years or so for the better critics, experts, and users and be able to assign weights to each of them. Then you could run the probability on their current predictions to make a meta-prediction. Hmm…

This will be fascinating to observe. As Scottferguson (and Nikki) said, there are many X-factors involved in prognostication that he will have to consider. Someone like Tom O’Neil or Sasha Stone has been around long enough to parse it out. It’s not clear if Nate Silver has the intenstinal fortitude for it. But it’s really saying something if La Finke herself is sniffing around and sharpening her teeth. She hasn’t decided if there’s another big dog in the yard to challenge her.