Entertainment world is kryptonite to our powers of prediction, ace analyst says

Predictability and the arts-and-entertainment world have an uneasy relationship.

Predictability is a given — like the mouthfeel of a Big Mac — when it comes to, say, the punch lines on "Two and a Half Men" or the beats of your average romantic comedy. Predictability is a boon to Hollywood when it comes to the box-office might of "Iron Man 3" or other similarly surefire superhero movies.

But the notion that you can engineer a hit by combining the right ingredients has led to an unfortunate predictability in the bland output of major movie studios and record labels, among others. Unpredictable members of the $100-million-plus-domestic-box-office club such as "Silver Linings Playbook" (romance among the mentally ill!) and "Life of Pi" (Indian boy on a boat with a tiger!) are exceptions in a business ruled by sequels, franchises and blowing-up-the-White-House formulas.

Then again, if the movie industry were so predictable, Disney wouldn't have had to take a reported $200 million write-down on last year's sci-fi flop "John Carter," and Warner Bros. might have made back its supposed $200million production budget on this spring's dud "Jack the Giant Slayer."

So on one hand, the entertainment world has access to more information and technology than ever before. On the other, what screenwriter/author William Goldman ("Butch Cassidy and the Sundance Kid," "Adventures in the Screen Trade") said decades ago still applies: "Nobody knows anything."

Since Nate Silver began crunching baseball numbers in Chicago before becoming the nation's most respected political stat-head, he has made an art out of making the previously unpredictable appear predictable. After graduating from the University of Chicago in 2000 with an economics degree, Silver, who is speaking Sunday at 1 p.m. at the Spertus Institute for Jewish Learning and Leadership on his book, "The Signal and the Noise: Why So Many Predictions Fail — But Some Don't," came up with the influential baseball statistic predictor PECOTA, and his "FiveThirtyEight" political blog, now a part of The New York Times, has been uncanny in forecasting the past two presidential elections.

The 35-year-old writer/analyst also has handicapped the past few Academy Awards races and has done … OK: He called four out of six winners in the top categories this year. In addition, he recently had a consulting gig with a cannot-be-named entertainment company that wanted to apply his predictive thinking to its output. Can his brand of analysis work where art meets commerce?

We decided to pick his brain on this topic — as well as the notion of whether art itself can be predicted — in a recent phone conversation, with Silver speaking from New York City, where he now lives. The following is an edited transcript.

Q: How much can statistical analysis apply to the entertainment world?

A: I think probably more than is taking place. I did a consulting project a couple of years ago for a big Hollywood company. In that company, you had some people who were very data driven and very progressive in their attitudes and some people who said, "This is art, you can't boil it down to science. We don't want to use math" kind of thing. So really it ran the whole gamut between those two polarized views.

At the same time, I think you have some of the same issues that you might have in a lot of industries, which is there's a tendency to think in the short term a lot. You have more and more films that are marketed toward your typically 22-year-old male audience, which is pretty likely to go out and see the films on opening weekend but may not give the film as much longevity. You have more and more sales made outside of the movie theater environment, and with social media, I think the buzz tends to matter a lot more, and films that are getting poor Twitter reaction, for instance, will have very, very sharp drop-off in their box office gross, not even from one week to the next but from the Friday night to the weekend even.

So this environment is changing. Statistics don't provide all the answers; you have to measure the uncertainty in a problem. But if you're not doing the stuff at all, then sooner or later your competitors will, and you really put yourself at a disadvantage.

Q: What are the statistics that people are paying too much attention to, and what are the ones that they're not paying enough attention to?

A: There maybe still is a somewhat old-fashioned fixation on opening weekend numbers, where that might not be all that large a part of the revenue pie and might not be as predictive as it used to be. … One analogy I sometimes make is what happened to the sport of boxing, of all things, where when they started having more and more of their bouts on pay-per-view, (they) kind of milked the revenue cow while they could, and you didn't have people growing up with boxing as a sport in their lives anymore, and it really kind of harmed the long-term audience.

Q: That's similar to the Blackhawks, who generated a lot more interest when more of their games became accessible on TV again.

A: Yeah, people are used to getting open access to things. For anyone in the media, this is a big challenge, where you have an audience that's used to having things on demand and not paying very much for them necessarily, which is obviously a threat to the industry in a lot of ways. But it also means you have to cater to it, and that means you have opportunities to distribute your product very, very cheaply, cut down on distribution costs and help build your brand. I really think the way things are now, differentiation is the key. People think, well, it's so easy to transmit information out there and distribute that it's a volume business. I think the reverse is true: that people have so much choice that they gravitate toward companies that are doing it well, and you'll have kind of a winner-take-all type environment in some fields.

Q: Which could be seen as a pro-art argument.

A: Yeah. The way I see it is, make the best film that you can, right? And then figure out how to market the hell out of it. Let your creatives be creative and do what they do best, and then find ways to reach your audience based on that. I think oftentimes there's kind of a co-mingling of different things, where you have studios who are trying to time minute by minute what the audience's reaction will be if you cut a scene in a different way, how that will change things. Those tactics I tend to be a little skeptical of.

Q: That reminds me of those test-audience squiggles that CNN runs during presidential debates. I feel I'm getting a lot of data that's more distracting than relevant.

A: One thing I kind of preach in the book is that in theory more data should always help you, though you might encounter kind of diminishing returns. In practice probably more data gives people more of an opportunity to get confused, more of an opportunity to cherry-pick the results they're looking at.

Q: Studios make calculations along the lines of "Bradley Cooper means X amount at the box office, and Mark Wahlberg means Y," but they don't necessarily calculate whether the movie will be any good.

A: What's also difficult in the motion picture business is it's hard to separate out the marketing budget from the film's success, by which I mean if you take a mediocre product and you market the hell out of it and you give it a good time slot, it's going to do relatively well, although that's (changing). If you'd taken a different film and invested the same resources and production and marketing behind the other film, which might have been inherently a better product, then it would have done as well or better instead. So separating out the cause and the effect is often very challenging, especially when you have people who are in essence acting on their predictions of what will work or what won't ahead of time. You can have self-fulfilling prophesies or self-denying prophesies in other cases.

Q: And people who will benefit if they can fulfill their prophesies.

A: Why are egregiously terrible films put out? It's because (there are) a lot of people whose careers would be in jeopardy if those projects were killed, so it's hard to stop that inertia sometimes toward a bad project once the ball gets rolling.

Q: Quentin Tarantino has talked about directors being like boxers and having peak periods before they go downhill. Bands are like this too; many put out a certain number of good albums before they lose it. Shouldn't there be a way to calculate which filmmakers or musicians will continue to put out worthy material?

A: You could look at Pitchfork (reviews) and how they evolve over time, or album sales, or something. But people who are in the arts, once they kind of achieve success, then it can be a tricky thing. You don't become as hungry anymore. Look, there are a lot more people who have one or maybe two great ideas and really do market themselves well — and by ideas I mean that very liberally in terms of a sound or a film or anything else. But it's a lot easier to have one great idea than be someone who has the aptitude to consistently be a genius.

I have become a foodie of sorts, but even restaurants have, like, a sweet spot that sometimes lasts for only a year or two, when the chef is really invested in it — and when that chef becomes well-known and famous and wants to scale up and opens another venture and the concept becomes kind of played out and it might become dumbed down, you see a decline in quality there. I think part of the problem, too, is basically in all of the arts — and I even encounter this a little bit myself in terms of what I do — there's always a demand in kind of a capitalist system to grow, let's grow, let's grow, and a lot of things when it comes to creativity are really hard to scale up.

A: Yeah, you talk to people who are getting tired of Spielberg now, but that was such a weird category this year because the nominations were all screwed up (with Ben Affleck not getting nominated for best picture winner "Argo").

Q: This was yet another illustration of William Goldman's "Nobody knows anything."

A: People like to lump things into the realm of the predictable and the unpredictable when it kind of exists along a spectrum. In baseball the metrics are very good and the stats are very advanced, but even the best hitter only gets on base 40 percent of the time. With something like predicting the Oscars, I think historically we've gotten something like 70-something percent of our predictions right. I don't frankly think there's any model that could do a lot better than that. It's just hard when you're dealing with a narrow group of people who are not revealing that much about their preferences, who are being lobbied actively by the different studios. It would be more helpful, too, if we knew things like which films were in second place or what percentage of the vote they got.

A: I don't know if I would self-consciously call it that. My parents always valued learning for learning's sake, and I think that's very much a part of Jewish culture. I think the fact that Jewish people view themselves as having been oppressed historically — which means you don't necessarily want to trust the mainstream view; you need to articulate your own path in life — I think that's something I embody.

Whether it's because I have a Jewish ethnicity or not, I'm not sure. I should mention that my mom's not Jewish, so officially there's debate about what I am, and my dad was very secular. But I do see it more as kind of an ethnic background instead of a religious tradition per se in that some of the same cultural advantages or stereotypes, however you want to put it, come through in my upbringing and my values.