The Netflix prize looks to be nearing completion since its 2006 announcement. Teams have competed to improve Netflix’s “suggested movies” algorithm by 10% (using your prior rentals, it predicts films that you might also like). The lack of success is from movies like Napoleon Dynamite. No matter how many regressions or models thrown at the algorithm, it’s impossible to predict whether someone would enjoy Napoleon Dynamite.

Two important points:

Napoleon Dynamite is popular, but its popularity has no rhyme or reason (e.g., celebrity cast, genre, type of humor).

No one (movie execs included) could have predicted Napoleon Dynamite’s popularity.

I’m reminded of a phenomenon called cumulative advantage, a theory that things are popular because they are popular (a feedback loop of sorts). Cumulative advantage has a big advocate: an academic, Duncan Watts. He argues that popularity is random and unpredictable–explaining Netflix’s inability to predict preference for Napoleon Dynamite. The idea is that popularity is not intrinsic (i.e., why Harry Potter was rejected by 8 publishers).

It’s black swan‘esque–consider music and movie execs scrambling to produce “hits.” Decades ago, broadcasting the latest singer on top 40 radio or casting Clark Gable in a film guaranteed popularity. But now, even an all-star headliner (i.e., Will Ferrell) means nothing.

Watts suggests that popularity has to do with our social nature–we desire to experience the best of everything by relying on others (i.e., Digg, most-viewed YouTube, most-reviewed Yelp), which compounds popularity and cumulative advantage. This interdependence makes predicting hits impossible, as individual taste have no impact.

So what’s this mean for marketers, as we are told that meeting individual customer needs are pivotal to success? Are customers irrationally relying on the influence of others?

My opinion: yes and no. If you’ve read Predictably Irrational by Dan Ariely, you’ll find many examples of consumer irrationality. Studies show that consumers will order different food and drinks based on whether they order first or last in a group. Another study shows a drop in cheating when people are forced to read the 10 commandments. No doubt, I can imagine social influence trumping personal preference in a variety of situations.

But I can’t imagine it affecting my decision for certain products, like toothpaste or detergent. In these situations, marketing frameworks are my friend.

If I was the marketing manager for Napoleon Dynamite, I would hope that I could influence popularity. But I’ve been drinking the marketing kool-aid for 6 years now. While I find it hard to believe that any marketing efforts would be inefficacous, the evidence is compelling: hundreds of statisticians, economists, and hobbyists competing for the $1M Netflix Prize could not predict consumers that would like Napoleon Dynamite. Who am I to think that my silly marketing research could do better?