Predicting Success or Failure

By Ellen Pearlman

Strategic Thinkers: Thomas Davenport and Jeanne HarrisCredentials: Davenport is the President's Distinguished Professor of Information Technology at Babson College; Harris is executive research fellow and director of research for the Accenture Institute for High Performance Business. They are the authors of Competing on Analytics: The New Science of WinningBig Idea: Prediction and recommendation technologies are proliferating, but they are not a substitute for decision-making.Article: "What People Want (and How to Predict It)" published by MIT Sloan Management Review, Winter 2009

Why is it that some people seem to have an uncanny ability to make smart decisions? We all know people like that who seem to lead a charmed life: they make the right choice at the right time and reap the rewards. But what about the rest of us - aren't there some useful tools to help make difficult decisions easier? There are, but how good are they?

In the current issue of MIT Sloan Management Review, Babson College Professor and author Thomas Davenport and Accenture Institute for High Performance researcher Jeanne Harris take a look at prediction and recommendation technologies to determine whether they are helpful in forecasting what customers want. They focus much of their article on consumer taste and whether it's a science or an art to figure out what the next great movie hit, gold record or best-selling toy will be.

To come to their conclusions, the authors reviewed the academic research on recommendation engines and conducted interviews with 18 organizations involved in predicting or recommending cultural products. They also interviewed movie studio executives, distributors and theatre companies about their use of predictive models. And while they focused on cultural products they note that any consumer product company will increasingly rely on these technologies to help guide them in making the right decisions. "No company will launch any expensive-to-create product or content offering without subjecting it to some form of systematic prediction or test," they say.

Traditionally, creators and distributors of cultural products have not used analytics to predict success. Instead they have relied on the instincts of tastemakers to determine what people will spend their money on. Today "the balance between art and science is shifting," the authors say, with the access to data and sophisticated technology providing options that didn't exist even a few years ago.

Recommendation engines are becoming popular with consumers because the number of choices they have to make is overwhelming. Producers too need to make smart decisions at a time when there are a plethora of products to buy and consumers are reluctant to open their wallets. In addition, the cost of producing many products has skyrocketed and developers can't afford to make expensive mistakes.

Technology has made prediction and recommendation offerings easier to use. Software can be integrated into online sites and mobile devices that make it easier to generate recommendations based on detailed customer-behavior data. This can give a noticeable lift to sales. For example, Overstock.com used a Gift Finder based on recommendation software developed by ChoiceStream during the 2006 holiday season and found its revenue up 250 percent from customers who used it.

There are many different predictive technologies available since companies like Amazon.com deployed the first generation of this type of technology: collaborative filtering (see the accompanying slide show link to slide show for a list of 10 of them). Each method has its own strengths and weaknesses: Some require a large amount of data to be useful, while others need a large number of independent participants to succeed. The authors say executives should use an approach that provides conservative recommendations "if people are buying your product one at a time," but if they are paying you a monthly fee they are probably "open to a recommendation engine that provides pleasant surprises."

New technologies are also in development-some measure biological indicators, such as heart rate and galvanic skin response-but whatever technology is deployed, it is less successful if it used after the product is created. In most of the cases the authors studied, they say, recommendation offerings were used to help customers choose from finished products.

Some companies are starting to deploy pre-creation approaches, especially in cases where production cycles are long and development costs high. For example, Epagogix is working on predicting the success of movie scripts before production begins using neural network analysis. They say the company is twice as successful as studios in predicting "turkeys" and "eagles."

Of course, human decision-making is not about to vanish, nor should it. "These systems are not a substitute for decision making, nor do they provide automatic, infallible answers," the authors say. But as long as new products are expensive and risky, executives will look for whatever help they can get from technology to increase their chances of success.

Book:Predictably Irrational: The Hidden Forces That Shape Our Decisions, by Dan Ariely, published by HarperCollins, February 2008. MIT behavioral economist Ariely refutes the common assumption that we behave in fundamentally rational ways. He explains how expectations, emotions, social norms, and other invisible, seemingly illogical forces skew our reasoning abilities.