Everyone Agrees: Crowdsourcing Sucks

Jon Burgstone was co-founder of SupplierMarket, acquired by Ariba for $1.1B. He now teaches at Berkeley, where he helped launch the Center for Entrepreneurship & Technology. He is co-author of Breakthrough Entrepreneurship.

You're in a crowded theater. Suddenly, a man in the front row has some sort of serious medical emergency--cardiac arrest, let's say. Which of the following choices seems like the best course of action?

A. Rush to the front of the house and begin treating him.

B. Shout out, "Is there a doctor in the house?"

C. Call 911.

D. Ask everyone in the theater to work together, develop a consensus diagnosis, and decide what to do.

If you answered "A," congratulations. You're a brave soul and a man or woman of action, the type to race toward the sound of the guns. (We hope you have some medical training, though.)

Did you answer "B" or "C"? Nice work--both are excellent choices.

But if you chose "D," perhaps you'd better read the rest of this article. In fairness, "D" is pretty close to what a lot of trendier, so-called business leaders seem to suggest you should do.

The Crowd Is Bad at Predictions

Recently, the U.S. Supreme Court upheld Obamacare. (You might have seen this somewhere in the news; it got just a little bit of coverage here and there.) What is especially interesting for our purposes, however, is that almost nobody predicted the actual outcome.

More to the point, pundits across the board failed to predict accurately. And, as The New York Times pointed out this week, the prediction-market website Intrade got things flat out wrong. Even as some Washington insiders began to hear gossipy speculation that maybe the court would in fact uphold the law, Intrade "continued to show about a 75% chance that the law's so-called mandate would be ruled unconstitutional, right up until the morning it was ruled constitutional."

So, why does this matter for you as an entrepreneur? Well, there's much talk these days about the "wisdom of crowds" and crowdsourcing. The U.S. government solicits crowdsourced ideas for products to solve problems in just about every federal program. (Check out http://challenge.gov if you haven't seen this before, or this article about how the Pentagon and intelligence agencies are trying to crowdsource proposals for better intelligence gathering.)

Even the much-touted "lean start-up" principle that encourages entrepreneurs to develop minimum viable products rests on "crowd knowledge." But how much does the crowd really know?

The plain truth is that the crowds predict the wrong outcomes--often. Look at these examples:

OK, so, insights: Ask a bunch of people to bet on what they think the best movies will be, and they might get it right. Ask them who will win an election, and they have very poor odds on it. And then take the health care case, for example. Asking the public to predict the votes of nine jurists they probably can't even name seems about as smart as trying to get the audience that is gnawing on Twizzlers or Milk Duds at the movie Ted to agree on how to handle a hypothetical heart-attack victim.

It stands to reason that if the individuals within the crowd actually have unique insights, then maybe there's value in their crowdsourced conclusions. If you were to gather opinions from serious students of the Supreme Court, then maybe you'd get better results. It also stands to reason that if you ask people things they're likely to know about personally--the movies they watch, which politicians they're going to vote for--you'll get better data.