5/24/2004 @ 12:00AM

Mass Intelligence

As Google understands, crowds do a better job of decision making than individuals.

When Google finally gets its gargantuan market capitalization later this year, it will turn its founders into billionaires and make individual investors everywhere swoon. But it will also validate an idea: The most valuable resource on the Internet is the collective intelligence of everyone who uses it.

Google has succeeded for a simple reason: It regularly finds the Web pages that are most valuable and puts them at the top of the list. The heart of the technology that lets it do this is the PageRank algorithm (after cofounder Larry E. Page), which essentially asks Web page producers to vote on which other pages are most worthwhile. Each link to a page counts as a vote. Google is a republic, rather than a pure democracy; sites that have more links into them are effectively given more voting power. But the principle is fundamentally democratic–let the masses decide. Given the Wild West nature of the Web, you’d think that this would lead to chaos or irrationality. Instead, it leads to a remarkable order.

How does this work? What Google is relying on is something I call the wisdom of crowds: Under the right circumstances, groups are smarter, make better decisions and are better at solving problems than even the smartest people within them. On any one problem a few people may outperform the group. But over time collective wisdom is near-impossible to beat. No one, you might say, knows more than everyone.

You can see this phenomenon at work in examples ranging from the trivial to the genuinely weighty. Finance professor Jack L. Treynor, for instance, devised an experiment in which he asked his students to guess how many jellybeans were in a jar and found that the group’s average guess was off by just 2% even though very, very few of the students were that close. Or consider the show Who Wants to Be a Millionaire. When a contestant on the show is stumped by a question, he has a couple of choices in asking for help: the audience or someone he’s designated as an expert. The experts do a reasonable job: They get the answer right 65% of the time. But the audience is close to perfect: It gets the answer right 91% of the time, even though it’s made up of people who have nothing better to do than sit in a TV studio and watch Regis Philbin.

More strikingly, groups also seem to do a good job of forecasting the future. The Iowa Electronic Markets is an online market where people can buy and sell contracts based on how they think presidential candidates will do. Since 1988, when it was created, the market’s forecasts have been more accurate than those of polls three-quarters of the time, even though poll questions and results are fashioned by professional pollsters, while the Iowa market is open to just about anyone.

The wisdom of crowds can be seen at the racetrack, where the odds on horses coincide very nicely with their probability of winning. (That is, if you look at a large collection of horses that went off at 4-1 odds, you find that 20% won.) And, of course, collective wisdom is also at work in markets, which is why it’s so hard to outperform the market over time. Just as Google’s PageRank encapsulates the knowledge of Web users, so does a market price embody, as the economist Friedrich Hayek suggested, all of the tacit knowledge and wisdom of investors and traders.

Of course, markets are also known for being subject to manias and panics, fads and mass hysterias. Why do these occur? For the crowd to be intelligent, the people within it have to be making decisions on their own, while drawing on diverse sources of information. During a bubble or panic, people’s decisions become dependent on others’–”If they’re selling, I better sell, too”–and diversity vanishes as people get caught up in the prevailing frenzy. But a crowd is wisest, it turns out, when its members act independently.

Google’s success, then, is far from an interesting quirk. Instead, it’s relevant to just about any problem-solving situation. As long as you’re asking a question that has a right answer–including questions like, “Should we acquire this company?” or “Is there a market for this new product?”–and as long as people are making judgments on their own, collective intelligence will get you the best answer possible. Google, and it shall be given.

James Surowiecki, author, The Wisdom of Crowds: Why The Many Are Smarter Than The Few And How Collective Wisdom Shapes Business, Economies, Societies And Nations.