How Google won the search wars

The idea that made Google the world's greatest search engine was all Larry Page and Sergey Brin. But the idea that set it on the path to becoming the world's greatest company wasn't theirs originally. It was borrowed, with some key modifications, from their biggest early rival.

In September 1998, Page and Brin founded Google based on an algorithm called PageRank that they had developed as Stanford Ph.D. students. Other search engines ranked pages based on the relevance of the text they contained, which allowed spammers to game the system by filling their pages with commonly searched keywords. PageRank dived much deeper, evaluating sites' authority and influence based on the number of other authoritative and influential sites that linked to them. Suddenly a search for "Honda" turned up Honda.com instead of, say, a porn site that had copied the word "Honda" 50 times in invisible type at the bottom of the page.

Within a year, it was clear to many in Silicon Valley that Google's algorithm was superior to those of more established search engines like AltaVista and Inktomi, whose software powered HotBot, Yahoo, and several other major Web portals. The tiny upstart won $25 million in venture-capital funding, which valued the company at $100 million. But it struggled to land contracts with the big portals, which were more focused on building up their homepages than improving their search results. The conventional wisdom at the time was that there wasn't much money in search -- the real revenue came from keeping people on your own pages, where they could see your banner ads.

Advertisement

If Google was going to make money from Google.com, and not just as a service provider to companies like Netscape, selling banner ads on its own search-results pages seemed the obvious play. But Brin and Page hated the idea. They thought banner ads were ugly and distracting. Worse, banner ads took time to load, and Google's founders possessed an almost religious devotion to efficiency and speed. In their seminal 1998 academic paper introducing the idea of Google, Page and Brin criticized advertising-funded search engines as "inherently biased towards the advertisers and away from the needs of consumers."

By late 1999, though, major Google investors such as Michael Moritz had started to get antsy, as John Battelle recounted in his excellent 2005 book "The Search." Google had about 50 employees at the time and was starting to catch on with Internet users, but it was bleeding cash. "We really couldn't figure out the business model," Moritz says in "The Search." "There was a period where things were looking pretty bleak."

That's when Page and Brin started to take a closer look at GoTo.com, a rival search engine that was beginning to reap huge revenues with a novel advertising model. GoTo.com had been founded by a serial entrepreneur named Bill Gross in February 1998, seven months before Google. As Gross explained to me in a phone interview, GoTo.com was actually his attempt to solve the same problem Page and Brin had been working on -- how to build a search engine whose results wouldn't be overrun by spam. But where Page and Brin tackled the problem with algorithms, Gross approached it from the business end.

Rather than build his own search engine from scratch, Gross took Inktomi's engine and introduced a twist: paid search. Like the way companies bought ads in the Yellow Pages, websites could pay for top placement on the GoTo.com results page for a given keyword. This would push down spam results, Gross reasoned, because companies would have an incentive to buy ads for search terms that were actually relevant to their products. Better yet, search ads would be far more cost-efficient than other types of online ads, because advertisers would be paying to reach only those people who were already searching for their products.

To drive home just how efficient these ads could be, Gross came up with an audacious pricing scheme. Instead of paying for page-views -- an old-media model that had come to dominate the Web -- advertisers would pay only when people actually clicked on their ads. And their placement on the GoTo.com results page would be determined through an auction, so that more desirable keywords would command higher prices, while less common keywords could be had for as little as a penny per click. As a search engine, GoTo.com had nothing on Google. But as a way of making money on searches, it was ingenious.

"The whole point of Internet advertising, I thought, was accountability," Gross says. "You could measure it, unlike with print ads. But here was everyone still selling ads the old way: buy a bunch of impressions, cross your fingers, and hope it turns out well."

Google followed the old model when it first dipped its toe into text-based search ads in January 2000. It didn't work particularly well, admits Jeff Levick, who joined Google's ad team a year later. (He's now chief of sales and marketing for Spotify.) "For the first two years of Google we were cold-calling people, trying to get them to buy keywords," Levick recalls.

Meanwhile, GoTo.com was taking off. It went public in June 1999, and in September 2000 it landed a coveted deal to syndicate its paid search results on AOL. But as Battelle recounts in The Search, such deals seemed to put GoTo's own homepage in competition with its syndication partners. GoTo's executives eventually convinced Gross to give up on his vision of making GoTo.com a destination portal in its own right; the company changed its name to Overture in 2001 and focused exclusively on running its ad network.

In late 2000, Google rolled out a new, self-service advertising product called AdWords that allowed businesses to purchase text ads on search-results pages. Soon after, Page and Brin met with Gross at a TED Conference, and Gross suggested a merger. No thanks, the Googlers told him. As Steven Levy recounts in his 2011 book In the Plex: How Google Thinks, Works, and Shapes Our Lives, Page and Brin "would have nothing to do with any system that mixed organic search results with ads." Then there was talk of a partnership, with Overture placing its auction-based ads alongside Google's own results. Overture would go on to partner successfully with Yahoo and MSN, but Page and Brin ultimately walked away. They reckoned they could build their own system to do essentially the same thing, but do it better.

And they did. In 2002 Google launched its own pay-per-click, auction-based search-advertising product, called AdWords Select. It was instant gold, and Google soon dropped the original AdWords altogether. As revenues rocketed, the company turned its first annual profit that year. Before long, AOL dropped Overture for Google, and Overture ended up being sold to Yahoo in 2003 for $1.63 billion. Meanwhile, Google was busy building on AdWords with AdSense, which allowed it to sell targeted ads on third-party websites. On Aug. 19, 2004, Google went public with a valuation of $27 billion. It's now worth close to $300 billion.

Did Google steal Overture's idea? Not really, says Gross. "We didn't patent the idea. So if we don't patent it, they can copy it." Yet Gross is convinced the concept of a cost-per-click, auction-based search-advertising system would have been patentable, in retrospect. By the time he thought to do it, though, it was too late. Overture did file for a slew of other patents peripheral to its system, and sued Google for infringement when it came out with the revamped AdWords in 2002. The case was settled in 2004, with Yahoo getting a big chunk of Google stock in exchange for the rights to Overture's intellectual property.

The ex-Googlers I talked to see it a little differently. Doug Edwards, an early marketing executive and author of the insider account "I'm Feeling Lucky: The Confessions of Google Employee Number 59," says that Google's executives saw the GoTo.com model as fundamentally flawed, because advertisers could bid up keywords irrelevant to their business, knowing that they would only have to pay if people clicked. As Levy explains in his book, Google addressed this by adding a "quality score" to each ad to punish those that were spammy or off-topic. Google also improved on Overture's auction model by replacing the standard "high-bid" auction with a variant on the Vickrey auction, which saves the winning bidder from overpaying.

"I don't recall a great deal of consternation about, 'Oh, this was someone else's intellectual property, we need to come up with something different,' " Edwards says of AdWords. "It was a case of finding an opportunity, finding a system that was broken, and finding a way to fix it and make it work."

Levick, the early ad-team member, agrees. "I think a lot of different companies were looking at marketplaces around that time, and Google was the one that really made it work," he told me in a phone interview. "It's no different from search. Google didn't invent search, they reinvented search. They didn't invent email, they reinvented email. You can take this all the way down to the self-driving car. Google doesn't imagine business products, they reimagine them."

Gross, for his part, seems comfortable, even happy, with how things turned out. And he has nothing but raves for Google. "I'm wildly proud of coming up with the paid-search model," he told me. "I didn't know how big it was at the time." Besides, Gross says, if Google didn't make billions with the pay-per-click auction model, it would have made its billions some other way. "I wish I had come up with the Google idea," he says. "The Google idea was the idea for organizing the world's information. Mine was just an idea for making money."