Google, you see, is not just a search engine. It's a multi-billion–dollar company that offers countless other internet services involving everything from news search and image search to video hosting, maps, finance, and even price-comparison shopping. And Google's search engine is, well, not just a search engine. Now controlling as much as 85 per cent of the search market, this de facto internet gateway is also a place where Google can deliver its own services to netizens across the globe. YouTube, Google Maps, Google Product Search, and any other Google service — as well as any service Google might build in future years — all have an obvious advantage over competitors.

This is particularly obvious when you consider Google's so-called Universal Search, where Google inserts links from its own services into prominent positions on its search-result pages. But the advantage extends across the length and breadth of the company's search engine, as Google — and Google alone —– has complete insight into how this internet gateway operates. Competing services are largely in the dark.

In its complaint to the European Union — filed in February 2010 —– UK-based price-comparison outfit Foundem accuses Google of "exploiting its dominance of search in ways that stifle innovation, suppress competition, and erode consumer choice." The complaint makes two arguments. It alleges that Google used "discriminatory penalties" to remove certain sites from its organic search results regardless of relevance, and it claims that Universal Search is transforming Google's search engine into an "immensely powerful marketing channel" for its own services.

Speaking with The Register this spring, Foundem founder and CEO Shivaun Raff summed up the complaint: "You have an overwhelmingly dominant search engine. If you add to that that search engine's ability to apply discriminatory penalties — they're discriminatory because some services are manually rendered immune through white lists — and you add the ability of that search engine to preferentially insert its own services at or near the top of the search results, all of that adds up to an unparalleled and unassailable competitive advantage."

Foundem, Raff says, has felt this firsthand. In 2006, algorithmic changes effectively removed Foundem from Google's organic search results and all but prevented the company from purchasing ad placement via Google AdWords. The company spent more than three years fighting for a return to Google's search engine, and it wasn't until Foundem took its story public — but before the EU complaint — that Google agreed to do so. In the meantime, Google introduced Universal Search, and according to Foundem, Mountain View has used the setup to boost its own price comparison service, Google Product Search, to the top of the market.

Google's ranking of leading UK price-comparison sites across a broad sample of product- and price-comparison–related search terms, as of 29 January 2010. Google Product Search results are shown in red. Other price comparison service results are shown in shades of green (source: Foundem)

Two other vertical-search outfits filed complaints with the EU: Ciao, now owned by Microsoft, and the France-based ejustice.fr. But Foundem is the complainant to go completely public with its story, and the meat of the EU's formal investigation looks to address Foundem's claims: "The Commission will investigate whether Google has abused a dominant market position in online search by allegedly lowering the ranking of unpaid search results of competing services which are specialised in providing users with specific online content such as price comparisons (so-called vertical search services) and by according preferential placement to the results of its own vertical search services in order to shut out competing services," reads Tuesday's statement from the European Commission.

Foundem has called on the EU to prevent Google from ever discriminating in favor of its own services and to ensure that when Google does displays links from its own services, they're clearly labelled as such. Its complaint also says that Google and other search engines should be more transparent about discriminatory penalties and provide a formal appeals process so that "errors" in results-placement can be quickly addressed.

Which is only reasonable.

The company does not want Google to expose its search and ad algorithms. Foundem realizes that such a move would allow sites to game the system. "We've never argued with the need for penalties. You need to have penalties because spam masquerades as original content. You need some way of removing it. This is a necessary part of any search engine process," Foundem CTO and founder Adam Raff told The Register following the European Commission's announcement. "What is completely unacceptable is when, with such penalties in place, the rationale behind those penalties [is not provided] and there's no mechanism for appeal."

The great whitelist hope

The playing field is nowhere close to level, Foundem says, unless Google opens up about how the system works. "At a minimum, we want more transparency," Adam Raff said.

One of the big sticking points is whitelists. Foundem says that Google uses whitelists to opt certain sites out of its standard algorithms. It says that Google support reps discussed these whitelists with the company on several occasions, and it provides email messages as evidence. Whitelists, Foundem argues, allow Google to discriminate against sites whenever it chooses.

But Google denies it uses whitelists. Company spokesman Adam Kovacevich reiterated this denial on the phone with The Register on Tuesday morning. It's this sort of contradiction, Foundem says, that causes outsiders so much pain.

It's also problematic, Foundem says, that Google apparently uses separate algorithms for Universal Search. According to Foundem, this is another means of discrimination. "Crucially, the placement of Google’s own services is subject to different algorithms and relevance criteria than those used to place everyone else’s services. This special treatment gives Google absolute discretion over how aggressively it favours its own services," the company says.

"Through this mechanism, Google can leverage its overwhelming dominance of search into virtually any market of its choosing, such as mapping, video, price comparison, travel search, financial search, property search, music downloads, and books."

Using Universal Search, Foundem says, Google has also used its discriminatory market power to squeeze out the likes of MapQuest in the online maps market:

Unique monthly US visitors to Google Maps and MapQuest between January 2007 and November 2009 (Source: ComScore)

And in much the same way, Foundem says, Google could potentially use Universal Search to dominate practically any market. "[The EU investigation] is really important because there are so many stakeholders in this. Any person or business that uses the internet is a stakeholder in the outcome of this case," Shivaun Raff told us on Tuesday morning. "Google can extend [Universal Search] to any service."

Naturally, Google paints a vastly different picture. As part of a long statement sent to The Register in response to the Commission's announcement, the company downplays claims that it affords its own services preferential treatment, saying that its sole aim is to serve users — i.e., the people who use its search engine.

"With respect to showing our 'own' services and 'preferential treatment,' our only goal is to provide the best answer for users — and sometimes the most useful answer isn’t 'ten blue links,' but a map for an address query, or a series of images for a query like 'pictures of Egyptian pyramids.'" the company says. "We often provide these results in the form of 'quick answers' at the top of the page, because our users want a quick answer."

That is undoubtedly a Google goal. But the company doesn't acknowledge that in showing these "answers," it is — in many cases — pushing its own services, including Google Maps and Google Product Search. The company's statement even seems to downplay the notion that it runs its own services (notice the "own" in quotes, above).

Pressed on this, Google spokesman Adam Kovacevich reiterated that Google is interested in providing users with the best answer, and he indicated that with Universal Search, Google is not showing its own content but merely content from others. This is true in some cases. But not always. When Google points to video with Universal Search, for instance, a vast majority comes from YouTube. Google Maps is part of Universal Search. And so on.

In the same conversation, Kovacevich criticized Foundem for not offering original content, saying that 79 per cent of its content is "duplicated" from other sites. Google's argument goes in several different directions. After criticizing the fundamental makeup of Foundem's site, it says it has no problem with vertical search engines and that it returned Foundem to Google's search results because the site improved in some way. At one point, Kovacevich even said that YouTube videos are not Google content. But the overarching message seems to be that as Google seeks to serve the user, Foundem is of little consequence.

By contrast, Foundem argues that Google is actively entering a new and separate market and pushed competitors out. "In a way, they're late to the party," Adam Raff said. "They're just now realizing how important vertical search is."

We also asked Google's Kovacevich if the company uses separate algorithms for Universal Search — as we've asked Google before — but he did not provide an answer. It's a telling omission — for many reasons — and it leads us back to Foundem's core complaint: there's not a level playing field, and, well, Google won't admit that there's not a level playing field.

"At some point, we hope that Google is going to stop confusing the issues and flanneling and start debating the issues," Shivaun Raff said. Foundem is also asking for oversight — but sensible oversight. "Transparency can be its own oversight," Adam Raff told us, "Without transparency, you need some kind of official authority to come in and [sort things out]. With reasonable transparency, people can provide their own oversight." ®