Achieving search neutrality

You would expect your web search results to be unbiased but are they? Are they completely neutral and without undocumented manipulation? The straight answer is no. As search-engines develop ever-more complex algorithms to drive up relevance and keep out the spammer, we have seen a full-blown fight between SEO organisations and the giants such as Google. The net result is that Google measures around 1,000 different components before awarding a site it’s position in search. Without knowing how these elements are measured, weighted and scored it is impossible for the average (or even advanced) searcher to make any real or contextual decision about the true relevance. Therefore all search engine results are skewed.

So what! You get the right answer, don’t you? What might you be missing? In the UK we seem to be lucky but lots of other countries actively ban certain sites in search and even demote sites that don’t advertise. As search engines are the tool of choice when users wish to find information you would imagine there is a heavy burden on them to provide fair, comprehensive and neutral results.

On a very obvious level, you tend to get different results in, say, Google when you search in London or in Frankfurt and even closer to home the results you get for some search terms are “localised” so I get to see plumbers in the CleckHuddersFax area. This is the search engines trying to layer additional information about me onto the search results and it sometimes works, but not always. I don’t know, because I am never told, how the results were created.

Founder of Google Sergey Brin, rather famously, quoted “The perfect search engine would be like the mind of God” and whilst I am not suggesting he was suggesting that Google were trying to create God I think he neatly summarised the challenge of producing the best results.

So can you create neutral results? Yes, but the answer would not be very meaningful and likely to be a nightmare of spam and porn. Assuming we don’t want to go back to crap-old days of 1997 Alta-Vista search we therefore must accept some shaping of the results. The problem in this shaping is that Google is not just a search engine, it also provides lots of other services such as email, video hosting, mapping, price comparison, local directory, software and a great deal more services that yield substantial financial return.

Would then Google shape it’s results to favour it’s own properties? There seems to be sufficient evidence that this is the case as the European Commission has recently launched an anti-trust investigation and we await the outcome but it’s probably fair to say that, for example, YouTube videos would sit higher in the ranking than Vimeo. With some of (potential) technical restrictions that Google place on YouTube the discussion around control and anti-trust could easier spread from search results to technology in general.

I suppose for the man or woman in the street there is no difference and I hope it stays that way or if it doesn’t then we can always use another search service and that would hurt Google doubly if sufficient people moved elsewhere.