Sunday, January 2. 2011

The lead up to the Christmas and New Year holidays required researching a number of consumer goods to buy, which of course meant using Google to search for them and ratings reviews thereof. But this year it really hit home just how badly Google's systems have been spammed, as typically anything on Page 1 of the search results was some form of SEO spam - most typically a site that doesn't actually sell you anything, just points to other sites (often doing the same thing) while slipping you some Ads (no doubt sold as "relevant"). The other main scamsite type is one that copies part of the relevant Wikipedia entry and throws lots of Ads at you. It wasn't just me who found this - Paul Kedrosky found the same:

Google has become a snake that too readily consumes its own keyword tail. Identify some words that show up in profitable searches -- from appliances, to mesothelioma suits, to kayak lessons -- churn out content cheaply and regularly, and you're done. On the web, no-one knows you're a content-grinder.

The result, however, is awful. Pages and pages of Google results that are just, for practical purposes, advertisements in the loose guise of articles, original or re-purposed. It hearkens back to the dark days of 1999, before Google arrived, when search had become largely useless, with results completely overwhelmed by spam and info-clutter.

And I can't believe Google doesn't know this - nor does Paul:

Google has to know this. The problem is too big and too obvious to miss. But it's hard to know what you can do algorithmically to solve the problem. Content creators are simply using Google against itself, feeding its hungry crawlers the sort of thing that Google loves to consume, to the detriment of search results and utility. For my part it has had a number of side-effects. One, I avoid searching for things that are likely to score high in Google keyword searches. Appliances are an example, but there are many more, most of which I use mechanisms other than broad search. Second, it has made me more willing to pay for things. In this case I ended up paying for a Consumer Reports review of dishwashers -- the opportunity cost of continuing to try to sort through the info-crap in Google results was simply too high.

Reading the comment's on Paul's blog post was interesting - you can parse the responses into 3 broad groups:

- Yes, we agree with you, and here are some tips on how to deal with it
- Yes, but its not poor Google's fault, its those evil spammers (ie Google has no way of changing their systems and is at the mercy of SEO)
- No, there is no problem, this is the best of all possible solutions (complete bollocks IMHO, it was definitely better a few years ago)

(Ignoring the ones trying to pimp their own products or agendas of course, and the end posts comparing the economics of online vs library copies of Consumer Reports.....)

Ignoring these comments, I have found my behaviour is exactly the same as Paul's , i.e. increasingly reaching for paid-for, edited research (Which? in the UK) as Google and some of the "comparison" sights (clearly flooded with Spam, Sock Puppets and Sleazeoids) become less and less credible. (Another aside - I had a gift voucher from Amazon, and searching for a book I wanted I found Page 1 was totally full of results for the book on Kindle, which was very irritating - they need to allow one to select e-book and/or book).

The interesting question to me is what happens if this gets worse, as Google risks attacks on 2 fronts:

(i) Other search engines decide to eschew Ads for accuracy and cut down the spamming, to gain market share. There is an article on Techcrunch today about Blekko, which appears to promise this.

(ii) The market for paid-for search and research grows - how much would you pay per month for a neutral search engine? Which? costs about £7 a month, would you pay that for a neutral engine?

Frankly, I don't believe that it is not possible to reduce this sort of spam, I think Google's problem is more that it is trying to navigate a line between income (systemically the more spam there is, the more Ad money it makes) and usefulness (how much spam can you run before the user walks away) and has veered too far to the spamside.

Update - this piece was ReTwt'd by Tim O'Reilly, and had the equivalent of a slashdotting so down went our server - fortunately those nice people over at Hacker News pointed to the proxies pronto. I liked two of the themes in the comments there:

(i) a lot of these sites must be known, ditto their pattern, so just a few weeks judicious "mechanical turk" work should have a large 80/20 impact

(ii) Google is like a monoculture, and thus parasites have a major impact once they have adapted to it - especially if Google has "lost the war". If search was more heterogenous, spamsites would find it more costly to scam every site. That is a very interesting argument against the level of Google market dominance

Searching for the New Search
The meme of Google's sputtering search product has certainly taken off, and that of course brings the apologistas out of the woodwork - one of the most amusing is this one on MediaBeat, which somewhat circularly argues that: I saw it coming first: I

Weblog: broadstuffTracked: Jan 13, 11:58

On the (rapidly) decreased uselessness of Google Search
You may recall we wrote a story just after the Christmas holidays about the "Increasing Uselessness of Google Search" - well, it got so far as being mentioned on TWIG (where Jeff Jarvis roundly dissed us for not knowing what we were talking about, despite

Weblog: broadstuffTracked: Jan 21, 20:46

Google blinks...
Just after Christmas we wrote a post on the "Increasing Uselessness of Google search" which (if we may say so ourselves) helped kick off a furore which still continues, the latest episode this week being a debate between Google, Bing and newcomer Blekko -

Weblog: broadstuffTracked: Feb 04, 12:08

More evidence of the Google Slippery Search Slope
It's interesting - ever since we wrote about the "Increasing Uselessness of Google Search" after the 2010 Christmas holidays, it looks like a whole lot of Googleflaws are coming out. Today there is a report in the NYT of how JC Penney gamed them over seve

Weblog: broadstuffTracked: Feb 13, 17:37

Google an an Akismet Almanac
Normally when we predict things will happen its over months or years, not weeks - so when we noted the things Google should to to get themsleves back on track we noted that they should, as a matter of Class A Fundamentals: Fundamentals - secure the nex

Weblog: broadstuffTracked: Feb 14, 22:25

http://broadstuff.com/archives/2430-unknown.html
On January 2nd we wrote a post on "The Increasing Uselessness of Googlev search" based on our experience on searching for commercial items over the holiday period. Matt Cutts of Google told us that we'd never had its so good with Spam and later that month

Weblog: broadstuffTracked: Feb 25, 21:00

http://broadstuff.com/archives/2431-unknown.html
On January 2nd we wrote a post on "The Increasing Uselessness of Googlev search" based on our experience on searching for commercial items over the holiday period. Matt Cutts of Google told us that we'd never had its so good with Spam and later that month

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.Enter the string from the spam-prevention image above: