A peek behind the curtain at Google

Udi Manber talks about search at Google in a recent post on the Google blog. If you’re interested in search or search engine optimization (SEO), the post is definitely worth a read. Udi discusses items from big (Google revamped how it computes PageRank in January) to small (in Hebrew, an acronym like IBM would be written as IB”M).

But you know what my favorite tidbit is? Udi talks a little about how the Search Quality group is organized. He mentions topics such as core ranking, evaluation, and webspam. This post makes it crystal clear that I have a limited role in overall search quality at Google. I can’t help but laugh when someone refers to me as Google’s head of search quality, because that’s not remotely close to true. I’m the head of the webspam team, which is just one part of the search quality group. Here’s how to think of search quality at Google:

As this hand-done bit of an org chart shows, webspam is just one group under the overall umbrella of search quality. The webspam group gets a lot of attention from the SEO community, but there are so many other people and teams that tackle search quality at Google — everything from synonyms and snippets to personalization and international search quality. I’m grateful to work with talented colleagues directly in my team, but I also really appreciate the chance to work with great people in the search quality team as a whole.

Anyway, check out Udi’s post and you’ll probably learn a thing or two about how we think about search quality at Google.

Tell us more about GOOG WebSpam Team. What Udi mentioned was too generic 🙂

“There is a whole team that concentrates on fighting webspam and other types of abuse. That team works on variety of issues from hidden text to off-topic pages stuffed with gibberish keywords, plus many other schemes that people use in an attempt to rank higher in our search results. The team spots new spam trends and works to counter those trends in scalable ways; like all other teams, they do it internationally. The webspam group works closely with the Google Webmaster Central team, so they can share insights with everyone and also listen to site owners.”

Matt I really thought Ubi’s post was probably too generic to be of practical help to most sites with problems. From the inside it probably appears that Google is bending over backwards to make absolutely sure almost no “innocent” sites get caught up in the SEO and Spam crossfire, but in practice most sites now attempt SEO in some form and many sites (and even companies) wind up damaged or destroyed without even knowing what hit them. The issue is the degree to which Google should share “what hit them”. Policy is to share nothing about algorithmic damage, and I think policy is still to define “being in the index” as “no penalty” which totally confuses anybody outside of SEO and even many of us who understand SEO quite well.

It’s the classic collateral damage argument – Google thinks this is necessary to protect the Algorithm, but I think long term this is a mistake and Google should expand the system of communication and community so there is at least a better explanation of the severe downranking penalties that leave sites in the index but out of view.

Towards a solution? Next time you do quality team hires have the new people play webmaster for a month before you share any info with them – have them work some sites, try to communicate with support, etc. This might help bring the outside frustrations…inside.

this is really good initiative taken by google search quality team to make peoples aware about their technology and how they fulfill their search needs.
But it should be expanded as there are so many things in google that no one is even heard about…
Thanks

Wow very well put Joseph, sometimes I’m not sure if Google knows how frustrating it is to see a site penalized or disappear from rankings and have no idea why. This is frustrating to us but even more so to small business owners who have even less of an idea what is going on and they are going to take their anger out on Google’s reputation.

Matt, any plans for the other heads of the search quality team to start blogs and be more open to contact?

I saw your point yesterday on a long ago post related to health care terms. Turned to google to find a photo and description of a particular bone fracture to convince someone of the severity of their condition. Was pleased with the quality of results and image search turned up several good results as well.

As I read your post above where you wrote “I can’t help but laugh”… I thought for a second that you might actually mention my latest video spoof…but, I, was, wrong. Oh, well.

A while back you mentioned you would be writing a blog post about Oatmeal’s “Widgets Go Wild” post (which is why I created the spoof video), can we be looking for that post on Off-Topic Link Bait anytime soon?

I was taken to see Eddie Izzard’s new stage show last night for my birthday – first thing he did was start talking about Universal Search – “what are all those tiny pictures doing in the Google Search results?” I was dying.

Harith, I thought that was a pretty good description. There are engineers that work on algorithmic solutions, but we’re also more than willing to ask some very smart, talented people to find and counteract new spam trends manually as well.

Dave Bascom, I was going for a cloud, but even the xkcd guy would laugh at my artistic abilities. There are other Googlers blogging.

Joe, I do think it’s great when Googlers have a personal website because they really get to see the other side of the situation too.

Michael D, it’s more of a nervous laugh, because I don’t want people to think that *I’m* claiming to be the head of search quality. It’s just a mistake that some reporters make. If anything, I’d really like to get more Googlers out front and center and doing different types of external communication. I do like that we sign our blog posts and that a diverse amount of Googlers get to participate though.

Dave Dugdale, the hazard of baiting more experienced search folks is that they may recognize the attempt and not bite. 🙂

It is great to get an insight into Google, and I love looking at the Google photos you see online from time to time. I wonder though if communication between the search engines in general and webmasters could be better, especially when major changes are implemented, but I realise why the SE’s wouldn’t want to be so forthcoming with information if it helps web spammers.

How long does it take the Webspam team to take down spam?
I submitted many times on this 1 link, and Google hasn’t removed it.
Google took down the other spam links in 1hr, but has still left this one.
The weird thing is that it’s the same link in the end!

I have reported the wiki link to Google for weeks, and it hasn’t been taken down. The culprit http://www.saveonrefinance.com seems to have a PR4, and I’m not sure why they haven’t been banned for the deceptive redirects.

That is a pretty “nebulous” organizational chart Matt ! I hope Google’s wage roll is more detailed (lol).
More seriously, whether your colleagues like it or not, you are among the worldwide webmaster community, not only Mr Spaminator, but a smiling face and strong voice representing Google. For remote folks (ten thousand milles and more away from Mountain View) you are the closest character to Mr Google !!

Thanks for feedback. It would be great to hear about the “social life” of WebSpam Team. Do you go out together sometimes. Do you have for example Table Soccer available for the team. How do you use to reward those intelligent hard working members of your team for a well done job, for example in connection with fighting back on paid links etc…

from what I could gather.. most of it stuff beyond PageRank seems “hacks” to get things working (yawn.. doesnt motivate me to send my CV.. seems pretty boring)
Given all the data, are there any “generic” patterns, scientific generalizations, algorithms that have emerged, which you folks would be willing to share? for the greater good? for the sake of science (Moma always used to say ‘never forget your roots’)

I would imagine that this perception is because you are so visible with this blog and your speaking engagements. You have opened our eyes to more people working on “search quality” at google, and yet now it’s a bigger mystery since we have no idea how many people work in that ( assumed larger ) team. I haven’t even began to think about that “other groups” mean…

Matt, that diagram is just asking to be misinterpreted. The way it appears from a purely visual standpoint, it looks like Webspam is half (or more) of Search Quality. I know that’s not what you intended, despite spam being a big problem, but that’s how it could be interpreted and sooner or later some idiot is going to start the rumor based on the diagram and everyone will follow the “logic” to a T. Just a thought.

Matt cutts I was wondering why my site was penalized so heavily during this update? It has been good for the last 5 years, now it just tanked for every keyword, the serps have such crap results and my site actually has content and a lot of natural links, it is http://www.cellphone-wallpapers.net

I was looking at GOOG webmaster tools I was ranking 1-4 for everything now I cant be found and I have emailed support to find out what is wrong so I can even possibly fix it, nothing, I don’t get it!!!

I was just practicing with a new video editing technique for my training videos and thought it might be fun to put that short video together (and maybe get a few links out of it too).

Since I don’t present at conferences like SMX, these spoof videos are a good way of meeting people at the conferences. At the last PubCon conference I had several people come up to me and say, “Hey aren’t you that guy that made that silly puppet video of Matt Cutts!”

Perhaps I will someday get a link out of you – even thought it would be off-topic! 🙂 If you get a chance in your Keynote at SMX in a couple of weeks you can address the Oatmeal Widgets Gone Wild post?

I’m quite impressed by the latest round of improvements over the last year. FINALLY I get search in another language…. i.e. search in Spanish for Estate Agents, can’t remember how long that’s been on my wishlist?!

The quality of translation has improved enormously in the latest changes to Google/translate. You’ve launch images analysis, well sort of, early days, it will get better.
Maps now has Thai and English script for Thailand (minor things like BKK is attached to the wrong airport, but still such a major improvement that I was able to enter the address of a house I was buying in badly transliterated Thai>English and it still suggested the correct street.)

Strikes me that voice audio and music are the next major things to get indexed… I know you could today analyze a tune, now if only I could hum it into my computer to get Google to find it… I recall some projects in that area (e.g. by recording the radio on smartphones).. And I know some tv programs have on the fly captioning so it must at least be partially possible to index audio. Perhaps even condense music style into a set of numbers, speed, melodicity etc.

A maps->Tom Tom bridge, again very useful. Slowly you’re indexing public transport, a step in the right direction IMHO, terrain is nice. Events next perhaps, I’d like to search what’s on in Paris in (time delimeter) and get a sensible answer from maps showing what’s happening where and when.

Likewise time delimiting in general search would be nice.
[movies london from June 1st – June 20th 2008]

Still not happy about the privacy handling in Google, but enough to move my public email address to a gmail server.

Lots of little improvements, no doubt lots more to come. How many grammes is 2 cups of flour, Google? What is sqrt(i), Google? 0.707106781 + 0.707106781 i ,….. erm well some features strike me as not being so pragmatically designed.

I love the “cloud” as it speaks volumes – it’s mysterious (hiding the Holy Grail known as the Google algorithm!), rising higher than the competition, when it rains it signals another Google Dance, etc.

Really appreciate how you share such good info with “the world.” Do know that I use your “Removing Pages from Google” video on YouTube in my SEO class. 🙂

there is no way thats keyword stuffing those are all phones that we support with carriers, its on the top 2/3 sites also that are ranked for the term (flycell and jamster) but i have it removed and ofcourse google didnt alert me telling me it was because of that even though i had emailed them…

Do you think Google now has new thing to work on? The latest thang that the SEO industry seems to think is perfectly dandy is the idea of publishing false and fake headlines and content in order to achieve maximum incoming links from blogs and news sites.

This certainly goes hand in hand with blackhats and how the SEO industry protects them and praises them. I guess I’m not surprised that a SEO is now praised for writing bogus and fake content, and did so on purpose in order to gain incoming links.

Maybe it’s now time for search quality to discount blog links, etc now? You cannot manually review content to make sure it’s not fake, so simply stop rewarding blogs who partner up with each other to spread deliberate lies.

Hi Matt,
I don’t know where to find an answer for my quandary. My website: watercolorart.com has disappeared, it doesn’t even show up in a search for the domain name. This is a site that had a #1 ranking for my search words (watercolor paintings, watercolors etc) since its inception ( 1998) until about 3 weeks ago. I don’t know why I’ve disappeared and why sites without relevance to the key words have now moved to the top, one site doesn’t even have any content and is merely a placeholder. How have I offended the Google Gods?

Apologies if this is not the correct forum to post, however I’m at a loss as to where to find help.

I also enjoyed reading the post. I can certainly understand the curiosity that people display regarding Google’s algorithms. But I also understand the need for secrecy, which as a user I care for the most. As a newbie web entrepreneur I have first hand experience with the difficulties involving getting visibility on the Internet and as the number of websites grows the more difficult this becomes. Google’s secrecy is an absolute necessity in fighting abuse attempts by cluttering the Internet with useless content in order to draw traffic for the sole purpose of having visitors clicking on Google’s own ads.
As an average surfer who searches for relevant information I am pleased though with Google’s concept of “relevancy” (at least as well as I understand it). I believe it helps me find what I am looking for more quickly in stead of having to go through too many irrelevant search results caused by auto-generated-content-websites. Then again, the code in such programs may actually get so intelligent eventually, that those site will provide what I was looking for – who knows.

I would have liked more info on how to get a good page listing but I know that isn’t happening. However, I agree with other posters. Google should at least tell people what not to do. Do anything you like – just don’t do this. People should be forewarned a little.

Of course common sense would say just play it cool. Make a small splash but don’t hit the water too hard. That kind of thing. When I do SEO work for a client one of my roles is that of therapist. I tell them they need to relax and lower impossible expectations. What else can I do? No one here has a magic wand. (I sure don’t.)