Usability Study: Users wrongly trust higher Google results

encyclo

A U.S. study suggests Internet users mistakenly have an inherent trust of Google search results that appear higher on a page. A College of Charleston eye tracking experiment revealed college students participating in the study trusted Google's ability to rank results by their true relevance to the query. When participants selected a link from Google's result pages, their decisions were strongly biased towards links higher in position, even if content was less relevant to the search query

ceestand

7:20 pm on Aug 23, 2007 (gmt 0)

"The way college students conduct online searches promotes a 'rich-get-richer' phenomenon, where popular sites get more hits regardless of relevance," he said.

Yes, I imagine this is true of all search engines. Since the user has little idea of the site's content prior to "hitting" them, by default the higher ranked sites would get hit first, and thus more often. Otherwise, the SE would have almost no value.

"This further cements the site's high rank, and makes it more difficult for lesser known sites to gain an audience."

Is this true? Do SEs count their own clickthrough rates towards a site's ranking? If so, eventually they'll create a self-fulfilling hubris prophecy.

Lord Majestic

7:55 pm on Aug 23, 2007 (gmt 0)

This is also true in real life.

europeforvisitors

8:24 pm on Aug 23, 2007 (gmt 0)

Maybe there's more to the study than the article suggests, but in any case, the result seems both predictable and obvious. Wouldn't there be a bias toward the top of the list in any list, not just search results? Or on any Web page, for that matter, not just on a SERP? Look at Jakob Nielsen's "F-shaped pattern for reading Web content" eyetracking study:

jtara

jimbeetle

pageoneresults

8:44 pm on Aug 23, 2007 (gmt 0)

I believe the AOL data that was leaked out a while back confirmed the same thing. In fact, this is something that has been prevalent for as far back as I can remember. If it were not, many of us would be without a career. ;)

ATF - Above The Fold

maximillianos

9:11 pm on Aug 23, 2007 (gmt 0)

Dagnabbit! I've been shooting for the bottom of the list all this time. Now they tell me.

rogerd

9:14 pm on Aug 23, 2007 (gmt 0)

It may not be part of the study results, but I think there's a belief that the top search result is, in fact, whatever the person searched for. If they typed in "Acme Widget Company", they will often assume that they have reached the website of that firm, and completely ignore the site content which show they are at "Big Widget Directory." I get contact form emails all the time that make that clear - people ignore all clues on the site (logos, headers, text, etc.) and forge ahead with their question.

rise2it

9:29 pm on Aug 23, 2007 (gmt 0)

RogerD has a great point.

There are many searches where my company comes up higher than the manufacturer, and we have gotten what has to be thousands of calls for things that has nothing to do with us.

They type in 'Acme Widgets', we come up #1, they assume we are the manufacturer - and they almost act like we are LIARS when we try to politely explain that we are NOT the manufacturer.

This has gone as far as people threatening to sue us while screaming that they KNOW we are the manufacturer - sometimes you just have to laugh.

As RogerD said, it doesn't matter what WE put on OUR page - they don't read it. They see the toll-free number and start dialing.

Waste of our time, waste of our money, but it's the price you pay...

[edited by: rise2it at 9:31 pm (utc) on Aug. 23, 2007]

maximillianos

9:30 pm on Aug 23, 2007 (gmt 0)

I agree with Rogerd. We get folks on our site all the time that think they are on an official site for whatever they were looking for, just because we were in the top few (or first). I get emails all the time from folks asking us to stop billing them for this or that... things that we have no idea what they are talking about.

europeforvisitors

9:53 pm on Aug 23, 2007 (gmt 0)

It may not be part of the study results, but I think there's a belief that the top search result is, in fact, whatever the person searched for. If they typed in "Acme Widget Company", they will often assume that they have reached the website of that firm

Aha! So that's why my travel-planning site gets job applications from cruise-ship personnel, requests for Murano glass repairs, and angry letters from animal-rights activists to the Swiss city that's famous for its bear pits.

carguy84

10:43 pm on Aug 23, 2007 (gmt 0)

In my experience, the 2nd and 3rd results are typically more relevant to what I'm searching for. I've noticed this to be especially true for programming-help type searches.

Beagle

10:57 pm on Aug 23, 2007 (gmt 0)

I have a friend who refuses to get an internet connection at home (she uses the computer someone gave her to play solitaire), so she does her surfing at the library. A couple of weeks ago, she made a typo in a URL and got the "Were you searching for...?" question. Yes, she was searching for that, so she clicked on the link thinking it would take her to the site she wanted. When she called me later to complain about it, she still thought the page of search results was the site she'd been looking for, and couldn't figure out why it gave her a bunch of useless links instead of the info she wanted.

rogerd

2:04 am on Aug 24, 2007 (gmt 0)

You should do something about those bear pits, EFV. If they track you down, you are liable to end up in there with the bears. ;)

Wlauzon

2:15 am on Aug 24, 2007 (gmt 0)

..They type in 'Acme Widgets', we come up #1, they assume we are the manufacturer - and they almost act like we are LIARS when we try to politely explain that we are NOT the manufacturer....

And I thought it was just us!

Over the past few years it has become totally obvious that many people have a severe reading comprehension problem when it comes to the internet. And very few really have any clue at all as to how search engines or almost any internet 'thingies' work.

And if anyone thinks it is 'old fogies lost in cyberspace', they would would have to see how many of the contacts we get are from students.

I suspect it was those same people that used to always call 'AAAAAAAAAA Widgets' in the Yellow Pages first - and why companies would actually name themselves something like that.

As much as we have tried to make it clear on our site that we SELL Acme Widgets, but we are NOT the Acme Widget company, we still get calls and emails for items we have never carried.

I don't know what can be done about it, as it seems that a certain percentage of people will always totally ignore what is on the page and just look for the nearest email link or 800 number on the page.

[edited by: Wlauzon at 2:17 am (utc) on Aug. 24, 2007]

amznVibe

2:43 am on Aug 24, 2007 (gmt 0)

You mean students think and react in a way that rewards a popularity contest? Who'da'thunk-it! So happy we have these studies in common sense.

econman

3:17 am on Aug 24, 2007 (gmt 0)

I think it would be a mistake to assume this study only reflects the behavior of students. I wouldn't be surprised if the same pattern holds true for many demographic groups -- aside from the most experienced users, WebmasterWorld members, etc.

Of course, if the first few results are NOT what they are looking for, they either reformulate their query, or perhaps they start scrutinizing the remaining snippets more carefully, as they work their way down the SERPs.

Another interesting phenomena -- the less targeted the page they click on, the more likely they will click on an ad when they don't find what they want on that site (that's one reason why MFA's are so profitable).

This is one of the truly odd things about the current web environment. While competitive pressures encourage the SE's and ad-supported information-providing sites to work at improving the quality of their results/content, in the short run the reward for that hard work is a decline in CTRs and revenues as relevancy and/or quality improves.

Marshall

4:35 am on Aug 24, 2007 (gmt 0)

This only supports my thoughts that people ONLY look at the link ranking and seldom, if ever, read the site description or look at the URI to see if it makes sense to the results.

In the the case of "Widget Whatever" lets say the top four results are: more_widgets.com, my_widgets.com, widget_whatever.com, and widgets_are_us.com. While three is the obvious match, who looks.

People are lazy. I moved a link on my site up from being about tenth in the hierarchy to fourth and the page views more than tripled. Did the page change? - No, but it's closer to the top.

Marshall

venice

5:31 am on Aug 24, 2007 (gmt 0)

they assume we are the manufacturer

Funny you mention this. We get calls every week for people wanting information regarding major landmarks in our area, just because there is info on our site. Comforting to know it's happening to others.

plumsauce

6:15 am on Aug 24, 2007 (gmt 0)

A College of Charleston eye tracking experiment revealed college students participating in the study trusted Google's ability to rank results by their true relevance to the query. When participants selected a link from Google's result pages, their decisions were strongly biased towards links higher in position, even if content was less relevant to the search query

says a lot about current educational standards.

vincevincevince

7:00 am on Aug 24, 2007 (gmt 0)

Positive reinforcement is the concept that a successful reaction to an action makes taking the same action in the future more likely.

In the past, those searchers have found what they were looking for from the higher ranked results. It makes sense that they will be inclined to try the same again.

If you had a great Korma at Al's Curry house on Thursday, you're more likely to go to Al's Curry house for your Madras rather than try the neighbouring curry place despite you having no knowledge about the Madras at Al's Curry house. It might even look less appetising than the Madras from the other place, but based upon positive reinforcement of the concept that food at Al's Curry house tastes good, you'll still buy it.

gibbergibber

8:53 am on Aug 24, 2007 (gmt 0)

This is an awful and misguided study, or at the very least this report makes it sound awful and misguided. Of course people are more likely to click on higher results... what else can they do?

Does anyone seriously expect people to visit sites in advance, decide their relevance by reading them, then go to Google to do their search and only click on the results they remember as being relevant?

The whole point of search engines is that you just type in a search term and click on the results in the order they are ranked. Yes, automated algo ranking is crude and often wrong, but what serious alternative is there?

Human-edited directories don't work because they are only about general topics, not necessarily specific phrases or bits of information, and it's impossible to keep up to date and accurate with all the billions of sites out there. At best a directory will tell you about a tiny percentage of quality sites, which is not even close to being a comprehensive look at all the info out there on the web.

Wlauzon

10:23 am on Aug 24, 2007 (gmt 0)

Of course people are more likely to click on higher results... what else can they do?

Well, reading is one option.

Thinking is another.

We have a page that describes animated widgets that just happens to use as a trademark a furry animal. It comes in at the top for animated widget searches. The 2nd entry after us is a page actually about the furry animals. It just happens that we also come in at #1 on the furry animal searches.

Yet looking at the logs for searches it is quite obvious that a high percentage of the hits we get are for people looking for the furry animal, but click on us instead because we are at the top. Even though the Google search line says "furry animal brand animated widgets" quite clearly in the description of the website. And the 2nd entry says quite clearly that is about the anatomy and habitat of the furry animal.

That is exactly what the study is describing. Yet you think there is nothing else they can do?

Yes, that might be a bit exaggerated, but not much. I see it happen quite often - often enough in the keyword search logs to know that people are just clicking blindly.

[edited by: Wlauzon at 10:26 am (utc) on Aug. 24, 2007]

Old_Honky

12:46 pm on Aug 24, 2007 (gmt 0)

Reading this thread has reinforced my conviction that it is impossible to underestimate the intelligence of the general public. As an old colleague of mine used to say "you can't educate pork".

It's not just google that has this problem, we are the UK distributor of a range of Italian Made steam cleaners. Most of our enquiry emails come from the USA and throughout the world which is annoying because we can't sell them anything and we often end up helping them by sending pdf's of the manual etc. Which takes up our time and has a cost.

We state in bold letters on our index page that we are the UK and Ireland Distributor for Italian Widget Co. We have a page with a map and links to all the other distributors worldwide which is one click away from every page, but because our plain and simple web site is more user friendly than the Italian all bells and whistles flash based site and we are number one on a "Company name" search and these "intellectually challenged" people pester us daily.

In the real world Yellow pages and Thompson Directories create similar problems, they have no category for "Distributors of steam cleaners" so they put us in with "electrical wholesalers" or "electrical repair shops". We get a fair amount of the great British public phoning to inquire about products that have nothing to do with us.

celgins

12:54 pm on Aug 24, 2007 (gmt 0)

People are lazy.

Exactly... and many website owners take advantage of that.

In fact, I would argue that most website owners have no desire to become authoritative or trustworthy sites, but would rather rank in positions 1,2,3,4 or 5 so that most users ... (who never scroll or navigate beyond page 1) ... will see their site.

So even though many users "trust" the sites in those top spots, the site owners wish to position themselves there because more attention means more money.

Another side: If you think about Webmasterworld, many users browse only the top few threads....(which has nothing to do with those threads being authoritative)... they just happen to be above the fold and lazy users hardly ever scroll.

MatthewHSE

1:15 pm on Aug 24, 2007 (gmt 0)

And if anyone thinks it is 'old fogies lost in cyberspace', they would would have to see how many of the contacts we get are from students.

My Dad is in his lower 50's. I'm in my mid 20's. Dad reads every word on a webpage. I either skim until I find what I'm looking for, or hit CTRL + F to find specific words. I can easily believe that the younger crowd are the main ones to make these kinds of mistakes (although I am always careful to make sure of my information before calling anyone - I've wound up on unrelated websites sometimes, but I've never bothered anyone by contacting them by mistake.)

Jbrookins

3:30 pm on Aug 24, 2007 (gmt 0)

Other breaking news items: The sky is blue, water is wet, and the earth is round.

Two problems that I see continuing on ad infinitum:

1. Roughly 50% of the population is below average intelligence. Cliche, sure, but that means there are an awful lot of stupid people out there considering that average intelligence isn't particularly impressive. (And just think! Really intelligent folks tend to breed far less these days...)

2. Currently, we (western society, at least) tends to be on information overload, unnecessarily hectic and goal oriented. Low attention spans and a general tendency to rush through things makes for a bad combination. Couple that with the fact that a search engine is SUPPOSED to feed you the most relevant results first and even intelligent people aren't paying nearly as much attention to detail as one would hope.

adfree

3:40 pm on Aug 24, 2007 (gmt 0)

"Users mistakenly..."

Wondering how often one wants to repeat a mistake... hmmm

pageoneresults

3:45 pm on Aug 24, 2007 (gmt 0)

It is a known fact that Internet surfers "scan" as opposed to in-depth reading when they are performing searches.

Also, the study didn't say how those searches were being performed. We're they exact phrase searches? Long tail searches? What?

If they were exact phrase and/or long tail stuff, the chances are that they will find what they are looking for ATF (Above The Fold) are very good. And, you can bet that Google is tracking all of that data.

I think the same way that the study suggests. But, I'm performing advanced search queries so I pretty much know that my ATF results are most likely going to be on target.

It all comes down to the search and how it was performed. If those users were trusting results from single word and generic word searches, there may be some issues. But, as soon as they click through, I think most users are "intelligent" enough to know that the destination is what they are, or are not looking for. Google can tell too. ;)