Subscribe to SearchCap

Time To End The Bull Search Engine Share Figures?

Yahoo’s up again in comScore’s latest search rankings. Reason? It seems largely due to non-search “slideshow” activity. Solution? Time to end the bullshit way search share figures are calculated. Chance of success? Probably nil, but it’s worth hoping for.

Today’s figures, for share of searches in the US handled by the major search engines, show Yahoo has continued to increase its share of searches for the third month in a row. The trend:

February 2010: 16.8

March 2010: 16.9% (up 0.1%)

April 2010: 17.7% (up 0.8%)

May 2010: 18.3% (up 0.6%)

How Yahoo Games Search Share

Prior to March 2010, Yahoo had seen a year of decline. What happened? One big change was that Yahoo started ramping up “slideshows” at Yahoo News to manufacture more searches. For example, if you’re on the Yahoo News home page, you’ll see a “Most Viewed Photos” section like this:

If you click on a photo, you get tossed into “slideshow” mode:

As you browse through the pictures, each click generates a new “search,” as you can see below:

In the example above, the person viewing the photo probably doesn’t realize they issued a query for “Police stand outside police headquarters.” They did.

Below the photo, they got news search results that don’t match the query they issued. But that’s OK. They probably didn’t notice the results were even there. In addition, they also got ads, none of which are targeted to the search query. Advertisers, take note.

Each click to get a new picture changes the URL to a new search query, thus making it seem when the person did a new search — even though they really didn’t.

Sure, way down at the bottom, the search results changed. Chances are, the user didn’t even look at these. Almost certainly they didn’t perceive what they did as a search. But it got counted that way, and Yahoo’s share has risen because of this.

Yahoo also says that it has ramped up the use of “trend” links like these on its home page:

Click on a trend link, and you get search results for that trend. Sprinkle links like these around Yahoo, and people apparently click a lot, Yahoo says, meaning more searches for them. Unlike with slideshows, at least these are more arguably search-like activities.

The Gaming Has Grown Searches

Let’s be clear. The two things above are the biggest drivers of Yahoo’s success. When I spoke with Shashi Seth, senior vice president of Yahoo search products yesterday, he made that clear.

Seth did argue that Yahoo has also been spending on radio ads to promote search since February. He sees those as “additive” to Yahoo’s recent successes.

I completely disagree. When search engines do ad campaigns, you usually see an immediate spike (if the ads work). More than a month after the ads started, the March 2010 figures showed a 0.1% increase in search share. Tiny. But in late March, Yahoo ramped up its use of slideshows. The following month, in April, it registered a 0.8% share. That’s a spike. That’s a spike related to slide shows and other activities to generate more searches on Yahoo itself, not by bringing in new people through radio ads.

Yahoo: We’re Just Doing What Microsoft Does

Counting slideshow activity as “searches” is bullshit in my books. Yahoo, oddly, kind of agrees. But Seth argues they weren’t the first to start the gaming. Blame Microsoft, Yahoo says. Yahoo’s just copying what they’ve been doing.

For example, here’s what you get at Bing Travel, an enticement to view slideshows:

Click on one of those, and yes, the same bullshit at Yahoo happens at Bing:

The picture you’re viewing is delivered complete with search results, even though you didn’t intend to conduct a search. Click through the slideshow, and you generate more searches for Bing.

Time For Game Reform?

What’s the solution to this gaming? You probably didn’t notice that both Yahoo and comScore put out press releases yesterday touching on the issues. This was damage control, anticipating that there would be a lot of complaining about Yahoo gaming the numbers when May 2010 figures came out today, as there was. The same thing happened when the April 2010 figures showed an increase.

The industry as a whole is changing rapidly, and as search continues to evolve, so will the measurements used to understand the market. People no longer search to find a list of blue links; they search to find answers in the shortest amount of time possible. We believe that surfacing the right information at the right time is more important than the number of total results delivered or number of traditional queries conducted.

As we push forward with new search features to bring relevant search results to more people in more places, we look forward to talking with folks like comScore and the industry at large about how to measure the new paradigm of search. Our goal is to help evolve measurement standards, definitions and metrics so the industry has what it needs to accurately understand trends in search share across different types of searches and different companies.

In July 2007, comScore revamped search measurement with the introduction of qSearch 2.0, which expanded the view of search beyond the traditional search sections to include searches at retailers, travel sites, directories, etc… anywhere that a user might submit a search. qSearch 2.0 captured the broad adoption of search technology throughout the web to improve the user experience.

Since that time, we’ve seen a wave of change across the web that has changed the very nature of a web page, from an object that is requested and delivered, to one that is a live platform that can integrate content from many sources. And search has changed along with it….

That said, the continued evolution of search and emerging innovations in how it is used to enhance user experience, calls for a thoughtful review of how we classify various types of searches, count them and report them. We want to ensure that we provide comprehensive and flexible measurement that meets the needs of the various constituencies in the digital marketplace. As our thinking evolves, we will include relevant stakeholders in the discussion and clearly communicate our thinking and rationale to the marketplace. While we will maintain the current method through the end of the second quarter to avoid reporting disruptions, we will aim to implement proposed revisions in the third quarter, ideally starting with the release of July data in the first half of August. Stay tuned to the comScore blog to find out more in the coming weeks.

They’re right. We do need better ways to measure search activity. But they’re wrong that this is somehow some new issue that’s come up.

Bad Metrics Aren’t New

I’ve been following search figures for a very, very long time — since 1996. Over the years, I’ve seen a shift from measuring search by the number of people who visited a search engine at least once per month (a laughable metric, but that’s how it was done) to the number of actual searches that were conducted. I’ve seen search engines not get counted because they weren’t considered search engines then spring onto the radar screen after a “reclassification.”

In addition, while search is evolving, we’ve long had search engines providing direct answers or “contextual” responses in the ways that the posts above would have you think are suddenly new. They aren’t. Search engines have used a variety of methods to try and “recirculate” people into doing more searches with things like related searches or answer box displays. What’s new is that they’re proving more effective gaming mechanisms than in the past.

Here we are more than three years after Microsoft actively used search gaming to generate new traffic, and we’ve not seen comScore nor any of the other major ratings services call for revisions on counting (or not counting) that activity. Why not?

The answer’s easy. Because it didn’t help in the long-term. Microsoft kept losing share, so it wasn’t worth trying to figure out whether games were helping them or not. They were clearly losing regardless.

Now Yahoo’s having success — and suggesting that Bing’s success is less to do all its marketing and more to do with slideshows. Cue the cry for reform again. But will it happen?

It should. Back in 2008, I started talking with some of the metrics companies about other ways that search could be measured. One thought is the number of search “sessions.” Not number of searches but a longer period of time when someone is exhibiting search activity. Go to Google four times per day for different things? Each visit probably generates several searches. Rather than count the number of searches, count the sessions.

That’s one suggestion. It’s not perfect, and we likely need a basket of metrics to fully appreciate what’s happening.

Will It Happen?

I hope we do see reform. I’ll push for it further myself, in follow up conversations with the various search engines and metrics companies over the company months. I’d like to see the big three come together on the issue. I’m just not that hopeful. Google, in particular, has little incentive to get involved.

Let’s be clear. With all the attention on Google from an anti-trust perspective, the company has a strong incentive to make it seems like it’s losing a bit in the search space. That’s especially so when from a search volume perspective, it’s doing just fine.

Consider today’s figures, which report search share in the United States for May 2010

Google: 63.7% (down 0.7% from April 2010)

Yahoo: 18.3% (up 0.6%)

Microsoft: 12.1% (up 0.4%)

Google’s in trouble! But then again, look at the number of searches generated:

Google: 10.158 billion (up 152 million)

Yahoo: 2.908 billion (up 156 million)

Bing: (up 101 million)

Google’s actually even with Yahoo on growth of actual searches and ahead of Bing, despite having a “share” that drops. It sounds weaker despite being strong.

Moreover, if the gaming keeps working for Yahoo and Bing (and gaming seldom has, in the long term), Google could crank up its own and crush the competition. It has so much search traffic that even the smallest changes might skyrocket the searches it handles. Perhaps it already has.

When we exclude the incremental contextual/slideshow searches from YHOO and MSFT, we get a very different picture of query results.

Google’s share actually increases from 66.1% in April to 66.4% in May, up 30 bps M/M and up 140bps Y/Y. We note that this is a record share for Google.

Yahoo!’s share falls from 16.9% in April to 16.6% in May, down 30 bps M/M and down over 340 bps Y/Y.

Microsoft’s share remains constant at 10.8% in April and May, up 280 bps Y/Y, and is the lowest share since Nov-09.

It’s unclear to me where Citi is getting these figures; I suspect they’ve asked comScore for a special run. Over at eWeek, there are similar figures even though these aren’t listed in the comScore press release. I’ll be checking on this further.

SMX Advanced is the only conference designed exclusively for experienced paid search advertisers and SEOs. You'll participate in experts-only sessions and network with fellow internet marketing thought leaders. Check out the tactic-packed agenda!

Sponsored

The latest analyses, insights and strategies that inspire CMOs and marketers everywhere.

http://www.search-usability.com/ Shari Thurow

Does anyone else LOVE it when Danny writes incredible headlines (headings) with great follow-up articles?

Can’t help it, Danny. I love when you call people out (myself included).

–Shari

http://www.researchandcompare.com amurphy59

The issue with these types of link based searches doesn’t just affect the numbers, it affects advertisers too.

The search of “police standing outside …” matched a mortgage ad. At my last company, we sold back products, most of which we sold 5 to 20 a month. We would get something like 50 to 500 clicks per month for keywords associated with such products.

Then one day, we got 1,000s of clicks for an obscure product … we were scratching our head, where did that come from? Well, we found out through a lot of digging and some very accommodating Google reps that it was from AOL. We then figured out that it was from AOL putting links on their home page for these obscure searches that we were bidding on to identify a potential buyer.

So, here’s the rub, we were bidding multiple dollars per click for those terms and were willing to do so because the product was a massage chair, that sells for 1000s of dollars. And only people that were really interested in buying a massage chair would put in the “exact phrase” of “Buy Massage Chair.”

The problem with AOL went away but would pop up from time to time and I have seen it far more often with Yahoo and MSN. I have simply assumed over time that this is why quality on Google, as measured by conversion rate, is so much higher than that of Yahoo or MSN.

So, when you say Advertisers take note … it is important that we all look at the details of what is happening to our accounts.

Great article. Thanks.

http://twitter.com/MichelleObama7 Clifford Bryan

I agree Shari. That headline was the bomb.

http://ifdebug.com Alistair Lattimore

While I completely agree that gaming search engine figures is grubby, at some level you’ve got to ask yourself why does anyone care really?

The share percentages suggest 15-20% for Yahoo! but in practical terms do they actually generate that percentage of revenue for many businesses? As an example of what I’m talking about, a hotel chain in Australia gets roughly 2% of their organic traffic from Yahoo! & a comparable percentage of organic revenue.

With figures like that, why does anyone really care if their market share is supposedly 15%, 30% or 80% – at the moment regardless of what their market share is reportedly – they don’t live up to those percentages in the real world – at least not for the business that I’m in contact with.

http://www.michael-martinez.com/ Michael Martinez

Search sessions would be just as bogus as “queries performed”. The metrics companies are only measuring pageviews at this time — which is just as laughable a metric as number of unique visitors per month.

The market is not composed who stays on the search engine sites — it is composed of who the search engines send to OTHER sites.

Counting Google, Bing, Yahoo!, Ask, and other service’s internal site searches in the market has just turned the whole metric into nonsense.

http://Camares.com/blog DebDiGregorio

Comscore, Shmomscore…Measuring searches is bogus because Google cannot deliver a satisfying user experience to save its life. People search a lot on Google CAUSE THEY CAN’T GET THE ANSWER THEY NEED. Google has trained users to go to Wikkipedia and other vortals that deliver.

http://www.kkti.com JVRudnick

wow…great post Danny! Love the fact that it appears that some se’s are trying to “game” the numbers….wouldn’t a thought that possible, till now!

makes ya wonder, eh?

:-(

Jim

CMO Resources from

Video

Analysis

Search, Meet Display. Display, Meet Search.

In the last couple of years, we’ve seen the display world become increasingly programmatic and biddable, whilst search marketers have been looking for new ways to optimize campaigns.