Understanding click-through rate (CTR) in the context of search satisfaction

Click-through rate (CTR) has historically been an important factor in gauging the quality of results in information retrieval tasks.

In SEO, there has long been a notion that Google uses a metric called Time-To-Long-Click (TTLC), first noted in 2013 by AJ Kohn in this wonderful article.

Since then, Google has released several research papers that elaborate on the complexity of measuring search quality due to their evolving nature.

Most notably:

Direct Answers

Positional bias

Expanding ad results

SERP features

SERP layout variations

All of these factors can have varying effects on how users interact and click (or don’t click) on Google results for a query. Google no doubt has various click models that set out expectations for how users should click based on search type and position.

This can be helpful in understanding outlier results either above or below the curve to help Google do a better job with satisfaction for all searches.

Search satisfaction

The reason this is important is that it can help us reframe our understanding of search result clicks away from CTR and TTLC and towards an understanding of search satisfaction.

This paper, along with accompanying code, attempts to use clicks, user attention, and satisfaction to distinguish how well the results performed for the user and to predict user action (which is a required feature in any click model).

The paper goes on to elaborate that the type of searches this model is useful for is long-tail informational searches, because “while a small number of head queries represent a big part of a search engine’s traffic, all modern search engines can answer these queries quite well.” (Citation)

Generally, the model looks at:

Attention: A model that looks at rank, serp item type, and the element’s location on the page in conjunction with click, mouse movement and satisfaction labels.

Clicks: A click probability model which takes into account SERP position and the knowledge that a result must have been seen to have been clicked.

Satisfaction: A model that uses search quality ratings along with user interaction with the various search elements to define the overall utility to the user of the page.

Are clicks really needed?

The most interesting aspect of this research is the concept that a search result does not actually need to receive a click to be useful.

Users may receive their answer from the search results and not require clicking through to a result, although the paper mentioned that, “while looking at the reasons specified by the raters we found out that 42% of the raters who said that they would click through on a SERP, indicated that their goal was ‘to confirm information already present in the summary.’” (Citation)

Another interesting (and obvious) takeaway across multiple research papers, is the importance of quality raters’ data in the training of models to predict search satisfaction.

None of this should be taken to assume that there is a direct impact on how clicks, attention, or other user-generated metrics affect search results. There have been a number of SEO tests with mixed results that tried to prove click impact on ranking.

At most there seems to be a temporary lift, if any at all. What this would suggest is that, being an evaluation metric, this type of model could be used in the training of internal systems which predict the ideal position of search results.

Click models

Aleksandr Chuklin, a Software Engineer at Google Research Europe and expert in Information Retrieval, published a paper and accompanying website in 2015 that evaluates various click models for web search.

The paper is interesting because it looks at the various models and underlines their various strengths and weaknesses. A few things of interest:

Models can:

Look at all results as equal.

Look at only results that would have been reviewed (top to bottom).

Look at multi-click single session instances.

Look at “perseverance” after a click (TTLC).

Look at the distance between current click and the last clicked document to predict user SERP browsing.

In addition, this gives some intuition into the fact that click models can be very helpful to Google beyond search satisfaction, by helping them understand the type of search.

Navigational queries are the most common queries in Google and click models can be used to determine navigational as opposed to informational and transactional queries. The click-through rate for these queries is more predictable than the latter two.

Wrapping up

Understanding click models and how Google uses them to evaluate the quality of search results can help us, as SEOs, understand variations in CTR when reviewing Google Search Console and Search Analytics data.

We often see that brand terms have a CTR of sixty to seventy percent (navigational), and that some results (that we may be ranking well for) have lower than expected clicks. Paul Shapiro looked into this in 2017 in a post that provided a metric (Modified z-score) for outliers in CTR as reported in Google Search Console.

Along with tools like this, it is important to understand more globally that Google has come a long way since ten blue links, and that many things have an impact on clicks, rather than just a compelling title tag.

Having established the importance of search satisfaction to Google, is there anything that SEOs can do to optimize for it?

Be aware that investigating whether CTR directly affects search is probably a rabbit hole: even if it did, the impact would more than likely be on longer tail non-transactional searches.

Google wants to give their users a great experience. Your listing is just a part of that – so make sure you add to the experience.

Want to stay on top of the latest search trends?

Related reading

Rich results are the search results which have a little extra panache, in which Google displays more information about the result rather than just the traditional title, URL and meta description. In this guide, we'll look at what's new with rich results in 2018, as well as how to give yourself the best chance of getting them.

Slow-loading web pages can slowly become the leading cause of high bounce rates, low user engagement, lost traffic opportunities, and abandoned sales journeys. If you have a WordPress site, there are a number of easy and effective methods you can use that will significantly increase your site's loading speed.

Where is the biggest opportunity in marketing at the moment? With 3.5 billion searches per day being carried out by Google, not to mention on vertical-specific websites like Amazon, YouTube and Pinterest, there is a huge sea of data available on customer intent which marketers should be taking advantage of.

Over the past decade, SEO has been growing into its position as a critical marketing channel for businesses. If you're new to this environment, or have new team members that need to be trained up, we've outlined some of the best (and importantly, free) SEO training courses and websites to take your knowledge to the next level.

Here we’ll take a look at the basic things you need to know in regards to search engine optimisation, a discipline that everyone in your organisation should at least be aware of, if not have a decent technical understanding.