Related Tags

Infinite scroll: its impact on SEO and how to fix it

Search engine spiders cannot scroll to the bottom of a page like an ordinary user, trigger the request for more content, and then wait to retrieve it for indexation.

Hence content only accessible via infinite scroll simply won’t be seen and therefore won’t make it into the respective search engine results listings.

The good news is there are easy solutions so that infinite scroll poses no problem for search, and in fact can also provide an opportunity to maximise user experience and maximise SEO value in tandem.

Infinite Scroll and SEO

What is it?

Most of us engage with infinite scrolling on a daily basis, most frequently with Facebook, Twitter and Pinterest.

It occurs when you scroll down to the bottom of a page of tweets, posts or other content and the page automatically updates to append more to the bottom, and does so again and again as you hit the bottom of the page each time (the scroll isn’t infinite however as the name misleadingly suggests).

A variation of infinite scroll is infinite ‘page’. Whilst the underlying architecture is broadly the same, the intention of each is quite different.

With an infinite scroll the expectation is that more and more content is added to the stream at regular intervals, whereas with an infinite page the expectation is that the narrative has a clear beginning and end and it won’t really be added to if at all (so even less infinite than the very finite infinite scroll).

A good example of an infinite page is the Renault Ze website which uses infinite scroll to tell a story and is not designed to regularly accommodate new content.

What are the SEO impacts?

Search engine spiders cannot scroll to the bottom of a page like an ordinary user, trigger the request for more content, and then wait to retrieve it for indexation.

Therefore, a web-page utilising infinite scroll will by default serve the search engine spider the first page of results and nothing more.

This means that lots of great content may never be seen by a search engine if there is no other mechanism in place to serve as a path to that content.

Content only accessible via the infinite scroll will simply not be seen and therefore will not make it into the respective search engine results listings.

You can check whether you have an infinite scroll or infinite page issue by emulating a search engine’s experience with your site.

You can do this by checking what Google has captured of your infinite scrolling page – just type in ‘cache:www.yourdomain.com’ into Google and click on ‘Text Only’ and see what is missing and what the crawl path to your content looks like.

Alternatively, you can (and should) also use a Web Developer extension on your browser that allows you to turn JavaScript off with just a few clicks and experience the website in that impaired condition.

What are the possible solutions?

Various sources suggest that infinite scroll should be avoided as it is bad for SEO. That is incorrect and unfair – anything poorly designed and executed is likely to be bad for SEO and infinite scroll isn’t inherently bad.

There are easy solutions so that infinite scroll poses no problem for SEO and in fact it can also provide an opportunity to maximise user experience and maximise SEO value at the same time.

The most popular and most effective solution is to implement some graceful degradation, otherwise known as progressive enhancement. This is where web techniques are used in a layered fashion so that every user receives a comparably functional and pleasant experience regardless of the capabilities of their browser or device.

So for basic browsers, a site with progressive enhancement will gracefully degrade the user experience without impacting accessibility negatively, and an advanced browser will find their user experience elevated if it meets various capability criteria.

A web page that offers infinite scrolling therefore should degrade when utilised by a browser or device with limited functionality, particularly one that does not, or has an inconsistent approach to executing JavaScript.

Search engine spiders would qualify in that browser/device category.

Therefore, when the page is viewed by a search engine spider the site should automatically provide a suitable replacement for the infinite scroll.

This is ultimately achieved by either placing the pagination within a noscript container in your page code or layering JavaScript, CSS, and mark-up independently of each other in your code so that the pagination is simply obfuscated progressively as a users’ device and browser allows for it (i.e. the pagination still resides on the page, even if you can’t see it when you’re infinitely scrolling in a contemporary browser).

This second approach would be considered the most desirable, mainly due to concerns over the relative value search engines place on content within a noscript container given it isn’t designed to be universally seen by all, and a general suspicion of the container due to its aggressive utilisation by spammers historically.

Google’s rel next/prev tags can also be used with this pagination to control how many of your first few pages you’d like Google to treat as being at the same level hierarchically.

This is useful when you’re not adding new content all the time and/or when the topic of your items (posts for instance) are highly searched for within search engines and will continue to be so for a while, so Google viewing them as residing on your homepage (or higher up than they actually reside) for longer, may be very desirable.

Some implementations take this progressive enhancement a step further and replace the infinite scroll functionality with an indexable archive, typically broken down by months of the current year, and then previous years, leading into a further breakdown into months on subsequent pages.

Mashable’s infinite scroll is a good example. It degrades for search engines to the most recent content, and then ends with this:

This achieves two complementary goals. Firstly, it means that a search engine can get to content that is below the infinite scroll trigger point. Secondly, this also acts as a great archive ensuring that there is a crawl path to every single piece of content ever produced by Mashable.

Logic can also be built into this archival mechanism to ensure that the crawl path (the number of clicks away from the homepage particularly) is reduced for the most popular pages (based on historical visits or shares, for instance).

Purists may be concerned with Mashable’s implementation as the site is ultimately showing the search engine different content than you are showing to most users.

This does not however qualify as cloaking as the intent is clearly not to game the search engines but rather to facilitate their traversal through the site because they cannot behave like a human being and keep scrolling until they reach the very bottom.

There is a line to be drawn here though, as the solution would also allow you to insert a footer into your pages for those people (and search engines) that cannot physically execute the infinite scroll.

Intent becomes very important at this stage; a footer that includes links to an ‘about us’ page, a sitemap and a copyright notice is clearly not designed to game any one network.

Conversely, if the obfuscated footer region includes lots of commercial keywords and looks as though it is designed just for search engine ranking purposes, you do run the risk of raising the suspicions of Google et al and incurring a penalty as a result.

Best practice with footers, within the infinite scroll and degradation context, would be to either create a sticky footer (see an example here), which you need to ensure is accessible to search engines itself.

Another option is to relocate the footer to a sidebar visible to everyone - see any Twitter page (Coca Cola’s here) for a great example of this, or any personal Facebook newsfeed (which are uncrawlable of course by search engines).

This second approach would be considered optimal as it is difficult to implement a sticky footer that users actually like and appreciate, and that ultimately adds value to both you and them.

Facebook incidentally is also a good example of how not to do this as Facebook brand pages have a footer that appears at the bottom of the page momentarily when you visit the page, and then disappears forever when the infinite scroll script kicks in.

Ultimately, infinite scroll has proved a valuable UI innovation for many brands and for many it has become a defining element of their publishing mechanism.

The hard part is determining whether your users want infinite scroll, whether it improves their engagement levels, and whether it is too much effort for too little return.

Making infinite scroll work from an SEO perspective is in fact the easy bit, assuming you select the optimal solution and implement it properly (see solutions grid below).

Great blog Andreas. We're currently in the process of redesigning our website and thinking of implementing infinite scrolling on our blog page so your tips will now definitely come into use for our web developer!

It is annoying that Google cannot build spiders to understand such a widespread phenomenon, or alternatively, they can understand it but they cannot be bothered to tell us rather like they failed to understand JS for donkeys years, except that they could.

As always, you are better off ignoring the inadequacies of Google and making the page wonderful. As another commenter pointed out, you probably dont want to use techniques like this for ecommerce pages.

over 4 years ago

Lenka Istvanova, Consultant at Seven League

Thanks a lot Andreas. Great insights and tips. Majority of Tumblr users use infinity scrolling. So I had a go and after reading your post I had a look at some of the brands/users. Well, some of them could be better of turning it off completely or apply your tips.

From the user/visitor point of view; I'm fan of infinite scrolling however only for telling the story rather than normal product page where products are added on a regular basis.

Interesting blog Andreas. I had been wondering about this. Surely it is only a mater of time until Google and other search engines will understand infinite scroll pages. Have you any thoughts on how difficult this would be and how long it may take Google to develop? My vote has to be with making content that works well for humans.

over 4 years ago

Sean Owens, MD at Willows Consulting

Very good at highlighting the issue of scroll on SEO.

It is bad that what is essentially a user experience enhancement ends off being a SEO bomb unless you are very careful.

over 4 years ago

James Barber, Head of Ecommerce at BoroughBox Ltd

Interesting post - we have implemented infinite scroll using a Magento extension which claims to not have any negative SEO effects (I'm new to the company so I'm currently trying to make sure that is the case).

We do currently have a potential usability issue though - the URLs are appended to display which page you're viewing, which breaks the back button so potentially confuses users.

On the whole I think that the benefits outweigh the negatives but will be doing some testing to confirm that.

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.