Testing the Mighty Mobilegeddon

SEO-pocalypse. Mobilegeddon. Last week’s algorithm update from Google, announced back in February, has been called many things. Did it live up to the hype?

This week-by-week comparison helps us begin to answer that question.

Quick Mobile SEO Refresher

There are two components to Google’s update. First, mobile-friendly websites, or at least sites Google considers to be mobile-friendly, will rank better in mobile search results across all languages. Google stated that the impact will be “significant” and has recommended using the Google Developers Mobile-Friendly Test to see if a website is mobile-friendly. You can also use the mobile section of the PageSpeed Insights report. However, these will only analyze one page at a time. To test your entire site, use the Google Webmaster Tools Mobile Usability Test.

Note: There are discrepancies in how these tests run, and you may receive contradictory results between tests or even on different runs of the same test. To learn more about why these tests are confusing, read this post.

Second, this algorithm update affects mobile app indexing. Android apps indexed via App Indexing will rank higher in mobile search results, assuming that the user has the app installed on their device and is signed in to the app. If you have an app, it will be worth indexing it to gain that ranking boost.

But most people are concerned with the mobile-friendly website part of the update, and that’s where our experiment comes in.

The Test

Our fantastic search marketing intern, Nick Eppinger, poured over client sites to benchmark performance before and after the algo update. He analyzed about 2 million sessions on 17 websites of various size and volume, checking testing the following one week before and one week after April 21, 2015.

Website – No/Responsive/Other

Mobile Badge – Yes/No

Google Mobile Test – Passed/Failed

Total Sessions – Before/After

Mobile Organic Sessions – Before/After

% Change – Before/After

Websites were divided into three buckets: static non-mobile friendly (which we called “no mobile”), responsive, and other, which includes mobile sudirectories, mobile subdomains (m.) and dynamic serving sites.

The mobile badge refers to Google’s tag in the search results that provides a preview for people using mobile devices. We used Google’s mobile testing tool for each site three times, noting the score each time. Some sites did not pass the first time… or second.

The Results

Here are several graphs and charts to outline the Mobilegeddon results on our 17 test subjects. Obviously, this is not a conclusive report based on the small sample size. We are simply trying to contribute to the conversation by sharing some of our data with the digital marketing community.

The chart below groups all mobile-friendly websites into one category and compares it to mobile-unfriendly websites. We can see that the mobile-friendly group had more total and mobile organic sessions, averaging a 4 percent increase for both.

It is worth pointing out that all mobile-unfriendly (labeled as No Mobile) websites had decreases in organic mobile sessions except for one, which remained constant. However, the traffic drops were not armageddon-like for most. The two affected most had drops in mobile organic traffic of 12 percent and 13 percent, both with total traffic drops of only 3 percent.

Nearly all before-and-afters changed by less than 13 percent, and most were less than 9 percent. There was only one outlier, a mobile-friendly website, with a change of +21 percent.

The following chart breaks the mobile-friendly group into two: responsive and other, which includes m-dots and dynamic serving mobile websites. Based off of this graph, it appears that the biggest winners with this change were non-responsive mobile-friendly websites.

Again, no one is saying this is THE DEFINITIVE MOBILEGEDDON STUDY. We’re just saying its a case study that might be interesting and helpful to fellow digital marketers. What are you saying? Let us know in the comments.

Receive Blog UpdatesGet notified when blog posts like this one are published!

Andrew Garberson is the Director of the Digital Marketing Strategy department. He has led digital marketing efforts in a variety of settings, including agency, entrepreneurial and nonprofit environments, and has master's degrees in business administration and mass communications. An Iowan at heart and Pittsburgher in spirit, Andrew commutes on his 10-speed most days between March and December -- after all, he's only human.