blog posts

You’re Doing it Wrong: Website Speed Test Strategy Edition

Website speed test tools have become a go-to part of search engine optimization. More than likely you already know the importance of page speed & performance as both a ranking and conversion factor.

Statistically, an often quoted study from Google shows that – on mobile – if your site takes more than 3 seconds to load, the probability of a user bouncing is 32%. If your site takes more than 5 seconds to load, the probability of a user bouncing is 90%. If your site takes more than 6 seconds to load, you may as well shut it down as the probability of a user bouncing jumps to 106%.

Keep in mind this study is from 2017, with the increase in internet speeds and overall decrease in attention span, I would expect this data to be even more dramatic in 2019 and beyond.

Obviously, this means that from time to time you need to be able to evaluate a website speed test and determine the risk/reward of improving on that performance. If you’ve done it before, you probably have a preferred tool in mind, if you haven’t, you’d probably search for how to do it.

There are a number of different website speed test tools on the market today, many of which are completely free to use. Some popular ones are Pingdom, GTmetrix, WebPageTest and PageSpeed Insights but there are many others out there that offer similar functionality.

Things to know about website speed test tools.

The goal of all these website speed test tools is similar. Either by using a real browser, or mimicking a browser, they make a real request to the site you provide as if they were a real site visitor. Then, they run tests to categorize various performance aspects of the site. At the end, they provide these performance metrics and usually a “letter score” like “A” or “B” to indicate if the performance is good or bad. However, some things to keep in mind about these tools are:

There is no standard methodology. Each testing tool uses its own logic to determine your page speed. Some are “mobile first” (measuring using a mobile device), some are “desktop first” (measuring using a regular computer). When using mobile, some will by default test using a 3G connection, some will test using a 4G connection. When using desktop, some tools throttle the performance based on a certain broadband connection and some just use their full available bandwidth. The takeaway is: Don’t expect different tools to provide consistent results.

Some tools allow you to select the location the test is performed from, so you can select a variety of locations or just the ones close to your most likely visitors. Other tools run all tests from a single location which may not be close to your web server or likely visitors. Keep in mind where you’re running the test from and whether or not that’s what your site visitors would see. The takeaway is: Be conscious of the location the tool is running from and weigh that in the speed that’s being reported.

The free versions of these tools don’t track your performance over time, they show your performance right now. Just because it’s fast right now or slow right now doesn’t mean that won’t change later in the day as traffic to the site or server changes. One of the biggest issues we see is the performance can change drastically over the course of the day depending on what else is happening on that server. The takeaway is: Just because it was fast when you ran the website speed test once doesn’t mean it’s fast on average.

Important metrics to watch for.

Depending on the tool you use, there is a lot of data and a lot of fancy charts thrown at you that it’s easy to get lost in. In this beginning stage, when you’re trying to understand how the site is truly performing, not necessarily implement fixes, there are two main data points to watch.

For the purposes of this explanation, we’ll use Pingdom as it’s one of the more popular tools and is easy to find the data we’re after. However, you should be able to find the same data on other tools as well.

Page Load Speed

The page load speed is the most obvious metric you’ll notice when you run each of these website speed tests and indicates the time in which that particular tool determines the page to be loaded. This measurement is often different between each of the tools and is one reason you’ll see different speeds indicated between different tools.

Page load speed in any of these tools is usually front and center. It’s one of the primary data points they want to show you. In Pingdom, you can find it in the spot just below the search box once the search is complete.

Think of “page load speed” as the point where after you’ve clicked on a search result, you get taken to an empty page for a short time, and then the details of that page being to fill in to the point where you can actually use the page. This metric includes both “internal resources” like images, stylesheets and javascript files, as well as “external resources” like Google Analytics, Youtube videos and Google Fonts.

It’s important to note that for this metric any delay loading these external resources can affect your page speed just as much as internal resources. Meaning if it takes 1.2 seconds to load the YouTube video because there’s a viral video going around somewhere else online, it will count against your page speed.

This page load speed is an indication of what a site visitor would see if testing from the location where the test was, at that particular time on that particular day, run using a similar computer and internet connection. This is a critical metric as it’s an indication of what your user will perceive as your website speed.

Time to first byte (TTFB)

Time to first byte has a tendency to get obscured in most of these testing tools but is an important metric to track. You can think of this metric as the time it takes for the request to reach the server, the server to process the request, and BEGIN to send the response back to the visitor (but not actually process the response or display any content). In other words, you can use this metric to get an understanding of how long it takes for the web server to process the request and begin sending information back.

To find the TTFB in Pingdom’s website speed test tool, scroll down to the “waterfall chart” where it lists all the requests that were made. The very first request should be for the page you requested. Mouse over that and look at the details of the request on the right.

Technically speaking the TTFB is the first five metrics together, so for the request below the TTFB would be 130.1 ms. Some people suggest just looking at the “Wait” time as that constitutes only the time it takes for the server to process the request and not any of the legwork done to make the connection and send the request. For comparison of two different servers, I prefer to include all of the five elements so that I’m including possible network lag if the servers are in different geographic regions. Use whichever version you’re most comfortable with but be consistent about it.

Keep in mind that this number by itself doesn’t mean that much as it tends to lead to finger pointing between the hosting company and the website operators. The site operator opens a ticket with the hosting company complaining of a slow TTFB and the hosting company closes the ticket with a note that the website itself is misconfigured or overly resource hungry.

However, there are two great ways to use this number to your advantage:

As a way to compare one server loading a site to another server loading the exact same site. If the site loads consistently on one server with a 500 ms TTFB and on another server with a 250 ms TTFB, you’re cutting the server processing time in half and can expect to see at least a 250 ms reduction in page load time for the end user by moving the website.

As a way to track TTFB consistency and deviation. Running these website speed tests over time and keeping track of the results will show you if there are times when the server is performing much slower than others. A server with high deviation is likely to be under heavier loads at different times of day leading to slower response times.

So… What now?

We’ve discussed how the testing tools work and the most important metrics to watch out for. We’ve also discussed how the testing tools only test at a single point in time from a single location. We’ve discussed how a single data point from a single location isn’t a great indicator of page speed. The next step would be to get more varied data to represent actual performance over the course of a period of time.

For instance, running the test every 15 minutes over the course of 24 hours would paint a much clearer picture of average page load speed and TTFB over the course of an entire day. Even better would be to run it every 15 minutes over 24 hours from multiple locations which helps paint a picture of average page load speed and TTFB across a geographic region relevant to your site visitors (e.g., Global, United States, Western Europe, etc.).

The problem is that with these free tools, you’ll be limited to how frequently you are able to run the requests. After you run too many too close together, you’ll likely receive an error message indicating you’ve reached your limit and to try again later.

Another thing to watch for if you run the tests too close together, you’ll notice the data doesn’t change as it’s actually relying on the data it pulled on the last run rather than a fresh request. I don’t know exactly how those limitations are defined but the best you can do might be to run the test once per hour from 2 different locations and then aggregating that data. Not perfect but still better than running a single test.

The SiteSpeed Way.

Because we deal with this regularly, we often get questions about how we handle it. The limitations of the free website speed test tools don’t lend themselves very well to tracking statistically significant data. Even in the paid version of the tools, you often can’t control which location or the number of locations that the tests are run from, so you’re stuck with a small dataset from a single location.

We opted to create our own proprietary tool that utilizes the open source Chromium (what Google’s Chrome browser is built on top of). We deploy our tool on multiple servers depending on the test we want to run and trigger it to run every 5-15 minutes until we tell it to stop. The data is then aggregated and we use it to run reports and see the big picture.

In the event we’re running a head-to-head test of a site on its current server compared to the same site on our servers, we’ll have these requests happening simultaneously and we are able to use the data to draw conclusions of the performance impact a site would see by moving.

Did we miss anything?

What did we miss? Do you still have questions? Drop a comment below or shoot a message using our contact form. We'd love to hear from you and get your website speed test strategy up to snuff.