January 2012 Site Performance Report

It’s a new year and it’s time for another report on how fast (or slow) our sites are. If you are new here, our previous report looked at the average load time for our four major types of pages, as well as the 95th percentile load time. The inspiration for this type of post came from our friends at Etsy.

Our methodology for this month’s update was the same as the last report, we looked at load times between 9AM and 5PM Eastern Time for 100% of the real users that visited www.wayfair.com during that period. To make things more readable I am going to break out the averages and 95th percentiles into two graphs. Here is a comparison of the average load times from January (bottom) and September:

Comparison of Average Performance (click to enlarge)

And now the 95th percentiles:

Comparison of 95th Percentile Performance (click to enlarge)

The obvious question is “what the heck happened?” Load times are higher pretty much across the board. The only exception is the homepage, which saw a minor increase in its average load time, but the 95th percentile dropped. Everything else is slower, and our search results page performance is just awful. There is a very simple reason for this – we stopped focusing on performance.

We had made performance a priority for a while – we treated it like a project, we set goals, and we achieved many of them. But then we made the mistake of resting somewhat on these achievements and moved on. Don’t misunderstand — nobody actually said, “we’re done, let’s forget about performance” but at the same time no one was actually dedicated to improving performance over the last 4 months, and only a few projects were explicitly designed to speed things up. Instead the relentless drive for new functionality (which usually ends up taxing our servers) took over and became the focus. And the results once again demonstrate that the natural trend for load time is up if you take your eyes off the target. On top of that, traffic on our sites is steadily increasing, adding further complexity to the situation (though in the end this is a good problem to have).

As with any growing company, there are a lot of projects pulling people in all directions. What we’ve learned is that if we’re not careful during this process and treat performance as a project instead of a lifestyle, then new feature development tends to take precedence over efforts to improve performance and stability — and we are currently paying the price for that. All is not lost though: we’re learning from this lesson and are currently forming a team to focus on performance and stability. Perhaps more importantly, we are also going to fold these activities into the daily life of project development. Our engineers need to understand the impact of the changes they make, and their projects need to be held accountable to meeting performance benchmarks on an ongoing basis. A project’s performance impact needs to be an integral part of the discussion with the business on how we attack a project – or assess whether it’s even worth doing at all.

We’re optimistic that these actions will lead us to a better place and reverse the trend we’re currently seeing. I’ll post another update in a few months, and I expect it will look quite a bit better than this one!