What we learned from testing the mobile load times of 200 top retail sites over 3G and LTE

Earlier today, I had an exciting opportunity to present some of the findings of Strangeloop’s newest study, the 2012 State of Mobile Ecommerce Performance at Velocity EU. I’ll be posting those slides shortly, but first want to take a moment to talk about this study and why we’re so excited about it.

Why test over cellular networks?

Last spring, we were sitting in the office talking about mobile performance measurement and why it’s so hard to get reliable real-world numbers, especially over cellular networks. This seemed like a glaring knowledge gap, but it’s an understandable gap given the massive variability in network performance. Last year, I did an informal latency survey and found that latency can fluctuate wildly, even in the same location at the same time.

Methodology: How to DIY your own RUM tests for mobile

So we set out to develop a reliable methodology for measuring mobile performance over cellular networks, which takes this variability into account. We also wanted to create a methodology that anyone could recreate fairly easily.