Buy me a cup of coffee

Hire me

Project: Measure Jetpack

Published October 6, 2015 by Arūnas

Whenever someone brings up Jetpack in a discussion, one thing is inevitable – someone calling it a bloated thing that is solely responsible for slowing their website, the whole internet and causing the global warming. The same thing recently happened at Advanced WordPress Facebook group, an it got me thinking.

WordPress developers either love or hate Jetpack. Some say it provides a lot of essential functionality in a single package and it is not slow. Others call it bloated, slow and full on unnecessary stuff. Both sides might have a point, but when it comes to cold hard data, nobody can provide some to prove their point. Everything is based on personal experiences and anecdotical evidence.

When BruteProtect got acquired by Jetpack, they did an interestingstudy of comparing Jetpack performance to that of similar popular plugins. But I think some of its assumptions and choices were flawed and the data was not quite granular enough to call it a proper study. And, of course they are pretty much an interested party in this, so being objective could be hard for them, too.

So that got me thinking – what if somebody did a proper study? Picked feature-wise comparable plugins, and ran tests module by module, with controls and clear methodology? I know, it sounds quite a lot of work, but that is exactly what students were invented for… It so happens, that as I was pondering this, some of my students aproached me, looking for an interesting research topic. Long story short, now I have a couple of students committed to this, and that is what we are going to try to do. All systems clear. Project “Measure Jetpack” is a go.

The Plan

Over the coming weeks, we are going to identify most proper, and most popular replacements for all Jetpack modules. By most proper, I mean ones that have a comparable feature set – just enough to provide the same functionality.

Then we are going to set up a bunch of identical WordPress sites – with some Lorem ipsum content. One will act as a benchmark, another one will be running Jetpack, and a third one will be running the other plugins.

Then we’ll start the measuring, with different plungins/modules activated and deactivated. We’ll be doing 1-to-1 comparisons, as well as some combos. We are going to measute load times, as well as some stats reported by Query Monitor – memory usage, database query count etc.

And then we are going to publish results – as a series of blog posts and articles in academic publications.

How Can You Help

First, offer any and all suggestions about methodology and things to measure.

Also, suggest plugins we should consider as feature-comparable to Jetpack modules. We are going to comb through WordPress.org directory ourselves, but hints and pointers are more than welcome.

This will take a lot of work, and I predict, that a lot of coffee will be consumed in the process. You can buy us a refil. Or ten. 🙂

Progress Reports

This will take a while. It is most definintely a side project, so we are not going to be working on it full time. If You want to follow our progress, you can follow this blog, or subscribe to my mailing list below. I am also going to post updates to Advanced WordPress on Facebook, and on my Twitter.