Cool devices like the Apple Watch, Fitbit, and the nearly ubiquitous Garmin Forerunners (at my local running group anyway) aim to track your biometrics, like heart rate, and calories burned, along with various other fun variables like pace, vertical oscillation, and elevation. There are even a few that try to predict your recovery time, race paces and even VO2 Max which would be very interesting (especially given the vomit inducing-ness of a real VO2 Max test), if they could be trusted. And that is a big "if."

When I got on board with these devices back in 2009, they weren't all that accurate. Even the cardiologists who study wearables at Stanford University went as far as to say that they thought of them a little bit like random number generators. They really didn't seem to be providing anything that was even close to your actual heart rate.

The folks at Stanford have recently tested seven newer fitness bands and now say those heart rate stats have gotten much much better. For most of the devices, the error rate was less than five percent. Which is considered medical grade—and good enough for your doctor!

Calories Burned

Where all the devices fell apart was in estimating the calories burned when compared to gold-standard lab measurements of energy expenditure.

The reason for the discrepancy could be that we all burn energy at different rates and that's hard to guess from simple parameters like weight and height which is generally the only thing we tell these devices when we first buy them. I mean, if I just go out and watch the runners in my neighbourhood, some are incredibly efficient and look simply elegant when they run (watch a video of triathlete Mirinda Carfrae for an example of this) while others are laboured, awkward and even look like they're burning a lot more calories to cover the same amount of ground (watch a video of me for an example of this).

The Test

At Stanford University they did an evaluation of seven devices using a group of 60 volunteers. They evaluated the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and the Samsung Gear S2.

The sixty volunteers, made up of 31 women and 29 men, wore the seven devices while walking or running on treadmills or using stationary bikes. Each volunteer’s heart was also measured with a medical-grade electrocardiograph (ECG). Then their metabolic rate was estimated with an instrument for measuring the oxygen and the carbon dioxide in their breath, which is a pretty good proxy for metabolism and energy expenditure.

Once they had that data, the results from the wearable devices were compared to the measurements from the two medical instruments.

In the end, they concluded, not all that shockingly, that some devices were more accurate than others. They also concluded that factors such as skin colour and body mass index (BMI) definitely affected the measurements. I don’t know about you but I have never had any device ask me about my skin tone… have you?