Are fitness trackers accurate?

Nate Meckes recognized that he needed to study the accuracy of activity monitors after wearing one. A shipment of the devices, known technically as accelerometers and designed to measure a person’s movement and energy expenditure, had arrived at Arizona State University, where Dr. Meckes was a researcher. To ensure they were operational, he slipped one over his hip and wore it throughout the day, including to an interminable meeting where he stood up and paced. “I’m not good at sitting still,” he says.

Checking his monitor afterward, though, he was flabbergasted. “It had recorded that I was not moving at all,” says Dr. Meckes, now an assistant professor at Azusa Pacific University in Azusa, Calif. The experience inspired him to set up an experiment examining how reliable such devices are.

The Jawbone Up is one of several fitness trackers on the market. [CREDIT: The New York Times]

Until recently, accelerometers, which use electronics to determine bodily movement and intensity, had been confined to research laboratories. But now, at-home users can choose from a variety of devices, sold under such brand names as Fitbit and Nike+ FuelBand. Some are worn on the hip; others on the arm or wrist. All sense movement and feed data into the device’s electronic brain, where proprietary equations determine how much energy someone is expending, meaning, in practical terms, how many calories they burn.

But by and large, users have had to take such measurements on faith. Unbiased, comparative studies of the devices haven’t been available.

Which makes Dr. Meckes’s experiment useful and timely, particularly since his results, presented last month in Indianapolis at the annual meeting of the American College of Sports Medicine, join those of a number of other new studies in raising concerns about just how well today’s activity monitors do their job.

For his experiment, Dr. Meckes gathered 16 adult volunteers and fitted each with three different monitors, two worn on the hip and one around the arm. The volunteers also donned portable masks that measure oxygen consumption, the gold standard in determining energy output.

The volunteers then threw themselves into a variety of activities in the university’s physiology lab, including walking on a treadmill, cleaning a simulated kitchen, standing up, typing at a computer and playing a board game.

All three of the devices accurately measured energy expenditure when the volunteers walked briskly, Dr. Meckes and his colleagues found; their estimates closely matched those of the oxygen-consumption monitor.

But the devices were far less reliable in tracking the energy costs of light-intensity activities like standing or cleaning, often misinterpreting them as physical immobility. Only the calorie cost of typing was overestimated, and only by the armband monitor, which considered the arm movements involved to be far more dynamic than they actually are.

These miscalculations echo those of the findings from several other new studies. One, also reported at the sports medicine meeting, involved 74 adults, young and old, who wore an armband accelerometer and a portable oxygen-consumption gauge while walking, jogging, riding a stationary bicycle, windmilling an arm ergometer, and completing so-called activities of daily living, like lifting boxes and sweeping.

Again, the accelerometer measured the more strenuous bodily movements, like jogging, fairly accurately. But it significantly underestimated subtler activities, like sweeping, and was “terrible” at measuring bicycle pedaling, which involves no arm movement, says Glenn Gaesser, the director of the Healthy Lifestyles Research Center at Arizona State University in Phoenix, who oversaw the study.

So, too, a study published last month in Medicine & Science in Sports & Exercise found that several hip-mounted accelerometers underestimated the energy involved in standing up, bicycling and walking or jogging uphill, says Ray Browning, an assistant professor of exercise science at Colorado State University in Fort Collins, who led the study.

The question, of course, is whether it matters if the devices are inaccurate, especially if they underestimate daily energy expenditure, and perhaps fiendishly spur some at-home users to move more, thinking that they’ve expended less energy than they actually have.

The studies’ researchers think the inaccuracies do matter. “There’s a growing consensus” among exercise scientists, Dr. Meckes says, “that people should spend less time in sedentary activities, like sitting,” and instead stand up, stroll or sweep more. But if people get the idea from their activity monitors that such activities don’t really count, in terms of movement and calorie expenditure, “it may be harder to get that message across,” he says.

The good news is that accelerometers are improving, Dr. Browning says. The algorithms underlying the devices’ measurements — which are developed by engineers using data from people wearing the devices — are constantly being refined. And researchers, including Dr. Browning, are exploring better monitor placement.

At Colorado State, for instance, he and his colleagues have created a prototype shoe-based accelerometer, which embeds the electronics in an insole. In his recent study, this device better captured changes in posture and foot pressure than hip-level accelerometers.

Still, the lesson at the moment for anyone who owns an accelerometer is that the device’s measurements are likely to be imperfect — which, says Dr. Gaesser, does not mean you should stash yours in a drawer. “They may not be accurate” for counting calories, he says. “But for many people, they’re inspirational, and if using one gets someone to move more, then as far as I’m concerned, it’s serving a good purpose.”

Last modified: June 12, 2013
All rights reserved. This copyrighted material may not be published without permissions. Links are encouraged.