The readings from these devices were compared to two more robust ways of tracking calories. In the first experiment, the participants spent 24 hours in a metabolic chamber where scientists could track oxygen consumption and carbon dioxide production, and assess calorie expenditure from that. In the second, researchers used urine samples to calculate participants' calorie expenditures in their normal lives.

In the metabolic chamber test, the Withings and Jawbone devices underestimated expenditure by about 270 calories while both Omron devices overestimated calories by about 200 calories. The other devices all fell in between that range, with the Panasonic and Epson devices the closest to the standard.

In the urine test, on the other hand, all the devices underestimated calorie output, by margins ranging from 69 calories short (the Omron Active Style Pro) to 590 calories short (the Jawbone UP24).

Researchers noted a few factors that might have contributed to the disconnect, including the fact that participants couldn't wear the trackers all the time. They also noted that they tested only a healthy weight population, and that future studies should include overweight participants as well.

In an accompanying commentary, Dr. Adam Schoenfeld, an internist at UCSF, wrote about how this study demonstrates the need for FDA regulation of mobile apps that are going to be used as medical devices.

"Use of health-related apps may pose little risk in healthy persons if the information collected is not used for medical decision-making, but these apps should still undergo basic testing for accuracy," he wrote. "For mHealth apps that are designed to play a role in medical care, documenting the validity and accuracy of measurements is crucial."

Plenty of recent studies, of varying degrees of robustness, have called into question the accuracy of fitness trackers. Particularly relevant to this study is one from February by the American Council on Exercise, which also found that commercial fitness trackers had accuracy problems in the realm of calorie tracking.