That, at least, is the conclusion of a team of researchers at the Stanford University School of Medicine, which ran tests on seven wrist-worn devices that offer users a range of health-related measurements.
The researchers evaluated the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and Samsung Gear S2 among a diverse group of volunteers, and published their findings this week. While many of the devices have been updated, retired, or even recalled, we can assume that many are still being used by the millions of people that bought them.
Looking at the team’s data, even the best performing device — the FitBit Surge — was way off the mark when counting burned calories, missing the real figure by 27 percent. The least accurate was the PulseOn, which was off by a whopping 93 percent.
“People are basing life decisions on the data provided by these devices,” Euan Ashley, a professor of cardiovascular medicine at Stanford and senior author of the research report, said in a release, adding that it can be hard for doctors to know how to respond to data offered by a patient with their own fitness tracker.
While the researchers couldn’t be certain as to why the energy expenditure data was so inaccurate, they noted that the devices each use proprietary algorithms for calculating such data. Anna Shcherbina, one of the report’s authors, commented that it’s “very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height and weight, etc.”
But before you toss your tracker in the trash, you might like to know that the findings weren’t all bad. For example, it was discovered that six of the devices measured heart rate with an error rate of less than 5 percent, while Samsung’s Gear S2 device had an error rate of 6.8 percent. These more accurate measurements can be put down to the fact that such data is measured directly rather than by way of an algorithm.
Ashley concluded that for the devices tested, a user can have a fair amount of confidence in the heart rate data, adding that you’ll clearly want to be very wary about designing your diet on the basis of the calories-count readout.
“The heart rate measurements performed far better than we expected, but the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me,” the researcher said.
We can certainly expect developments in software and technology to improve the accuracy of tracker data and look forward to seeing the results of additional research on more recently released — and future — fitness-focused devices.
