JP Onnela, PhD, an associate professor of biostatistics at the Boston-based Harvard T.H. Chan School of Public Health, stopped using Apple Watches in his research after spotting several inconsistencies in the data collected by the device, according to a July 27 report by The Verge.
For his research, he is collaborating with the department of neurosurgery at Boston-based Brigham and Women's Hospital, and he wanted to use commercially available products. Hassan Dawood, a research fellow at Brigham and Women's, exported his heart rate data stored on his Apple Watch twice. In both instances, the data was from December 2018 to September 2020, but it was exported about eight months apart.
"These algorithms are what we would call black boxes — they're not transparent," Dr. Onnela said. "So it's impossible to know what's in them."
Wearable devices don't share raw data. Instead, they show data after it has been collected and filtered. Companies can also change their algorithms which can affect the consistency with data collected at a future date.
The datasets from Mr. Dawood's watch were surprisingly different, Dr. Onnela said, so it may have been filtered differently. Inconsistency makes it difficult to launch research with commercial devices because each device at different time periods can output different results.
These inconsistencies made his team back away from using Apple Watches in their study. He said commercial wearables can only be studied if raw data is available or, at a bare minimum, researchers are notified before the algorithm is changed.