Recommendations for Determining the Validity of Consumer Wearables and Smartphones for the Estimation of Energy Expenditure: Expert Statement and Checklist of the INTERLIVE Network

Composite activity type and stride-specific energy expenditure estimation model for thigh-worn accelerometry

BACKGROUND

Accurately measuring energy expenditure during physical activity outside of the laboratory is challenging, especially on a large scale. Thigh-worn accelerometers have gained popularity due to the possibility to accurately detect physical activity types. The use of machine learning techniques for activity classification and energy expenditure prediction may improve accuracy over current methods. Here, we developed a novel composite energy expenditure estimation model by combining an activity classification model with a stride specific energy expenditure model for walking, running, and cycling.

METHODS

We first trained a supervised deep learning activity classification model using pooled data from available adult accelerometer datasets. The composite energy expenditure model was then developed and validated using additional data based on a sample of 69 healthy adult participants (49% female; age = 25.2 ± 5.8 years) who completed a standardised activity protocol with indirect calorimetry as the reference measure.

RESULTS

The activity classification model showed an overall accuracy of 99.7% across all five activity types during validation. The composite model for estimating energy expenditure achieved a mean absolute percentage error of 10.9%. For running, walking, and cycling, the composite model achieved a mean absolute percentage error of 6.6%, 7.9% and 16.1%, respectively.

CONCLUSIONS

The integration of thigh-worn accelerometers with machine learning models provides a highly accurate method for classifying physical activity types and estimating energy expenditure. Our novel composite model approach improves the accuracy of energy expenditure measurements and supports better monitoring and assessment methods in non-laboratory settings.

Machine learning prediction of pulmonary oxygen uptake from muscle oxygen in cycling

The purpose of this study was to test whether a machine learning model can accurately predict VO2 across different exercise intensities by combining muscle oxygen (MO2) with heart rate (HR). Twenty young highly trained athletes performed the following tests: a ramp incremental exercise, three submaximal constant intensity exercises, and three severe intensity exhaustive exercises. A Machine Learning model was trained to predict VO2, with model inputs including heart rate, MO2 in the left (LM) and right legs (RM). All models demonstrated equivalent results, with the accuracy of predicting VO2 at different exercise intensities varying among different models. The LM+RM+HR model performed the best across all intensities, with low bias in predicted VO2 for all intensity exercises (0.08 ml/kg/min, 95% limits of agreement: -5.64 to 5.81), and a very strong correlation (r = 0.94, p < 0.001) with measured VO2. Furthermore, the accuracy of predicting VO2 using LM+HR or RM+HR was higher than using LM+RM, and higher than the accuracy of predicting VO2 using LM, RM, or HR alone. This study demonstrates the potential of a machine learning model combining MO2 and HR to predict VO2 with minimal bias, achieving accurate predictions of VO2 for different intensity levels of exercise.

Comparison of raw accelerometry data from ActiGraph, Apple Watch, Garmin, and Fitbit using a mechanical shaker table

The purpose of this study was to evaluate the reliability and validity of the raw accelerometry output from research-grade and consumer wearable devices compared to accelerations produced by a mechanical shaker table. Raw accelerometry data from a total of 40 devices (i.e., n = 10 ActiGraph wGT3X-BT, n = 10 Apple Watch Series 7, n = 10 Garmin Vivoactive 4S, and n = 10 Fitbit Sense) were compared to reference accelerations produced by an orbital shaker table at speeds ranging from 0.6 Hz (4.4 milligravity-mg) to 3.2 Hz (124.7mg). Two-way random effects absolute intraclass correlation coefficients (ICC) tested inter-device reliability. Pearson product moment, Lin's concordance correlation coefficient (CCC), absolute error, mean bias, and equivalence testing were calculated to assess the validity between the raw estimates from the devices and the reference metric. Estimates from Apple, ActiGraph, Garmin, and Fitbit were reliable, with ICCs = 0.99, 0.97, 0.88, and 0.88, respectively. Estimates from ActiGraph, Apple, and Fitbit devices exhibited excellent concordance with the reference CCCs = 0.88, 0.83, and 0.85, respectively, while estimates from Garmin exhibited moderate concordance CCC = 0.59 based on the mean aggregation method. ActiGraph, Apple, and Fitbit produced similar absolute errors = 16.9mg, 21.6mg, and 22.0mg, respectively, while Garmin produced higher absolute error = 32.5mg compared to the reference. ActiGraph produced the lowest mean bias 0.0mg (95%CI = -40.0, 41.0). Equivalence testing revealed raw accelerometry data from all devices were not statistically significantly within the equivalence bounds of the shaker speed. Findings from this study provide evidence that raw accelerometry data from Apple, Garmin, and Fitbit devices can be used to reliably estimate movement; however, no estimates were statistically significantly equivalent to the reference. Future studies could explore device-agnostic and harmonization methods for estimating physical activity using the raw accelerometry signals from the consumer wearables studied herein.

A Device Agnostic Approach to Predict Children's Activity from Consumer Wearable Accelerometer Data: A Proof-of-Concept Study

INTRODUCTION

This study examined the potential of a device agnostic approach for predicting physical activity from consumer wearable accelerometry compared with a research-grade accelerometry.

METHODS

Seventy-five 5- to 12-year-olds (58% male, 63% White) participated in a 60-min protocol. Children wore wrist-placed consumer wearables (Apple Watch Series 7 and Garmin Vivoactive 4) and a research-grade device (ActiGraph GT9X) concurrently with an indirect calorimeter (COSMED K5). Activity intensities (i.e., inactive, light, moderate-to-vigorous physical activity) were estimated via indirect calorimetry (criterion), and the Hildebrand thresholds were applied to the raw accelerometer data from the consumer wearables and research-grade device. Epoch-by-epoch (e.g., weighted sensitivity, specificity) and discrepancy (e.g., mean bias, absolute error) analyses evaluated agreement between accelerometry-derived and criterion estimates. Equivalence testing evaluated the equivalence of estimates produced by the consumer wearables and ActiGraph.

RESULTS

Estimates produced by the raw accelerometry data from ActiGraph, Apple, and Garmin produced similar criterion agreement with weighted sensitivity = 68.2% (95% confidence interval (CI), 67.1%-69.3%), 73.0% (95% CI, 71.8%-74.3%), and 66.6% (95% CI, 65.7%-67.5%), respectively, and weighted specificity = 84.4% (95% CI, 83.6%-85.2%), 82.0% (95% CI, 80.6%-83.4%), and 75.3% (95% CI, 74.7%-75.9%), respectively. Apple Watch produced the lowest mean bias (inactive, -4.0 ± 4.5; light activity, 2.1 ± 4.0) and absolute error (inactive, 4.9 ± 3.4; light activity, 3.6 ± 2.7) for inactive and light physical activity minutes. For moderate-to-vigorous physical activity, ActiGraph produced the lowest mean bias (1.0 ± 2.9) and absolute error (2.8 ± 2.4). No ActiGraph and consumer wearable device estimates were statistically significantly equivalent.

CONCLUSIONS

Raw accelerometry estimated inactive and light activity from wrist-placed consumer wearables performed similarly to, if not better than, a research-grade device, when compared with indirect calorimetry. This proof-of-concept study highlights the potential of device-agnostic methods for quantifying physical activity intensity via consumer wearables.

Large Estimate Variations in Assessed Energy Expenditure and Physical Activity Levels during Active Virtual Reality Gaming: A Short Report

The purpose of the study was to compare methods for estimating energy expenditure (EE) and physical activity (PA) intensity during a 30 min session of active virtual reality (VR) gaming. Eight individuals (age = 25.4 ± 2.0 yrs) participated, with a maximal oxygen consumption (VO2max) of 41.3 ± 5.7 mL∙kg−1∙min−1. All tests were conducted over two days. An incremental test to determine the VO2max when running was performed on day 1, while 30 min of active VR gaming was performed on day 2. The instruments used for EE estimations and PA measurements were indirect calorimetry, a heart rate (HR) monitor, and waist- and wrist-worn accelerometer. Compared to indirect calorimetry, waist-worn accelerometers underestimated EE (mean difference: −157.3 ± 55.9 kcal, p < 0.01) and PA levels. HR-based equations overestimated EE (mean difference: 114.8 ± 39.0 kcal, p < 0.01 and mean difference: 141.0 ± 81.6 kcal, p < 0.01). The wrist-worn accelerometer was the most accurate in estimating EE (mean difference: 23.9 ± 45.4 kcal, p = 0.95). The large variations in EE have implications for population-based surveillance of PA levels and for clinical studies using active VR gaming.

Validity of three smartwatches in estimating energy expenditure during outdoor walking and running

Commercially wrist-worn devices often present inaccurate estimations of energy expenditure (EE), with large between-device differences. We aimed to assess the validity of the Apple Watch Series 6 (AW), Garmin FENIX 6 (GF) and Huawei Watch GT 2e (HW) in estimating EE during outdoor walking and running. Twenty young normal-weight Chinese adults concurrently wore three index devices randomly positioned at both wrists during walking at 6 km/h and running at 10 km/h for 2 km on a 400- meter track. As a criterion, EE was assessed by indirect calorimetry (COSMED K5). For walking, EE from AW and GF was significantly higher than that obtained by the K5 (p < 0.001 and 0.002, respectively), but not for HW (p = 0.491). The mean absolute percentage error (MAPE) was 19.8% for AW, 32.0% for GF, and 9.9% for HW, respectively. The limits of agreement (LoA) were 44.1, 150.1 and 48.6 kcal for AW, GF, and HW respectively. The intraclass correlation coefficient (ICC) was 0.821, 0.216 and 0.760 for AW, GF, and HW, respectively. For running, EE from AW and GF were significantly higher than the K5 (p < 0.001 and 0.001, respectively), but not for HW (p = 0.946). The MAPE was 24.4%, 21.8% and 11.9% for AW, GF and HW, respectively. LoA were 62.8, 89.4 and 65.6 kcal for AW, GF and HW, respectively. The ICC was 0.741, 0.594, and 0.698 for AW, GF and HW, respectively. The results indicate that the tested smartwatches show a moderate validity in EE estimations for outdoor walking and running.

Wearable activity trackers-advanced technology or advanced marketing?

Wearable devices represent one of the most popular trends in health and fitness. Rapid advances in wearable technology present a dizzying display of possible functions: from thermometers and barometers, magnetometers and accelerometers, to oximeters and calorimeters. Consumers and practitioners utilize wearable devices to track outcomes, such as energy expenditure, training load, step count, and heart rate. While some rely on these devices in tandem with more established tools, others lean on wearable technology for health-related outcomes, such as heart rhythm analysis, peripheral oxygen saturation, sleep quality, and caloric expenditure. Given the increasing popularity of wearable devices for both recreation and health initiatives, understanding the strengths and limitations of these technologies is increasingly relevant. Need exists for continued evaluation of the efficacy of wearable devices to accurately and reliably measure purported outcomes. The purposes of this review are (1) to assess the current state of wearable devices using recent research on validity and reliability, (2) to describe existing gaps between physiology and technology, and (3) to offer expert interpretation for the lay and professional audience on how best to approach wearable technology and employ it in the pursuit of health and fitness. Current literature demonstrates inconsistent validity and reliability for various metrics, with algorithms not publicly available or lacking high-quality validation studies. Advancements in wearable technology should consider standardizing validation metrics, providing transparency in used algorithms, and improving how technology can be tailored to individuals. Until then, it is prudent to exercise caution when interpreting metrics reported from consumer-wearable devices.