Miles to go before we sleep…a step toward transparent evaluation of consumer sleep tracking devices

Evaluating Accuracy in Five Commercial Sleep-Tracking Devices Compared to Research-Grade Actigraphy and Polysomnography

The development of consumer sleep-tracking technologies has outpaced the scientific evaluation of their accuracy. In this study, five consumer sleep-tracking devices, research-grade actigraphy, and polysomnography were used simultaneously to monitor the overnight sleep of fifty-three young adults in the lab for one night. Biases and limits of agreement were assessed to determine how sleep stage estimates for each device and research-grade actigraphy differed from polysomnography-derived measures. Every device, except the Garmin Vivosmart, was able to estimate total sleep time comparably to research-grade actigraphy. All devices overestimated nights with shorter wake times and underestimated nights with longer wake times. For light sleep, absolute bias was low for the Fitbit Inspire and Fitbit Versa. The Withings Mat and Garmin Vivosmart overestimated shorter light sleep and underestimated longer light sleep. The Oura Ring underestimated light sleep of any duration. For deep sleep, bias was low for the Withings Mat and Garmin Vivosmart while other devices overestimated shorter and underestimated longer times. For REM sleep, bias was low for all devices. Taken together, these results suggest that proportional bias patterns in consumer sleep-tracking technologies are prevalent and could have important implications for their overall accuracy.

Biomarkers linking habitual short sleep duration with risk of cardiometabolic disease: current progress and future directions

Approximately one in three adults in the United States sleeps less than the recommended 7 h per night. Decades of epidemiological data and data from experimental sleep restriction studies demonstrate short sleep duration is associated with adverse cardiometabolic risk, including risk of type 2 diabetes and cardiovascular disease. However, the precise mechanisms underlying this risk are not fully elucidated and there is a lack of sleep-based interventions designed to mitigate such risk. One strategy to overcome these limitations is to develop biomarkers that link habitual short sleep duration with adverse cardiometabolic risk. Such biomarkers could inform biochemical mechanisms, identify new targets for interventions, support precision medicine by identifying individuals most likely to benefit from sleep-based interventions, and ultimately lead to improved cardiometabolic health in people with habitual short sleep durations. Early progress demonstrates proof-of-principle that omics-based technologies are a viable approach to create biochemical signatures (biomarkers) of short sleep duration, primarily derived from acute studies of experimental sleep restriction. Yet, much work remains. Notably, studies that translate early findings from experimental sleep restriction to free-living adults with habitual short sleep duration have high potential to advance the field. Such studies also create an exciting opportunity for larger randomized controlled trials that simultaneously identify biomarkers of habitual short sleep duration and evaluate the efficacy of sleep-based interventions. Ultimately, early progress in developing molecular biomarkers of short sleep duration combined with the prior decades of progress in the sleep and metabolism fields provide the foundation for exciting progress in the biomarker development space.

Performance of Four Commercial Wearable Sleep-Tracking Devices Tested Under Unrestricted Conditions at Home in Healthy Young Adults

PURPOSE

Commercial wearable sleep-tracking devices are growing in popularity and in recent studies have performed well against gold standard sleep measurement techniques. However, most studies were conducted in controlled laboratory conditions. We therefore aimed to test the performance of devices under naturalistic unrestricted home sleep conditions.

PARTICIPANTS AND METHODS

Healthy young adults (n = 21; 12 women, 9 men; 29.0 ± 5.0 years, mean ± SD) slept at home under unrestricted conditions for 1 week using a set of commercial wearable sleep-tracking devices and completed daily sleep diaries. Devices included the Fatigue Science Readiband, Fitbit Inspire HR, Oura ring, and Polar Vantage V Titan. Participants also wore a research-grade actigraphy watch (Philips Respironics Actiwatch 2) for comparison. To assess performance, all devices were compared with a high performing mobile sleep electroencephalography headband device (Dreem 2). Analyses included epoch-by-epoch and sleep summary agreement comparisons.

RESULTS

Devices accurately tracked sleep-wake summary metrics (ie, time in bed, total sleep time, sleep efficiency, sleep latency, wake after sleep onset) on most nights but performed best on nights with higher sleep efficiency. Epoch-by-epoch sensitivity (for sleep) and specificity (for wake), respectively, were as follows: Actiwatch (0.95, 0.35), Fatigue Science (0.94, 0.40), Fitbit (0.93, 0.45), Oura (0.94, 0.41), and Polar (0.96, 0.35). Sleep stage-tracking performance was mixed, with high variability.

CONCLUSION

As in previous studies, all devices were better at detecting sleep than wake, and most devices compared favorably to actigraphy in wake detection. Devices performed best on nights with more consolidated sleep patterns. Unrestricted sleep TIB differences were accurately tracked on most nights. High variability in sleep stage-tracking performance suggests that these devices, in their current form, are still best utilized for tracking sleep-wake outcomes and not sleep stages. Most commercial wearables exhibited promising performance for tracking sleep-wake in real-world conditions, further supporting their consideration as an alternative to actigraphy.

Past, Present, and Future of Multisensory Wearable Technology to Monitor Sleep and Circadian Rhythms

Movement-based sleep-wake detection devices (i.e., actigraphy devices) were first developed in the early 1970s and have repeatedly been validated against polysomnography, which is considered the “gold-standard” of sleep measurement. Indeed, they have become important tools for objectively inferring sleep in free-living conditions. Standard actigraphy devices are rooted in accelerometry to measure movement and make predictions, via scoring algorithms, as to whether the wearer is in a state of wakefulness or sleep. Two important developments have become incorporated in newer devices. First, additional sensors, including measures of heart rate and heart rate variability and higher resolution movement sensing through triaxial accelerometers, have been introduced to improve upon traditional, movement-based scoring algorithms. Second, these devices have transcended scientific utility and are now being manufactured and distributed to the general public. This review will provide an overview of: (1) the history of actigraphic sleep measurement, (2) the physiological underpinnings of heart rate and heart rate variability measurement in wearables, (3) the refinement and validation of both standard actigraphy and newer, multisensory devices for real-world sleep-wake detection, (4) the practical applications of actigraphy, (5) important limitations of actigraphic measurement, and lastly (6) future directions within the field.

Challenges and Opportunities for Applying Wearable Technology to Sleep

Wearable technology has a history in sleep research dating back to the 1970s. Because modern wearable technology is relatively cheap and widely used by the general population, this represents an opportunity to leverage wearable devices to advance sleep medicine and research. However, there is a lack of published validation studies designed to quantify device performance against accepted gold standards, especially across different populations. Recommendations for conducting performance assessments and using wearable devices are now published with the goal of standardizing wearable device implementation and advancing the field.