Comparison of three actigraphic algorithms used to evaluate sleep in patients with obstructive sleep apnea

Performance evaluation of Fitbit Charge 3 and actigraphy vs. polysomnography: Sensitivity, specificity, and reliability across participants and nights

GOAL AND AIMS

Compare the accuracy and reliability of sleep/wake classification between the Fitbit Charge 3 and the Micro Motionlogger actigraph when applying either the Cole-Kripke or Sadeh scoring algorithms. Accuracy was established relative to simultaneous Polysomnography recording. Focus technology: Fitbit Charge 3 and actigraphy. Reference technology: Polysomnography.

SAMPLE

Twenty-one university students (10 females).

DESIGN

Simultaneous Fitbit Charge 3, actigraphy, and polysomnography were recorded over 3 nights at the participants' homes.

CORE ANALYTICS

Total sleep time, wake after sleep onset, sensitivity, specificity, positive predictive value, and negative predictive value.

ADDITIONAL ANALYTICS AND EXPLORATORY ANALYSES

Variability of specificity and negative predictive value across subjects and across nights.

CORE OUTCOMES

Fitbit Charge 3 and actigraphy using the Cole-Kripke or Sadeh algorithms exhibited similar sensitivity in classifying sleep segments relative to polysomnography (sensitivity of 0.95, 0.96, and 0.95, respectively). Fitbit Charge 3 was significantly more accurate in classifying wake segments (specificity of 0.69, 0.33, and 0.29, respectively). Fitbit Charge 3 also exhibited significantly higher positive predictive value than actigraphy (0.99 vs. 0.97 and 0.97, respectively) and a negative predictive value that was significantly higher only relative to the Sadeh algorithm (0.41 vs. 0.25, respectively).

IMPORTANT ADDITIONAL OUTCOMES

Fitbit Charge 3 exhibited significantly lower standard deviation in specificity values across subjects and negative predictive value across nights.

CORE CONCLUSION

This study demonstrates that Fitbit Charge 3 is more accurate and reliable in identifying wake segments than the examined FDA-approved Micro Motionlogger actigraphy device. The results also highlight the need to create devices that record and save raw multi-sensor data, which are necessary for developing open-source sleep or wake classification algorithms.

Exploring Predictors of Sleep State Misperception in Women with Posttraumatic Stress Disorder

OBJECTIVES

Insomnia is a common symptom of posttraumatic stress disorder (PTSD) that is resistant to first-line cognitive behavioral interventions. However, research suggests that, among individuals with PTSD, self-reported sleep impairment is typically more severe than what is objectively observed, a phenomenon termed sleep state misperception. Relatively little research has examined which individuals with PTSD are most likely to exhibit sleep state misperception. This study explored clinical predictors of sleep state misperception in a sample of 43 women with PTSD and clinically significant sleep impairment.

METHOD

During a baseline assessment, participants' PTSD symptoms were assessed using a clinical interview and their sleep was assessed using the Pittsburgh Sleep Quality Index (PSQI). Objective sleep, self-reported sleep, and PTSD symptoms were then assessed over a 1-week period using actigraphy and daily diaries.

RESULTS

Consistent with previous research, women in the study exhibited total sleep time (TST), sleep efficiency (SE), and sleep onset latency (SOL) sleep state misperception. For TST and SE, but not SOL, discrepancies between actigraphy and the PSQI were associated with each clinician-rated PTSD symptom cluster, whereas discrepancies between actigraphy and daily diary were only associated with clinician-rated reexperiencing symptoms. The only self-reported PTSD symptom that was uniquely associated with sleep state misperception was nightmares. This association was no longer significant after controlling for sleep-related anxiety.

CONCLUSIONS

Results suggest that women with more severe reexperiencing symptoms of PTSD, particularly nightmares, may be more likely to exhibit TST and SE sleep state misperception, perhaps due to associated sleep-related anxiety.

On the Unification of Common Actigraphic Data Scoring Algorithms

Actigraphy is a well-known, inexpensive method to investigate human movement patterns. Sleep and circadian rhythm studies are among the most popular applications of actigraphy. In this study, we investigate seven common sleep-wake scoring algorithms designed for actigraphic data, namely Cole-Kripke algorithm, two versions of Sadeh algorithm, Sazonov algorithm, Webster algorithm, UCSD algorithm and Scripps Clinic algorithm. We propose a unified mathematical framework describing five of them. One of the observed novelties is that five of these algorithms are in fact equivalent to low-pass FIR filters with very similar characteristics. We also provide explanations about the role of some factors defining these algorithms, as none were given by their Authors who followed empirical procedures. Proposed framework provides a robust mathematical description of discussed algorithms, which for the first time allows one to fully understand their operation and basics.

The development, inter-rater agreement and performance of a hierarchical procedure for setting the rest-interval in actigraphy data

The aim of this study was to develop and empirically test a hierarchical procedure for defining rest intervals in actigraphy data.

BACKGROUND

This is a two-part study. The aim of study 1 was to identify common practices for setting rest intervals in actigraphy research and investigate whether standardized guidelines for setting the rest interval exist, as a base to develop a new procedure for defining rest intervals in actigraphy. The aim of study 2 was to empirically test this procedure (The Rest Interval Setting, RISE Procedure). The RISE procedure was applied to a dataset of 537 nights from the sleep study SLEEPIC.

PARTICIPANTS

Participants (N = 55) were aged 19-33 (M = 22.7, SD = 3.0).

METHODS

Study 1: Structured overview of the methods used to correct actigraphy data. Study 2: Three scorers independently applied the RISE procedure to the dataset.

RESULTS

Study 1 demonstrated that methods and reporting practices are inconsistent and that there is a need for a standardized procedure for setting the rest interval. The results in study 2 revealed that using the new procedure for setting rest intervals provided high agreement between scorers for both rest onsets (α= 0.975) and offsets (α= 0.998). Applying the procedure to the dataset resulted in a shortening of the rest interval by 36 min and 19 s on average. There were significant changes (p < 0.001) in all sleep estimate outcomes after applying the RISE procedure.

CONCLUSION

Methods for processing and reporting actigraphy data are highly inconsistent across studies. Here we present empirical support for a new standardized procedure for setting the rest interval, which is likely to improve transparency and reproducibility in achigraphy research.

Tracheal Sound Analysis Using a Deep Neural Network to Detect Sleep Apnea

STUDY OBJECTIVES

Portable devices for home sleep apnea testing are often limited by their inability to discriminate sleep/wake status, possibly resulting in underestimations. Tracheal sound (TS), which can be visualized as a spectrogram, carries information about apnea/hypopnea and sleep/wake status. We hypothesized that image analysis of all-night TS recordings by a deep neural network (DNN) would be capable of detecting breathing events and classifying sleep/wake status. The aim of this study is to develop a DNN-based system for sleep apnea testing and validate it using a large sampling of polysomnography (PSG) data.

METHODS

PSG examinations for the evaluation of sleep-disordered breathing (SDB) were performed for 1,852 patients: 1,548 PSG records were used to develop the system, and the remaining 304 records were used for validation. TS spectrogram images were obtained every 60 seconds and labeled with the PSG scoring results (breathing event and sleep/wake status), then introduced to DNN learning. Two different DNNs were trained for breathing status and sleep/wake status, respectively.

RESULTS

A DNN with convolutional layers showed the best performance for discriminating breathing status. The same DNN structure was trained for sleep/wake discrimination. In the validation study, the DNN analysis was capable of discriminating the sleep/wake status with reasonable accuracy. The diagnostic sensitivity, specificity, and area under the receiver operating characteristic curves for diagnosis of SDB with apnea-hypopnea index of > 5, 15, and 30 were 0.98, 0.76, and 0.99; 0.97, 0.90, and 0.99; and 0.92, 0.94, and 0.98, respectively.

CONCLUSIONS

The developed system using the TS DNN analysis has a good performance for SDB testing.

CITATION

Nakano H, Furukawa T, Tanigawa T. Tracheal sound analysis using a deep neural network to detect sleep apnea. J Clin Sleep Med. 2019;15(8): 1125-1133.

Wearable Sleep Technology in Clinical and Research Settings

: The accurate assessment of sleep is critical to better understand and evaluate its role in health and disease. The boom in wearable technology is part of the digital health revolution and is producing many novel, highly sophisticated and relatively inexpensive consumer devices collecting data from multiple sensors and claiming to extract information about users' behaviors, including sleep. These devices are now able to capture different biosignals for determining, for example, HR and its variability, skin conductance, and temperature, in addition to activity. They perform 24/7, generating overwhelmingly large data sets (big data), with the potential of offering an unprecedented window on users' health. Unfortunately, little guidance exists within and outside the scientific sleep community for their use, leading to confusion and controversy about their validity and application. The current state-of-the-art review aims to highlight use, validation and utility of consumer wearable sleep-trackers in clinical practice and research. Guidelines for a standardized assessment of device performance is deemed necessary, and several critical factors (proprietary algorithms, device malfunction, firmware updates) need to be considered before using these devices in clinical and sleep research protocols. Ultimately, wearable sleep technology holds promise for advancing understanding of sleep health; however, a careful path forward needs to be navigated, understanding the benefits and pitfalls of this technology as applied in sleep research and clinical sleep medicine.

Validation of a Device for the Ambulatory Monitoring of Sleep Patterns: A Pilot Study on Parkinson's Disease

The development of wearable devices has increase interest in the use of ambulatory methods to detect sleep disorders more objectively than those permitted by subjective scales evaluating sleep quality, while subjects maintain their usual lifestyle. This study aims to validate an ambulatory circadian monitoring (ACM) device for the detection of sleep and wake states and apply it to the evaluation of sleep quality in patients with Parkinson disease (PD). A polysomnographic validation study was conducted on a group of patients with different sleep disorders in a preliminary phase, followed by a pilot study to apply this methodology to PD patients. The ACM device makes it possible to estimate the main sleep parameters very accurately, as demonstrated by: (a) the lack of significant differences between the mean values detected by PSG and ACM in time in bed (TIB), total sleep time (TST), sleep efficiency (SE), and time awake after sleep onset (WASO); (b) the slope of the correlation lines between the parameters estimated by the two procedures, very close to 1, which demonstrates the linearity of the predictions; (c) the low bias value in the estimates obtained through ACM. Sleep in PD is associated with lower distal skin temperature, efficiency and overall sleep time; greater WASO, activity during sleep and duration of naps and a worse circadian function index. In summary, the ACM device has proven to be clinically useful to evaluate sleep in an objective manner, thanks to the integrated management of different complementary variables, having advantages over conventional actigraphy.

Sleep-wake detection using recurrence quantification analysis

Actigraphy is a method for monitoring the movements of the nondominant arm, and the technology has found applications ranging from clinical devices to smart wristbands. Time series obtained from actigraphy data is used in chronobiology to define the sleep-wake cycle, as well as in sleep medicine to evaluate an individual's sleep quality. In the study described in this paper, an algorithm based on recurrence quantification analysis (RQA) was applied to a time series obtained from a commercial actigraph, which was used to collect raw data alongside polysomnography (PSG), generally considered as the gold standard for assessing sleep quality. The central hypothesis is that transitions between sleep and wakefulness are not purely random events, but are strongly influenced by two internal processes: the homeostatic pressure and the circadian cycle. On the basis of this premise, application of RQA to time series as an estimator of this system should lead to improved results and allow more reliable investigations than a purely empirical approach. To compare the results from the RQA algorithm and those from PSG, we present a detailed statistical analysis involving a bias evaluation of the two methods following an approach suggested by Bland and Altman, a comparison of data processed using the kappa coefficient, and a comparison of consolidated sleep quality data using the p -value.

Use of Actigraphy for the Evaluation of Sleep Disorders and Circadian Rhythm Sleep-Wake Disorders: An American Academy of Sleep Medicine Systematic Review, Meta-Analysis, and GRADE Assessment

INTRODUCTION

The purpose of this systematic review is to provide supporting evidence for a clinical practice guideline on the use of actigraphy.

METHODS

The American Academy of Sleep Medicine commissioned a task force of experts in sleep medicine. A systematic review was conducted to identify studies that compared the use of actigraphy, sleep logs, and/or polysomnography. Statistical analyses were performed to determine the clinical significance of using actigraphy as an objective measure of sleep and circadian parameters. Finally, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) process was used to assess the evidence for making recommendations.

RESULTS

The literature search resulted in 81 studies that met inclusion criteria; all 81 studies provided data suitable for statistical analyses. These data demonstrate that actigraphy provides consistent objective data that is often unique from patient-reported sleep logs for some sleep parameters in adult and pediatric patients with suspected or diagnosed insomnia, circadian rhythm sleep-wake disorders, sleep-disordered breathing, central disorders of hypersomnolence, and adults with insufficient sleep syndrome. These data also demonstrate that actigraphy is not a reliable measure of periodic limb movements in adult and pediatric patients. The task force provided a detailed summary of the evidence along with the quality of evidence, the balance of benefits and harms, patient values and preferences, and resource use considerations.

Identifying bedrest using 24-h waist or wrist accelerometry in adults

Objectives

To adapt and refine a previously-developed youth-specific algorithm to identify bedrest for use in adults. The algorithm is based on using an automated decision tree (DT) analysis of accelerometry data.

Design

Healthy adults (n = 141, 85 females, 19–69 years-old) wore accelerometers on the waist, with a subset also wearing accelerometers on the dominant wrist (n = 45). Participants spent ≈24-h in a whole-room indirect calorimeter equipped with a force-platform floor to detect movement.

Methods

Minute-by-minute data from recordings of waist-worn or wrist-worn accelerometers were used to identify bedrest and wake periods. Participants were randomly allocated to development (n = 69 and 23) and validation (n = 72 and 22) groups for waist-worn and wrist-worn accelerometers, respectively. The optimized DT algorithm parameters were block length, threshold, bedrest-start trigger, and bedrest-end trigger. Differences between DT classification and synchronized objective classification by the room calorimeter to bedrest or wake were assessed for sensitivity, specificity, and accuracy using a Receiver Operating Characteristic (ROC) procedure applied to 1-min epochs (n = 92,543 waist; n = 30,653 wrist).

Results

The optimal algorithm parameter values for block length were 60 and 45 min, thresholds 12.5 and 400 counts/min, bedrest-start trigger 120 and 400 counts/min, and bedrest-end trigger 1,200 and 1,500 counts/min, for the waist and wrist-worn accelerometers, respectively. Bedrest was identified correctly in the validation group with sensitivities of 0.819 and 0.912, specificities of 0.966 and 0.923, and accuracies of 0.755 and 0.859 by the waist and wrist-worn accelerometer, respectively. The DT algorithm identified bedrest/sleep with greater accuracy than a commonly used automated algorithm (Cole-Kripke) for wrist-worn accelerometers (p<0.001).

Conclusions

The adapted DT accurately identifies bedrest in data from accelerometers worn by adults on either the wrist or waist. The automated bedrest/sleep detection DT algorithm for both youth and adults is openly accessible as a package “PhysActBedRest” for the R-computer language.

Actigraphy-based sleep estimation in adolescents and adults: a comparison with polysomnography using two scoring algorithms

OBJECTIVES

Actigraphy is widely used to estimate sleep-wake time, despite limited information regarding the comparability of different devices and algorithms. We compared estimates of sleep-wake times determined by two wrist actigraphs (GT3X+ versus Actiwatch Spectrum [AWS]) to in-home polysomnography (PSG), using two algorithms (Sadeh and Cole-Kripke) for the GT3X+ recordings.

SUBJECTS AND METHODS

Participants included a sample of 35 healthy volunteers (13 school children and 22 adults, 46% male) from Boston, MA, USA. Twenty-two adults wore the GT3X+ and AWS simultaneously for at least five consecutive days and nights. In addition, actigraphy and PSG were concurrently measured in 12 of these adults and another 13 children over a single night. We used intraclass correlation coefficients (ICCs), epoch-by-epoch comparisons, paired t-tests, and Bland-Altman plots to determine the level of agreement between actigraphy and PSG, and differences between devices and algorithms.

RESULTS

Each actigraph showed comparable accuracy (0.81-0.86) for sleep-wake estimation compared to PSG. When analyzing data from the GT3X+, the Cole-Kripke algorithm was more sensitive (0.88-0.96) to detect sleep, but less specific (0.35-0.64) to detect wake than the Sadeh algorithm (sensitivity: 0.82-0.91, specificity: 0.47-0.68). Total sleep time measured using the GT3X+ with both algorithms was similar to that obtained by PSG (ICC=0.64-0.88). In contrast, agreement between the GT3X+ and PSG wake after sleep onset was poor (ICC=0.00-0.10). In adults, the GT3X+ using the Cole-Kripke algorithm provided data comparable to the AWS (mean bias=3.7±19.7 minutes for total sleep time and 8.0±14.2 minutes for wake after sleep onset).

CONCLUSION

The two actigraphs provided comparable and accurate data compared to PSG, although both poorly identified wake episodes (i.e., had low specificity). Use of actigraphy scoring algorithm influenced the mean bias and level of agreement in sleep-wake times estimates. The GT3X+, when analyzed by the Cole-Kripke, but not the Sadeh algorithm, provided comparable data to the AWS.

Accuracy of computer algorithms and the human eye in scoring actigraphy

PURPOSE

The purpose of this study is to determine the optimal scoring method and parameter settings of actigraphy by comparison to simultaneous polysomnography (PSG).

METHODS

Fifteen studies of simultaneous PSG and actigraphy were completed in adolescents (mean age = 16.3 years) and analyzed. Scoring actigraphy by the human eye was compared to a commercial computerized algorithm using various parameters. The PSG was considered the reference standard.

RESULTS

There was a better correlation between actigraphy and PSG sleep start/end, total sleep time, wake after sleep onset, and sleep efficiency when the rest period was determined by the human (mean r = 0.640) rather than auto-set by the software (r = 0.406). The best results came when the rest intervals were set based on the PSG (r = 0.694). Scoring the printed actogram by the human eye was superior to the auto analyses as well (r = 0.575). Higher correlations and lower biases were obtained from lower wake threshold settings (low and medium) and higher immobility times (10 and 15 min).

CONCLUSIONS

Visual scoring by simple inspection of the actigraphy tracing had a reasonable correlation with the gold standard PSG. Accurate determination of the rest interval is important in scoring actigraphy. Scoring actigraphy by the human eye is superior to this computer algorithm when auto-setting major rest periods. A low wake threshold and 10-15 min of immobility for sleep onset and sleep end yield the most accurate computerized results. Auto-setting major rest intervals should be avoided to set start/end of rest intervals; adjustments for artifacts and/or a sleep diary for comparison are helpful.