RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale

Sleep disorder diagnosis relies on the analysis of polysomnography (PSG) records. As a preliminary step of this examination, sleep stages are systematically determined. In practice, sleep stage classification relies on the visual inspection of 30-second epochs of polysomnography signals. Numerous automatic approaches have been developed to replace this tedious and expensive task. Although these methods demonstrated better performance than human sleep experts on specific datasets, they remain largely unused in sleep clinics. The main reason is that each sleep clinic uses a specific PSG montage that most automatic approaches cannot handle out-of-the-box. Moreover, even when the PSG montage is compatible, publications have shown that automatic approaches perform poorly on unseen data with different demographics. To address these issues, we introduce RobustSleepNet, a deep learning model for automatic sleep stage classification able to handle arbitrary PSG montages. We trained and evaluated this model in a leave-one-out-dataset fashion on a large corpus of 8 heterogeneous sleep staging datasets to make it robust to demographic changes. When evaluated on an unseen dataset, RobustSleepNet reaches 97% of the F1 of a model explicitly trained on this dataset. Hence, RobustSleepNet unlocks the possibility to perform high-quality out-of-the-box automatic sleep staging with any clinical setup. We further show that finetuning RobustSleepNet, using a part of the unseen dataset, increases the F1 by 2% when compared to a model trained specifically for this dataset. Therefore, finetuning might be used to reach a state-of-the-art level of performance on a specific population.

The Dreem Headband compared to polysomnography for electroencephalographic signal acquisition and sleep staging

Study Objectives

The development of ambulatory technologies capable of monitoring brain activity during sleep longitudinally is critical for advancing sleep science. The aim of this study was to assess the signal acquisition and the performance of the automatic sleep staging algorithms of a reduced-montage dry-electroencephalographic (EEG) device (Dreem headband, DH) compared to the gold-standard polysomnography (PSG) scored by five sleep experts.

Methods

A total of 25 subjects who completed an overnight sleep study at a sleep center while wearing both a PSG and the DH simultaneously have been included in the analysis. We assessed (1) similarity of measured EEG brain waves between the DH and the PSG; (2) the heart rate, breathing frequency, and respiration rate variability (RRV) agreement between the DH and the PSG; and (3) the performance of the DH’s automatic sleep staging according to American Academy of Sleep Medicine guidelines versus PSG sleep experts manual scoring.

Results

The mean percentage error between the EEG signals acquired by the DH and those from the PSG for the monitoring of α was 15 ± 3.5%, 16 ± 4.3% for β, 16 ± 6.1% for λ, and 10 ± 1.4% for θ frequencies during sleep. The mean absolute error for heart rate, breathing frequency, and RRV was 1.2 ± 0.5 bpm, 0.3 ± 0.2 cpm, and 3.2 ± 0.6%, respectively. Automatic sleep staging reached an overall accuracy of 83.5 ± 6.4% (F1 score: 83.8 ± 6.3) for the DH to be compared with an average of 86.4 ± 8.0% (F1 score: 86.3 ± 7.4) for the 5 sleep experts.

Conclusions

These results demonstrate the capacity of the DH to both monitor sleep-related physiological signals and process them accurately into sleep stages. This device paves the way for, large-scale, longitudinal sleep studies.

Clinical Trial Registration

NCT03725943.

Dreem Open Datasets: Multi-Scored Sleep Datasets to Compare Human and Automated Sleep Staging

Sleep stage classification constitutes an important element of sleep disorder diagnosis. It relies on the visual inspection of polysomnography records by trained sleep technologists. Automated approaches have been designed to alleviate this resource-intensive task. However, such approaches are usually compared to a single human scorer annotation despite an inter-rater agreement of about 85% only. The present study introduces two publicly-available datasets, DOD-H including 25 healthy volunteers and DOD-O including 55 patients suffering from obstructive sleep apnea (OSA). Both datasets have been scored by 5 sleep technologists from different sleep centers. We developed a framework to compare automated approaches to a consensus of multiple human scorers. Using this framework, we benchmarked and compared the main literature approaches to a new deep learning method, SimpleSleepNet, which reach state-of-the-art performances while being more lightweight. We demonstrated that many methods can reach human-level performance on both datasets. SimpleSleepNet achieved an F1 of 89.9% vs 86.8% on average for human scorers on DOD-H, and an F1 of 88.3% vs 84.8% on DOD-O. Our study highlights that state-of-the-art automated sleep staging outperforms human scorers performance for healthy volunteers and patients suffering from OSA. Considerations could be made to use automated approaches in the clinical setting.

1211 Assessing the Accuracy of a Dry-EEG Headband for Measuring Brain Activity, Heart Rate, Breathing and Automatic Sleep Staging

Introduction

The development of new sleep study devices, adapted for daily use, is necessary for diagnosis of sleep disorders. However, this requires to be both suitable for daily use and capable of recording accurate electrophysiological data. This study assesses the signal acquisition of a comfortable sleep headband, using dry electrodes, and the performance of its automatic sleep staging algorithms compared to the gold-standard clinical PSG scored by 4 sleep experts.

Methods

42 participants slept at a sleep center wearing both the Dreem headband (DH) and a PSG simultaneously. We measured 1) the EEG signal similarity between both devices, 2) heart rate, breathing frequency and respiration rate variability (RRV) agreement, and 3) the performance of the headband automatic sleep scoring compared to PSG sleep experts manual scoring.

Results

Results demonstrate a strong correlation between the EEG signals acquired by the headband and those from the PSG, and the signals acquired by the headband enable monitoring of alpha (r= 0.75 ± 0.11), beta (r= 0.74 ± 0.14), delta (r = 0.78 ± 0.16), and theta (r = 0.63 ± 0.15) frequencies during sleep. The mean absolute error for heart rate, breathing frequency, and RRV was 2.2 ± 0.8 bpm, 0.3 ± 0.2 cpm and 3.1 ± 0.4 %, respectively. Automatic Sleep Staging reached an overall accuracy of 84.1 ± 7.5% (F1 score: 83.0 ± 8.4) for the headband to be compared with an average of 86.4 ± 5.5% (F1 score: 86.5 ± 5.5) for the 4 sleep experts.

Conclusion

These results demonstrate the capacity of the headband to both precisely monitor sleep-related physiological signals and process them accurately into sleep stages. This device paves the way for high-quality, large-scale, longitudinal sleep studies.

Support

This Study has been supported by Dreem sas.

0352 A Large-Scale EEG Study at Home to Objectivise Effects of Ageing on Slow Wave Sleep and Process S

Introduction

Several studies have shown slow wave sleep (SWS) is altered with ageing. However, most of these studies have been conducted in-lab and usually over a single night. In this study, we assessed the evolution of process S with ageing by analysing the dynamics of endogenous and auditory-evoked slow waves in a large population.

Methods

300 participants (200 M, 20 - 70 y.o.) were selected from volunteers users wearing a sleep headband for at least 3 nights, meeting the criteria of high signal quality and having no subjective sleep complaints nor being shift-workers. The Dreem headband is a connected device able to monitor EEG signals as well as pulse and movement and performs sleep staging in real-time automatically. Slow waves were detected as large negative deflections on the filtered EEG signals during NREM sleep. The auditory evoked slow waves were done using a previously validated closed-loop procedure.

Results

In our study, age was strongly correlated with N3 sleep duration (r=-0.34, p<0.0001), slow wave amplitude (r=-0.25, p<0.0001), and slow wave density (r=-0.40, p<0.0001). The slope of the slow wave activity, representing the process S here, was significantly decreased (r=-0.32, p<0.0001). This effect was mainly due to changes in the density of slow waves in the first 2 hours of sleep (r=-0.41, p<0.0001). Finally, our results show a decrease in the probability of auditory evoked slow waves (r=-0.43, p<0.0001).

Conclusion

These results confirmed the in-lab studies showing a heterogeneous alteration of homoeostatic process S with age, as well as a general decrease of slow wave occurrences, that is observed in parallel of a decrease of the probability of evoking slow waves, suggesting a global change in the system responsible for slow wave generation.

Support

This study was supported by Dreem sas and ANR, FLAG ERA 2015, HPB SLOW-Dyn

0546 Sleep Science at Home - Delivering Sleep Assessment and Digital CBT-i at Scale

Introduction

Cognitive-behavioral therapy for insomnia (CBT-I) is the current first-line treatment for insomnia disorder, recommended by the AASM and SRS. Digital versions of CBT-I have been developed and validated to address the need for implementation at scale but still suffer from poor accessibility and compliance. Therefore, the aim of this open-label, Real-World Study (RWS) was to assess the engagement and efficacy of a next-generation CBT-i 6-weeks program.

Methods

1304 subjects were included in the analysis between Dec 23rd, 2018 and December, 14th 2019. The main inclusion criteria were having an Insomnia Severity Index ISI ≥ 15 and completion of one week of Dreem program. The variables have been measured by the Dreem headband (DH) for objective variables, and on subjects’ answers to questionnaires for subjective ones.

Results

The retention during this RWS was 70.4 % (Pre: n = 1304 and Week 4: n = 935). The program led to a clinically significant decrease of 7.42 points on the ISI (p < 0.001). The obj-WASO was reduced by 35% (n = 359, p < 0.001), obj-Awakenings were reduced by 37% (n = 359 p < 0.001), obj-SE was increased by 2.56 points (n = 305, p < 0.001) and obj-SOL was reduced by 22% (n = 359, p < 0.001). The subj-SOL was reduced by 41% (n = 176, p < 0.001), subj-SE was increased by 8.9 points (n =168, p < 0.001), subj-SD was increased by 16% (baseline: 307.50 ± 88.86 min; post 357.07 ± 91.24 min, subj-SD (n = 174, p < 0.001).

Conclusion

The results of this RWS suggest this insomnia program has a high engagement compared to other digital CBT-I programs and is as effective as traditional in-person CBT-I. This new generation of Insomnia therapy combining hardware, software and therapist serves as an efficient and engaging treatment implementable at scale.

Support

This study has been supported by Dreem sas.

1210 Development Of An Auditory Neurofeedback During Sleep Onset Process

Introduction

Biofeedback is proposed as an alternative method to help patients with insomnia reducing their anxiety. Some studies have shown that auditory neurofeedback can be effective at reducing sleep-onset latency. However, the AASM sleep stage classification only describes the sleep-onset as a binary state (i.e. wake or N1) which makes it not adapted for neurofeedback. We introduced a simple 4-stages classification for sleep-onset, on 10 seconds EEG epoch. The aim of this study was to develop an automatic method to detect these stages, and an online algorithm embedded in the Dreem headband (DH) that adapted the auditory feedback based on the current stage.

Methods

Fourteen subjects underwent an overnight PSG monitoring, from which the first sleep-onset period was extracted. We defined the simple 4-stages classification for sleep-onset on 10 seconds EEG epoch as following: SO1) > 75% of the epoch covered by alpha frequencies SO2) between 25% and 50% of the window covered with alpha frequencies, SO3) Alpha frequencies covered less than 25% and theta frequencies covered less than 30% of the epoch, and SO4) Theta frequency covered more than 30% of the epoch. For the manual scoring, 4 sleep scorers have been given the instructions and a Q&A session after scoring the first two records. For the algorithm, a sound triggering algorithm was linked to a neural network trained on the scored data, to dynamically adapt the sound to the sleep-onset stage.

Results

The scorers reached an average agreement of 68 + 15% over all the records. The neural network reached an accuracy of 68%. Per state the accuracy was: 71 ± 32% (S1), 52 ± 22% (S2), 54 ± 23% (S3), 79 ± 21% (S4). The automatic neurofeedback was able to adapt sound stimulations in real-time based on stages and was well perceived among first testers.

Conclusion

The results of this preliminary work show that we can reach a higher agreement by reducing the epoch duration and use this classification to produce automatic biofeedback during the sleep onset period. Further studies using a data-driven method should be conducted.

Support

This study supported by Dreem sas.

0616 Validation of a Sleep Headband for Detecting Obstructive Sleep Apnea

Introduction

Polysomnography (PSG) is the gold-standard to diagnose obstructive sleep apnea (OSA). OSA severity diagnosis is defined by the apnea-hypopnea index (AHI) defined as the number of apnea and hypopnea events measured per hour of sleep. The Dreem2 headband (DH) is a self-administered, easy to use device that measure EEG, breathing frequency, heart rate and sound at-home. In our study, we assessed the performance of the DH to automatically detects OSA compared to 3 sleep’s experts scoring on PSG.

Methods

41 subjects (8 females, 42.6 ± 13.7 y.o.) having a suspicion of OSA performed a night at-home wearing both a PSG and the DH. Each PSG record was scored for apnea and hypopnea events by 3 independent trained sleep experts following AASM guidelines. The deep learning approach DOSED, was trained on the DH signals using the manual apnea scoring. 10-fold cross-validation was used to provide predictions for each of the 41 subjects with the DH.

Results

We observed an average AHI expert’s scoring of 13.6 ± 10.1 CI[10.5, 16.5] compared to 12.9 ± 10.3 CI[9.6, 15.8] for the DH. Both, the correlation between the 3 scorers (r= 0.88, p < 0.001) and the DH and the scorers (r=0.79, p< 0.001) were significant. The specificity and sensitivity to detect mild OSA (AHI ≤ 5) was 84.4 % and 96.4 % for the DH and 86.5 % and 86.0% for the scorers.

Conclusion

The results show that the DH using deep learning can detect OSA with an accuracy similar to the sleep experts. The use of DH paves the way for longitudinal monitoring of patients with a suspicion of OSA and its accessibility could lead to better screening of the general population.

Support

This Study has been supported by Dreem sas.

AI vs Humans for the diagnosis of sleep apnea

Polysomnography (PSG) is the gold standard for diagnosing sleep obstructive apnea (OSA). It allows monitoring of breathing events throughout the night. The detection of these events is usually done by trained sleep experts. However, this task is tedious, highly time-consuming and subject to important inter-scorer variability. In this study, we adapted our state-of-the-art deep learning method for sleep event detection, DOSED, to the detection of sleep breathing events in PSG for the diagnosis of OSA. We used a dataset of 52 PSG recordings with apnea-hypopnea event scoring from 5 trained sleep experts. We assessed the performance of the automatic approach and compared it to the inter-scorer performance for both the diagnosis of OSA severity and, at the microscale, for the detection of single breathing events. We observed that human sleep experts reached an average accuracy of 75% while the automatic approach reached 81% for sleep apnea severity diagnosis. The F1 score for individual event detection was 0.55 for experts and 0.57 for the automatic approach, on average. These results demonstrate that the automatic approach can perform at a sleep expert level for the diagnosis of OSA.

Towards a Flexible Deep Learning Method for Automatic Detection of Clinically Relevant Multi-Modal Events in the Polysomnogram

Much attention has been given to automatic sleep staging algorithms in past years, but the detection of discrete events in sleep studies is also crucial for precise characterization of sleep patterns and possible diagnosis of sleep disorders. We propose here a deep learning model for automatic detection and annotation of arousals and leg movements. Both of these are commonly seen during normal sleep, while an excessive amount of either is linked to disrupted sleep patterns, excessive daytime sleepiness impacting quality of life, and various sleep disorders. Our model was trained on 1,485 subjects and tested on 1,000 separate recordings of sleep. We tested two different experimental setups and found optimal arousal detection was attained by including a recurrent neural network module in our default model with a dynamic default event window (F1 = 0.75), while optimal leg movement detection was attained using a static event window (F1 = 0.65). Our work show promise while still allowing for improvements. Specifically, future research will explore the proposed model as a general-purpose sleep analysis model.

Performance of an Ambulatory Dry-EEG Device for Auditory Closed-Loop Stimulation of Sleep Slow Oscillations in the Home Environment

Recent research has shown that auditory closed-loop stimulation can enhance sleep slow oscillations (SO) to improve N3 sleep quality and cognition. Previous studies have been conducted in lab environments. The present study aimed to validate and assess the performance of a novel ambulatory wireless dry-EEG device (WDD), for auditory closed-loop stimulation of SO during N3 sleep at home. The performance of the WDD to detect N3 sleep automatically and to send auditory closed-loop stimulation on SO were tested on 20 young healthy subjects who slept with both the WDD and a miniaturized polysomnography (part 1) in both stimulated and sham nights within a double blind, randomized and crossover design. The effects of auditory closed-loop stimulation on delta power increase were assessed after one and 10 nights of stimulation on an observational pilot study in the home environment including 90 middle-aged subjects (part 2).The first part, aimed at assessing the quality of the WDD as compared to a polysomnograph, showed that the sensitivity and specificity to automatically detect N3 sleep in real-time were 0.70 and 0.90, respectively. The stimulation accuracy of the SO ascending-phase targeting was 45 ± 52°. The second part of the study, conducted in the home environment, showed that the stimulation protocol induced an increase of 43.9% of delta power in the 4 s window following the first stimulation (including evoked potentials and SO entrainment effect). The increase of SO response to auditory stimulation remained at the same level after 10 consecutive nights. The WDD shows good performances to automatically detect in real-time N3 sleep and to send auditory closed-loop stimulation on SO accurately. These stimulation increased the SO amplitude during N3 sleep without any adaptation effect after 10 consecutive nights. This tool provides new perspectives to figure out novel sleep EEG biomarkers in longitudinal studies and can be interesting to conduct broad studies on the effects of auditory stimulation during sleep.