Staging Sleep in Polysomnograms: Analysis of Inter-Scorer Variability

Personalization of Automatic Sleep Scoring: How Best to Adapt Models to Personal Domains in Wearable EEG

Wearable EEG enables us to capture large amounts of high-quality sleep data for diagnostic purposes. To make full use of this capacity we need high-performance automatic sleep scoring models. To this end, it has been noted that domain mismatch between recording equipment can be considerable, e.g. PSG to wearable EEG, but a previously observed benefit from personalizing models to individual subjects further indicates a personal domain in sleep EEG. In this work, we have investigated the extent of such a personal domain in wearable EEG, and review supervised and unsupervised approaches to personalization as found in the literature. We investigated the personalization effect of the unsupervised Adversarial Domain Adaptation and implemented an unsupervised method based on statistics alignment. No beneficial personalization effect was observed using these unsupervised methods. We find that supervised personalization leads to a substantial performance improvement on the target subject ranging from 15% Cohen's Kappa for subjects with poor performance ( ) to roughly 2% on subjects with high performance ( ). This improvement was present for models trained on both small and large data sets, indicating that even high-performance models benefit from supervised personalization. We found that this personalization can be beneficially regularized using Kullback-Leibler regularization, leading to lower variance with negligible cost to improvement. Based on the experiments, we recommend model personalization using Kullback-Leibler regularization.

Improved sleep stage predictions by deep learning of photoplethysmogram and respiration patterns

Sleep staging is a crucial tool for diagnosing and monitoring sleep disorders, but the standard clinical approach using polysomnography (PSG) in a sleep lab is time-consuming, expensive, uncomfortable, and limited to a single night. Advancements in sensor technology have enabled home sleep monitoring, but existing devices still lack sufficient accuracy to inform clinical decisions. To address this challenge, we propose a deep learning architecture that combines a convolutional neural network and bidirectional long short-term memory to accurately classify sleep stages. By supplementing photoplethysmography (PPG) signals with respiratory sensor inputs, we demonstrated significant improvements in prediction accuracy and Cohen's kappa (k) for 2- (92.7 %; k = 0.768), 3- (80.2 %; k = 0.714), 4- (76.8 %, k = 0.550), and 5-stage (76.7 %, k = 0.616) sleep classification using raw data. This relatively translatable approach, with a less intensive AI model and leveraging only a few, inexpensive sensors, shows promise in accurately staging sleep. This has potential for diagnosing and managing sleep disorders in a more accessible and practical manner, possibly even at home.

Sleep patterns and risk of chronic disease as measured by long-term monitoring with commercial wearable devices in the All of Us Research Program

Poor sleep health is associated with increased all-cause mortality and incidence of many chronic conditions. Previous studies have relied on cross-sectional and self-reported survey data or polysomnograms, which have limitations with respect to data granularity, sample size and longitudinal information. Here, using objectively measured, longitudinal sleep data from commercial wearable devices linked to electronic health record data from the All of Us Research Program, we show that sleep patterns, including sleep stages, duration and regularity, are associated with chronic disease incidence. Of the 6,785 participants included in this study, 71% were female, 84% self-identified as white and 71% had a college degree; the median age was 50.2 years (interquartile range = 35.7, 61.5) and the median sleep monitoring period was 4.5 years (2.5, 6.5). We found that rapid eye movement sleep and deep sleep were inversely associated with the odds of incident atrial fibrillation and that increased sleep irregularity was associated with increased odds of incident obesity, hyperlipidemia, hypertension, major depressive disorder and generalized anxiety disorder. Moreover, J-shaped associations were observed between average daily sleep duration and hypertension, major depressive disorder and generalized anxiety disorder. These findings show that sleep stages, duration and regularity are all important factors associated with chronic disease development and may inform evidence-based recommendations on healthy sleeping habits.

Refining sleep staging accuracy: Transfer learning coupled with scorability models

STUDY OBJECTIVES

This study aimed to 1) improve sleep staging accuracy through transfer learning, to achieve or exceede human inter-expert agreement; 2) introduce a scorability model to assess the quality and trustworthiness of automated sleep staging.

METHODS

A deep neural network (base model) was trained on a large multi-site polysomnography (PSG) dataset from the United States. Transfer learning was used to calibrate the model to a reduced montage and limited samples from the Korean Genome and Epidemiology Study (KoGES) dataset. Model performance was compared to inter-expert reliability among three human experts. A scorability assessment was developed to predict the agreement between the model and human experts.

RESULTS

Initial sleep staging by the base model showed lower agreement with experts (κ=0.55) compared to inter-expert agreement (κ=0.62). Calibration with 324 randomly sampled training cases matched expert agreement levels. Further targeted sampling improved performance, with models exceeding inter-expert agreement (κ=0.70). The scorability assessment, combining biosignal quality and model confidence features, predicted model-expert agreement moderately well (R²=0.42). Recordings with higher scorability scores demonstrated greater model-expert agreement than inter-expert agreement. Even with lower scorability scores, model performance was comparable to inter-expert agreement.

CONCLUSIONS

Fine-tuning a pre-trained neural network through targeted transfer learning significantly enhances sleep staging performance for an atypical montage, achieving and surpassing human expert agreement levels. The introduction of a scorability assessment provides a robust measure of reliability, ensuring quality control and enhancing the practical application of the system before deployment. This approach marks an important advancement in automated sleep analysis, demonstrating the potential for AI to exceed human performance in clinical settings.

Performance Investigation of Somfit Sleep Staging Algorithm

Purpose

To investigate accuracy of the sleep staging algorithm in a new miniaturized home sleep monitoring device - Compumedics® Somfit. Somfit is attached to patient's forehead and combines channels specified for a pulse arterial tonometry (PAT)-based home sleep apnea testing (HSAT) device with the neurological signals. Somfit sleep staging deep learning algorithm is based on convolutional neural network architecture.

Patients and Methods

One hundred and ten participants referred for sleep investigation with suspected or preexisting obstructive sleep apnea (OSA) in need of a review were enrolled into the study involving simultaneous recording of full overnight polysomnography (PSG) and Somfit data. The recordings were conducted at three centers in Australia. The reported statistics include standard measures of agreement between Somfit automatic hypnogram and consensus PSG hypnogram.

Results

Overall percent agreement across five sleep stages (N1, N2, N3, REM, and wake) between Somfit automatic and consensus PSG hypnograms was 76.14 (SE: 0.79). The percent agreements between different pairs of sleep technologists' PSG hypnograms varied from 74.36 (1.93) to 85.50 (0.64), with interscorer agreement being greater for scorers from the same sleep laboratory. The estimate of kappa between Somfit and consensus PSG was 0.672 (0.002). Percent agreement for sleep/wake discrimination was 89.30 (0.37). The accuracy of Somfit sleep staging algorithm varied with increasing OSA severity - percent agreement was 79.67 (1.87) for the normal subjects, 77.38 (1.06) for mild OSA, 74.83 (1.79) for moderate OSA and 72.93 (1.68) for severe OSA.

Conclusion

Agreement between Somfit and PSG hypnograms was non-inferior to PSG interscorer agreement for a number of scorers, thus confirming acceptability of electrode placement at the center of the forehead. The directions for algorithm improvement include additional arousal detection, integration of motion and oximetry signals and separate inference models for individual sleep stages.

Physiologic Consequences of Upper Airway Obstruction in Sleep Apnea

OSA is diagnosed and managed by a metric called the apnea-hypopnea index (AHI). The AHI quantifies the number of respiratory events (apnea or hypopnea), disregarding important information on the characteristics and physiologic consequences of respiratory events, including degrees of ventilatory deficit and associated hypoxemia, cardiac autonomic response, and cortical activity. The oversimplification of the disorder by the AHI is considered one of the reasons for divergent findings on the associations of OSA and cardiovascular disease (CVD) in observational and randomized controlled trial studies. Prospective observational cohort studies have demonstrated strong associations of OSA with several cardiovascular diseases, and randomized controlled trials of CPAP intervention have not been able to detect a benefit of CPAP to reduce the risk of CVD. Over the last several years, novel methodologies have been proposed to better quantify the magnitude of OSA-related breathing disturbance and its physiologic consequences. As a result, stronger associations with cardiovascular and neurocognitive outcomes have been observed. In this review, we focus on the methods that capture polysomnographic heterogeneity of OSA.

Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review

Purpose

This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain (ID) and out-of-domain (OOD) data, and considering subjects' diagnoses.

Patients and Methods

A total of 19,578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of an additional 8832 PSGs, covering a full spectrum of ages (0-91 years) and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician.

Results

U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve κ ≥ 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians' workload, and facilitating near-perfect agreement.

Conclusion

Inter-scorer variability limits the accuracy of the scoring algorithms to ~80%. By integrating an uncertainty estimation with U-Sleep, we enhance the review of predicted hypnograms, to align with the scoring taste of a responsible physician. Validated across ID and OOD data and various sleep-disorders, our approach offers a strategy to boost automated scoring tools' usability in clinical settings.

Revisiting the maintenance of wakefulness test: from intra-/inter-scorer agreement to normative values in patients treated for obstructive sleep apnea

The Maintenance of Wakefulness Test is widely used to objectively assess sleepiness and make safety-related decisions, but its interpretation is subjective and normative values remain debated. Our work aimed to determine normative thresholds in non-subjectively sleepy patients with well-treated obstructive sleep apnea, and to assess intra- and inter-scorer variability. We included maintenance of wakefulness tests of 141 consecutive patients with treated obstructive sleep apnea (90% men, mean (SD) age 47.5 (9.2) years, mean (SD) pre-treatment apnea-hypopnea index of 43.8 (20.3) events/h). Sleep onset latencies were independently scored by two experts. Discordant scorings were reviewed to reach a consensus and half of the cohort was double-scored by each scorer. Intra- and inter-scorer variability was assessed using Cohen's kappa for 40, 33, and 19 min mean sleep latency thresholds. Consensual mean sleep latencies were compared between four groups according to subjective sleepiness (Epworth Sleepiness Scale score < versus ≥11) and residual apnea-hypopnea index (< versus ≥15 events/h). In well-treated non-sleepy patients (n = 76), the consensual mean (SD) sleep latency was 38.4 (4.2) min (lower normal limit [mean - 2SD] = 30 min), and 80% of them did not fall asleep. Intra-scorer agreement on mean sleep latency was high but inter-scorer was only fair (Cohen's kappa 0.54 for 33-min threshold, 0.27 for 19-min threshold), resulting in changes in latency category in 4%-12% of patients. A higher sleepiness score but not the residual apnea-hypopnea index was significantly associated with a lower mean sleep latency. Our findings suggest a higher than usually accepted normative threshold (30 min) in this context and emphasise the need for more reproducible scoring approaches.

ZleepAnlystNet: a novel deep learning model for automatic sleep stage scoring based on single-channel raw EEG data using separating training

Numerous models for sleep stage scoring utilizing single-channel raw EEG signal have typically employed CNN and BiLSTM architectures. While these models, incorporating temporal information for sequence classification, demonstrate superior overall performance, they often exhibit low per-class performance for N1-stage, necessitating an adjustment of loss function. However, the efficacy of such adjustment is constrained by the training process. In this study, a pioneering training approach called separating training is introduced, alongside a novel model, to enhance performance. The developed model comprises 15 CNN models with varying loss function weights for feature extraction and 1 BiLSTM for sequence classification. Due to its architecture, this model cannot be trained using an end-to-end approach, necessitating separate training for each component using the Sleep-EDF dataset. Achieving an overall accuracy of 87.02%, MF1 of 82.09%, Kappa of 0.8221, and per-class F1-socres (W 90.34%, N1 54.23%, N2 89.53%, N3 88.96%, and REM 87.40%), our model demonstrates promising performance. Comparison with sleep technicians reveals a Kappa of 0.7015, indicating alignment with reference sleep stags. Additionally, cross-dataset validation and adaptation through training with the SHHS dataset yield an overall accuracy of 84.40%, MF1 of 74.96% and Kappa of 0.7785 when tested with the Sleep-EDF-13 dataset. These findings underscore the generalization potential in model architecture design facilitated by our novel training approach.

Evaluating reliability in wearable devices for sleep staging

Sleep is crucial for physical and mental health, but traditional sleep quality assessment methods have limitations. This scoping review analyzes 35 articles from the past decade, evaluating 62 wearable setups with varying sensors, algorithms, and features. Our analysis indicates a trend towards combining accelerometer and photoplethysmography (PPG) data for out-of-lab sleep staging. Devices using only accelerometer data are effective for sleep/wake detection but fall short in identifying multiple sleep stages, unlike those incorporating PPG signals. To enhance the reliability of sleep staging wearables, we propose five recommendations: (1) Algorithm validation with equity, diversity, and inclusion considerations, (2) Comparative performance analysis of commercial algorithms across multiple sleep stages, (3) Exploration of feature impacts on algorithm accuracy, (4) Consistent reporting of performance metrics for objective reliability assessment, and (5) Encouragement of open-source classifier and data availability. Implementing these recommendations can improve the accuracy and reliability of sleep staging algorithms in wearables, solidifying their value in research and clinical settings.

Multicentre sleep-stage scoring agreement in the Sleep Revolution project

Determining sleep stages accurately is an important part of the diagnostic process for numerous sleep disorders. However, as the sleep stage scoring is done manually following visual scoring rules there can be considerable variation in the sleep staging between different scorers. Thus, this study aimed to comprehensively evaluate the inter-rater agreement in sleep staging. A total of 50 polysomnography recordings were manually scored by 10 independent scorers from seven different sleep centres. We used the 10 scorings to calculate a majority score by taking the sleep stage that was the most scored stage for each epoch. The overall agreement for sleep staging was κ = 0.71 and the mean agreement with the majority score was 0.86. The scorers were in perfect agreement in 48% of all scored epochs. The agreement was highest in rapid eye movement sleep (κ = 0.86) and lowest in N1 sleep (κ = 0.41). The agreement with the majority scoring varied between the scorers from 81% to 91%, with large variations between the scorers in sleep stage-specific agreements. Scorers from the same sleep centres had the highest pairwise agreements at κ = 0.79, κ = 0.85, and κ = 0.78, while the lowest pairwise agreement between the scorers was κ = 0.58. We also found a moderate negative correlation between sleep staging agreement and the apnea-hypopnea index, as well as the rate of sleep stage transitions. In conclusion, although the overall agreement was high, several areas of low agreement were also found, mainly between non-rapid eye movement stages.

Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement

STUDY OBJECTIVES

To efficiently improve the scoring competency of scorers with varying levels of experience across regions in Taiwan, we developed a training program with a cloud-based polysomnography scoring platform to evaluate and improve interscorer agreement.

METHODS

A total of 70 scorers from 34 sleep centers in Taiwan (job tenure: 0.5-39.0 years) completed a scoring test. All scorers scored a 742-epoch (30 s/epoch) overnight polysomnography recording of a patient with a moderate apnea-hypopnea index. Subsequently, 8 scoring experts delivered 8 interactive online lectures (each lasting 30 minutes). The training program included identifying scoring weaknesses, highlighting the latest scoring rules, and providing physicians' perspectives. Afterward, the scorers completed the second scoring test on the same participant. Changes in agreement from the first to second scoring test were identified. Sleep staging, sleep parameters, and respiratory events were considered for evaluating scoring agreement.

RESULTS

The scorers' agreement in overall sleep stage scoring significantly increased from 74.6 to 82.3% (median score). The proportion of scorers with an agreement of ≥ 80% increased from 20.0% (14/70) to 58.6% (41/70) after the online training program. In addition, the scorers' agreement in overall respiratory-event scoring increased to 88.8% (median score) after training. The scorers with a job tenure of 2.0-4.9 years exhibited the highest level of improvement in overall sleep staging (their median agreement increased from 72.8 to 84.9%; P < .001).

CONCLUSIONS

Our interactive online training program efficiently targeted the scorers' scoring weaknesses identified in the first scoring test, leading to substantial improvements in scoring proficiency.

CITATION

Liao Y-S, Wu M-C, Li C-X, Lin W-K, Lin C-Y, Liang S-F. Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement. J Clin Sleep Med. 2024;20(2):271-278.

Evaluating Accuracy in Five Commercial Sleep-Tracking Devices Compared to Research-Grade Actigraphy and Polysomnography

The development of consumer sleep-tracking technologies has outpaced the scientific evaluation of their accuracy. In this study, five consumer sleep-tracking devices, research-grade actigraphy, and polysomnography were used simultaneously to monitor the overnight sleep of fifty-three young adults in the lab for one night. Biases and limits of agreement were assessed to determine how sleep stage estimates for each device and research-grade actigraphy differed from polysomnography-derived measures. Every device, except the Garmin Vivosmart, was able to estimate total sleep time comparably to research-grade actigraphy. All devices overestimated nights with shorter wake times and underestimated nights with longer wake times. For light sleep, absolute bias was low for the Fitbit Inspire and Fitbit Versa. The Withings Mat and Garmin Vivosmart overestimated shorter light sleep and underestimated longer light sleep. The Oura Ring underestimated light sleep of any duration. For deep sleep, bias was low for the Withings Mat and Garmin Vivosmart while other devices overestimated shorter and underestimated longer times. For REM sleep, bias was low for all devices. Taken together, these results suggest that proportional bias patterns in consumer sleep-tracking technologies are prevalent and could have important implications for their overall accuracy.

Neurophysiological Parameters Influencing Sleep-Wake Discrepancy in Insomnia Disorder: A Preliminary Analysis on Alpha Rhythm during Sleep Onset

Sleep state misperception (SSM) is a common issue in insomnia disorder (ID), causing a discrepancy between objective and subjective sleep/wake time estimation and increased daytime impairments. In this context, the hyperarousal theory assumes that sustained central nervous system activation contributes to the SSM. This study investigates factors influencing SSM during sleep latency (SL) and total sleep time (TST). Objective polysomnographic sleep variables (the alpha density index, latency-to-sleep stages and the first K-complex, and Rapid Eye Movement (REM) arousal density) and subjective sleep indices, taken from sleep diaries, were analyzed in 16 ID patients. Correlation analyses revealed a positive association between the degree of SL misperception (SLm) and the percentage of epochs that contained a visually scored stereotyped alpha rhythm during objective SL. A regression analysis showed that the REM arousal density and alpha density index significantly predicted TST misperception (TSTm). Furthermore, the degree of SLm was associated with an increased probability of transitioning from stage 1 of non-REM sleep to wakefulness during subjective SL. These findings support the role of hyperarousal in SSM and highlight the importance of alpha activity in unravelling the heterogeneous underpinnings of SSM.

A deep learning algorithm model to automatically score and grade obstructive sleep apnea in adult polysomnography

Objective

Polysomnography (PSG) is unique in diagnosing sleep disorders, notably obstructive sleep apnea (OSA). Despite its advantages, manual PSG data grading is time-consuming and laborious. Thus, this research evaluated a deep learning-based automated scoring system for respiratory events in sleep-disordered breathing patients.

Methods

A total of 1000 case PSG data were enrolled to develop a deep learning algorithm. Of the 1000 data, 700 were distributed for training, 200 for validation, and 100 for testing. The respiratory events scoring deep learning model is composed of five sequential layers: an initial layer of perceptrons, followed by three consecutive layers of long short-term memory cells, and ultimately, an additional two layers of perceptrons.

Results

The PSG data of 100 patients (simple snoring, mild, moderate, and severe OSA; n = 25 in each group) were selected for validation and testing of the deep learning model. The algorithm demonstrated high sensitivity (95% CI: 98.06-98.51) and specificity (95% CI: 95.46-97.79) across all OSA severities in detecting apnea/hypopnea events, compared to manual PSG analysis. The deep learning model's area under the curve values for predicting OSA in apnea-hypopnea index ≥ 5, 15, and 30 groups were 0.9402, 0.9388, and 0.9442, respectively, showing no significant differences between each group.

Conclusion

The deep learning algorithm employed in our study showed high accuracy in identifying apnea/hypopnea episodes and assessing the severity of OSA, suggesting the potential for enhancing both the efficiency and accuracy of automated respiratory event scoring in PSG through advanced deep learning techniques.

Standardized image-based polysomnography database and deep learning algorithm for sleep-stage classification

STUDY OBJECTIVES

Polysomnography (PSG) scoring is labor-intensive, subjective, and often ambiguous. Recently several deep learning (DL) models for automated sleep scoring have been developed, they are tied to a fixed amount of input channels and resolution. In this study, we constructed a standardized image-based PSG dataset in order to overcome the heterogeneity of raw signal data obtained from various PSG devices and various sleep laboratory environments.

METHODS

All individually exported European data format files containing raw signals were converted into images with an annotation file, which contained the demographics, diagnoses, and sleep statistics. An image-based DL model for automatic sleep staging was developed, compared with a signal-based model, and validated in an external dataset.

RESULTS

We constructed 10253 image-based PSG datasets using a standardized format. Among these, 7745 diagnostic PSG data were used to develop our DL model. The DL model using the image dataset showed similar performance to the signal-based dataset for the same subject. The overall DL accuracy was greater than 80%, even with severe obstructive sleep apnea. Moreover, for the first time, we showed explainable DL in the field of sleep medicine as visualized key inference regions using Eigen-class activation maps. Furthermore, when a DL model for sleep scoring performs external validation, we achieved a relatively good performance.

CONCLUSIONS

Our main contribution demonstrates the availability of a standardized image-based dataset, and highlights that changing the data sampling rate or number of sensors may not require retraining, although performance decreases slightly as the number of sensors decreases.

Enhanced sleep staging with artificial intelligence: a validation study of new software for sleep scoring

Manual sleep staging (MSS) using polysomnography is a time-consuming task, requires significant training, and can lead to significant variability among scorers. STAGER is a software program based on machine learning algorithms that has been developed by Medibio Limited (Savage, MN, USA) to perform automatic sleep staging using only EEG signals from polysomnography. This study aimed to extensively investigate its agreement with MSS performed during clinical practice and by three additional expert sleep technicians. Forty consecutive polysomnographic recordings of patients referred to three US sleep clinics for sleep evaluation were retrospectively collected and analyzed. Three experienced technicians independently staged the recording using the electroencephalography, electromyography, and electrooculography signals according to the American Academy of Sleep Medicine guidelines. The staging initially performed during clinical practice was also considered. Several agreement statistics between the automatic sleep staging (ASS) and MSS, among the different MSSs, and their differences were calculated. Bootstrap resampling was used to calculate 95% confidence intervals and the statistical significance of the differences. STAGER's ASS was most comparable with, or statistically significantly better than the MSS, except for a partial reduction in the positive percent agreement in the wake stage. These promising results indicate that STAGER software can perform ASS of inpatient polysomnographic recordings accurately in comparison with MSS.

Local field potentials and single unit dynamics in motor cortex of unconstrained macaques during different behavioral states

Different sleep stages have been shown to be vital for a variety of brain functions, including learning, memory, and skill consolidation. However, our understanding of neural dynamics during sleep and the role of prominent LFP frequency bands remain incomplete. To elucidate such dynamics and differences between behavioral states we collected multichannel LFP and spike data in primary motor cortex of unconstrained macaques for up to 24 h using a head-fixed brain-computer interface (Neurochip3). Each 8-s bin of time was classified into awake-moving (Move), awake-resting (Rest), REM sleep (REM), or non-REM sleep (NREM) by using dimensionality reduction and clustering on the average spectral density and the acceleration of the head. LFP power showed high delta during NREM, high theta during REM, and high beta when the animal was awake. Cross-frequency phase-amplitude coupling typically showed higher coupling during NREM between all pairs of frequency bands. Two notable exceptions were high delta-high gamma and theta-high gamma coupling during Move, and high theta-beta coupling during REM. Single units showed decreased firing rate during NREM, though with increased short ISIs compared to other states. Spike-LFP synchrony showed high delta synchrony during Move, and higher coupling with all other frequency bands during NREM. These results altogether reveal potential roles and functions of different LFP bands that have previously been unexplored.

An approach for determining the reliability of manual and digital scoring of sleep stages

STUDY OBJECTIVES

Inter-scorer variability in sleep staging is largely due to equivocal epochs that contain features of more than one stage. We propose an approach that recognizes the existence of equivocal epochs and evaluates scorers accordingly.

METHODS

Epoch-by-epoch staging was performed on 70 polysomnograms by six qualified technologists and by a digital system (Michele Sleep Scoring [MSS]). Probability that epochs assigned the same stage by only two of the six technologists (minority score) resulted from random occurrence of two errors was calculated and found to be <5%, thereby indicating that the stage assigned is an acceptable variant for the epoch. Acceptable stages were identified in each epoch as stages assigned by at least two technologists. Percent agreement between each technologist and the other five technologists, acting as judges, was determined. Agreement was considered to exist if the stage assigned by the tested scorer was one of the acceptable stages for the epoch. Stage assigned by MSS was likewise considered in agreement if included in the acceptable stages made by the technologists.

RESULTS

Agreement of technologists tested against five qualified judges increased from 80.8% (range 70.5%-86.4% among technologists) when using the majority rule, to 96.1 (89.8%-98.5%) by the proposed approach. Agreement between unedited MSS and same judges was 90.0% and increased to 92.1% after brief editing.

CONCLUSIONS

Accounting for equivocal epochs provides a more accurate estimate of a scorer's (human or digital) competence in scoring sleep stages and reduces inter-scorer disagreements. The proposed approach can be implemented in sleep-scoring training and accreditation programs.

Decentralized Data-Privacy Preserving Deep-Learning Approaches for Enhancing Inter-Database Generalization in Automatic Sleep Staging

Automatic sleep staging has been an active field of development. Despite multiple efforts, the area remains a focus of research interest. Indeed, while promising results have reported in past literature, uptake of automatic sleep scoring in the clinical setting remains low. One of the current issues regards the difficulty to generalization performance results beyond the local testing scenario, i.e. across data from different clinics. Issues derived from data-privacy restrictions, that generally apply in the medical domain, pose additional difficulties in the successful development of these methods. We propose the use of several decentralized deep-learning approaches, namely ensemble models and federated learning, for robust inter-database performance generalization and data-privacy preservation in automatic sleep staging scenario. Specifically, we explore four ensemble combination strategies (max-voting, output averaging, size-proportional weighting, and Nelder-Mead) and present a new federated learning algorithm, so-called sub-sampled federated stochastic gradient descent (ssFedSGD). To evaluate generalization capabilities of such approaches, experimental procedures are carried out using a leaving-one-database-out direct-transfer scenario on six independent and heterogeneous public sleep staging databases. The resulting performance is compared with respect to two baseline approaches involving single-database and centralized multiple-database derived models. Our results show that proposed decentralized learning methods outperform baseline local approaches, and provide similar generalization results to centralized database-combined approaches. We conclude that these methods are more preferable choices, as they come with additional advantages concerning improved scalability, flexible design, and data-privacy preservation.

Long-term ear-EEG monitoring of sleep - A case study during shift work

The interest in sleep as a potential clinical biomarker is growing, but the standard method of sleep assessment, polysomnography, is expensive, time consuming, and requires a lot of expert assistance for both set-up and interpretation. To make sleep analysis more available both in research and in the clinic, there is a need for a reliable wearable device for sleep staging. In this case study, we test ear-electroencephalography. A wearable, where electrodes are placed in the outer ear, as a platform for longitudinal at-home recording of sleep. We explore the usability of the ear-electroencephalography in a shift work case with alternating sleep conditions. We find the ear-electroencephalography platform to be reliable both in terms of showing substantial agreement to polysomnography after long-time use (with an overall agreement, using Cohen's kappa, of 0.72) and by being unobtrusive enough to wear during night shift conditions. We find that fractions of non-rapid eye movement sleep and transition probability between sleep stages show great potential as sleep metrics when exploring quantitative differences in sleep architecture between shifting sleep conditions. This study shows that the ear-electroencephalography platform holds great potential as a reliable wearable for quantifying sleep "in the wild", pushing this technology further towards clinical adaptation.

Challenges of Applying Automated Polysomnography Scoring at Scale

Automatic polysomnography analysis can be leveraged to shorten scoring times, reduce associated costs, and ultimately improve the overall diagnosis of sleep disorders. Multiple and diverse strategies have been attempted for implementation of this technology at scale in the routine workflow of sleep centers. The field, however, is complex and presents unsolved challenges in a number of areas. Recent developments in computer science and artificial intelligence are nevertheless closing the gap. Technological advances are also opening new pathways for expanding our current understanding of the domain and its analysis.

MSED: A Multi-Modal Sleep Event Detection Model for Clinical Sleep Analysis

Clinical sleep analysis require manual analysis of sleep patterns for correct diagnosis of sleep disorders. However, several studies have shown significant variability in manual scoring of clinically relevant discrete sleep events, such as arousals, leg movements, and sleep disordered breathing (apneas and hypopneas). We investigated whether an automatic method could be used for event detection and if a model trained on all events (joint model) performed better than corresponding event-specific models (single-event models). We trained a deep neural network event detection model on 1653 individual recordings and tested the optimized model on 1000 separate hold-out recordings. F1 scores for the optimized joint detection model were 0.70, 0.63, and 0.62 for arousals, leg movements, and sleep disordered breathing, respectively, compared to 0.65, 0.61, and 0.60 for the optimized single-event models. Index values computed from detected events correlated positively with manual annotations (r2 = 0.73, r2 = 0.77, r2 = 0.78, respectively). We furthermore quantified model accuracy based on temporal difference metrics, which improved overall by using the joint model compared to single-event models. Our automatic model jointly detects arousals, leg movements and sleep disordered breathing events with high correlation with human annotations. Finally, we benchmark against previous state-of-the-art multi-event detection models and found an overall increase in F1 score with our proposed model despite a 97.5% reduction in model size.

Evaluation of consensus sleep stage scoring of dysregulated sleep in Parkinson's disease

OBJECTIVE

Sleep dysregulation in Parkinson's disease (PD) has been hypothesized to occur, in part, from dysfunction in the basal ganglia-cortical circuit. Assessment of this relationship requires accurate sleep stage determination, a known challenge in this clinical population. Our objective was to optimize the consensus on the sleep staging process and reduce interrater variability in a cohort of advanced PD subjects.

METHODS

Fifteen PD subjects were enrolled from three sites in a clinical trial that involved recordings from subthalamic nucleus (STN) deep brain stimulation (DBS) leads (NCT04620551). Video polysomnography (vPSG) data for a total of 45 nights were analyzed. Four experienced scorers independently scored data on initial review. Epochs with less than 75% consensus were flagged for secondary review. In secondary review of discordant epochs, two of the original scorers re-assessed epochs, from which the final consensus stage was derived.

RESULTS

Sleep stage classification agreement averaged 83.10% across all sleep stages on initial scoring (IS), and on secondary consensus scoring (CS) review, agreement reached 96.58%. Greatest disagreement was noted in determination of awake epochs (33.6% of discordant epochs) and non-rapid-eye-movement stage 2 (N2) epochs (31.8% of discordant epochs). Scoring discrepancy was resolved with direct measurement of cortical frequency and amplitudes, physiologic context of the epoch, and video review.

CONCLUSION

Our method of multi-level initial and then secondary consensus review scoring resulted in consensus scoring agreement superior to conventional standards. This work features a custom-engineered vPSG software and review platform for integration of consensus sleep stage scoring in a multi-site clinical trial.

AI-Driven sleep staging from actigraphy and heart rate

Sleep is an important indicator of a person's health, and its accurate and cost-effective quantification is of great value in healthcare. The gold standard for sleep assessment and the clinical diagnosis of sleep disorders is polysomnography (PSG). However, PSG requires an overnight clinic visit and trained technicians to score the obtained multimodality data. Wrist-worn consumer devices, such as smartwatches, are a promising alternative to PSG because of their small form factor, continuous monitoring capability, and popularity. Unlike PSG, however, wearables-derived data are noisier and far less information-rich because of the fewer number of modalities and less accurate measurements due to their small form factor. Given these challenges, most consumer devices perform two-stage (i.e., sleep-wake) classification, which is inadequate for deep insights into a person's sleep health. The challenging multi-class (three, four, or five-class) staging of sleep using data from wrist-worn wearables remains unresolved. The difference in the data quality between consumer-grade wearables and lab-grade clinical equipment is the motivation behind this study. In this paper, we present an artificial intelligence (AI) technique termed sequence-to-sequence LSTM for automated mobile sleep staging (SLAMSS), which can perform three-class (wake, NREM, REM) and four-class (wake, light, deep, REM) sleep classification from activity (i.e., wrist-accelerometry-derived locomotion) and two coarse heart rate measures-both of which can be reliably obtained from a consumer-grade wrist-wearable device. Our method relies on raw time-series datasets and obviates the need for manual feature selection. We validated our model using actigraphy and coarse heart rate data from two independent study populations: the Multi-Ethnic Study of Atherosclerosis (MESA; N = 808) cohort and the Osteoporotic Fractures in Men (MrOS; N = 817) cohort. SLAMSS achieves an overall accuracy of 79%, weighted F1 score of 0.80, 77% sensitivity, and 89% specificity for three-class sleep staging and an overall accuracy of 70-72%, weighted F1 score of 0.72-0.73, 64-66% sensitivity, and 89-90% specificity for four-class sleep staging in the MESA cohort. It yielded an overall accuracy of 77%, weighted F1 score of 0.77, 74% sensitivity, and 88% specificity for three-class sleep staging and an overall accuracy of 68-69%, weighted F1 score of 0.68-0.69, 60-63% sensitivity, and 88-89% specificity for four-class sleep staging in the MrOS cohort. These results were achieved with feature-poor inputs with a low temporal resolution. In addition, we extended our three-class staging model to an unrelated Apple Watch dataset. Importantly, SLAMSS predicts the duration of each sleep stage with high accuracy. This is especially significant for four-class sleep staging, where deep sleep is severely underrepresented. We show that, by appropriately choosing the loss function to address the inherent class imbalance, our method can accurately estimate deep sleep time (SLAMSS/MESA: 0.61±0.69 hours, PSG/MESA ground truth: 0.60±0.60 hours; SLAMSS/MrOS: 0.53±0.66 hours, PSG/MrOS ground truth: 0.55±0.57 hours;). Deep sleep quality and quantity are vital metrics and early indicators for a number of diseases. Our method, which enables accurate deep sleep estimation from wearables-derived data, is therefore promising for a variety of clinical applications requiring long-term deep sleep monitoring.

Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring

STUDY OBJECTIVES

To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers.

METHODS

We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6-12 scorers, to compare sleep stage probabilities (hypnodensity; i.e. the probability of each sleep stage being assigned to a given epoch) as the primary output, as well as a single sleep stage per epoch assigned by hierarchical majority rule.

RESULTS

The percentage of epochs with 100% agreement across scorers was 46 ± 9%, 38 ± 10% and 32 ± 9% for the datasets with 6, 9, and 12 scorers, respectively. The mean intra-class correlation coefficient between sleep stage probabilities from auto- and manual-scoring was 0.91, representing excellent reliability. Within each dataset, agreement between auto-scoring and consensus manual-scoring was significantly higher than agreement between manual-scoring and consensus manual-scoring (0.78 vs. 0.69; 0.74 vs. 0.67; and 0.75 vs. 0.67; all p < 0.01).

CONCLUSIONS

Analysis of scoring performed by multiple scorers reveals that sleep stage ambiguity is the rule rather than the exception. Probabilities of the sleep stages determined by artificial intelligence auto-scoring provide an excellent estimate of this ambiguity. Compared to consensus manual-scoring, sleep staging derived from auto-scoring is for each individual PSG noninferior to manual-scoring meaning that auto-scoring output is ready for interpretation without the need for manual adjustment.

Interpretation and further development of the hypnodensity representation of sleep structure

Objective.The recently-introduced hypnodensity graph provides a probability distribution over sleep stages per data window (i.e. an epoch). This work explored whether this representation reveals continuities that can only be attributed to intra- and inter-rater disagreement of expert scorings, or also to co-occurrence of sleep stage-dependent features within one epoch.Approach.We proposed a simplified model for time series like the ones measured during sleep, and a second model to describe the annotation process by an expert. Generating data according to these models, enabled controlled experiments to investigate the interpretation of the hypnodensity graph. Moreover, the influence of both the supervised training strategy, and the used softmax non-linearity were investigated. Polysomnography recordings of 96 healthy sleepers (of which 11 were used as independent test set), were subsequently used to transfer conclusions to real data.Main results.A hypnodensity graph, predicted by a supervised neural classifier, represents the probability with which the sleep expert(s) assigned a label to an epoch. It thus reflects annotator behavior, and is thereby only indirectly linked to the ratio of sleep stage-dependent features in the epoch. Unsupervised training was shown to result in hypnodensity graph that were slightly less dependent on this annotation process, resulting in, on average, higher-entropy distributions over sleep stages (H_unsupervised= 0.41 versusH_supervised= 0.29). Moreover, pre-softmax predictions were, for both training strategies, found to better reflect the ratio of sleep stage-dependent characteristics in an epoch, as compared to the post-softmax counterparts (i.e. the hypnodensity graph). In real data, this was observed from the linear relation between pre-softmax N3 predictions and the amount of delta power.Significance.This study provides insights in, and proposes new, representations of sleep that may enhance our comprehension about sleep and sleep disorders.

Screening the risk of obstructive sleep apnea by utilizing supervised learning techniques based on anthropometric features and snoring events

Objectives

Obstructive sleep apnea (OSA) is typically diagnosed by polysomnography (PSG). However, PSG is time-consuming and has some clinical limitations. This study thus aimed to establish machine learning models to screen for the risk of having moderate-to-severe and severe OSA based on easily acquired features.

Methods

We collected PSG data on 3529 patients from Taiwan and further derived the number of snoring events. Their baseline characteristics and anthropometric measures were obtained, and correlations among the collected variables were investigated. Next, six common supervised machine learning techniques were utilized, including random forest (RF), extreme gradient boosting (XGBoost), k-nearest neighbor (kNN), support vector machine (SVM), logistic regression (LR), and naïve Bayes (NB). First, data were independently separated into a training and validation dataset (80%) and a test dataset (20%). The approach with the highest accuracy in the training and validation phase was employed to classify the test dataset. Next, feature importance was investigated by calculating the Shapley value of every factor, which represented the impact on OSA risk screening.

Results

The RF produced the highest accuracy (of >70%) in the training and validation phase in screening for both OSA severities. Hence, we employed the RF to classify the test dataset, and results showed a 79.32% accuracy for moderate-to-severe OSA and 74.37% accuracy for severe OSA. Snoring events and the visceral fat level were the most and second most essential features of screening for OSA risk.

Conclusions

The established model can be considered for screening for the risk of having moderate-to-severe or severe OSA.

An accessible and versatile deep learning-based sleep stage classifier

Manual sleep scoring for research purposes and for the diagnosis of sleep disorders is labor-intensive and often varies significantly between scorers, which has motivated many attempts to design automatic sleep stage classifiers. With the recent introduction of large, publicly available hand-scored polysomnographic data, and concomitant advances in machine learning methods to solve complex classification problems with supervised learning, the problem has received new attention, and a number of new classifiers that provide excellent accuracy. Most of these however have non-trivial barriers to use. We introduce the Greifswald Sleep Stage Classifier (GSSC), which is free, open source, and can be relatively easily installed and used on any moderately powered computer. In addition, the GSSC has been trained to perform well on a large variety of electrode set-ups, allowing high performance sleep staging with portable systems. The GSSC can also be readily integrated into brain-computer interfaces for real-time inference. These innovations were achieved while simultaneously reaching a level of accuracy equal to, or exceeding, recent state of the art classifiers and human experts, making the GSSC an excellent choice for researchers in need of reliable, automatic sleep staging.

Benchmarking performance of an automatic polysomnography scoring system in a population with suspected sleep disorders

Aim

The current gold standard for measuring sleep disorders is polysomnography (PSG), which is manually scored by a sleep technologist. Scoring a PSG is time-consuming and tedious, with substantial inter-rater variability. A deep-learning-based sleep analysis software module can perform autoscoring of PSG. The primary objective of the study is to validate the accuracy and reliability of the autoscoring software. The secondary objective is to measure workflow improvements in terms of time and cost via a time motion study.

Methodology

The performance of an automatic PSG scoring software was benchmarked against the performance of two independent sleep technologists on PSG data collected from patients with suspected sleep disorders. The technologists at the hospital clinic and a third-party scoring company scored the PSG records independently. The scores were then compared between the technologists and the automatic scoring system. An observational study was also performed where the time taken for sleep technologists at the hospital clinic to manually score PSGs was tracked, along with the time taken by the automatic scoring software to assess for potential time savings.

Results

Pearson's correlation between the manually scored apnea-hypopnea index (AHI) and the automatically scored AHI was 0.962, demonstrating a near-perfect agreement. The autoscoring system demonstrated similar results in sleep staging. The agreement between automatic staging and manual scoring was higher in terms of accuracy and Cohen's kappa than the agreement between experts. The autoscoring system took an average of 42.7 s to score each record compared with 4,243 s for manual scoring. Following a manual review of the auto scores, an average time savings of 38.6 min per PSG was observed, amounting to 0.25 full-time equivalent (FTE) savings per year.

Conclusion

The findings indicate a potential for a reduction in the burden of manual scoring of PSGs by sleep technologists and may be of operational significance for sleep laboratories in the healthcare setting.

Screening for Obstructive Sleep Apnea Risk by Using Machine Learning Approaches and Anthropometric Features

Obstructive sleep apnea (OSA) is a global health concern and is typically diagnosed using in-laboratory polysomnography (PSG). However, PSG is highly time-consuming and labor-intensive. We, therefore, developed machine learning models based on easily accessed anthropometric features to screen for the risk of moderate to severe and severe OSA. We enrolled 3503 patients from Taiwan and determined their PSG parameters and anthropometric features. Subsequently, we compared the mean values among patients with different OSA severity and considered correlations among all participants. We developed models based on the following machine learning approaches: logistic regression, k-nearest neighbors, naïve Bayes, random forest (RF), support vector machine, and XGBoost. Collected data were first independently split into two data sets (training and validation: 80%; testing: 20%). Thereafter, we adopted the model with the highest accuracy in the training and validation stage to predict the testing set. We explored the importance of each feature in the OSA risk screening by calculating the Shapley values of each input variable. The RF model achieved the highest accuracy for moderate to severe (84.74%) and severe (72.61%) OSA. The level of visceral fat was found to be a predominant feature in the risk screening models of OSA with the aforementioned levels of severity. Our machine learning models can be employed to screen for OSA risk in the populations in Taiwan and in those with similar craniofacial structures.

Computer-assisted analysis of polysomnographic recordings improves inter-scorer associated agreement and scoring times

STUDY OBJECTIVES

To investigate inter-scorer agreement and scoring time differences associated with visual and computer-assisted analysis of polysomnographic (PSG) recordings.

METHODS

A group of 12 expert scorers reviewed 5 PSGs that were independently selected in the context of each of the following tasks: (i) sleep staging, (ii) scoring of leg movements, (iii) detection of respiratory (apneic-related) events, and (iv) of electroencephalographic (EEG) arousals. All scorers independently reviewed the same recordings, hence resulting in 20 scoring exercises per scorer from an equal amount of different subjects. The procedure was repeated, separately, using the classical visual manual approach and a computer-assisted (semi-automatic) procedure. Resulting inter-scorer agreement and scoring times were examined and compared among the two methods.

RESULTS

Computer-assisted sleep scoring showed a consistent and statistically relevant effect toward less time required for the completion of each of the PSG scoring tasks. Gain factors ranged from 1.26 (EEG arousals) to 2.41 (leg movements). Inter-scorer kappa agreement was also consistently increased with the use of supervised semi-automatic scoring. Specifically, agreement increased from Κ = 0.76 to K = 0.80 (sleep stages), Κ = 0.72 to K = 0.91 (leg movements), Κ = 0.55 to K = 0.66 (respiratory events), and Κ = 0.58 to Κ = 0.65 (EEG arousals). Inter-scorer agreement on the examined set of diagnostic indices did also show a trend toward higher Interclass Correlation Coefficient scores when using the semi-automatic scoring approach.

CONCLUSIONS

Computer-assisted analysis can improve inter-scorer agreement and scoring times associated with the review of PSG studies resulting in higher efficiency and overall quality in the diagnosis sleep disorders.

Development of a non-contact sleep monitoring system for children

Daily monitoring is important, even for healthy children, because sleep plays a critical role in their development and growth. Polysomnography is necessary for sleep monitoring. However, measuring sleep requires specialized equipment and knowledge and is difficult to do at home. In recent years, smartwatches and other devices have been developed to easily measure sleep. However, they cannot measure children's sleep, and contact devices may disturb their sleep.

A non-contact method of measuring sleep is the use of video during sleep. This is most suitable for the daily monitoring of children’s sleep, as it is simple and inexpensive. However, the algorithms have been developed only based on adult sleep, whereas children’s sleep is known to differ considerably from that of adults.

For this reason, we conducted a non-contact estimation of sleep stages for children using video. The participants were children between the ages of 0–6 years old. We estimated the four stages of sleep using the body movement information calculated from the videos recorded. Six parameters were calculated from body movement information. As children’s sleep is known to change significantly as they grow, estimation was divided into two groups (0–2 and 3–6 years).

The results show average estimation accuracies of 46.7 ± 6.6 and 49.0 ± 4.8% and kappa coefficients of 0.24 ± 0.11 and 0.28 ± 0.06 in the age groups of 0–2 and 3–6 years, respectively. This performance is comparable to or better than that reported in previous adult studies.

SleepSEEG: Automatic sleep scoring using intracranial EEG recordings only

OBJECTIVE

To perform automatic sleep scoring based only on intracranial EEG, without the need for scalp electroencephalography (EEG), electrooculography (EOG) and electromyography (EMG), in order to study sleep, epilepsy, and their interaction.

APPROACH

Data from 33 adult patients was used for development and training of the automatic scoring algorithm using both oscillatory and non-oscillatory spectral features. The first step consisted in unsupervised clustering of channels based on feature variability. For each cluster the classification was done in two steps, a multiclass tree followed by binary classification trees to distinguish the more challenging stage N1. The test data consisted in 11 patients, in whom the classification was done independently for each channel and then combined to get a single stage per epoch.

MAIN RESULTS

An overall agreement of 78% was observed in the test set between the sleep scoring of the algorithm and two human experts scoring based on scalp EEG, EOG and EMG. Balanced sensitivity and specificity were obtained for the different sleep stages. The performance was excellent for stages W, N2, and N3, and good for stage R, but with high variability across patients. The performance for the challenging stage N1 was poor, but at a similar level as for published algorithms based on scalp EEG. High confidence epochs in different stages (other than N1) can be identified with median per patient specificity >80%.

SIGNIFICANCE

The automatic algorithm can perform sleep scoring of long term recordings of patients with intracranial electrodes undergoing presurgical evaluation in the absence of scalp EEG, EOG and EMG, which are normally required to define sleep stages but are difficult to use in the context of intracerebral studies. It also constitutes a valuable tool to generate hypotheses regarding local aspects of sleep, and will be significant for sleep evaluation in clinical epileptology and neuroscience research.

Objective underpinnings of self-reported sleep quality in middle-aged and older adults: The importance of N2 and wakefulness

STUDY OBJECTIVES

The measurable aspects of brain function (polysomnography, PSG) that are correlated with sleep satisfaction are poorly understood. Using recent developments in automated sleep scoring, which remove the within- and between-rater error associated with human scoring, we examine whether PSG measures are associated with sleep satisfaction.

DESIGN AND SETTING

A single night of PSG data was compared to contemporaneously collected measures of sleep satisfaction with Random Forest regressions. Whole and partial night PSG data were scored using a novel machine learning algorithm.

PARTICIPANTS

Community-dwelling adults (N = 3165) who participated in the Sleep Heart Health Study.

INTERVENTIONS

None.

MEASUREMENTS AND RESULTS

Models explained 30% of sleep depth and 27% of sleep restfulness, with a similar top four predictors: minutes of N2 sleep, sleep efficiency, age, and minutes of wake after sleep onset (WASO). With increasing self-reported sleep quality, there was a progressive increase in N2 and decrease in WASO of similar magnitude, without systematic changes in N1, N3 or REM sleep. In comparing those with the best and worst self-reported sleep satisfaction, there was a range of approximately 30 min more N2, 30 min less WASO, an improvement of sleep efficiency of 7-8%, and an age span of 3-5 years. Examination of sleep most proximal to morning awakening revealed no greater explanatory power than the whole-night data set.

CONCLUSIONS

Higher N2 and concomitant lower wake is associated with improved sleep satisfaction. Interventions that specifically target these may be suitable for improving the self-reported sleep experience.

Diagnosis of Sleep Apnoea Using a Mandibular Monitor and Machine Learning Analysis: One-Night Agreement Compared to in-Home Polysomnography

Background

The capacity to diagnose obstructive sleep apnoea (OSA) must be expanded to meet an estimated disease burden of nearly one billion people worldwide. Validated alternatives to the gold standard polysomnography (PSG) will improve access to testing and treatment. This study aimed to evaluate the diagnosis of OSA, using measurements of mandibular movement (MM) combined with automated machine learning analysis, compared to in-home PSG.

Methods

40 suspected OSA patients underwent single overnight in-home sleep testing with PSG (Nox A1, ResMed, Australia) and simultaneous MM monitoring (Sunrise, Sunrise SA, Belgium). PSG recordings were manually analysed by two expert sleep centres (Grenoble and London); MM analysis was automated. The Obstructive Respiratory Disturbance Index calculated from the MM monitoring (MM-ORDI) was compared to the PSG (PSG-ORDI) using intraclass correlation coefficient and Bland-Altman analysis. Receiver operating characteristic curves (ROC) were constructed to optimise the diagnostic performance of the MM monitor at different PSG-ORDI thresholds (5, 15, and 30 events/hour).

Results

31 patients were included in the analysis (58% men; mean (SD) age: 48 (15) years; BMI: 30.4 (7.6) kg/m2). Good agreement was observed between MM-ORDI and PSG-ORDI (median bias 0.00; 95% CI −23.25 to + 9.73 events/hour). However, for 15 patients with no or mild OSA, MM monitoring overestimated disease severity (PSG-ORDI &lt; 5: MM-ORDI mean overestimation + 5.58 (95% CI + 2.03 to + 7.46) events/hour; PSG-ORDI &gt; 5–15: MM-ORDI overestimation + 3.70 (95% CI −0.53 to + 18.32) events/hour). In 16 patients with moderate-severe OSA (n = 9 with PSG-ORDI 15–30 events/h and n = 7 with a PSG-ORD &gt; 30 events/h), there was an underestimation (PSG-ORDI &gt; 15: MM-ORDI underestimation −8.70 (95% CI −28.46 to + 4.01) events/hour). ROC optimal cut-off values for PSG-ORDI thresholds of 5, 15, 30 events/hour were: 9.53, 12.65 and 24.81 events/hour, respectively. These cut-off values yielded a sensitivity of 88, 100 and 79%, and a specificity of 100, 75, 96%. The positive predictive values were: 100, 80, 95% and the negative predictive values 89, 100, 82%, respectively.

Conclusion

The diagnosis of OSA, using MM with machine learning analysis, is comparable to manually scored in-home PSG. Therefore, this novel monitor could be a convenient diagnostic tool that can easily be used in the patients’ own home.

Clinical Trial Registration

https://clinicaltrials.gov, identifier NCT04262557

Association of a novel EEG metric of sleep depth/intensity with attention-deficit/hyperactivity, learning, and internalizing disorders and their pharmacotherapy in adolescence

STUDY OBJECTIVES

Psychiatric/learning disorders are associated with sleep disturbances, including those arising from abnormal cortical activity. The odds ratio product (ORP) is a standardized electroencephalogram metric of sleep depth/intensity validated in adults, while ORP data in youth are lacking. We tested ORP as a measure of sleep depth/intensity in adolescents with and without psychiatric/learning disorders.

METHODS

Four hundred eighteen adolescents (median 16 years) underwent a 9-hour, in-lab polysomnography. Of them, 263 were typically developing (TD), 89 were unmedicated, and 66 were medicated for disorders including attention-deficit/hyperactivity (ADHD), learning (LD), and internalizing (ID). Central ORP during non-rapid eye movement (NREM) sleep was the primary outcome. Secondary/exploratory outcomes included central and frontal ORP during NREM stages, in the 9-seconds following arousals (ORP-9), in the first and second halves of the night, during REM sleep and wakefulness.

RESULTS

Unmedicated youth with ADHD/LD had greater central ORP than TD during stage 3 and in central and frontal regions during stage 2 and the second half of the sleep period, while ORP in youth with ADHD/LD on stimulants did not significantly differ from TD. Unmedicated youth with ID did not significantly differ from TD in ORP, while youth with ID on antidepressants had greater central and frontal ORP than TD during NREM and REM sleep, and higher ORP-9.

CONCLUSIONS

The greater ORP in unmedicated youth with ADHD/LD, and normalized levels in those on stimulants, suggests ORP is a useful metric of decreased NREM sleep depth/intensity in ADHD/LD. Antidepressants are associated with greater ORP/ORP-9, suggesting these medications induce cortical arousability.

Interrater reliability of sleep stage scoring: a meta-analysis

STUDY OBJECTIVES

We evaluated the interrater reliabilities of manual polysomnography sleep stage scoring. We included all studies that employed Rechtschaffen and Kales rules or American Academy of Sleep Medicine standards. We sought the overall degree of agreement and those for each stage.

METHODS

The keywords were "Polysomnography (PSG)," "sleep staging," "Rechtschaffen and Kales (R&K)," "American Academy of Sleep Medicine (AASM)," "interrater (interscorer) reliability," and "Cohen's kappa." We searched PubMed, OVID Medline, EMBASE, the Cochrane library, KoreaMed, KISS, and the MedRIC. The exclusion criteria included automatic scoring and pediatric patients. We collected data on scorer histories, scoring rules, numbers of epochs scored, and the underlying diseases of the patients.

RESULTS

A total of 101 publications were retrieved; 11 satisfied the selection criteria. The Cohen's kappa for manual, overall sleep scoring was 0.76, indicating substantial agreement (95% confidence interval, 0.71-0.81; P < .001). By sleep stage, the figures were 0.70, 0.24, 0.57, 0.57, and 0.69 for the W, N1, N2, N3, and R stages, respectively. The interrater reliabilities for stage N2 and N3 sleep were moderate, and that for stage N1 sleep was only fair.

CONCLUSIONS

We conducted a meta-analysis to generalize the variation in manual scoring of polysomnography and provide reference data for automatic sleep stage scoring systems. The reliability of manual scorers of polysomnography sleep stages was substantial. However, for certain stages, the results were poor; validity requires improvement.

CITATION

Lee YJ, Lee JY, Cho JH, Choi JH. Interrater reliability of sleep stage scoring: a meta-analysis. J Clin Sleep Med. 2022;18(1):193-202.

Automated Scoring of Sleep and Associated Events

Conventionally, sleep and associated events are scored visually by trained technologists according to the rules summarized in the American Academy of Sleep Medicine Manual. Since its first publication in 2007, the manual was continuously updated; the most recent version as of this writing was published in 2020. Human expert scoring is considered as gold standard, even though there is increasing evidence of limited interrater reliability between human scorers. Significant advances in machine learning have resulted in powerful methods for addressing complex classification problems such as automated scoring of sleep and associated events. Evidence is increasing that these autoscoring systems deliver performance comparable to manual scoring and offer several advantages to visual scoring: (1) avoidance of the rather expensive, time-consuming, and difficult visual scoring task that can be performed only by well-trained and experienced human scorers, (2) attainment of consistent scoring results, and (3) proposition of added value such as scoring in real time, sleep stage probabilities per epoch (hypnodensity), estimates of signal quality and sleep/wake-related features, identifications of periods with clinically relevant ambiguities (confidence trends), configurable sensitivity and rule settings, as well as cardiorespiratory sleep staging for home sleep apnea testing. This chapter describes the development of autoscoring systems since the first attempts in the 1970s up to the most recent solutions based on deep neural network approaches which achieve an accuracy that allows to use the autoscoring results directly for review and interpretation by a physician.

Manual Annotation of Time in Bed Using Free-Living Recordings of Accelerometry Data

With the emergence of machine learning for the classification of sleep and other human behaviors from accelerometer data, the need for correctly annotated data is higher than ever. We present and evaluate a novel method for the manual annotation of in-bed periods in accelerometer data using the open-source software Audacity^®, and we compare the method to the EEG-based sleep monitoring device Zmachine^® Insight+ and self-reported sleep diaries. For evaluating the manual annotation method, we calculated the inter- and intra-rater agreement and agreement with Zmachine and sleep diaries using interclass correlation coefficients and Bland–Altman analysis. Our results showed excellent inter- and intra-rater agreement and excellent agreement with Zmachine and sleep diaries. The Bland–Altman limits of agreement were generally around ±30 min for the comparison between the manual annotation and the Zmachine timestamps for the in-bed period. Moreover, the mean bias was minuscule. We conclude that the manual annotation method presented is a viable option for annotating in-bed periods in accelerometer data, which will further qualify datasets without labeling or sleep records.

Deconstructing Commercial Wearable Technology: Contributions toward Accurate and Free-Living Monitoring of Sleep

Despite prolific demands and sales, commercial sleep assessment is primarily limited by the inability to "measure" sleep itself; rather, secondary physiological signals are captured, combined, and subsequently classified as sleep or a specific sleep state. Using markedly different approaches compared with gold-standard polysomnography, wearable companies purporting to measure sleep have rapidly developed during recent decades. These devices are advertised to monitor sleep via sensors such as accelerometers, electrocardiography, photoplethysmography, and temperature, alone or in combination, to estimate sleep stage based upon physiological patterns. However, without regulatory oversight, this market has historically manufactured products of poor accuracy, and rarely with third-party validation. Specifically, these devices vary in their capacities to capture a signal of interest, process the signal, perform physiological calculations, and ultimately classify a state (sleep vs. wake) or sleep stage during a given time domain. Device performance depends largely on success in all the aforementioned requirements. Thus, this review provides context surrounding the complex hardware and software developed by wearable device companies in their attempts to estimate sleep-related phenomena, and outlines considerations and contributing factors for overall device success.

Interrater sleep stage scoring reliability between manual scoring from two European sleep centers and automatic scoring performed by the artificial intelligence-based Stanford-STAGES algorithm

STUDY OBJECTIVES

The objective of this study was to evaluate interrater reliability between manual sleep stage scoring performed in 2 European sleep centers and automatic sleep stage scoring performed by the previously validated artificial intelligence-based Stanford-STAGES algorithm.

METHODS

Full night polysomnographies of 1,066 participants were included. Sleep stages were manually scored in Berlin and Innsbruck sleep centers and automatically scored with the Stanford-STAGES algorithm. For each participant, we compared (1) Innsbruck to Berlin scorings (INN vs BER); (2) Innsbruck to automatic scorings (INN vs AUTO); (3) Berlin to automatic scorings (BER vs AUTO); (4) epochs where scorers from Innsbruck and Berlin had consensus to automatic scoring (CONS vs AUTO); and (5) both Innsbruck and Berlin manual scorings (MAN) to the automatic ones (MAN vs AUTO). Interrater reliability was evaluated with several measures, including overall and sleep stage-specific Cohen's κ.

RESULTS

Overall agreement across participants was substantial for INN vs BER (κ = 0.66 ± 0.13), INN vs AUTO (κ = 0.68 ± 0.14), CONS vs AUTO (κ = 0.73 ± 0.14), and MAN vs AUTO (κ = 0.61 ± 0.14), and moderate for BER vs AUTO (κ = 0.55 ± 0.15). Human scorers had the highest disagreement for N1 sleep (κ_N1 = 0.40 ± 0.16 for INN vs BER). Automatic scoring had lowest agreement with manual scorings for N1 and N3 sleep (κ_N1 = 0.25 ± 0.14 and κ_N3 = 0.42 ± 0.32 for MAN vs AUTO).

CONCLUSIONS

Interrater reliability for sleep stage scoring between human scorers was in line with previous findings, and the algorithm achieved an overall substantial agreement with manual scoring. In this cohort, the Stanford-STAGES algorithm showed similar performances to the ones achieved in the original study, suggesting that it is generalizable to new cohorts. Before its integration in clinical practice, future independent studies should further evaluate it in other cohorts.

Automatic analysis of single-channel sleep EEG in a large spectrum of sleep disorders

STUDY OBJECTIVES

To assess the performance of the single-channel automatic sleep staging (AS) software ASEEGA in adult patients diagnosed with various sleep disorders.

METHODS

Sleep recordings were included of 95 patients (38 women, 40.5 ± 13.7 years) diagnosed with insomnia (n = 23), idiopathic hypersomnia (n = 24), narcolepsy (n = 24), and obstructive sleep apnea (n = 24). Visual staging (VS) was performed by two experts (VS1 and VS2) according to the American Academy of Sleep Medicine rules. AS was based on the analysis of a single electroencephalogram channel (Cz-Pz), without any information from electro-oculography nor electromyography. The epoch-by-epoch agreement (concordance and Conger's coefficient [κ]) was compared pairwise (VS1-VS2, AS-VS1, AS-VS2) and between AS and consensual VS. Sleep parameters were also compared.

RESULTS

The pairwise agreements were: between AS and VS1, 78.6% (κ = 0.70); AS and VS2, 75.0% (0.65); and VS1 and VS2, 79.5% (0.72). Agreement between AS and consensual VS was 85.6% (0.80), with the following distribution: insomnia 85.5% (0.80), narcolepsy 83.8% (0.78), idiopathic hypersomnia 86.1% (0.68), and obstructive sleep disorder 87.2% (0.82). A significant low-amplitude scorer effect was observed for most sleep parameters, not always driven by the same scorer. Hypnograms obtained with AS and VS exhibited very close sleep organization, except for 80% of rapid eye movement sleep onset in the group diagnosed with narcolepsy missed by AS.

CONCLUSIONS

Agreement between AS and VS in sleep disorders is comparable to that reported in healthy individuals and to interexpert agreement in patients. ASEEGA could therefore be considered as a complementary sleep stage scoring tool in clinical practice, after improvement of rapid eye movement sleep onset detection.

The Aging Slow Wave: A Shifting Amalgam of Distinct Slow Wave and Spindle Coupling Subtypes Define Slow Wave Sleep Across the Human Lifespan

STUDY OBJECTIVES

Slow wave and spindle coupling supports memory consolidation, and loss of coupling is linked with cognitive decline and neurodegeneration. Coupling is proposed to be a possible biomarker of neurological disease, yet little is known about the different subtypes of coupling that normally occur throughout human development and aging. Here we identify distinct subtypes of spindles within slow wave upstates and describe their relationships with sleep stage across the human lifespan.

METHODS

Coupling within a cross-sectional cohort of 582 subjects was quantified from stages N2 and N3 sleep across ages 6-88 years old. Results were analyzed across the study population via mixed model regression. Within a subset of subjects, we further utilized coupling to identify discrete subtypes of slow waves by their coupled spindles.

RESULTS

Two different subtypes of spindles were identified during the upstates of (distinct) slow waves: an "early-fast" spindle, more common in stage N2 sleep, and a "late-fast" spindle, more common in stage N3. We further found stages N2 and N3 sleep contain a mixture of discrete subtypes of slow waves, each identified by their unique coupled-spindle timing and frequency. The relative contribution of coupling subtypes shifts across the human lifespan, and a deeper sleep phenotype prevails with increasing age.

CONCLUSIONS

Distinct subtypes of slow waves and coupled spindles form the composite of slow wave sleep. Our findings support a model of sleep-dependent synaptic regulation via discrete slow wave/spindle coupling subtypes and advance a conceptual framework for the development of coupling-based biomarkers in age-associated neurological disease.

Maturational trajectories of non-rapid eye movement slow wave activity and odds ratio product in a population-based sample of youth

BACKGROUND

Brain maturation is reflected in the sleep electroencephalogram (EEG) by a decline in non-rapid eye movement (NREM) slow wave activity (SWA) throughout adolescence and a related decrease in sleep depth. However, this trajectory and its sex and pubertal differences lack replication in population-based samples. We tested age-related changes in SWA (0.4-4 Hz) power and odds ratio product (ORP), a standardized measure of sleep depth.

METHODS

We analyzed the sleep EEG of 572 subjects aged 6-21 y (48% female, 26% racial/ethnic minority) and 332 subjects 5-12 y followed-up at 12-22 y. Multivariable-adjusted analyses tested age-related cross-sectional and longitudinal trajectories of SWA and ORP.

RESULTS

SWA remained stable from age 6 to 10, decreased between ages 11 and 17, and plateaued from age 18 to 21 (p-cubic<0.001); females showed a longitudinal decline 23% greater than males by 13 y, while males experienced a steeper slope after 14 y and their longitudinal decline was 21% greater by 19 y. More mature adolescents (75% female) experienced a greater longitudinal decline in SWA than less mature adolescents by 14 y. ORP showed an age-related increasing trajectory (p-linear<0.001) with no sex or pubertal differences.

CONCLUSIONS

We provide population-level evidence for the maturational decline and sex and pubertal differences in SWA in the transition from childhood to adolescence, while introducing ORP as a novel metric in youth. Along with previous studies, the distinct trajectories observed suggest that age-related changes in SWA reflect brain maturation and local/synaptic processes during this developmental period, while those of ORP may reflect global/state control of NREM sleep depth.

Automatic sleep stage classification with deep residual networks in a mixed-cohort setting

STUDY OBJECTIVES

Sleep stage scoring is performed manually by sleep experts and is prone to subjective interpretation of scoring rules with low intra- and interscorer reliability. Many automatic systems rely on few small-scale databases for developing models, and generalizability to new datasets is thus unknown. We investigated a novel deep neural network to assess the generalizability of several large-scale cohorts.

METHODS

A deep neural network model was developed using 15,684 polysomnography studies from five different cohorts. We applied four different scenarios: (1) impact of varying timescales in the model; (2) performance of a single cohort on other cohorts of smaller, greater, or equal size relative to the performance of other cohorts on a single cohort; (3) varying the fraction of mixed-cohort training data compared with using single-origin data; and (4) comparing models trained on combinations of data from 2, 3, and 4 cohorts.

RESULTS

Overall classification accuracy improved with increasing fractions of training data (0.25%: 0.782 ± 0.097, 95% CI [0.777-0.787]; 100%: 0.869 ± 0.064, 95% CI [0.864-0.872]), and with increasing number of data sources (2: 0.788 ± 0.102, 95% CI [0.787-0.790]; 3: 0.808 ± 0.092, 95% CI [0.807-0.810]; 4: 0.821 ± 0.085, 95% CI [0.819-0.823]). Different cohorts show varying levels of generalization to other cohorts.

CONCLUSIONS

Automatic sleep stage scoring systems based on deep learning algorithms should consider as much data as possible from as many sources available to ensure proper generalization. Public datasets for benchmarking should be made available for future research.

Sleep-wake parameters can be detected in patients with chronic stroke using a multisensor accelerometer: a validation study

STUDY OBJECTIVES

Sleep-wake dysfunction is bidirectionally associated with the pathogenesis and evolution of stroke. Longitudinal and prospective measurement of sleep after chronic stroke remains poorly characterized because of a lack of validated objective and ambulatory sleep measurement tools in neurological populations. This study aimed to validate a multisensor sleep monitor, the SenseWear Armband (SWA), in patients with ischemic stroke and control patients using at-home polysomnography.

METHODS

Twenty-eight radiologically confirmed patients with ischemic stroke (aged 69.61 ± 7.35 years; mean = 4.1 years poststroke) and 16 control patients (aged 73.75 ± 7.10 years) underwent overnight at-home polysomnography in tandem with the SWA. Lin's concordance correlation coefficient and reduced major axis regressions were employed to assess concordance of SWA vs polysomnography-measured total sleep time, sleep efficiency, sleep onset latency, and wake after sleep onset. Subsequently, data were converted to 30-second epochs to match at-home polysomnography. Epoch-by-epoch agreement between SWA and at-home polysomnography was estimated using crude agreement, Cohen's kappa, sensitivity, and specificity.

RESULTS

Total sleep time was the most robustly quantified sleep-wake variable (concordance correlation coefficient = 0.49). The SWA performed poorest for sleep measures requiring discrimination of wakefulness (sleep onset latency; concordance correlation coefficient = 0.16). The sensitivity of the SWA was high (95.90%) for patients with stroke and for control patients (95.70%). The specificity of the SWA was fair-moderate for patients with stroke (40.45%) and moderate for control patients (45.60%). Epoch-by-epoch agreement rate was fair (78%) in patients with stroke and fair (74%) in controls.

CONCLUSIONS

The SWA shows promise as an ambulatory tool to estimate macro parameters of sleep-wake; however, agreement at an epoch level is only moderate-fair. Use of the SWA warrants caution when it is used as a diagnostic tool or in populations with significant sleep-wake fragmentation.

Efficient embedded sleep wake classification for open-source actigraphy

This study presents a thorough analysis of sleep/wake detection algorithms for efficient on-device sleep tracking using wearable accelerometric devices. It develops a novel end-to-end algorithm using convolutional neural network applied to raw accelerometric signals recorded by an open-source wrist-worn actigraph. The aim of the study is to develop an automatic classifier that: (1) is highly generalizable to heterogenous subjects, (2) would not require manual features' extraction, (3) is computationally lightweight, embeddable on a sleep tracking device, and (4) is suitable for a wide assortment of actigraphs. Hereby, authors analyze sleep parameters, such as total sleep time, waking after sleep onset and sleep efficiency, by comparing the outcomes of the proposed algorithm to the gold standard polysomnographic concurrent recordings. The relatively substantial agreement (Cohen's kappa coefficient, median, equal to 0.78 ± 0.07) and the low-computational cost (2727 floating-point operations) make this solution suitable for an on-board sleep-detection approach.

Automatic Sleep Stage Classification of Children with Sleep-Disordered Breathing Using the Modularized Network

PURPOSE

To develop an automatic sleep stage analysis model for children and evaluate the effect of the model on the diagnosis of sleep-disordered breathing (SDB).

PATIENTS AND METHODS

Three hundred and forty-four SDB patients aged between 2 to 18 years who completed polysomnography (PSG) to assess the severity of the disease were enrolled in this study. We developed deep neural networks to stage sleep from electroencephalography (EEG), electrooculography (EOG) and electromyogram (EMG). The model performance was estimated by accuracy, precision, recall, F1-score, and Cohen's Kappa coefficient (ĸ). And we compared the difference in calculation of sleep parameters among the technicians, the model ensemble, and the single-channel EEG model.

RESULTS

The numbers of raw data divided into training, validation, and testing were 240, 36, and 68, respectively. The best performance appeared in the model ensemble of which the accuracy was 83.36% (ĸ=0.7817) in 5-stages, and the accuracy was 96.76% (ĸ=0.8236) in 2-stages. The single-channel EEG model showed the classification satisfyingly as well. There was no significant difference in TST, SE, SOL, time in W, time in N1+N2, time in N3, and OAHI between technician and the model (P>0.05). On the datasets from sleep-EDF-13 and sleep-EDF-18, the average classification accuracies achieved were 92.76% and 91.94% in 5-stages by using the proposed method, respectively.

CONCLUSION

This research established the model for pediatric automatic sleep stage classification with satisfying reliability and generalizability. In addition, it could be applied for calculating quantitative sleep parameters and evaluating the severity of SDB.

Uncovering the structure of clinical EEG signals with self-supervised learning

OBJECTIVE

Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG), where labeling can be costly in terms of specialized expertise and human processing time. Consequently, deep learning architectures designed to learn on EEG data have yielded relatively shallow models and performances at best similar to those of traditional feature-based approaches. However, in most situations, unlabeled data is available in abundance. By extracting information from this unlabeled data, it might be possible to reach competitive performance with deep neural networks despite limited access to labels.

APPROACH

We investigated self-supervised learning (SSL), a promising technique for discovering structure in unlabeled data, to learn representations of EEG signals. Specifically, we explored two tasks based on temporal context prediction as well as contrastive predictive coding on two clinically-relevant problems: EEG-based sleep staging and pathology detection. We conducted experiments on two large public datasets with thousands of recordings and performed baseline comparisons with purely supervised and hand-engineered approaches.

MAIN RESULTS

Linear classifiers trained on SSL-learned features consistently outperformed purely supervised deep neural networks in low-labeled data regimes while reaching competitive performance when all labels were available. Additionally, the embeddings learned with each method revealed clear latent structures related to physiological and clinical phenomena, such as age effects.

SIGNIFICANCE

We demonstrate the benefit of SSL approaches on EEG data. Our results suggest that self-supervision may pave the way to a wider use of deep learning models on EEG data.