Attention-based CNN-BiLSTM for sleep state classification of spatiotemporal wide-field calcium imaging data

BACKGROUND

Wide-field calcium imaging (WFCI) with genetically encoded calcium indicators allows for spatiotemporal recordings of neuronal activity in mice. When applied to the study of sleep, WFCI data are manually scored into the sleep states of wakefulness, non-REM (NREM) and REM by use of adjunct EEG and EMG recordings. However, this process is time-consuming, invasive and often suffers from low inter- and intra-rater reliability. Therefore, an automated sleep state classification method that operates on spatiotemporal WFCI data is desired.

NEW METHOD

A hybrid network architecture consisting of a convolutional neural network (CNN) to extract spatial features of image frames and a bidirectional long short-term memory network (BiLSTM) with attention mechanism to identify temporal dependencies among different time points was proposed to classify WFCI data into states of wakefulness, NREM and REM sleep.

RESULTS

Sleep states were classified with an accuracy of 84 % and Cohen's κ of 0.64. Gradient-weighted class activation maps revealed that the frontal region of the cortex carries more importance when classifying WFCI data into NREM sleep while posterior area contributes most to the identification of wakefulness. The attention scores indicated that the proposed network focuses on short- and long-range temporal dependency in a state-specific manner.

COMPARISON WITH EXISTING METHOD

On a held out, repeated 3-hour WFCI recording, the CNN-BiLSTM achieved a κ of 0.67, comparable to a κ of 0.65 corresponding to the human EEG/EMG-based scoring.

CONCLUSIONS

The CNN-BiLSTM effectively classifies sleep states from spatiotemporal WFCI data and will enable broader application of WFCI in sleep research.

Severe obstructive sleep apnea in children with syndromic craniosynostosis: analysis of pulse transit time

STUDY OBJECTIVES

We examined the association between pulse transit time (PTT) and obstructive sleep apnea (OSA) in children with syndromic craniosynostosis (SCS), where OSA is a common problem and may cause cardiorespiratory disturbance.

METHODS

A retrospective study of children (age < 18 years) with SCS and moderate-to-severe OSA (ie, obstructive apnea-hypopnea index ≥ 5) or no OSA (obstructive apnea-hypopnea index < 1) who underwent overnight polysomnography. Children without SCS and normal polysomnography were included as controls. Reference intervals for PTT were computed by nonparametric bootstrap analysis. Based on reference intervals of controls, the sensitivity and specificity of PTT to detect OSA were determined. In a linear mixed model, the explanatory variables assessed were sex, age, sleep stage, and time after obstructive events.

RESULTS

In all 68 included children (19 with SCS with OSA, 30 with SCS without OSA, 19 controls), obstructive events occurred throughout all sleep stages, most prominently during rapid eye movement (REM) sleep and non-REM sleep stages N1 and N2, with evident PTT changes. The greatest reductions were observed 4-8 seconds after an event (P < .05). In SCS with OSA, PTT reference intervals were lower during all sleep stages compared with SCS without OSA. The highest sensitivity was observed during N1 (55.5%), and the highest specificity during REM sleep (76.5%). The lowest PTT values were identified during N1.

CONCLUSIONS

Obstructive events occur throughout all sleep stages with transient reductions in PTT. However, PTT as a variable for OSA detection is limited by its sensitivity and specificity.

CITATION

Yang S, van Twist E, van Heesch GGM, et al. Severe obstructive sleep apnea in children with syndromic craniosynostosis: analysis of pulse transit time. J Clin Sleep Med. 2024;20(8):1233-1240.

Performance Investigation of Somfit Sleep Staging Algorithm

Purpose

To investigate accuracy of the sleep staging algorithm in a new miniaturized home sleep monitoring device - Compumedics® Somfit. Somfit is attached to patient's forehead and combines channels specified for a pulse arterial tonometry (PAT)-based home sleep apnea testing (HSAT) device with the neurological signals. Somfit sleep staging deep learning algorithm is based on convolutional neural network architecture.

Patients and Methods

One hundred and ten participants referred for sleep investigation with suspected or preexisting obstructive sleep apnea (OSA) in need of a review were enrolled into the study involving simultaneous recording of full overnight polysomnography (PSG) and Somfit data. The recordings were conducted at three centers in Australia. The reported statistics include standard measures of agreement between Somfit automatic hypnogram and consensus PSG hypnogram.

Results

Overall percent agreement across five sleep stages (N1, N2, N3, REM, and wake) between Somfit automatic and consensus PSG hypnograms was 76.14 (SE: 0.79). The percent agreements between different pairs of sleep technologists' PSG hypnograms varied from 74.36 (1.93) to 85.50 (0.64), with interscorer agreement being greater for scorers from the same sleep laboratory. The estimate of kappa between Somfit and consensus PSG was 0.672 (0.002). Percent agreement for sleep/wake discrimination was 89.30 (0.37). The accuracy of Somfit sleep staging algorithm varied with increasing OSA severity - percent agreement was 79.67 (1.87) for the normal subjects, 77.38 (1.06) for mild OSA, 74.83 (1.79) for moderate OSA and 72.93 (1.68) for severe OSA.

Conclusion

Agreement between Somfit and PSG hypnograms was non-inferior to PSG interscorer agreement for a number of scorers, thus confirming acceptability of electrode placement at the center of the forehead. The directions for algorithm improvement include additional arousal detection, integration of motion and oximetry signals and separate inference models for individual sleep stages.

Revisiting the maintenance of wakefulness test: from intra-/inter-scorer agreement to normative values in patients treated for obstructive sleep apnea

The Maintenance of Wakefulness Test is widely used to objectively assess sleepiness and make safety-related decisions, but its interpretation is subjective and normative values remain debated. Our work aimed to determine normative thresholds in non-subjectively sleepy patients with well-treated obstructive sleep apnea, and to assess intra- and inter-scorer variability. We included maintenance of wakefulness tests of 141 consecutive patients with treated obstructive sleep apnea (90% men, mean (SD) age 47.5 (9.2) years, mean (SD) pre-treatment apnea-hypopnea index of 43.8 (20.3) events/h). Sleep onset latencies were independently scored by two experts. Discordant scorings were reviewed to reach a consensus and half of the cohort was double-scored by each scorer. Intra- and inter-scorer variability was assessed using Cohen's kappa for 40, 33, and 19 min mean sleep latency thresholds. Consensual mean sleep latencies were compared between four groups according to subjective sleepiness (Epworth Sleepiness Scale score < versus ≥11) and residual apnea-hypopnea index (< versus ≥15 events/h). In well-treated non-sleepy patients (n = 76), the consensual mean (SD) sleep latency was 38.4 (4.2) min (lower normal limit [mean - 2SD] = 30 min), and 80% of them did not fall asleep. Intra-scorer agreement on mean sleep latency was high but inter-scorer was only fair (Cohen's kappa 0.54 for 33-min threshold, 0.27 for 19-min threshold), resulting in changes in latency category in 4%-12% of patients. A higher sleepiness score but not the residual apnea-hypopnea index was significantly associated with a lower mean sleep latency. Our findings suggest a higher than usually accepted normative threshold (30 min) in this context and emphasise the need for more reproducible scoring approaches.

The impact of study type and sleep measurement on oxygen desaturation index calculation

STUDY OBJECTIVES

The oxygen desaturation index (ODI) is an important measure of sleep-disordered breathing during polysomnography (PSG); however, the AASM Manual (V3) does not specify whether to include oxygen desaturations occurring during wake epochs. Additionally, an ODI obtained from PSG can differ from an ODI using home sleep apnea tests (HSATs) that do not measure sleep, hampering diagnostic and treatment decision reliability. This study aimed to (1) compare an ODI that included all desaturations with an ODI that excluded desaturations occurring during wake epochs in PSG and (2) compare ODIs obtained from PSG with HSAT.

METHODS

100 consecutive PSGs for investigation of obstructive sleep apnea were compared. ODIs were calculated including all desaturations (ODIall) and by excluding desaturations entirely during wake epochs (ODIsleep). Additionally, we compared ODIall with an ODI calculated using monitoring time as the denominator (ODIHSAT).

RESULTS

The median (interquartile range) 3% ODI for ODIall was 22.8 (13.1, 44.1) events/h and ODIsleep was 17.6 (11.5, 35.2) events/h (median difference: -3.9 events/h [-8.2, -0.9]; 21.0% [8.7%, 33.2%]). This discrepancy was larger with increasing ODI and decreasing sleep efficiency. The ODIHSAT was 17.4 (11.3, 35.2) events/h and the median reduction in ODIHSAT vs ODIall was -4.5 (-10.9, -2.0) events/h (21.6%; 11.1%, 33.8).

CONCLUSIONS

ODI was significantly reduced when desaturations in wake epochs were excluded, and when ODI was based on monitoring time rather than sleep time, with the potential for underestimation of disease severity. Results suggest that ODI can differ substantially depending on the calculation and study type used, and that there is a need for standardization to ensure consistent diagnosis and treatment outcomes.

CITATION

Whenn CB, Wilson DL, Ruehland WR, Churchward TJ, Worsnop C, Tolson J. The impact of study type and sleep measurement on oxygen desaturation index calculation. J Clin Sleep Med. 2024;20(5):709-717.

ZleepAnlystNet: a novel deep learning model for automatic sleep stage scoring based on single-channel raw EEG data using separating training

Numerous models for sleep stage scoring utilizing single-channel raw EEG signal have typically employed CNN and BiLSTM architectures. While these models, incorporating temporal information for sequence classification, demonstrate superior overall performance, they often exhibit low per-class performance for N1-stage, necessitating an adjustment of loss function. However, the efficacy of such adjustment is constrained by the training process. In this study, a pioneering training approach called separating training is introduced, alongside a novel model, to enhance performance. The developed model comprises 15 CNN models with varying loss function weights for feature extraction and 1 BiLSTM for sequence classification. Due to its architecture, this model cannot be trained using an end-to-end approach, necessitating separate training for each component using the Sleep-EDF dataset. Achieving an overall accuracy of 87.02%, MF1 of 82.09%, Kappa of 0.8221, and per-class F1-socres (W 90.34%, N1 54.23%, N2 89.53%, N3 88.96%, and REM 87.40%), our model demonstrates promising performance. Comparison with sleep technicians reveals a Kappa of 0.7015, indicating alignment with reference sleep stags. Additionally, cross-dataset validation and adaptation through training with the SHHS dataset yield an overall accuracy of 84.40%, MF1 of 74.96% and Kappa of 0.7785 when tested with the Sleep-EDF-13 dataset. These findings underscore the generalization potential in model architecture design facilitated by our novel training approach.

Comparison of OPPO Watch Sleep Analyzer and Polysomnography for Obstructive Sleep Apnea Screening

Objective

To evaluate the clinical performance of the OPPO Watch (OW) Sleep Analyzer (OWSA) on OSA screening with polysomnography reference.

Methods

We recruited 350 participants using OWSA and PSG simultaneously in a sleep laboratory. The respiratory event index (REI) derived from OWSA and the apnea-hypopnea index (AHI) provided by PSG were compared. SHapley Additive exPlanation (SHAP) values were calculated to explain the model of OWSA.

Results

The OWSA-REI (26.5±18.5 events/h) correlated well with PSG-AHI (33.2±25.7 events/h; r = 0.91, p < 0.001), with an intraclass correlation coefficient (ICC) of 0.83. Using a threshold of AHI ≥15 events/h, the sensitivity, specificity, accuracy, and area under the curve (AUC) were 86.1%, 86.7%, 86.3%, and 0.94, respectively. Bland-Altman analysis showed that OWSA-REI and PSG-AHI were in good agreement (Mean Difference: -6.7, 95% CI:16.0 to -29.3 events/h). In addition, the effectiveness of the models in OWSA were also explained by visualizing SHAP values.

Conclusion

The OWSA demonstrated a reasonable performance for OSA screening in the clinical setting. In light of this, it is possible for smartwatches to become a complementary tool to PSG, which is particularly useful for larger-scale preliminary screenings.

Multicentre sleep-stage scoring agreement in the Sleep Revolution project

Determining sleep stages accurately is an important part of the diagnostic process for numerous sleep disorders. However, as the sleep stage scoring is done manually following visual scoring rules there can be considerable variation in the sleep staging between different scorers. Thus, this study aimed to comprehensively evaluate the inter-rater agreement in sleep staging. A total of 50 polysomnography recordings were manually scored by 10 independent scorers from seven different sleep centres. We used the 10 scorings to calculate a majority score by taking the sleep stage that was the most scored stage for each epoch. The overall agreement for sleep staging was κ = 0.71 and the mean agreement with the majority score was 0.86. The scorers were in perfect agreement in 48% of all scored epochs. The agreement was highest in rapid eye movement sleep (κ = 0.86) and lowest in N1 sleep (κ = 0.41). The agreement with the majority scoring varied between the scorers from 81% to 91%, with large variations between the scorers in sleep stage-specific agreements. Scorers from the same sleep centres had the highest pairwise agreements at κ = 0.79, κ = 0.85, and κ = 0.78, while the lowest pairwise agreement between the scorers was κ = 0.58. We also found a moderate negative correlation between sleep staging agreement and the apnea-hypopnea index, as well as the rate of sleep stage transitions. In conclusion, although the overall agreement was high, several areas of low agreement were also found, mainly between non-rapid eye movement stages.

Standardized image-based polysomnography database and deep learning algorithm for sleep-stage classification

STUDY OBJECTIVES

Polysomnography (PSG) scoring is labor-intensive, subjective, and often ambiguous. Recently several deep learning (DL) models for automated sleep scoring have been developed, they are tied to a fixed amount of input channels and resolution. In this study, we constructed a standardized image-based PSG dataset in order to overcome the heterogeneity of raw signal data obtained from various PSG devices and various sleep laboratory environments.

METHODS

All individually exported European data format files containing raw signals were converted into images with an annotation file, which contained the demographics, diagnoses, and sleep statistics. An image-based DL model for automatic sleep staging was developed, compared with a signal-based model, and validated in an external dataset.

RESULTS

We constructed 10253 image-based PSG datasets using a standardized format. Among these, 7745 diagnostic PSG data were used to develop our DL model. The DL model using the image dataset showed similar performance to the signal-based dataset for the same subject. The overall DL accuracy was greater than 80%, even with severe obstructive sleep apnea. Moreover, for the first time, we showed explainable DL in the field of sleep medicine as visualized key inference regions using Eigen-class activation maps. Furthermore, when a DL model for sleep scoring performs external validation, we achieved a relatively good performance.

CONCLUSIONS

Our main contribution demonstrates the availability of a standardized image-based dataset, and highlights that changing the data sampling rate or number of sensors may not require retraining, although performance decreases slightly as the number of sensors decreases.

Enhanced sleep staging with artificial intelligence: a validation study of new software for sleep scoring

Manual sleep staging (MSS) using polysomnography is a time-consuming task, requires significant training, and can lead to significant variability among scorers. STAGER is a software program based on machine learning algorithms that has been developed by Medibio Limited (Savage, MN, USA) to perform automatic sleep staging using only EEG signals from polysomnography. This study aimed to extensively investigate its agreement with MSS performed during clinical practice and by three additional expert sleep technicians. Forty consecutive polysomnographic recordings of patients referred to three US sleep clinics for sleep evaluation were retrospectively collected and analyzed. Three experienced technicians independently staged the recording using the electroencephalography, electromyography, and electrooculography signals according to the American Academy of Sleep Medicine guidelines. The staging initially performed during clinical practice was also considered. Several agreement statistics between the automatic sleep staging (ASS) and MSS, among the different MSSs, and their differences were calculated. Bootstrap resampling was used to calculate 95% confidence intervals and the statistical significance of the differences. STAGER's ASS was most comparable with, or statistically significantly better than the MSS, except for a partial reduction in the positive percent agreement in the wake stage. These promising results indicate that STAGER software can perform ASS of inpatient polysomnographic recordings accurately in comparison with MSS.

An approach for determining the reliability of manual and digital scoring of sleep stages

STUDY OBJECTIVES

Inter-scorer variability in sleep staging is largely due to equivocal epochs that contain features of more than one stage. We propose an approach that recognizes the existence of equivocal epochs and evaluates scorers accordingly.

METHODS

Epoch-by-epoch staging was performed on 70 polysomnograms by six qualified technologists and by a digital system (Michele Sleep Scoring [MSS]). Probability that epochs assigned the same stage by only two of the six technologists (minority score) resulted from random occurrence of two errors was calculated and found to be <5%, thereby indicating that the stage assigned is an acceptable variant for the epoch. Acceptable stages were identified in each epoch as stages assigned by at least two technologists. Percent agreement between each technologist and the other five technologists, acting as judges, was determined. Agreement was considered to exist if the stage assigned by the tested scorer was one of the acceptable stages for the epoch. Stage assigned by MSS was likewise considered in agreement if included in the acceptable stages made by the technologists.

RESULTS

Agreement of technologists tested against five qualified judges increased from 80.8% (range 70.5%-86.4% among technologists) when using the majority rule, to 96.1 (89.8%-98.5%) by the proposed approach. Agreement between unedited MSS and same judges was 90.0% and increased to 92.1% after brief editing.

CONCLUSIONS

Accounting for equivocal epochs provides a more accurate estimate of a scorer's (human or digital) competence in scoring sleep stages and reduces inter-scorer disagreements. The proposed approach can be implemented in sleep-scoring training and accreditation programs.

Modeling the Impact of Inter-Rater Disagreement on Sleep Statistics Using Deep Generative Learning

Sleep staging is the process by which an overnight polysomnographic measurement is segmented into epochs of 30 seconds, each of which is annotated as belonging to one of five discrete sleep stages. The resulting scoring is graphically depicted as a hypnogram, and several overnight sleep statistics are derived, such as total sleep time and sleep onset latency. Gold standard sleep staging as performed by human technicians is time-consuming, costly, and comes with imperfect inter-scorer agreement, which also results in inter-scorer disagreement about the overnight statistics. Deep learning algorithms have shown promise in automating sleep scoring, but struggle to model inter-scorer disagreement in sleep statistics. To that end, we introduce a novel technique using conditional generative models based on Normalizing Flows that permits the modeling of the inter-rater disagreement of overnight sleep statistics, termed U-Flow. We compare U-Flow to other automatic scoring methods on a hold-out test set of 70 subjects, each scored by six independent scorers. The proposed method achieves similar sleep staging performance in terms of accuracy and Cohen's kappa on the majority-voted hypnograms. At the same time, U-Flow outperforms the other methods in terms of modeling the inter-rater disagreement of overnight sleep statistics. The consequences of inter-rater disagreement about overnight sleep statistics may be great, and the disagreement potentially carries diagnostic and scientifically relevant information about sleep structure. U-Flow is able to model this disagreement efficiently and can support further investigations into the impact inter-rater disagreement has on sleep medicine and basic sleep research.

Challenges of Applying Automated Polysomnography Scoring at Scale

Automatic polysomnography analysis can be leveraged to shorten scoring times, reduce associated costs, and ultimately improve the overall diagnosis of sleep disorders. Multiple and diverse strategies have been attempted for implementation of this technology at scale in the routine workflow of sleep centers. The field, however, is complex and presents unsolved challenges in a number of areas. Recent developments in computer science and artificial intelligence are nevertheless closing the gap. Technological advances are also opening new pathways for expanding our current understanding of the domain and its analysis.

MSED: A Multi-Modal Sleep Event Detection Model for Clinical Sleep Analysis

Clinical sleep analysis require manual analysis of sleep patterns for correct diagnosis of sleep disorders. However, several studies have shown significant variability in manual scoring of clinically relevant discrete sleep events, such as arousals, leg movements, and sleep disordered breathing (apneas and hypopneas). We investigated whether an automatic method could be used for event detection and if a model trained on all events (joint model) performed better than corresponding event-specific models (single-event models). We trained a deep neural network event detection model on 1653 individual recordings and tested the optimized model on 1000 separate hold-out recordings. F1 scores for the optimized joint detection model were 0.70, 0.63, and 0.62 for arousals, leg movements, and sleep disordered breathing, respectively, compared to 0.65, 0.61, and 0.60 for the optimized single-event models. Index values computed from detected events correlated positively with manual annotations (r2 = 0.73, r2 = 0.77, r2 = 0.78, respectively). We furthermore quantified model accuracy based on temporal difference metrics, which improved overall by using the joint model compared to single-event models. Our automatic model jointly detects arousals, leg movements and sleep disordered breathing events with high correlation with human annotations. Finally, we benchmark against previous state-of-the-art multi-event detection models and found an overall increase in F1 score with our proposed model despite a 97.5% reduction in model size.

Automatic monitoring of obstructive sleep apnea based on multi-modal signals by phone and smartwatch

Obstructive Sleep Apnea (OSA) is the most common sleep-related breathing disorder, with an overall population prevalence ranging from 9% to 38%, and it is associated with many cardiovascular diseases. The diagnosis of OSA requires polysomnography (PSG) testing, which is unsuitable for large-scale preliminary screening due to its high cost and discomfort to wear. Therefore, a simple and inexpensive screening method would be of great value. This study presents a novel at-home OSA screening method using a smartwatch and a smartphone to obtain several physiological signals, snoring segments, and questionnaire information during a whole night's sleep. The proposed method can distinguish four OSA risk levels based on machine learning (ML) classifications; the system was validated by conducting an in-hospital study on 350 subjects with sleep disorders. The estimated OSA risk levels are in good agreement with the OSA severity diagnosed by PSG (correlation with apnea-hypopnea index (AHI) = 0.92), and an encouraging classification performance is achieved (accuracy = 88.1%, 84.5%, 85.1%, sensitivity = 89.1%, 84.2%, 85.6% for mild, moderate and severe OSA). These findings reveal that wearable devices have the potential for large-scale OSA screening.

Automated Sleep Stages Classification Using Convolutional Neural Network From Raw and Time-Frequency Electroencephalogram Signals: Systematic Evaluation Study

BACKGROUND

Most existing automated sleep staging methods rely on multimodal data, and scoring a specific epoch requires not only the current epoch but also a sequence of consecutive epochs that precede and follow the epoch.

OBJECTIVE

We proposed and tested a convolutional neural network called SleepInceptionNet, which allows sleep classification of a single epoch using a single-channel electroencephalogram (EEG).

METHODS

SleepInceptionNet is based on our systematic evaluation of the effects of different EEG preprocessing methods, EEG channels, and convolutional neural networks on automatic sleep staging performance. The evaluation was performed using polysomnography data of 883 participants (937,975 thirty-second epochs). Raw data of individual EEG channels (ie, frontal, central, and occipital) and 3 specific transformations of the data, including power spectral density, continuous wavelet transform, and short-time Fourier transform, were used separately as the inputs of the convolutional neural network models. To classify sleep stages, 7 sequential deep neural networks were tested for the 1D data (ie, raw EEG and power spectral density), and 16 image classifier convolutional neural networks were tested for the 2D data (ie, continuous wavelet transform and short-time Fourier transform time-frequency images).

RESULTS

The best model, SleepInceptionNet, which uses time-frequency images developed by the continuous wavelet transform method from central single-channel EEG data as input to the InceptionV3 image classifier algorithm, achieved a Cohen κ agreement of 0.705 (SD 0.077) in reference to the gold standard polysomnography.

CONCLUSIONS

SleepInceptionNet may allow real-time automated sleep staging in free-living conditions using a single-channel EEG, which may be useful for on-demand intervention or treatment during specific sleep stages.

Deep learning-based algorithm accurately classifies sleep stages in preadolescent children with sleep-disordered breathing symptoms and age-matched controls

Introduction

Visual sleep scoring has several shortcomings, including inter-scorer inconsistency, which may adversely affect diagnostic decision-making. Although automatic sleep staging in adults has been extensively studied, it is uncertain whether such sophisticated algorithms generalize well to different pediatric age groups due to distinctive EEG characteristics. The preadolescent age group (10-13-year-olds) is relatively understudied, and thus, we aimed to develop an automatic deep learning-based sleep stage classifier specifically targeting this cohort.

Methods

A dataset (n = 115) containing polysomnographic recordings of Icelandic preadolescent children with sleep-disordered breathing (SDB) symptoms, and age and sex-matched controls was utilized. We developed a combined convolutional and long short-term memory neural network architecture relying on electroencephalography (F4-M1), electrooculography (E1-M2), and chin electromyography signals. Performance relative to human scoring was further evaluated by analyzing intra- and inter-rater agreements in a subset (n = 10) of data with repeat scoring from two manual scorers.

Results

The deep learning-based model achieved an overall cross-validated accuracy of 84.1% (Cohen's kappa κ = 0.78). There was no meaningful performance difference between SDB-symptomatic (n = 53) and control subgroups (n = 52) [83.9% (κ = 0.78) vs. 84.2% (κ = 0.78)]. The inter-rater reliability between manual scorers was 84.6% (κ = 0.78), and the automatic method reached similar agreements with scorers, 83.4% (κ = 0.76) and 82.7% (κ = 0.75).

Conclusion

The developed algorithm achieved high classification accuracy and substantial agreements with two manual scorers; the performance metrics compared favorably with typical inter-rater reliability between manual scorers and performance reported in previous studies. These suggest that our algorithm may facilitate less labor-intensive and reliable automatic sleep scoring in preadolescent children.

Computer-assisted analysis of polysomnographic recordings improves inter-scorer associated agreement and scoring times

STUDY OBJECTIVES

To investigate inter-scorer agreement and scoring time differences associated with visual and computer-assisted analysis of polysomnographic (PSG) recordings.

METHODS

A group of 12 expert scorers reviewed 5 PSGs that were independently selected in the context of each of the following tasks: (i) sleep staging, (ii) scoring of leg movements, (iii) detection of respiratory (apneic-related) events, and (iv) of electroencephalographic (EEG) arousals. All scorers independently reviewed the same recordings, hence resulting in 20 scoring exercises per scorer from an equal amount of different subjects. The procedure was repeated, separately, using the classical visual manual approach and a computer-assisted (semi-automatic) procedure. Resulting inter-scorer agreement and scoring times were examined and compared among the two methods.

RESULTS

Computer-assisted sleep scoring showed a consistent and statistically relevant effect toward less time required for the completion of each of the PSG scoring tasks. Gain factors ranged from 1.26 (EEG arousals) to 2.41 (leg movements). Inter-scorer kappa agreement was also consistently increased with the use of supervised semi-automatic scoring. Specifically, agreement increased from Κ = 0.76 to K = 0.80 (sleep stages), Κ = 0.72 to K = 0.91 (leg movements), Κ = 0.55 to K = 0.66 (respiratory events), and Κ = 0.58 to Κ = 0.65 (EEG arousals). Inter-scorer agreement on the examined set of diagnostic indices did also show a trend toward higher Interclass Correlation Coefficient scores when using the semi-automatic scoring approach.

CONCLUSIONS

Computer-assisted analysis can improve inter-scorer agreement and scoring times associated with the review of PSG studies resulting in higher efficiency and overall quality in the diagnosis sleep disorders.

Association Between Sleep Microstructure and Incident Hypertension in a Population‐Based Sample: The HypnoLaus Study

Background

Poor sleep quality is associated with increased incident hypertension. However, few studies have investigated the impact of objective sleep structure parameters on hypertension. This study investigated the association between sleep macrostructural and microstructural parameters and incident hypertension in a middle‐ to older‐aged sample.

Methods and Results

Participants from the HypnoLaus population‐based cohort without hypertension at baseline were included. Participants had at‐home polysomnography at baseline, allowing assessment of sleep macrostructure (nonrapid eye movement sleep stages 1, 2, and 3; rapid eye movement sleep stages; and total sleep time) and microstructure including power spectral density of electroencephalogram in nonrapid eye movement sleep and spindles characteristics (density, duration, frequency, amplitude) in nonrapid eye movement sleep stage 2. Associations between sleep macrostructure and microstructure parameters at baseline and incident clinical hypertension over a mean follow‐up of 5.2 years were assessed with multiple‐adjusted logistic regression. A total of 1172 participants (42% men; age 55±10 years) were included. Of these, 198 (17%) developed hypertension. After adjustment for confounders, no sleep macrostructure features were associated with incident hypertension. However, low absolute delta and sigma power were significantly associated with incident hypertension where participants in the lowest quartile of delta and sigma had a 1.69‐fold (95% CI, 1.00–2.89) and 1.72‐fold (95% CI, 1.05–2.82) increased risk of incident hypertension, respectively, versus those in the highest quartile. Lower spindle density (odds ratio, 0.87; 95% CI, 0.76–0.99) and amplitude (odds ratio, 0.98; 95% CI, 0.95–1.00) were also associated with higher incident hypertension.

Conclusions

Sleep microstructure is associated with incident hypertension. Slow‐wave activity and sleep spindles, 2 hallmarks of objective sleep continuity and quality, were inversely and consistently associated with incident hypertension. This supports the protective role of sleep continuity in the development of hypertension.

Slow wave synchronization and sleep state transitions

Spontaneous synchronization over large networks is ubiquitous in nature, ranging from inanimate to biological systems. In the human brain, neuronal synchronization and de-synchronization occur during sleep, with the greatest degree of neuronal synchronization during slow wave sleep (SWS). The current sleep classification schema is based on electroencephalography and provides common criteria for clinicians and researchers to describe stages of non-rapid eye movement (NREM) sleep as well as rapid eye movement (REM) sleep. These sleep stage classifications have been based on convenient heuristic criteria, with little consideration of the accompanying normal physiological changes across those same sleep stages. To begin to resolve those inconsistencies, first focusing only on NREM sleep, we propose a simple cluster synchronization model to explain the emergence of SWS in healthy people without sleep disorders. We apply the empirical mode decomposition (EMD) analysis to quantify slow wave activity in electroencephalograms, and provide quantitative evidence to support our model. Based on this synchronization model, NREM sleep can be classified as SWS and non-SWS, such that NREM sleep can be considered as an intrinsically bistable process. Finally, we develop an automated algorithm for SWS classification. We show that this new approach can unify brain wave dynamics and their corresponding physiologic changes.

Adherence Index: sleep depth and nocturnal hypoventilation predict long-term adherence with positive airway pressure therapy in severe obstructive sleep apnea

STUDY OBJECTIVES

Treatment of obstructive sleep apnea with positive airway pressure (PAP) devices is limited by poor long-term adherence. Early identification of individual patients' probability of long-term PAP adherence would help in their management. We determined whether conventional polysomnogram (PSG) scoring and measures of sleep depth based on the odds ratio product would predict adherence with PAP therapy 12 months after it was started.

METHODS

Patients with obstructive sleep apnea referred to an academic sleep center had split-night PSG, arterial blood gases, and a sleep questionnaire. Multiple linear regression analysis of conventional PSG scoring and the odds ratio product both during diagnostic PSG and PAP titration provided an "Adherence Index," which was correlated with PAP use 12 months later.

RESULTS

Patients with obstructive sleep apnea (n = 236, apnea-hypopnea index 72.2 ± 34.1 events/h) were prescribed PAP therapy (82% received continuous PAP, 18% received bilevel PAP). Each patient's adherence with PAP therapy 12 months later was categorized as "never used," "quit using," "poor adherence," and "good adherence." PSG measures that were most strongly correlated with PAP adherence were apnea-hypopnea index and odds ratio product during nonrapid eye movement sleep; the additional contribution of nocturnal hypoxemia to this correlation was confined to those with chronic hypoventilation treated with bilevel PAP. The Adherence Index derived from these measures, during both diagnostic PSG and PAP titration, was strongly correlated with PAP adherence 12 months later.

CONCLUSIONS

Long-term adherence with PAP therapy can be predicted from diagnostic PSG in patients with severe obstructive sleep apnea, which may facilitate a precision-based approach to PAP management.

CITATION

Younes MK, Beaudin AE, Raneri JK, Gerardy BJ, Hanly PJ. Adherence Index: sleep depth and nocturnal hypoventilation predict long-term adherence with positive airway pressure therapy in severe obstructive sleep apnea. J Clin Sleep Med. 2022:18(8):1933-1944.

SleepSEEG: Automatic sleep scoring using intracranial EEG recordings only

OBJECTIVE

To perform automatic sleep scoring based only on intracranial EEG, without the need for scalp electroencephalography (EEG), electrooculography (EOG) and electromyography (EMG), in order to study sleep, epilepsy, and their interaction.

APPROACH

Data from 33 adult patients was used for development and training of the automatic scoring algorithm using both oscillatory and non-oscillatory spectral features. The first step consisted in unsupervised clustering of channels based on feature variability. For each cluster the classification was done in two steps, a multiclass tree followed by binary classification trees to distinguish the more challenging stage N1. The test data consisted in 11 patients, in whom the classification was done independently for each channel and then combined to get a single stage per epoch.

MAIN RESULTS

An overall agreement of 78% was observed in the test set between the sleep scoring of the algorithm and two human experts scoring based on scalp EEG, EOG and EMG. Balanced sensitivity and specificity were obtained for the different sleep stages. The performance was excellent for stages W, N2, and N3, and good for stage R, but with high variability across patients. The performance for the challenging stage N1 was poor, but at a similar level as for published algorithms based on scalp EEG. High confidence epochs in different stages (other than N1) can be identified with median per patient specificity >80%.

SIGNIFICANCE

The automatic algorithm can perform sleep scoring of long term recordings of patients with intracranial electrodes undergoing presurgical evaluation in the absence of scalp EEG, EOG and EMG, which are normally required to define sleep stages but are difficult to use in the context of intracerebral studies. It also constitutes a valuable tool to generate hypotheses regarding local aspects of sleep, and will be significant for sleep evaluation in clinical epileptology and neuroscience research.

Patient-Ventilator Synchronization During Non-invasive Ventilation: A Pilot Study of an Automated Analysis System

Background: Patient-ventilator synchronization during non-invasive ventilation (NIV) can be assessed by visual inspection of flow and pressure waveforms but it remains time consuming and there is a large inter-rater variability, even among expert physicians. SyncSmart™ software developed by Breas Medical (Mölnycke, Sweden) provides an automatic detection and scoring of patient-ventilator asynchrony to help physicians in their daily clinical practice. This study was designed to assess performance of the automatic scoring by the SyncSmart software using expert clinicians as a reference in patient with chronic respiratory failure receiving NIV.

Methods: From nine patients, 20 min data sets were analyzed automatically by SyncSmart software and reviewed by nine expert physicians who were asked to score auto-triggering (AT), double-triggering (DT), and ineffective efforts (IE). The study procedure was similar to the one commonly used for validating the automatic sleep scoring technique. For each patient, the asynchrony index was computed by automatic scoring and each expert, respectively. Considering successively each expert scoring as a reference, sensitivity, specificity, positive predictive value (PPV), κ-coefficients, and agreement were calculated.

Results: The asynchrony index assessed by SynSmart was not significantly different from the one assessed by the experts (18.9 ± 17.7 vs. 12.8 ± 9.4, p = 0.19). When compared to an expert, the sensitivity and specificity provided by SyncSmart for DT, AT, and IE were significantly greater than those provided by an expert when compared to another expert.

Conclusions:SyncSmart software is able to score asynchrony events within the inter-rater variability. When the breathing frequency is not too high (<24), it therefore provides a reliable assessment of patient-ventilator asynchrony; AT is over detected otherwise.

Deep Learning Application to Clinical Decision Support System in Sleep Stage Classification

Recently, deep learning for automated sleep stage classification has been introduced with promising results. However, as many challenges impede their routine application, automatic sleep scoring algorithms are not widely used. Typically, polysomnography (PSG) uses multiple channels for higher accuracy; however, the disadvantages include a requirement for a patient to stay one or more nights in the lab wearing uncomfortable sensors and wires. To avoid the inconvenience caused by the multiple channels, we aimed to develop a deep learning model for use in clinical decision support systems (CDSSs) and combined convolutional neural networks and a transformer for the supervised learning of three classes of sleep stages only with single-channel EEG data (C4-M1). The data for training, validation, and test were derived from 1590, 341, and 343 polysomnography recordings, respectively. The developed model yielded an overall accuracy of 91.4%, comparable with that of human experts. Based on the severity of obstructive sleep apnea, the model’s accuracy was 94.3%, 91.9%, 91.9%, and 90.6% in normal, mild, moderate, and severe cases, respectively. Our deep learning model enables accurate and rapid delineation of three-class sleep staging and could be useful as a CDSS for application in real-world clinical practice.

Automated sleep state classification of wide-field calcium imaging data via multiplex visibility graphs and deep learning

BACKGROUND

Wide-field calcium imaging (WFCI) allows for monitoring of cortex-wide neural dynamics in mice. When applied to the study of sleep, WFCI data are manually scored into the sleep states of wakefulness, non-REM (NREM) and REM by use of adjunct EEG and EMG recordings. However, this process is time-consuming and often suffers from low inter- and intra-rater reliability and invasiveness. Therefore, an automated sleep state classification method that operates on WFCI data alone is needed.

NEW METHOD

A hybrid, two-step method is proposed. In the first step, spatial-temporal WFCI data is mapped to multiplex visibility graphs (MVGs). Subsequently, a two-dimensional convolutional neural network (2D CNN) is employed on the MVGs to be classified as wakefulness, NREM and REM.

RESULTS

Sleep states were classified with an accuracy of 84% and Cohen's κ of 0.67. The method was also effectively applied on a binary classification of wakefulness/sleep (accuracy=0.82, κ = 0.62) and a four-class wakefulness/sleep/anesthesia/movement classification (accuracy=0.74, κ = 0.66). Gradient-weighted class activation maps revealed that the CNN focused on short- and long-term temporal connections of MVGs in a sleep state-specific manner. Sleep state classification performance when using individual brain regions was highest for the posterior area of the cortex and when cortex-wide activity was considered.

COMPARISON WITH EXISTING METHOD

On a 3-hour WFCI recording, the MVG-CNN achieved a κ of 0.65, comparable to a κ of 0.60 corresponding to the human EEG/EMG-based scoring.

CONCLUSIONS

The hybrid MVG-CNN method accurately classifies sleep states from WFCI data and will enable future sleep-focused studies with WFCI.

Exclusion of EEG-based arousals in wake epochs of polysomnography leads to underestimation of the arousal index

STUDY OBJECTIVES

There is an internal contradiction in current American Academy of Sleep Medicine (AASM) standards for arousal index (AI) calculation in polysomnography (PSG), in that arousals in sleep and wake epochs are counted, but only sleep time is used in the denominator. This study aimed to investigate the impact of including arousals scored in wake epochs on the AI.

METHODS

We compared arousal indices including (AI_inc) vs. excluding (AI_exc) awake-epoch arousals from 100 consecutive PSGs conducted for investigation of possible OSA. To determine the AI that most closely approximated 'truth', AI_inc and AI_exc were compared to an AI calculated from continuous sleep analysis (AI_cont) in a 20 PSG subgroup.

RESULTS

The median (IQR) increase in AI_inc was 5.2/h (3.5, 8.1) vs. AI_exc (AI_inc = 28.0/h (18.4, 38.9) vs. AI_exc = 22.9/h (13.1, 31.3)), equating to an increase of 25.3% (15.6, 40.8). As the AI increased, the difference increased (p < .001), with decreasing sleep efficiency and increasing AHI the strongest predictors of the difference between AI_exc and AI_inc. The absolute AI_exc-AI_cont difference (7.7/h (5.1, 13.6)) was significantly greater than the AI_inc-AI_cont difference (1.2/h (0.6, 5.7); z = -3.099, p = .002).

CONCLUSIONS

There was a notable increase in AI when including wake-epoch arousals, particularly with more severe OSA or reduced sleep efficiency. However, the AI including wake-epoch arousals best matched the 'true' continuous sleep scoring arousal index. Our study informs clinical and research practice, highlights epoch scoring pitfalls, and supports the current AASM standard arousal reporting approach for future standards.

Interrater reliability of sleep stage scoring: a meta-analysis

STUDY OBJECTIVES

We evaluated the interrater reliabilities of manual polysomnography sleep stage scoring. We included all studies that employed Rechtschaffen and Kales rules or American Academy of Sleep Medicine standards. We sought the overall degree of agreement and those for each stage.

METHODS

The keywords were "Polysomnography (PSG)," "sleep staging," "Rechtschaffen and Kales (R&K)," "American Academy of Sleep Medicine (AASM)," "interrater (interscorer) reliability," and "Cohen's kappa." We searched PubMed, OVID Medline, EMBASE, the Cochrane library, KoreaMed, KISS, and the MedRIC. The exclusion criteria included automatic scoring and pediatric patients. We collected data on scorer histories, scoring rules, numbers of epochs scored, and the underlying diseases of the patients.

RESULTS

A total of 101 publications were retrieved; 11 satisfied the selection criteria. The Cohen's kappa for manual, overall sleep scoring was 0.76, indicating substantial agreement (95% confidence interval, 0.71-0.81; P < .001). By sleep stage, the figures were 0.70, 0.24, 0.57, 0.57, and 0.69 for the W, N1, N2, N3, and R stages, respectively. The interrater reliabilities for stage N2 and N3 sleep were moderate, and that for stage N1 sleep was only fair.

CONCLUSIONS

We conducted a meta-analysis to generalize the variation in manual scoring of polysomnography and provide reference data for automatic sleep stage scoring systems. The reliability of manual scorers of polysomnography sleep stages was substantial. However, for certain stages, the results were poor; validity requires improvement.

CITATION

Lee YJ, Lee JY, Cho JH, Choi JH. Interrater reliability of sleep stage scoring: a meta-analysis. J Clin Sleep Med. 2022;18(1):193-202.

A Method for Identifying Ground Truth Labels in Regression Problems using Annotator Precision

We propose a novel method for deriving ground truth labels for regression problems that considers the precision of annotators separately for each label. This method ensures that higher performing annotators contribute more to the final landmark position which is in contrast to conventional methods that assume all annotators are equally accurate in completing the set task. In addition to describing the novel method, a set of preliminary experimental results is also provided, comparing the performance of the precision method to that of the global mean.

An open-source, high-performance tool for automated sleep staging

The clinical and societal measurement of human sleep has increased exponentially in recent years. However, unlike other fields of medical analysis that have become highly automated, basic and clinical sleep research still relies on human visual scoring. Such human-based evaluations are time-consuming, tedious, and can be prone to subjective bias. Here, we describe a novel algorithm trained and validated on +30,000 hr of polysomnographic sleep recordings across heterogeneous populations around the world. This tool offers high sleep-staging accuracy that matches human scoring accuracy and interscorer agreement no matter the population kind. The software is designed to be especially easy to use, computationally low-demanding, open source, and free. Our hope is that this software facilitates the broad adoption of an industry-standard automated sleep staging software package.

Inter-database validation of a deep learning approach for automatic sleep scoring

STUDY OBJECTIVES

Development of inter-database generalizable sleep staging algorithms represents a challenge due to increased data variability across different datasets. Sharing data between different centers is also a problem due to potential restrictions due to patient privacy protection. In this work, we describe a new deep learning approach for automatic sleep staging, and address its generalization capabilities on a wide range of public sleep staging databases. We also examine the suitability of a novel approach that uses an ensemble of individual local models and evaluate its impact on the resulting inter-database generalization performance.

METHODS

A general deep learning network architecture for automatic sleep staging is presented. Different preprocessing and architectural variant options are tested. The resulting prediction capabilities are evaluated and compared on a heterogeneous collection of six public sleep staging datasets. Validation is carried out in the context of independent local and external dataset generalization scenarios.

RESULTS

Best results were achieved using the CNN_LSTM_5 neural network variant. Average prediction capabilities on independent local testing sets achieved 0.80 kappa score. When individual local models predict data from external datasets, average kappa score decreases to 0.54. Using the proposed ensemble-based approach, average kappa performance on the external dataset prediction scenario increases to 0.62. To our knowledge this is the largest study by the number of datasets so far on validating the generalization capabilities of an automatic sleep staging algorithm using external databases.

CONCLUSIONS

Validation results show good general performance of our method, as compared with the expected levels of human agreement, as well as to state-of-the-art automatic sleep staging methods. The proposed ensemble-based approach enables flexible and scalable design, allowing dynamic integration of local models into the final ensemble, preserving data locality, and increasing generalization capabilities of the resulting system at the same time.

Detailed Assessment of Sleep Architecture With Deep Learning and Shorter Epoch-to-Epoch Duration Reveals Sleep Fragmentation of Patients With Obstructive Sleep Apnea

Traditional sleep staging with non-overlapping 30-second epochs overlooks multiple sleep-wake transitions. We aimed to overcome this by analyzing the sleep architecture in more detail with deep learning methods and hypothesized that the traditional sleep staging underestimates the sleep fragmentation of obstructive sleep apnea (OSA) patients. To test this hypothesis, we applied deep learning-based sleep staging to identify sleep stages with the traditional approach and by using overlapping 30-second epochs with 15-, 5-, 1-, or 0.5-second epoch-to-epoch duration. A dataset of 446 patients referred for polysomnography due to OSA suspicion was used to assess differences in the sleep architecture between OSA severity groups. The amount of wakefulness increased while REM and N3 decreased in severe OSA with shorter epoch-to-epoch duration. In other OSA severity groups, the amount of wake and N1 decreased while N3 increased. With the traditional 30-second epoch-to-epoch duration, only small differences in sleep continuity were observed between the OSA severity groups. With 1-second epoch-to-epoch duration, the hazard ratio illustrating the risk of fragmented sleep was 1.14 (p = 0.39) for mild OSA, 1.59 (p < 0.01) for moderate OSA, and 4.13 (p < 0.01) for severe OSA. With shorter epoch-to-epoch durations, total sleep time and sleep efficiency increased in the non-OSA group and decreased in severe OSA. In conclusion, more detailed sleep analysis emphasizes the highly fragmented sleep architecture in severe OSA patients which can be underestimated with traditional sleep staging. The results highlight the need for a more detailed analysis of sleep architecture when assessing sleep disorders.

Automatic analysis of single-channel sleep EEG in a large spectrum of sleep disorders

STUDY OBJECTIVES

To assess the performance of the single-channel automatic sleep staging (AS) software ASEEGA in adult patients diagnosed with various sleep disorders.

METHODS

Sleep recordings were included of 95 patients (38 women, 40.5 ± 13.7 years) diagnosed with insomnia (n = 23), idiopathic hypersomnia (n = 24), narcolepsy (n = 24), and obstructive sleep apnea (n = 24). Visual staging (VS) was performed by two experts (VS1 and VS2) according to the American Academy of Sleep Medicine rules. AS was based on the analysis of a single electroencephalogram channel (Cz-Pz), without any information from electro-oculography nor electromyography. The epoch-by-epoch agreement (concordance and Conger's coefficient [κ]) was compared pairwise (VS1-VS2, AS-VS1, AS-VS2) and between AS and consensual VS. Sleep parameters were also compared.

RESULTS

The pairwise agreements were: between AS and VS1, 78.6% (κ = 0.70); AS and VS2, 75.0% (0.65); and VS1 and VS2, 79.5% (0.72). Agreement between AS and consensual VS was 85.6% (0.80), with the following distribution: insomnia 85.5% (0.80), narcolepsy 83.8% (0.78), idiopathic hypersomnia 86.1% (0.68), and obstructive sleep disorder 87.2% (0.82). A significant low-amplitude scorer effect was observed for most sleep parameters, not always driven by the same scorer. Hypnograms obtained with AS and VS exhibited very close sleep organization, except for 80% of rapid eye movement sleep onset in the group diagnosed with narcolepsy missed by AS.

CONCLUSIONS

Agreement between AS and VS in sleep disorders is comparable to that reported in healthy individuals and to interexpert agreement in patients. ASEEGA could therefore be considered as a complementary sleep stage scoring tool in clinical practice, after improvement of rapid eye movement sleep onset detection.

Large-scale assessment of consistency in sleep stage scoring rules among multiple sleep centers using an interpretable machine learning algorithm

STUDY OBJECTIVES

Polysomnography is the gold standard in identifying sleep stages; however, there are discrepancies in how technicians use the standards. Because organizing meetings to evaluate this discrepancy and/or reach a consensus among multiple sleep centers is time-consuming, we developed an artificial intelligence system to efficiently evaluate the reliability and consistency of sleep scoring and hence the sleep center quality.

METHODS

An interpretable machine learning algorithm was used to evaluate the interrater reliability (IRR) of sleep stage annotation among sleep centers. The artificial intelligence system was trained to learn raters from 1 hospital and was applied to patients from the same or other hospitals. The results were compared with the experts' annotation to determine IRR. Intracenter and intercenter assessments were conducted on 679 patients without sleep apnea from 6 sleep centers in Taiwan. Centers with potential quality issues were identified by the estimated IRR.

RESULTS

In the intracenter assessment, the median accuracy ranged from 80.3%-83.3%, with the exception of 1 hospital, which had an accuracy of 72.3%. In the intercenter assessment, the median accuracy ranged from 75.7%-83.3% when the 1 hospital was excluded from testing and training. The performance of the proposed method was higher for the N2, awake, and REM sleep stages than for the N1 and N3 stages. The significant IRR discrepancy of the 1 hospital suggested a quality issue. This quality issue was confirmed by the physicians in charge of the 1 hospital.

CONCLUSIONS

The proposed artificial intelligence system proved effective in assessing IRR and hence the sleep center quality.

A comparison of automated and manual sleep staging and respiratory event recognition in a portable sleep diagnostic device with in-lab sleep study

STUDY OBJECTIVES

The objectives were to develop and validate an algorithm for editing WatchPAT scoring and assess the accuracy in an unselected clinical population as well as age and sex substrata.

METHODS

Two hundred sixty-two participants were enrolled to undergo WatchPAT simultaneously with in-lab polysomnography (PSG) recordings for developing (n = 30), optimizing (n = 62), and validating (n = 170) an algorithm to review and edit respiratory events and sleep architecture of WatchPAT recordings, which was based on visual inspection of WatchPAT signals. Apnea-hypopnea index (AHI) and sleep indices were compared with PSG-derived and automated WatchPAT indices.

RESULTS

Although estimation of total sleep time (TST) was comparable between automated and manual algorithm, estimation of rapid eye movement (REM) sleep time was markedly improved with manual editing from 0.48, 23.0 min (-43.9 to 89.8) to 0.64, 18.3 min (-32.6 to 69.1) (correlation with PSG, mean difference [reference range] from PSG, respectively). Manual scoring also improved correlation and agreement with PSG AHI from 0.65, 2.5 events/h (-24.0 to 28.9) to 0.81, -4.5 events/h (-22.5 to 13.6) as well as concordance for categorical agreement of sleep-disordered breathing severity and concordance for detecting severe REM-related sleep-disordered breathing. Interscorer reliabilities were excellent for TST and AHI, while good for REM sleep time. The automated algorithm performed better in younger than in older patients, while performed similarly between men and women with respect to concordance statistics. The manual algorithm markedly improved concordances more in older patients and women than in their counterparts.

CONCLUSIONS

Our manual editing algorithm improves correlation and agreement with PSG-derived sleep and breathing indices. Sex and age influence the accuracy of automated analysis and the performance of manual editing on AHI concordance.

Sleep Diagnostics for Home Monitoring of Sleep Apnea Patients

Objectives: Sleep time information is essential for monitoring of obstructive sleep apnea (OSA), as the severity assessment depends on the number of breathing disturbances per hour of sleep. However, clinical procedures for sleep monitoring rely on numerous uncomfortable sensors, which could affect sleeping patterns. Therefore, an automated method to identify sleep intervals from unobtrusive data is required. However, most unobtrusive sensors suffer from data loss and sensitivity to movement artifacts. Thus, current sleep detection methods are inadequate, as these require long intervals of good quality. Moreover, sleep monitoring of OSA patients is often less reliable due to heart rate disturbances, movement and sleep fragmentation. The primary aim was to develop a sleep-wake classifier for sleep time estimation of suspected OSA patients, based on single short-term segments of their cardiac and respiratory signals. The secondary aim was to define metrics to detect OSA patients directly from their predicted sleep-wake pattern and prioritize them for clinical diagnosis. Methods: This study used a dataset of 183 suspected OSA patients, of which 36 test subjects. First, a convolutional neural network was designed for sleep-wake classification based on healthier patients (AHI < 10). It employed single 30 s epochs of electrocardiograms and respiratory inductance plethysmograms. Sleep information and Total Sleep Time (TST) was derived for all patients using the short-term segments. Next, OSA patients were detected based on the average confidence of sleep predictions and the percentage of sleep-wake transitions in the predicted sleep architecture. Results: Sleep-wake classification on healthy, mild and moderate patients resulted in moderate κ scores of 0.51, 0.49, and 0.48, respectively. However, TST estimates decreased in accuracy with increasing AHI. Nevertheless, severe patients were detected with a sensitivity of 78% and specificity of 89%, and prioritized for clinical diagnosis. As such, their inaccurate TST estimate becomes irrelevant. Excluding detected OSA patients resulted in an overall estimated TST with a mean bias error of 21.9 (± 55.7) min and Pearson correlation of 0.74 to the reference. Conclusion: The presented framework offered a realistic tool for unobtrusive sleep monitoring of suspected OSA patients. Moreover, it enabled fast prioritization of severe patients for clinical diagnosis.

MetaSleepLearner: A Pilot Study on Fast Adaptation of Bio-Signals-Based Sleep Stage Classifier to New Individual Subject Using Meta-Learning

Identifying bio-signals based-sleep stages requires time-consuming and tedious labor of skilled clinicians. Deep learning approaches have been introduced in order to challenge the automatic sleep stage classification conundrum. However, the difficulties can be posed in replacing the clinicians with the automatic system due to the differences in many aspects found in individual bio-signals, causing the inconsistency in the performance of the model on every incoming individual. Thus, we aim to explore the feasibility of using a novel approach, capable of assisting the clinicians and lessening the workload. We propose the transfer learning framework, entitled MetaSleepLearner, based on Model Agnostic Meta-Learning (MAML), in order to transfer the acquired sleep staging knowledge from a large dataset to new individual subjects (source code is available at https://github.com/IoBT-VISTEC/MetaSleepLearner). The framework was demonstrated to require the labelling of only a few sleep epochs by the clinicians and allow the remainder to be handled by the system. Layer-wise Relevance Propagation (LRP) was also applied to understand the learning course of our approach. In all acquired datasets, in comparison to the conventional approach, MetaSleepLearner achieved a range of 5.4% to 17.7% improvement with statistical difference in the mean of both approaches. The illustration of the model interpretation after the adaptation to each subject also confirmed that the performance was directed towards reasonable learning. MetaSleepLearner outperformed the conventional approaches as a result from the fine-tuning using the recordings of both healthy subjects and patients. This is the first work that investigated a non-conventional pre-training method, MAML, resulting in a possibility for human-machine collaboration in sleep stage classification and easing the burden of the clinicians in labelling the sleep stages through only several epochs rather than an entire recording.

Automatic sleep stage classification with deep residual networks in a mixed-cohort setting

STUDY OBJECTIVES

Sleep stage scoring is performed manually by sleep experts and is prone to subjective interpretation of scoring rules with low intra- and interscorer reliability. Many automatic systems rely on few small-scale databases for developing models, and generalizability to new datasets is thus unknown. We investigated a novel deep neural network to assess the generalizability of several large-scale cohorts.

METHODS

A deep neural network model was developed using 15,684 polysomnography studies from five different cohorts. We applied four different scenarios: (1) impact of varying timescales in the model; (2) performance of a single cohort on other cohorts of smaller, greater, or equal size relative to the performance of other cohorts on a single cohort; (3) varying the fraction of mixed-cohort training data compared with using single-origin data; and (4) comparing models trained on combinations of data from 2, 3, and 4 cohorts.

RESULTS

Overall classification accuracy improved with increasing fractions of training data (0.25%: 0.782 ± 0.097, 95% CI [0.777-0.787]; 100%: 0.869 ± 0.064, 95% CI [0.864-0.872]), and with increasing number of data sources (2: 0.788 ± 0.102, 95% CI [0.787-0.790]; 3: 0.808 ± 0.092, 95% CI [0.807-0.810]; 4: 0.821 ± 0.085, 95% CI [0.819-0.823]). Different cohorts show varying levels of generalization to other cohorts.

CONCLUSIONS

Automatic sleep stage scoring systems based on deep learning algorithms should consider as much data as possible from as many sources available to ensure proper generalization. Public datasets for benchmarking should be made available for future research.

Deep learning enables sleep staging from photoplethysmogram for patients with suspected sleep apnea

Study Objectives

Accurate identification of sleep stages is essential in the diagnosis of sleep disorders (e.g. obstructive sleep apnea [OSA]) but relies on labor-intensive electroencephalogram (EEG)-based manual scoring. Furthermore, long-term assessment of sleep relies on actigraphy differentiating only between wake and sleep periods without identifying specific sleep stages and having low reliability in identifying wake periods after sleep onset. To address these issues, we aimed to develop an automatic method for identifying the sleep stages from the photoplethysmogram (PPG) signal obtained with a simple finger pulse oximeter.

Methods

PPG signals from the diagnostic polysomnographies of susptected OSA patients (n = 894) were utilized to develop a combined convolutional and recurrent neural network. The deep learning model was trained individually for three-stage (wake/NREM/REM), four-stage (wake/N1+N2/N3/REM), and five-stage (wake/N1/N2/N3/REM) classification of sleep.

Results

The three-stage model achieved an epoch-by-epoch accuracy of 80.1% with Cohen’s κ of 0.65. The four- and five-stage models achieved 68.5% (κ = 0.54), and 64.1% (κ = 0.51) accuracies, respectively. With the five-stage model, the total sleep time was underestimated with a mean bias error (SD) of of 7.5 (55.2) minutes.

Conclusion

The PPG-based deep learning model enabled accurate estimation of sleep time and differentiation between sleep stages with a moderate agreement to manual EEG-based scoring. As PPG is already included in ambulatory polygraphic recordings, applying the PPG-based sleep staging could improve their diagnostic value by enabling simple, low-cost, and reliable monitoring of sleep and help assess otherwise overlooked conditions such as REM-related OSA.

Ontology-Based Decision Support Tool for Automatic Sleep Staging Using Dual-Channel EEG Data

Sleep staging has attracted significant attention as a critical step in auxiliary diagnosis of sleep disease. To avoid subjectivity of doctor’s manual sleep staging, and to realize scientific management of massive physiological data, an ontology-based decision support tool is proposed. The tool implements an automated procedure for sleep staging using dual-channel electroencephalogram (EEG) signals. First of all, it encodes EEG features, sleep-related concepts and other contextual information to “EEG-Sleep ontology”. Secondly, a rule-set is constructed based on a data mining technique. Finally, the first two steps are processed in a reasoning engine which is automatically assign each 30 s epoch (segment) sleep stage to one of five possible sleep stages: WA, NREM1, NREM2, SWS and REM. The rule set is obtained using EEG data taken from the Sleep-EDF database [EXPANDED] according to the random forest algorithm (RF), we prove that the performance of the proposed method with 89.12% accuracy, and 0.81 Kappa statistics is superior to other algorithms such as Bayesian network, C4.5, support vector machine, and multilayer perceptron. Additionally, our proposed approach improved performance when compared to other studies using a small subset of the Sleep-EDF database [EXPANDED].

Deep transfer learning for improving single-EEG arousal detection

Datasets in sleep science present challenges for machine learning algorithms due to differences in recording setups across clinics. We investigate two deep transfer learning strategies for overcoming the channel mismatch problem for cases where two datasets do not contain exactly the same setup leading to degraded performance in single-EEG models. Specifically, we train a baseline model on multivariate polysomnography data and subsequently replace the first two layers to prepare the architecture for single-channel electroencephalography data. Using a fine-tuning strategy, our model yields similar performance to the baseline model (F1=0.682 and F1=0.694, respectively), and was significantly better than a comparable single-channel model. Our results are promising for researchers working with small databases who wish to use deep learning models pre-trained on larger databases.

Reliability of Family Dogs' Sleep Structure Scoring Based on Manual and Automated Sleep Stage Identification

Non-invasive polysomnography recording on dogs has been claimed to produce data comparable to those for humans regarding sleep macrostructure, EEG spectra and sleep spindles. While functional parallels have been described relating to both affective (e.g., emotion processing) and cognitive (e.g., memory consolidation) domains, methodologically relevant questions about the reliability of sleep stage scoring still need to be addressed. In Study 1, we analyzed the effects of different coders and different numbers of visible EEG channels on the visual scoring of the same polysomnography recordings. The lowest agreement was found between independent coders with different scoring experience using full (3 h-long) recordings of the whole dataset, and the highest agreement within-coder, using only a fraction of the original dataset (randomly selected 100 epochs (i.e., 100 × 20 s long segments)). The identification of drowsiness was found to be the least reliable, while that of non-REM (rapid eye movement, NREM) was the most reliable. Disagreements resulted in no or only moderate differences in macrostructural and spectral variables. Study 2 targeted the task of automated sleep EEG time series classification. Supervised machine learning (ML) models were used to help the manual annotation process by reliably predicting if the dog was sleeping or awake. Logistic regression models (LogREG), gradient boosted trees (GBT) and convolutional neural networks (CNN) were set up and trained for sleep state prediction from already collected and manually annotated EEG data. The evaluation of the individual models suggests that their combination results in the best performance: ~0.9 AUC test scores.

Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks

STUDY OBJECTIVES

Polysomnography (PSG) scoring is labor intensive and suffers from variability in inter- and intra-rater reliability. Automated PSG scoring has the potential to reduce the human labor costs and the variability inherent to this task. Deep learning is a form of machine learning that uses neural networks to recognize data patterns by inspecting many examples rather than by following explicit programming.

METHODS

A sleep staging classifier trained using deep learning methods scored PSG data from the Sleep Heart Health Study (SHHS). The training set was composed of 42 560 hours of PSG data from 5213 patients. To capture higher-order data, spectrograms were generated from electroencephalography, electrooculography, and electromyography data and then passed to the neural network. A holdout set of 580 PSGs not included in the training set was used to assess model accuracy and discrimination via weighted F1-score, per-stage accuracy, and Cohen's kappa (K).

RESULTS

The optimal neural network model was composed of spectrograms in the input layer feeding into convolutional neural network layers and a long short-term memory layer to achieve a weighted F1-score of 0.87 and K = 0.82.

CONCLUSIONS

The deep learning sleep stage classifier demonstrates excellent accuracy and agreement with expert sleep stage scoring, outperforming human agreement on sleep staging. It achieves comparable or better F1-scores, accuracy, and Cohen's kappa compared to literature for automated sleep stage scoring of PSG epochs. Accurate automated scoring of other PSG events may eventually allow for fully automated PSG scoring.

Sleep Staging Monitoring Based on Sonar Smartphone Technology

This paper presents the validation results of a new non-contact ultrasonic technology, which employs inaudible Sonar to monitor the movements and respiration of a subject in bed. Sleep monitoring can be achieved by placing a smartphone onto the bedside table and starting a custom app. The app employs sophisticated and novel proprietary algorithms to identify sleep stages: Wake (W), Light Sleep (N1, N2 sleep), Deep Sleep (N3 sleep), Rapid Eye Movement (REM) Sleep or Absence.The sleep staging performance of the app were assessed by testing it against expert manually scored polysomnography (PSG) of 38 subjects gathered in a sleep laboratory. As a secondary assessment, on the same dataset, the performance of the app is compared to that of a reference non-contact device, the S+ by ResMed.Performance across different sleep stage detections was balanced, exceeding the agreement typically reported for actigraphy based devices [1], [2] thanks to a significantly higher sensitivity for all sleep stages. Furthermore, the performance of the app was found to be comparable to the S+ by ResMed product [3], [4].The combination of unobtrusive non-contact sensing and accurate sleep quality assessment, coupled with removal of the requirement to purchase a custom device to enable monitoring of sleep, enables consumers to measure their sleep in the home environment in a zero-cost and accessible manner, while providing sleep staging information not otherwise available with actigraphy based devices.