Performance of an automated polysomnography scoring system versus computer-assisted manual scoring

Temporal and sleep stage-dependent agreement in manual scoring of respiratory events

Obstructive sleep apnea diagnosis is based on the manual scoring of respiratory events. The agreement in the manual scoring of the respiratory events lacks an in-depth investigation as most of the previous studies reported only the apnea-hypopnea index or overall agreement, and not temporal, second-by-second or event subtype agreement. We hypothesized the temporal and subtype agreement to be low because the event duration or subtypes are not generally considered in current clinical practice. The data comprised 50 polysomnography recordings scored by 10 experts. The respiratory event agreement between the scorers was calculated using kappa statistics in a second-by-second manner. Obstructive sleep apnea severity categories (no obstructive sleep apnea/mild/moderate/severe) were compared between scorers. The Fleiss' kappa value for binary (event/no event) respiratory event scorings was 0.32. When calculated separately within N1, N2, N3 and R, the Fleiss' kappa values were 0.12, 0.23, 0.22 and 0.23, respectively. Binary analysis conducted separately for the event subtypes showed the highest Fleiss' kappa for hypopneas to be 0.26. In 34% of the participants, the obstructive sleep apnea severity category was the same regardless of the scorer, whereas in the rest of the participants the category changed depending on the scorer. Our findings indicate that the agreement of manual scoring of respiratory events depends on the event type and sleep stage. The manual scoring has discrepancies, and these differences affect the obstructive sleep apnea diagnosis. This is an alarming finding, as ultimately these differences in the scorings affect treatment decisions.

An ensemble method for improving robustness against the electrode contact problems in automated sleep stage scoring

In-home automated scoring systems are in high demand; however, the current systems are not widely adopted in clinical settings. Problems with electrode contact and restriction on measurable signals often result in unstable and inaccurate scoring for clinical use. To address these issues, we propose a method based on ensemble of small sleep stage scoring models with different input signal sets. By excluding models that employ problematic signals from the voting process, our method can mitigate the effects of electrode contact failure. Comparative experiments demonstrated that our method could reduce the impact of contact problems and improve scoring accuracy for epochs with problematic signals by 8.3 points, while also decreasing the deterioration in scoring accuracy from 7.9 to 0.3 points compared to typical methods. Additionally, we confirmed that assigning different input sets to small models did not diminish the advantages of the ensemble but instead increased its efficacy. The proposed model can improve overall scoring accuracy and minimize the effect of problematic signals simultaneously, making in-home sleep stage scoring systems more suitable for clinical practice.

Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review

Purpose

This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain (ID) and out-of-domain (OOD) data, and considering subjects' diagnoses.

Patients and Methods

A total of 19,578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of an additional 8832 PSGs, covering a full spectrum of ages (0-91 years) and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician.

Results

U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve κ ≥ 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians' workload, and facilitating near-perfect agreement.

Conclusion

Inter-scorer variability limits the accuracy of the scoring algorithms to ~80%. By integrating an uncertainty estimation with U-Sleep, we enhance the review of predicted hypnograms, to align with the scoring taste of a responsible physician. Validated across ID and OOD data and various sleep-disorders, our approach offers a strategy to boost automated scoring tools' usability in clinical settings.

A sleep staging model on wavelet-based adaptive spectrogram reconstruction and light weight CNN

Effective methods for automatic sleep staging are important for diagnosis and treatment of sleep disorders. EEG has weak signal properties and complex frequency components during the transition of sleep stages. Wavelet-based adaptive spectrogram reconstruction (WASR) by seed growth is utilized to capture dominant time-frequency patterns of sleep EEG. We introduced variant energy from Teager operator in WASR to capture hidden dynamic patterns of EEG, which produced additional spectrograms. These spectrograms enabled a light weight CNN to detect and extract finer details of different sleep stages, which improved the feature representation of EEG. With specially designed depthwise separable convolution, the light weight CNN achieved more robust sleep stage classification. Experimental results on Sleep-EDF 20 dataset showed that our proposed model yielded overall accuracy of 87.6%, F1-score of 82.1%, and Cohen kappa of 0.83, which is competitive compared with baselines with reduced computation cost.

Achieving Better Understanding of Obstructive Sleep Apnea Treatment Effects on Cardiovascular Disease Outcomes through Machine Learning Approaches: A Narrative Review

Obstructive sleep apnea (OSA) affects almost a billion people worldwide and is associated with a myriad of adverse health outcomes. Among the most prevalent and morbid are cardiovascular diseases (CVDs). Nonetheless, randomized controlled trials (RCTs) of OSA treatment have failed to show improvements in CVD outcomes. A major limitation in our field is the lack of precision in defining OSA and specifically subgroups with the potential to benefit from therapy. Further, this has called into question the validity of using the time-honored apnea-hypopnea index as the ultimate defining criteria for OSA. Recent applications of advanced statistical methods and machine learning have brought to light a variety of OSA endotypes and phenotypes. These methods also provide an opportunity to understand the interaction between OSA and comorbid diseases for better CVD risk stratification. Lastly, machine learning and specifically heterogeneous treatment effects modeling can help uncover subgroups with differential outcomes after treatment initiation. In an era of data sharing and big data, these techniques will be at the forefront of OSA research. Advanced data science methods, such as machine-learning analyses and artificial intelligence, will improve our ability to determine the unique influence of OSA on CVD outcomes and ultimately allow us to better determine precision medicine approaches in OSA patients for CVD risk reduction. In this narrative review, we will highlight how team science via machine learning and artificial intelligence applied to existing clinical data, polysomnography, proteomics, and imaging can do just that.

Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement

STUDY OBJECTIVES

To efficiently improve the scoring competency of scorers with varying levels of experience across regions in Taiwan, we developed a training program with a cloud-based polysomnography scoring platform to evaluate and improve interscorer agreement.

METHODS

A total of 70 scorers from 34 sleep centers in Taiwan (job tenure: 0.5-39.0 years) completed a scoring test. All scorers scored a 742-epoch (30 s/epoch) overnight polysomnography recording of a patient with a moderate apnea-hypopnea index. Subsequently, 8 scoring experts delivered 8 interactive online lectures (each lasting 30 minutes). The training program included identifying scoring weaknesses, highlighting the latest scoring rules, and providing physicians' perspectives. Afterward, the scorers completed the second scoring test on the same participant. Changes in agreement from the first to second scoring test were identified. Sleep staging, sleep parameters, and respiratory events were considered for evaluating scoring agreement.

RESULTS

The scorers' agreement in overall sleep stage scoring significantly increased from 74.6 to 82.3% (median score). The proportion of scorers with an agreement of ≥ 80% increased from 20.0% (14/70) to 58.6% (41/70) after the online training program. In addition, the scorers' agreement in overall respiratory-event scoring increased to 88.8% (median score) after training. The scorers with a job tenure of 2.0-4.9 years exhibited the highest level of improvement in overall sleep staging (their median agreement increased from 72.8 to 84.9%; P < .001).

CONCLUSIONS

Our interactive online training program efficiently targeted the scorers' scoring weaknesses identified in the first scoring test, leading to substantial improvements in scoring proficiency.

CITATION

Liao Y-S, Wu M-C, Li C-X, Lin W-K, Lin C-Y, Liang S-F. Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement. J Clin Sleep Med. 2024;20(2):271-278.

A deep learning algorithm model to automatically score and grade obstructive sleep apnea in adult polysomnography

Objective

Polysomnography (PSG) is unique in diagnosing sleep disorders, notably obstructive sleep apnea (OSA). Despite its advantages, manual PSG data grading is time-consuming and laborious. Thus, this research evaluated a deep learning-based automated scoring system for respiratory events in sleep-disordered breathing patients.

Methods

A total of 1000 case PSG data were enrolled to develop a deep learning algorithm. Of the 1000 data, 700 were distributed for training, 200 for validation, and 100 for testing. The respiratory events scoring deep learning model is composed of five sequential layers: an initial layer of perceptrons, followed by three consecutive layers of long short-term memory cells, and ultimately, an additional two layers of perceptrons.

Results

The PSG data of 100 patients (simple snoring, mild, moderate, and severe OSA; n = 25 in each group) were selected for validation and testing of the deep learning model. The algorithm demonstrated high sensitivity (95% CI: 98.06-98.51) and specificity (95% CI: 95.46-97.79) across all OSA severities in detecting apnea/hypopnea events, compared to manual PSG analysis. The deep learning model's area under the curve values for predicting OSA in apnea-hypopnea index ≥ 5, 15, and 30 groups were 0.9402, 0.9388, and 0.9442, respectively, showing no significant differences between each group.

Conclusion

The deep learning algorithm employed in our study showed high accuracy in identifying apnea/hypopnea episodes and assessing the severity of OSA, suggesting the potential for enhancing both the efficiency and accuracy of automated respiratory event scoring in PSG through advanced deep learning techniques.

An approach for determining the reliability of manual and digital scoring of sleep stages

STUDY OBJECTIVES

Inter-scorer variability in sleep staging is largely due to equivocal epochs that contain features of more than one stage. We propose an approach that recognizes the existence of equivocal epochs and evaluates scorers accordingly.

METHODS

Epoch-by-epoch staging was performed on 70 polysomnograms by six qualified technologists and by a digital system (Michele Sleep Scoring [MSS]). Probability that epochs assigned the same stage by only two of the six technologists (minority score) resulted from random occurrence of two errors was calculated and found to be <5%, thereby indicating that the stage assigned is an acceptable variant for the epoch. Acceptable stages were identified in each epoch as stages assigned by at least two technologists. Percent agreement between each technologist and the other five technologists, acting as judges, was determined. Agreement was considered to exist if the stage assigned by the tested scorer was one of the acceptable stages for the epoch. Stage assigned by MSS was likewise considered in agreement if included in the acceptable stages made by the technologists.

RESULTS

Agreement of technologists tested against five qualified judges increased from 80.8% (range 70.5%-86.4% among technologists) when using the majority rule, to 96.1 (89.8%-98.5%) by the proposed approach. Agreement between unedited MSS and same judges was 90.0% and increased to 92.1% after brief editing.

CONCLUSIONS

Accounting for equivocal epochs provides a more accurate estimate of a scorer's (human or digital) competence in scoring sleep stages and reduces inter-scorer disagreements. The proposed approach can be implemented in sleep-scoring training and accreditation programs.

Automatic neonatal sleep stage classification: A comparative study

Sleep is an essential feature of living beings. For neonates, it is vital for their mental and physical development. Sleep stage cycling is an important parameter to assess neonatal brain and physical development. Therefore, it is crucial to administer newborn's sleep in the neonatal intensive care unit (NICU). Currently, Polysomnography (PSG) is used as a gold standard method for classifying neonatal sleep patterns, but it is expensive and requires a lot of human involvement. Over the last two decades, multiple researchers are working on automatic sleep stage classification algorithms using electroencephalography (EEG), electrocardiography (ECG), and video. In this study, we present a comprehensive review of existing algorithms for neonatal sleep, their limitations and future recommendations. Additionally, a brief comparison of the extracted features, classification algorithms and evaluation parameters is reported in the proposed study.

Automatic sleep staging by a hybrid model based on deep 1D-ResNet-SE and LSTM with single-channel raw EEG signals

Sleep staging is crucial for assessing sleep quality and diagnosing sleep disorders. Recent advances in deep learning methods with electroencephalogram (EEG) signals have shown remarkable success in automatic sleep staging. However, the use of deeper neural networks may lead to the issues of gradient disappearance and explosion, while the non-stationary nature and low signal-to-noise ratio of EEG signals can negatively impact feature representation. To overcome these challenges, we proposed a novel lightweight sequence-to-sequence deep learning model, 1D-ResNet-SE-LSTM, to classify sleep stages into five classes using single-channel raw EEG signals. Our proposed model consists of two main components: a one-dimensional residual convolutional neural network with a squeeze-and-excitation module to extract and reweight features from EEG signals, and a long short-term memory network to capture the transition rules among sleep stages. In addition, we applied the weighted cross-entropy loss function to alleviate the class imbalance problem. We evaluated the performance of our model on two publicly available datasets; Sleep-EDF Expanded consists of 153 overnight PSG recordings collected from 78 healthy subjects and ISRUC-Sleep includes 100 PSG recordings collected from 100 subjects diagnosed with various sleep disorders, and obtained an overall accuracy rate of 86.39% and 81.97%, respectively, along with corresponding macro average F1-scores of 81.95% and 79.94%. Our model outperforms existing sleep staging models in terms of overall performance metrics and per-class F1-scores for several sleep stages, particularly for the N1 stage, where it achieves F1-scores of 59.00% and 55.53%. The kappa coefficient is 0.812 and 0.766 for the Sleep-EDF Expanded and ISRUC-Sleep datasets, respectively, indicating strong agreement with certified sleep experts. We also investigated the effect of different weight coefficient combinations and sequence lengths of EEG epochs used as input to the model on its performance. Furthermore, the ablation study was conducted to evaluate the contribution of each component to the model's performance. The results demonstrate the effectiveness and robustness of the proposed model in classifying sleep stages, and highlights its potential to reduce human clinicians' workload, making sleep assessment and diagnosis more effective. However, the proposed model is subject to several limitations. Firstly, the model is a sequence-to-sequence network, which requires input sequences of EEG epochs. Secondly, the weight coefficients in the loss function could be further optimized to balance the classification performance of each sleep stage. Finally, apart from the channel attention mechanism, incorporating more advanced attention mechanisms could enhance the model's effectiveness.

Adoption of Transformer Neural Network to Improve the Diagnostic Performance of Oximetry for Obstructive Sleep Apnea

Scoring polysomnography for obstructive sleep apnea diagnosis is a laborious, long, and costly process. Machine learning approaches, such as deep neural networks, can reduce scoring time and costs. However, most methods require prior filtering and preprocessing of the raw signal. Our work presents a novel method for diagnosing obstructive sleep apnea using a transformer neural network with learnable positional encoding, which outperforms existing state-of-the-art solutions. This approach has the potential to improve the diagnostic performance of oximetry for obstructive sleep apnea and reduce the time and costs associated with traditional polysomnography. Contrary to existing approaches, our approach performs annotations at one-second granularity. Allowing physicians to interpret the model's outcome. In addition, we tested different positional encoding designs as the first layer of the model, and the best results were achieved using a learnable positional encoding based on an autoencoder with structural novelty. In addition, we tried different temporal resolutions with various granularity levels from 1 to 360 s. All experiments were carried out on an independent test set from the public OSASUD dataset and showed that our approach outperforms current state-of-the-art solutions with a satisfactory AUC of 0.89, accuracy of 0.80, and F1-score of 0.79.

Challenges of Applying Automated Polysomnography Scoring at Scale

Automatic polysomnography analysis can be leveraged to shorten scoring times, reduce associated costs, and ultimately improve the overall diagnosis of sleep disorders. Multiple and diverse strategies have been attempted for implementation of this technology at scale in the routine workflow of sleep centers. The field, however, is complex and presents unsolved challenges in a number of areas. Recent developments in computer science and artificial intelligence are nevertheless closing the gap. Technological advances are also opening new pathways for expanding our current understanding of the domain and its analysis.

A system based on machine learning for improving sleep

Closed-loop auditory stimulation is one of the well-known and emerging sensory stimulation techniques, which achieves the purpose of sleep regulation by driving the EEG slow oscillation (SO, <1 Hz) through auditory stimulation. The main challenge is to accurately identify the stimulation timing and provide feedback in real-time, which has high requirements on the response time and recognition accuracy of the closed-loop auditory stimulation system. To reduce the impact of systematic errors on the regulation results, most traditional closed-loop auditory stimulation systems try to identify a single feature to determine the timing of stimulus delivery and reduce the system feedback delay by simplifying the calculation. Unlike existing closed-loop regulation systems that identify specific brain features, this paper proposes a closed-loop auditory stimulation sleep regulation system deploying machine learning. The process is: through online sleep real-time automatic staging, tracking the sleep stage to provide feedback quickly, and continuously offering external auditory stimulation at a specific SO phase. This paper uses this system to conduct sleep auditory stimulation regulation experiments on ten subjects. The experimental results show that the sleep closed-loop regulation system proposed in this paper can achieve consistency (effective for almost all subjects in the experiment) and immediate (taking effect immediately after stimulation) modulation effects on SOs. More importantly, the proposed method is superior to existing advanced methods. Therefore, the system designed in this paper has great potential to be more reliable and flexible in sleep regulation.

Automatic sleep-stage classification of heart rate and actigraphy data using deep and transfer learning approaches

Manual sleep-stage scoring based on full-night polysomnography data recorded in a sleep lab has been the gold standard of clinical sleep medicine. This costly and time-consuming approach is unfit for long-term studies as well as assessment of sleep on a population level. With the vast amount of physiological data becoming available from wrist-worn devices, deep learning techniques provide an opportunity for fast and reliable automatic sleep-stage classification tasks. However, training a deep neural network requires large annotated sleep databases, which are not available for long-term epidemiological studies. In this paper, we introduce an end-to-end temporal convolutional neural network able to automatically score sleep stages from raw heartbeat RR interval (RRI) and wrist actigraphy data. Moreover, a transfer learning approach enables the training of the network on a large public database (Sleep Heart Health Study, SHHS) and its subsequent application to a much smaller database recorded by a wristband device. The transfer learning significantly shortens training time and improves sleep-scoring accuracy from 68.9% to 73.8% and inter-rater reliability (Cohen's kappa) from 0.51 to 0.59. We also found that for the SHHS database, automatic sleep-scoring accuracy using deep learning shows a logarithmic relationship with the training size. Although deep learning approaches for automatic sleep scoring are not yet comparable to the inter-rater reliability among sleep technicians, performance is expected to significantly improve in the near future when more large public databases become available. We anticipate those deep learning techniques, when combined with our transfer learning approach, will leverage automatic sleep scoring of physiological data from wearable devices and enable the investigation of sleep in large cohort studies.

Autonomic arousal detection and cardio-respiratory sleep staging improve the accuracy of home sleep apnea tests

Introduction: The apnea-hypopnea index (AHI), defined as the number of apneas and hypopneas per hour of sleep, is still used as an important index to assess sleep disordered breathing (SDB) severity, where hypopneas are confirmed by the presence of an oxygen desaturation or an arousal. Ambulatory polygraphy without neurological signals, often referred to as home sleep apnea testing (HSAT), can potentially underestimate the severity of sleep disordered breathing (SDB) as sleep and arousals are not assessed. We aim to improve the diagnostic accuracy of HSATs by extracting surrogate sleep and arousal information derived from autonomic nervous system activity with artificial intelligence. Methods: We used polysomnographic (PSG) recordings from 245 subjects (148 with simultaneously recorded HSATs) to develop and validate a new algorithm to detect autonomic arousals using artificial intelligence. A clinically validated auto-scoring algorithm (Somnolyzer) scored respiratory events, cortical arousals, and sleep stages in PSGs, and provided respiratory events and sleep stages from cardio-respiratory signals in HSATs. In a four-fold cross validation of the newly developed algorithm, we evaluated the accuracy of the estimated arousal index and HSAT-derived surrogates for the AHI. Results: The agreement between the autonomic and cortical arousal index was moderate to good with an intraclass correlation coefficient of 0.73. When using thresholds of 5, 15, and 30 to categorize SDB into none, mild, moderate, and severe, the addition of sleep and arousal information significantly improved the classification accuracy from 70.2% (Cohen's κ = 0.58) to 80.4% (κ = 0.72), with a significant reduction of patients where the severity category was underestimated from 18.8% to 7.3%. Discussion: Extracting sleep and arousal information from autonomic nervous system activity can improve the diagnostic accuracy of HSATs by significantly reducing the probability of underestimating SDB severity without compromising specificity.

Validation of a single-use and reusable home sleep apnea test based on peripheral arterial tonometry compared to laboratory polysomnography for the diagnosis of obstructive sleep apnea

STUDY OBJECTIVES

The objective of this study was to independently validate a disposable and a reusable home sleep apnea test (HSAT) recording on peripheral arterial tonometry, compared to laboratory polysomnography (PSG), for the diagnosis of obstructive sleep apnea (OSA).

METHODS

115 participants undergoing PSG for suspected OSA were recruited and fitted with the two study devices (NightOwl, Ectosense, Belgium). After exclusions were applied and device failures were removed, data from 100 participants were analyzed. HSAT-derived apnea-hypopnea index (AHI), OSA severity category, total sleep time, and oxygen desaturation index 3% were compared to PSG.

RESULTS

Both study devices demonstrated satisfactory levels of agreement with minimal mean bias for determination of AHI and oxygen desaturation index 3% (disposable: AHI mean bias 2.04 events/h [95% limits of agreement -20.9 to 25.0], oxygen desaturation index 3% mean bias -0.21/h [-18.1 to 17.7]; reusable: AHI mean bias 2.91 events/h [-16.9 to 22.7], oxygen desaturation index 3% mean bias 0.77/h [-15.7 to 17.3]). Level of agreement diminished at higher AHI levels although misclassification for severe OSA occurred infrequently. Total sleep time level of agreement for the reusable HSAT was also satisfactory with minimal mean bias (4.18 minutes, -125.1 to 112.4), but the disposable HSAT was impacted by studies with high signal rejection (23.7 minutes, -132.7 to 180.1). Categorization of OSA severity demonstrated moderate agreement with laboratory PSG, with a kappa of 0.52 and 0.57 for the disposable and reusable HSATs respectively.

CONCLUSIONS

The two HSAT devices were comparable and performed well compared to laboratory PSG for the diagnosis of OSA.

CLINICAL TRIAL REGISTRATION

Registry: Australian New Zealand Clinical Trials Registry; Identifier: ANZCTR12621000444886.

CITATION

Lyne CJ, Hamilton GS, Turton ARE, et al. Validation of a single-use and reusable home sleep apnea test based on peripheral arterial tonometry compared to laboratory polysomnography for the diagnosis of obstructive sleep apnea. J Clin Sleep Med. 2023;19(8):1429-1435.

Contribution of obstructive sleep apnea to disrupted sleep in a large clinical cohort of patients with suspected obstructive sleep apnea

STUDY OBJECTIVES

The response of sleep depth to CPAP in patients with OSA is unpredictable. The odds-ratio-product (ORP) is a continuous index of sleep depth and wake propensity that distinguishes different sleep depths within sleep stages, and different levels of vigilance during stage wake. When expressed as fractions of time spent in different ORP deciles, nine distinctive patterns are found. Only three of these are associated with OSA. We sought to determine whether sleep depth improves on CPAP exclusively in patients with these three ORP patterns.

METHODS

ORP was measured during the diagnostic and therapeutic components of 576 split-night polysomnographic (PSG) studies. ORP architecture in the diagnostic section was classified into one of the nine possible ORP patterns and the changes in sleep architecture were determined on CPAP for each of these patterns. ORP architecture was similarly determined in the first half of 760 full-night diagnostic PSG studies and the changes in the second half were measured to control for differences in sleep architecture between the early and late portions of sleep time in the absence of CPAP.

RESULTS

Frequency of the three ORP patterns increased progressively with the apnea-hypopnea index. Sleep depth improved significantly on CPAP only in the three ORP patterns associated with OSA. Changes in CPAP in the other six patterns, or in full diagnostic PSG studies, were insignificant or paradoxical.

CONCLUSIONS

ORP architecture types can identify patients in whom OSA adversely affects sleep and whose sleep is expected to improve on CPAP therapy.

A deep learning model based on the combination of convolutional and recurrent neural networks to enhance pulse oximetry ability to classify sleep stages in children with sleep apnea

Characterization of sleep stages is essential in the diagnosis of sleep-related disorders but relies on manual scoring of overnight polysomnography (PSG) recordings, which is onerous and labor-intensive. Accordingly, we aimed to develop an accurate deep-learning model for sleep staging in children suffering from pediatric obstructive sleep apnea (OSA) using pulse oximetry signals. For this purpose, pulse rate (PR) and blood oxygen saturation (SpO2) from 429 childhood OSA patients were analyzed. A CNN-RNN architecture fed with PR and SpO2 signals was developed to automatically classify wake (W), non-Rapid Eye Movement (NREM), and REM sleep stages. This architecture was composed of: (i) a convolutional neural network (CNN), which learns stage-related features from raw PR and SpO2 data; and (ii) a recurrent neural network (RNN), which models the temporal distribution of the sleep stages. The proposed CNN-RNN model showed a high performance for the automated detection of W/NREM/REM sleep stages (86.0% accuracy and 0.743 Cohen's kappa). Furthermore, the total sleep time estimated for each children using the CNN-RNN model showed high agreement with the manually derived from PSG (intra-class correlation coefficient = 0.747). These results were superior to previous works using CNN-based deep-learning models for automatic sleep staging in pediatric OSA patients from pulse oximetry signals. Therefore, the combination of CNN and RNN allows to obtain additional information from raw PR and SpO2 data related to sleep stages, thus being useful to automatically score sleep stages in pulse oximetry tests for children evaluated for suspected OSA.Clinical Relevance-This research establishes the usefulness of a CNN-RNN architecture to automatically score sleep stages in pulse oximetry tests for pediatric OSA diagnosis.

The sleep homeostatic response to sleep deprivation in humans is heritable

STUDY OBJECTIVES

Following sleep deprivation, increases in delta power have historically been used to index increases in sleep pressure. Research in mice has demonstrated that the homeostatic delta power response to sleep deprivation is heritable. Whether this is true in humans is unknown. In the present study, we used delta power and ORP, a novel measure of sleep depth, to investigate the effects of acute sleep deprivation on sleep depth and to assess the heritability of sleep homeostasis in humans.

METHODS

ORP and delta power were examined during baseline and recovery sleep following 38 h of sleep deprivation in 57 monozygotic and 38 dizygotic same-sex twin pairs. Two complementary methods were used to estimate the trait heritability of sleep homeostasis.

RESULTS

During recovery sleep, ORP was lower and delta power was higher than at baseline, indicating deeper sleep. However, at the end of the recovery night, delta power reached baseline levels but ORP demonstrated incomplete recovery. Both ORP and delta power showed a broad sense heritability of sleep homeostasis following sleep deprivation. The classical approach demonstrated an h2 estimate of 0.43 for ORP and 0.73 for delta power. Mixed-effect multilevel models showed that the proportion of variance attributable to additive genetic transmission was 0.499 (95% CI = 0.316-0.682; p < .0001) for ORP and 0.565 (95% CI = 0.403-0.726; p < .0001 for delta power.

CONCLUSIONS

These results demonstrate that the homeostatic response to sleep deprivation is a heritable trait in humans and confirm ORP as a robust measure of sleep depth.

Sleep stage classification in EEG signals using the clustering approach based probability distribution features coupled with classification algorithms

Sleep scoring is one of the primary tasks for the classification of sleep stages in Electroencephalogram (EEG) signals. Manual visual scoring of sleep stages is time-consuming as well as being dependent on the experience of a highly qualified sleep expert. This paper aims to address these issues by developing a new method to automatically classify sleep stages in EEG signals. In this research, a robust method has been presented based on the clustering approach, coupled with probability distribution features, to identify six sleep stages with the use of EEG signals. Using this method, each 30-second EEG signal is firstly segmented into small epochs and then each epoch is divided into 60 sub-segments. Each sub-segment is decomposed into five levels by using a discrete wavelet transform (DWT) to obtain the approximation and detailed coefficient. The wavelet coefficient of each level is clustered using the k-means algorithm. Subsequently, features are extracted based on the probability distribution for each wavelet coefficient. The extracted features then are forwarded to the least squares support vector machine classifier (LS-SVM) to identify sleep stages. Comparisons with several existing methods are also made in this study. The proposed method for the classification of the sleep stages achieves an average accuracy rate of 97.4%. It can be an effective tool for sleep stages classification and can be useful for doctors and neurologists for diagnosing sleep disorders.

Automated Sleep Stages Classification Using Convolutional Neural Network From Raw and Time-Frequency Electroencephalogram Signals: Systematic Evaluation Study

BACKGROUND

Most existing automated sleep staging methods rely on multimodal data, and scoring a specific epoch requires not only the current epoch but also a sequence of consecutive epochs that precede and follow the epoch.

OBJECTIVE

We proposed and tested a convolutional neural network called SleepInceptionNet, which allows sleep classification of a single epoch using a single-channel electroencephalogram (EEG).

METHODS

SleepInceptionNet is based on our systematic evaluation of the effects of different EEG preprocessing methods, EEG channels, and convolutional neural networks on automatic sleep staging performance. The evaluation was performed using polysomnography data of 883 participants (937,975 thirty-second epochs). Raw data of individual EEG channels (ie, frontal, central, and occipital) and 3 specific transformations of the data, including power spectral density, continuous wavelet transform, and short-time Fourier transform, were used separately as the inputs of the convolutional neural network models. To classify sleep stages, 7 sequential deep neural networks were tested for the 1D data (ie, raw EEG and power spectral density), and 16 image classifier convolutional neural networks were tested for the 2D data (ie, continuous wavelet transform and short-time Fourier transform time-frequency images).

RESULTS

The best model, SleepInceptionNet, which uses time-frequency images developed by the continuous wavelet transform method from central single-channel EEG data as input to the InceptionV3 image classifier algorithm, achieved a Cohen κ agreement of 0.705 (SD 0.077) in reference to the gold standard polysomnography.

CONCLUSIONS

SleepInceptionNet may allow real-time automated sleep staging in free-living conditions using a single-channel EEG, which may be useful for on-demand intervention or treatment during specific sleep stages.

Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring

STUDY OBJECTIVES

To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers.

METHODS

We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6-12 scorers, to compare sleep stage probabilities (hypnodensity; i.e. the probability of each sleep stage being assigned to a given epoch) as the primary output, as well as a single sleep stage per epoch assigned by hierarchical majority rule.

RESULTS

The percentage of epochs with 100% agreement across scorers was 46 ± 9%, 38 ± 10% and 32 ± 9% for the datasets with 6, 9, and 12 scorers, respectively. The mean intra-class correlation coefficient between sleep stage probabilities from auto- and manual-scoring was 0.91, representing excellent reliability. Within each dataset, agreement between auto-scoring and consensus manual-scoring was significantly higher than agreement between manual-scoring and consensus manual-scoring (0.78 vs. 0.69; 0.74 vs. 0.67; and 0.75 vs. 0.67; all p < 0.01).

CONCLUSIONS

Analysis of scoring performed by multiple scorers reveals that sleep stage ambiguity is the rule rather than the exception. Probabilities of the sleep stages determined by artificial intelligence auto-scoring provide an excellent estimate of this ambiguity. Compared to consensus manual-scoring, sleep staging derived from auto-scoring is for each individual PSG noninferior to manual-scoring meaning that auto-scoring output is ready for interpretation without the need for manual adjustment.

Central Sleep Apnea Detection by Means of Finger Photoplethysmography

Obstructive Sleep Apnea (OSA) and Central Sleep Apnea (CSA) are two types of Sleep Apnea (SA) with different etiologies and treatment options. Home sleep apnea testing based on photoplethysmography-derived peripheral arterial tonometry (PAT HSAT) has become the most widely deployed outpatient SA diagnostic method. Being able to differentiate between CSA and OSA based solely on photoplethysmography-data would further increase PAT HSAT's clinical utility. The present work proposes a method to detect CSA using finger photoplethysmography (PPG) data and evaluates the proposed method against simultaneous in-lab polysomnography (PSG).

METHODS

For 266 patients with a suspicion of SA, concurrent in-lab PSG and PPG data were acquired. The respiratory information embedded in the PPG data was extracted and used to train an ensemble of trees classifiers that predicts the central or obstructive nature of each respiratory event. The classifier performance was evaluated using patient-wise leave-one-out cross-validation where an expert analysis of the PSG served as ground truth. A second, independent analysis of the PSG was also evaluated against the ground truth to allow benchmarking of the PPG-based method.

RESULTS

The method achieved a sensitivity of 81%, a specificity of 99%, a positive predictive value of 90%, and a negative predictive value of 98% at the central apnea-hypopnea index cutoff of 10 events per hour of sleep.

CONCLUSION AND SIGNIFICANCE

The present study aimed to evaluate a method to detect CSA in SA patients using only PPG data which could be used to flag CSA which in turn may aid in more optimal therapy decision making.

Benchmarking performance of an automatic polysomnography scoring system in a population with suspected sleep disorders

Aim

The current gold standard for measuring sleep disorders is polysomnography (PSG), which is manually scored by a sleep technologist. Scoring a PSG is time-consuming and tedious, with substantial inter-rater variability. A deep-learning-based sleep analysis software module can perform autoscoring of PSG. The primary objective of the study is to validate the accuracy and reliability of the autoscoring software. The secondary objective is to measure workflow improvements in terms of time and cost via a time motion study.

Methodology

The performance of an automatic PSG scoring software was benchmarked against the performance of two independent sleep technologists on PSG data collected from patients with suspected sleep disorders. The technologists at the hospital clinic and a third-party scoring company scored the PSG records independently. The scores were then compared between the technologists and the automatic scoring system. An observational study was also performed where the time taken for sleep technologists at the hospital clinic to manually score PSGs was tracked, along with the time taken by the automatic scoring software to assess for potential time savings.

Results

Pearson's correlation between the manually scored apnea-hypopnea index (AHI) and the automatically scored AHI was 0.962, demonstrating a near-perfect agreement. The autoscoring system demonstrated similar results in sleep staging. The agreement between automatic staging and manual scoring was higher in terms of accuracy and Cohen's kappa than the agreement between experts. The autoscoring system took an average of 42.7 s to score each record compared with 4,243 s for manual scoring. Following a manual review of the auto scores, an average time savings of 38.6 min per PSG was observed, amounting to 0.25 full-time equivalent (FTE) savings per year.

Conclusion

The findings indicate a potential for a reduction in the burden of manual scoring of PSGs by sleep technologists and may be of operational significance for sleep laboratories in the healthcare setting.

Sleep calibrates atherosclerotic cardiovascular disease

Sleep modulates cardiovascular health, and recent studies have begun to uncover underlying mechanistic links. An integrated translational approach that combines animal models and human trials will enrich scientific discovery, improve therapy, and help to alleviate the global burden of insufficient sleep and cardiovascular disease.

Computer-assisted analysis of polysomnographic recordings improves inter-scorer associated agreement and scoring times

STUDY OBJECTIVES

To investigate inter-scorer agreement and scoring time differences associated with visual and computer-assisted analysis of polysomnographic (PSG) recordings.

METHODS

A group of 12 expert scorers reviewed 5 PSGs that were independently selected in the context of each of the following tasks: (i) sleep staging, (ii) scoring of leg movements, (iii) detection of respiratory (apneic-related) events, and (iv) of electroencephalographic (EEG) arousals. All scorers independently reviewed the same recordings, hence resulting in 20 scoring exercises per scorer from an equal amount of different subjects. The procedure was repeated, separately, using the classical visual manual approach and a computer-assisted (semi-automatic) procedure. Resulting inter-scorer agreement and scoring times were examined and compared among the two methods.

RESULTS

Computer-assisted sleep scoring showed a consistent and statistically relevant effect toward less time required for the completion of each of the PSG scoring tasks. Gain factors ranged from 1.26 (EEG arousals) to 2.41 (leg movements). Inter-scorer kappa agreement was also consistently increased with the use of supervised semi-automatic scoring. Specifically, agreement increased from Κ = 0.76 to K = 0.80 (sleep stages), Κ = 0.72 to K = 0.91 (leg movements), Κ = 0.55 to K = 0.66 (respiratory events), and Κ = 0.58 to Κ = 0.65 (EEG arousals). Inter-scorer agreement on the examined set of diagnostic indices did also show a trend toward higher Interclass Correlation Coefficient scores when using the semi-automatic scoring approach.

CONCLUSIONS

Computer-assisted analysis can improve inter-scorer agreement and scoring times associated with the review of PSG studies resulting in higher efficiency and overall quality in the diagnosis sleep disorders.

XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging

Automating sleep staging is vital to scale up sleep assessment and diagnosis to serve millions experiencing sleep deprivation and disorders and enable longitudinal sleep monitoring in home environments. Learning from raw polysomnography signals and their derived time-frequency image representations has been prevalent. However, learning from multi-view inputs (e.g., both the raw signals and the time-frequency images) for sleep staging is difficult and not well understood. This work proposes a sequence-to-sequence sleep staging model, XSleepNet,¹ that is capable of learning a joint representation from both raw signals and time-frequency images. Since different views may generalize or overfit at different rates, the proposed network is trained such that the learning pace on each view is adapted based on their generalization/overfitting behavior. In simple terms, the learning on a particular view is speeded up when it is generalizing well and slowed down when it is overfitting. View-specific generalization/overfitting measures are computed on-the-fly during the training course and used to derive weights to blend the gradients from different views. As a result, the network is able to retain the representation power of different views in the joint features which represent the underlying distribution better than those learned by each individual view alone. Furthermore, the XSleepNet architecture is principally designed to gain robustness to the amount of training data and to increase the complementarity between the input views. Experimental results on five databases of different sizes show that XSleepNet consistently outperforms the single-view baselines and the multi-view baseline with a simple fusion strategy. Finally, XSleepNet also outperforms prior sleep staging methods and improves previous state-of-the-art results on the experimental databases.

Certainty about uncertainty in sleep staging: a theoretical framework

Sleep stage classification is an important tool for the diagnosis of sleep disorders. Because sleep staging has such a high impact on clinical outcome, it is important that it is done reliably. However, it is known that uncertainty exists in both expert scorers and automated models. On average, the agreement between human scorers is only 82.6%. In this study, we provide a theoretical framework to facilitate discussion and further analyses of uncertainty in sleep staging. To this end, we introduce two variants of uncertainty, known from statistics and the machine learning community: aleatoric and epistemic uncertainty. We discuss what these types of uncertainties are, why the distinction is useful, where they arise from in sleep staging, and provide recommendations on how this framework can improve sleep staging in the future.

Multi-scale ResNet and BiGRU automatic sleep staging based on attention mechanism

Sleep staging is the basis of sleep evaluation and a key step in the diagnosis of sleep-related diseases. Despite being useful, the existing sleep staging methods have several disadvantages, such as relying on artificial feature extraction, failing to recognize temporal sequence patterns in the long-term associated data, and reaching the accuracy upper limit of sleep staging. Hence, this paper proposes an automatic Electroencephalogram (EEG) sleep signal staging model, which based on Multi-scale Attention Residual Nets (MAResnet) and Bidirectional Gated Recurrent Unit (BiGRU). The proposed model is based on the residual neural network in deep learning. Compared with the traditional residual learning module, the proposed model additionally uses the improved channel and spatial feature attention units and convolution kernels of different sizes in parallel at the same position. Thus, multiscale feature extraction of the EEG sleep signals and residual learning of the neural networks is performed to avoid network degradation. Finally, BiGRU is used to determine the dependence between the sleep stages and to realize the automatic learning of sleep data staging features and sleep cycle extraction. According to the experiment, the classification accuracy and kappa coefficient of the proposed method on sleep-EDF data set are 84.24% and 0.78, which are respectively 0.24% and 0.21 higher than the traditional residual net. At the same time, this paper also verified the proposed method on UCD and SHHS data sets, and the figure of classification accuracy is 79.34% and 81.6%, respectively. Compared to related existing studies, the recognition accuracy is significantly improved, which validates the effectiveness and generalization performance of the proposed method.

A multicentric validation study of a novel home sleep apnea test based on peripheral arterial tonometry

STUDY OBJECTIVES

This paper reports on the multicentric validation of a novel FDA-cleared home sleep apnea test based on peripheral arterial tonometry (PAT HSAT).

METHODS

One hundred sixty-seven participants suspected of having obstructive sleep apnea (OSA) were included in a multicentric cohort. All patients underwent simultaneous polysomnography (PSG) and PAT HSAT, and all PSG data were independently double scored using both the recommended 1A rule for hypopnea, requiring a 3% desaturation or arousal (3% Rule), and the acceptable 1B rule for hypopnea, requiring a 4% desaturation (4% Rule). The double-scoring of PSG enabled a comparison of the agreement between PAT HSAT and PSG to the inter-rater agreement of PSG. Clinical endpoint parameters were selected to evaluate the device's ability to determine the OSA severity category. Finally, a correction for near-boundary apnea-hypopnea index values was proposed to adequately handle the inter-rater variability of the PSG benchmark.

RESULTS

For both the 3% and the 4% Rules, most endpoint parameters showed a close agreement with PSG. The 4-way OSA severity categorization accuracy of PAT HSAT was strong, but nevertheless lower than the inter-rater agreement of PSG (70% vs 77% for the 3% Rule and 78% vs 81% for the 4% Rule).

CONCLUSIONS

This paper reported on a multitude of robust endpoint parameters, in particular OSA severity categorization accuracies, while also benchmarking clinical performances against double-scored PSG. This study demonstrated strong agreement of PAT HSAT with PSG. The results of this study also suggest that different brands of PAT HSAT may have distinct clinical performance characteristics.

Association of a novel EEG metric of sleep depth/intensity with attention-deficit/hyperactivity, learning, and internalizing disorders and their pharmacotherapy in adolescence

STUDY OBJECTIVES

Psychiatric/learning disorders are associated with sleep disturbances, including those arising from abnormal cortical activity. The odds ratio product (ORP) is a standardized electroencephalogram metric of sleep depth/intensity validated in adults, while ORP data in youth are lacking. We tested ORP as a measure of sleep depth/intensity in adolescents with and without psychiatric/learning disorders.

METHODS

Four hundred eighteen adolescents (median 16 years) underwent a 9-hour, in-lab polysomnography. Of them, 263 were typically developing (TD), 89 were unmedicated, and 66 were medicated for disorders including attention-deficit/hyperactivity (ADHD), learning (LD), and internalizing (ID). Central ORP during non-rapid eye movement (NREM) sleep was the primary outcome. Secondary/exploratory outcomes included central and frontal ORP during NREM stages, in the 9-seconds following arousals (ORP-9), in the first and second halves of the night, during REM sleep and wakefulness.

RESULTS

Unmedicated youth with ADHD/LD had greater central ORP than TD during stage 3 and in central and frontal regions during stage 2 and the second half of the sleep period, while ORP in youth with ADHD/LD on stimulants did not significantly differ from TD. Unmedicated youth with ID did not significantly differ from TD in ORP, while youth with ID on antidepressants had greater central and frontal ORP than TD during NREM and REM sleep, and higher ORP-9.

CONCLUSIONS

The greater ORP in unmedicated youth with ADHD/LD, and normalized levels in those on stimulants, suggests ORP is a useful metric of decreased NREM sleep depth/intensity in ADHD/LD. Antidepressants are associated with greater ORP/ORP-9, suggesting these medications induce cortical arousability.

SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

BACKGROUND

Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments.

METHODS

Towards interpretability, this work proposes a sequence-to-sequence sleep staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the models decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the models decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection.

RESULTS

Making sense of the transformers self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts.

CONCLUSION

Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes.

SIGNIFICANCE

Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

Deep Learning Application to Clinical Decision Support System in Sleep Stage Classification

Recently, deep learning for automated sleep stage classification has been introduced with promising results. However, as many challenges impede their routine application, automatic sleep scoring algorithms are not widely used. Typically, polysomnography (PSG) uses multiple channels for higher accuracy; however, the disadvantages include a requirement for a patient to stay one or more nights in the lab wearing uncomfortable sensors and wires. To avoid the inconvenience caused by the multiple channels, we aimed to develop a deep learning model for use in clinical decision support systems (CDSSs) and combined convolutional neural networks and a transformer for the supervised learning of three classes of sleep stages only with single-channel EEG data (C4-M1). The data for training, validation, and test were derived from 1590, 341, and 343 polysomnography recordings, respectively. The developed model yielded an overall accuracy of 91.4%, comparable with that of human experts. Based on the severity of obstructive sleep apnea, the model’s accuracy was 94.3%, 91.9%, 91.9%, and 90.6% in normal, mild, moderate, and severe cases, respectively. Our deep learning model enables accurate and rapid delineation of three-class sleep staging and could be useful as a CDSS for application in real-world clinical practice.

Traffic noise-induced changes in wake-propensity measured with the Odds-Ratio Product (ORP)

Nocturnal traffic noise can disrupt sleep and impair physical and mental restoration, but classical sleep scoring techniques may not fully capture subtle yet clinically relevant alterations of sleep induced by noise. We used a validated continuous measure of sleep depth and quality based on automatic analysis of physiologic sleep data, termed Wake Propensity (WP), to investigate temporal changes of sleep in response to nocturnal noise events in 3-s epochs. Seventy-two healthy participants (mean age 40.3 years, range 18-71 years, 40 females, 32 males) slept for 11 nights in a laboratory, during which we measured sleep with polysomnography. In 8 nights, participants were exposed to 40, 80 or 120 road, rail and/or aircraft noise events with maximum noise levels of 45-65 dB L_AS,max during 8-h sleep opportunities. We analyzed sleep macrostructure and event-related change of WP during noise exposure with linear mixed models. Nocturnal traffic noise led to event-related shifts towards wakefulness and less deep, more unstable sleep (increase in WP relative to pre-noise baseline ranging from +29.5% at 45 dB to +38.3% at 65 dB; type III effect p < 0.0001). Sleep depth decreased dynamically with increasing noise level, peaking when L_AS,max was highest. This change in WP was stronger and occurred more quickly for events where the noise onset was more rapid (road and rail) compared to more gradually time-varying noise (aircraft). Sleep depth did not immediately recover to pre-noise WP, leading to decreased sleep stability across the night compared to quiet nights, which was greater with an increasing number of noise events (standardized β = 0.053, p = 0.003). Further, WP was more sensitive to noise than classical arousals. Results demonstrate the usefulness of WP as a measure of the effects of external stimuli on sleep, and show WP is a more sensitive measure of noise-induced sleep disruption than traditional methods of sleep analysis.

Interrater reliability of sleep stage scoring: a meta-analysis

STUDY OBJECTIVES

We evaluated the interrater reliabilities of manual polysomnography sleep stage scoring. We included all studies that employed Rechtschaffen and Kales rules or American Academy of Sleep Medicine standards. We sought the overall degree of agreement and those for each stage.

METHODS

The keywords were "Polysomnography (PSG)," "sleep staging," "Rechtschaffen and Kales (R&K)," "American Academy of Sleep Medicine (AASM)," "interrater (interscorer) reliability," and "Cohen's kappa." We searched PubMed, OVID Medline, EMBASE, the Cochrane library, KoreaMed, KISS, and the MedRIC. The exclusion criteria included automatic scoring and pediatric patients. We collected data on scorer histories, scoring rules, numbers of epochs scored, and the underlying diseases of the patients.

RESULTS

A total of 101 publications were retrieved; 11 satisfied the selection criteria. The Cohen's kappa for manual, overall sleep scoring was 0.76, indicating substantial agreement (95% confidence interval, 0.71-0.81; P < .001). By sleep stage, the figures were 0.70, 0.24, 0.57, 0.57, and 0.69 for the W, N1, N2, N3, and R stages, respectively. The interrater reliabilities for stage N2 and N3 sleep were moderate, and that for stage N1 sleep was only fair.

CONCLUSIONS

We conducted a meta-analysis to generalize the variation in manual scoring of polysomnography and provide reference data for automatic sleep stage scoring systems. The reliability of manual scorers of polysomnography sleep stages was substantial. However, for certain stages, the results were poor; validity requires improvement.

CITATION

Lee YJ, Lee JY, Cho JH, Choi JH. Interrater reliability of sleep stage scoring: a meta-analysis. J Clin Sleep Med. 2022;18(1):193-202.

Automated Scoring of Sleep and Associated Events

Conventionally, sleep and associated events are scored visually by trained technologists according to the rules summarized in the American Academy of Sleep Medicine Manual. Since its first publication in 2007, the manual was continuously updated; the most recent version as of this writing was published in 2020. Human expert scoring is considered as gold standard, even though there is increasing evidence of limited interrater reliability between human scorers. Significant advances in machine learning have resulted in powerful methods for addressing complex classification problems such as automated scoring of sleep and associated events. Evidence is increasing that these autoscoring systems deliver performance comparable to manual scoring and offer several advantages to visual scoring: (1) avoidance of the rather expensive, time-consuming, and difficult visual scoring task that can be performed only by well-trained and experienced human scorers, (2) attainment of consistent scoring results, and (3) proposition of added value such as scoring in real time, sleep stage probabilities per epoch (hypnodensity), estimates of signal quality and sleep/wake-related features, identifications of periods with clinically relevant ambiguities (confidence trends), configurable sensitivity and rule settings, as well as cardiorespiratory sleep staging for home sleep apnea testing. This chapter describes the development of autoscoring systems since the first attempts in the 1970s up to the most recent solutions based on deep neural network approaches which achieve an accuracy that allows to use the autoscoring results directly for review and interpretation by a physician.

Polysomnographic Markers of Obstructive Sleep Apnea Severity and Cancer-Related Mortality: A Large Retrospective Multicenter Clinical Cohort Study

RATIONALE

The evidence for an association between cancer survival and obstructive sleep apnea (OSA) remains under-explored.

OBJECTIVES

To evaluate an association between markers of OSA severity (respiratory disturbances, hypoxemia, and sleep fragmentation) and cancer-related mortality in individuals with previously diagnosed cancer.

METHODS

We conducted a multicenter retrospective cohort study using linked clinical and provincial health administrative data on consecutive adults who underwent a diagnostic sleep study between 1994 and 2017 in four Canadian academic hospitals and were previously diagnosed with cancer through the Ontario Cancer Registry. Multivariable cause-specific Cox regressions were utilized to address the research objective.

RESULTS

We included 2,222 subjects. Over a median follow-up time of 5.6 years (IQR: 2.7-9.1), 261/2,222 (11.7%) individuals with prevalent cancer died from cancer-related causes, which accounted for 44.2% (261/590) of all-cause death. Controlling for age, sex, alcohol use disorder, prior heart failure, COPD, hypertension, diabetes, treatment for OSA, clinic site, year of the sleep study, and time since the cancer diagnosis, measures of hypoxemia and sleep fragmentation, but not apnea-hypopnea index were significantly associated with the cancer-specific mortality: % time spent with SaO2 <90% (HR per 5% increase: 1.05; 95% CI: 1.01-1.09); mean SaO2 (HR per 3% increase: 0.79; 0.68-0.92); and % of Stage 1 Sleep (HR per 16% increase: 1.27; 1.07-1.51).

CONCLUSION

In a large clinical cohort of adults with suspected OSA and previously diagnosed cancer, measures of nocturnal hypoxemia and sleep fragmentation as markers of OSA severity were significantly associated with cancer-related mortality, suggesting the need for more targeted risk awareness.

Past, Present, and Future of Multisensory Wearable Technology to Monitor Sleep and Circadian Rhythms

Movement-based sleep-wake detection devices (i.e., actigraphy devices) were first developed in the early 1970s and have repeatedly been validated against polysomnography, which is considered the “gold-standard” of sleep measurement. Indeed, they have become important tools for objectively inferring sleep in free-living conditions. Standard actigraphy devices are rooted in accelerometry to measure movement and make predictions, via scoring algorithms, as to whether the wearer is in a state of wakefulness or sleep. Two important developments have become incorporated in newer devices. First, additional sensors, including measures of heart rate and heart rate variability and higher resolution movement sensing through triaxial accelerometers, have been introduced to improve upon traditional, movement-based scoring algorithms. Second, these devices have transcended scientific utility and are now being manufactured and distributed to the general public. This review will provide an overview of: (1) the history of actigraphic sleep measurement, (2) the physiological underpinnings of heart rate and heart rate variability measurement in wearables, (3) the refinement and validation of both standard actigraphy and newer, multisensory devices for real-world sleep-wake detection, (4) the practical applications of actigraphy, (5) important limitations of actigraphic measurement, and lastly (6) future directions within the field.

An open-source, high-performance tool for automated sleep staging

The clinical and societal measurement of human sleep has increased exponentially in recent years. However, unlike other fields of medical analysis that have become highly automated, basic and clinical sleep research still relies on human visual scoring. Such human-based evaluations are time-consuming, tedious, and can be prone to subjective bias. Here, we describe a novel algorithm trained and validated on +30,000 hr of polysomnographic sleep recordings across heterogeneous populations around the world. This tool offers high sleep-staging accuracy that matches human scoring accuracy and interscorer agreement no matter the population kind. The software is designed to be especially easy to use, computationally low-demanding, open source, and free. Our hope is that this software facilitates the broad adoption of an industry-standard automated sleep staging software package.

Statistical uncertainty quantification to augment clinical decision support: a first implementation in sleep medicine

Machine learning has the potential to change the practice of medicine, particularly in areas that require pattern recognition (e.g. radiology). Although automated classification is unlikely to be perfect, few modern machine learning tools have the ability to assess their own classification confidence to recognize uncertainty that might need human review. Using automated single-channel sleep staging as a first implementation, we demonstrated that uncertainty information (as quantified using Shannon entropy) can be utilized in a "human in the loop" methodology to promote targeted review of uncertain sleep stage classifications on an epoch-by-epoch basis. Across 20 sleep studies, this feedback methodology proved capable of improving scoring agreement with the gold standard over automated scoring alone (average improvement in Cohen's Kappa of 0.28), in a fraction of the scoring time compared to full manual review (60% reduction). In summary, our uncertainty-based clinician-in-the-loop framework promotes the improvement of medical classification accuracy/confidence in a cost-effective and economically resourceful manner.

Phenotyping REM OSA by means of peripheral arterial tone-based home sleep apnea testing and polysomnography: A critical assessment of the sensitivity and specificity of both methods

The clinical relevance of rapid eye movement sleep-related obstructive sleep apnea (REM OSA) is supported by its associated adverse health outcomes and impact on optimal treatment strategies. To date, no assessment of REM OSA phenotyping performance has been conducted for any type of sleep testing technology. The objective of this study was to assess this for polysomnography and peripheral arterial tone-based home sleep apnea testing (PAT HSAT). In a dataset comprising 261 participants, the sensitivity and specificity of the agreement on REM OSA phenotyping was assessed for two independent scorings of polysomnography and a synchronously administered PAT HSAT. The sensitivity and specificity of REM OSA phenotyping were 0.87 and 0.89, respectively, for the polysomnography inter-scorer comparison, and 0.68 and 0.97 for the PAT HSAT on a single-night basis, using the conventional minimum required rapid eye movement sleep time of 30 min. Polysomnography-based REM OSA phenotyping was found to be sensitive and specific even for a single-night testing protocol. Peripheral arterial tone-based REM OSA phenotyping showed a lower sensitivity but a slightly higher specificity compared to polysomnography. In order to increase performance and conclusiveness of peripheral arterial tone-based REM OSA phenotyping, a multi-night protocol of 2-5 nights could be considered. Finally, the minimum required rapid eye movement sleep time could be lowered from the conventional 30 min to 15 min without significantly lowering REM OSA phenotyping sensitivity and specificity, while increasing the level of phenotyping conclusiveness.

Effects of Sedatives on Sleep Architecture Measured With Odds Ratio Product in Critically Ill Patients

Supplemental Digital Content is available in the text.

OBJECTIVES:

Evaluation of sleep quality in critically ill patients is difficult using conventional scoring criteria. The aim of this study was to examine sleep in critically ill patients with and without light sedation using the odds ratio product, a validated continuous metric of sleep depth (0 = deep sleep; 2.5 = full wakefulness) that does not rely on the features needed for conventional staging.

DESIGN:

Retrospective study.

SETTINGS:

A 16-bed medical-surgical ICU.

PATIENTS:

Twenty-three mechanically ventilated patients who had previously undergone two nocturnal sleep studies, one without and one with sedation (propofol, n = 12; dexmedetomidine, n = 11).

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

Sleep architecture was evaluated with odds ratio product analysis by the distribution of 30-second epochs with different odds ratio product values. Electroencephalogram spectral patterns and frequency of wake intrusions (3-s odds ratio product > 1.75) were measured at different odds ratio product levels. Thirty-seven normal sleepers were used as controls. Compared with normal sleepers, unsedated critically ill patients spent little time in stable sleep (percent odds ratio product < 1.0: 31% vs 63%; p < 0.001), whereas most of the time were either in stage wake (odds ratio product > 1.75) or in a transitional state (odds ratio product 1.0–1.75), characterized by frequent wake intrusions. Propofol and dexmedetomidine had comparable effects on sleep. Sedation resulted in significant shift in odds ratio product distribution toward normal; percent odds ratio product less than 1.0 increased by 54% (p = 0.006), and percent odds ratio product greater than 1.75 decreased by 48% (p = 0.013). In six patients (26%), sedation failed to improve sleep.

CONCLUSIONS:

In stable critically ill unsedated patients, sleep quality is poor with frequent wake intrusions and little stable sleep. Light sedation with propofol or dexmedetomidine resulted in a shift in sleep architecture toward normal in most, but not all, patients.

Maturational trajectories of non-rapid eye movement slow wave activity and odds ratio product in a population-based sample of youth

BACKGROUND

Brain maturation is reflected in the sleep electroencephalogram (EEG) by a decline in non-rapid eye movement (NREM) slow wave activity (SWA) throughout adolescence and a related decrease in sleep depth. However, this trajectory and its sex and pubertal differences lack replication in population-based samples. We tested age-related changes in SWA (0.4-4 Hz) power and odds ratio product (ORP), a standardized measure of sleep depth.

METHODS

We analyzed the sleep EEG of 572 subjects aged 6-21 y (48% female, 26% racial/ethnic minority) and 332 subjects 5-12 y followed-up at 12-22 y. Multivariable-adjusted analyses tested age-related cross-sectional and longitudinal trajectories of SWA and ORP.

RESULTS

SWA remained stable from age 6 to 10, decreased between ages 11 and 17, and plateaued from age 18 to 21 (p-cubic<0.001); females showed a longitudinal decline 23% greater than males by 13 y, while males experienced a steeper slope after 14 y and their longitudinal decline was 21% greater by 19 y. More mature adolescents (75% female) experienced a greater longitudinal decline in SWA than less mature adolescents by 14 y. ORP showed an age-related increasing trajectory (p-linear<0.001) with no sex or pubertal differences.

CONCLUSIONS

We provide population-level evidence for the maturational decline and sex and pubertal differences in SWA in the transition from childhood to adolescence, while introducing ORP as a novel metric in youth. Along with previous studies, the distinct trajectories observed suggest that age-related changes in SWA reflect brain maturation and local/synaptic processes during this developmental period, while those of ORP may reflect global/state control of NREM sleep depth.

Spindles are highly heritable as identified by different spindle detectors

STUDY OBJECTIVES

Sleep spindles, a defining feature of stage N2 sleep, are maximal at central electrodes and are found in the frequency range of the electroencephalogram (EEG) (sigma 11-16 Hz) that is known to be heritable. However, relatively little is known about the heritability of spindles. Two recent studies investigating the heritability of spindles reported moderate heritability, but with conflicting results depending on scalp location and spindle type. The present study aimed to definitively assess the heritability of sleep spindle characteristics.

METHODS

We utilized the polysomnography data of 58 monozygotic and 40 dizygotic same-sex twin pairs to identify heritable characteristics of spindles at C3/C4 in stage N2 sleep including density, duration, peak-to-peak amplitude, and oscillation frequency. We implemented and tested a variety of spindle detection algorithms and used two complementary methods of estimating trait heritability.

RESULTS

We found robust evidence to support strong heritability of spindles regardless of detector method (h2 > 0.8). However not all spindle characteristics were equally heritable, and each spindle detection method produced a different pattern of results.

CONCLUSIONS

The sleep spindle in stage N2 sleep is highly heritable, but the heritability differs for individual spindle characteristics and depends on the spindle detector used for analysis.

Expert-level automated sleep staging of long-term scalp electroencephalography recordings using deep learning

STUDY OBJECTIVES

Develop a high-performing, automated sleep scoring algorithm that can be applied to long-term scalp electroencephalography (EEG) recordings.

METHODS

Using a clinical dataset of polysomnograms from 6,431 patients (MGH-PSG dataset), we trained a deep neural network to classify sleep stages based on scalp EEG data. The algorithm consists of a convolutional neural network for feature extraction, followed by a recurrent neural network that extracts temporal dependencies of sleep stages. The algorithm's inputs are four scalp EEG bipolar channels (F3-C3, C3-O1, F4-C4, and C4-O2), which can be derived from any standard PSG or scalp EEG recording. We initially trained the algorithm on the MGH-PSG dataset and used transfer learning to fine-tune it on a dataset of long-term (24-72 h) scalp EEG recordings from 112 patients (scalpEEG dataset).

RESULTS

The algorithm achieved a Cohen's kappa of 0.74 on the MGH-PSG holdout testing set and cross-validated Cohen's kappa of 0.78 after optimization on the scalpEEG dataset. The algorithm also performed well on two publicly available PSG datasets, demonstrating high generalizability. Performance on all datasets was comparable to the inter-rater agreement of human sleep staging experts (Cohen's kappa ~ 0.75 ± 0.11). The algorithm's performance on long-term scalp EEGs was robust over a wide age range and across common EEG background abnormalities.

CONCLUSION

We developed a deep learning algorithm that achieves human expert level sleep staging performance on long-term scalp EEG recordings. This algorithm, which we have made publicly available, greatly facilitates the use of large long-term EEG clinical datasets for sleep-related research.

Effects of Weight Loss on Obstructive Sleep Apnea Severity. Ten-Year Results of the Sleep AHEAD Study

Rationale: Weight loss is recommended to treat obstructive sleep apnea (OSA).Objectives: To determine whether the initial benefit of intensive lifestyle intervention (ILI) for weight loss on OSA severity is maintained at 10 years.Methods: Ten-year follow-up polysomnograms of 134 of 264 adults in Sleep AHEAD (Action for Health in Diabetes) with overweight/obesity, type 2 diabetes mellitus, and OSA were randomized to ILI for weight loss or diabetes support and education (DSE).Measurements and Main Results: Change in apnea-hypopnea index (AHI) was measured. Mean ± SE weight losses of ILI participants of 10.7 ± 0.7, 7.4 ± 0.7, 5.1 ± 0.7, and 7.1 ± 0.8 kg at 1, 2, 4, and 10 years, respectively, were significantly greater than the 1-kg weight loss at 1, 2, and 4 years and 3.5 ± 0.8 kg weight loss at 10 years for the DSE group (P values ≤ 0.0001). AHI was lower with ILI than DSE by 9.7, 8.0, and 7.9 events/h at 1, 2, and 4 years, respectively (P values ≤ 0.0004), and 4.0 events/h at 10 years (P = 0.109). Change in AHI over time was related to amount of weight loss, baseline AHI, visit year (P values < 0.0001), and intervention independent of weight change (P = 0.01). OSA remission at 10 years was more common with ILI (34.4%) than DSE (22.2%).Conclusions: Participants with OSA and type 2 diabetes mellitus receiving ILI for weight loss had reduced OSA severity at 10 years. No difference in OSA severity was present between ILI and DSE groups at 10 years. Improvement in OSA severity over the 10-year period with ILI was related to change in body weight, baseline AHI, and intervention independent of weight change.