1
|
Cheng H, Yang Y, Shi J, Li Z, Feng Y, Wang X. Comparison of automated deep neural network against manual sleep stage scoring in clinical data. Comput Biol Med 2024; 179:108855. [PMID: 39029432 DOI: 10.1016/j.compbiomed.2024.108855] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 07/04/2024] [Accepted: 07/05/2024] [Indexed: 07/21/2024]
Abstract
OBJECTIVE To compare the accuracy and generalizability of an automated deep neural network and the Philip Sleepware G3™ Somnolyzer system (Somnolyzer) for sleep stage scoring using American Academy of Sleep Medicine (AASM) guidelines. METHODS Sleep recordings from 104 participants were analyzed by a convolutional neural network (CNN), the Somnolyzer and skillful technicians. Evaluation metrics were derived for different combinations of sleep stages. A further comparison between the Somnolyzer and the CNN model using a single-channel signal as input was also performed. Sleep recordings from 263 participants with a lower prevalence of OSA served as a cross-validation dataset to validate the generalizability of the CNN model. RESULTS The overall agreement between automated and manual scoring for sleep staging in 104 participants outperformed that of the Somnolyzer according to various metrics (accuracy: 81.81 % vs. 77.07 %; F1: 76.36 % vs. 73.80 %; Cohen's kappa: 0.7403 vs. 0.6848). The results showed that the left electrooculography (EOG) single-channel model had minor advantages over the Somnolyzer. In terms of consistency with manual sleep staging, the CNN model demonstrated superior performance in identifying more pronounced sleep transitions, particularly in the N2 stage and sleep latency metrics. Conversely, the Somnolyzer showed enhanced proficiency in the analysis of REM stages, notably in measuring REM latency. The accuracy in the cross-validation set of 263 participants was also above 80 %. CONCLUSIONS The CNN-based automated deep neural network outperformed the Somnolyzer and is sufficiently accurate for sleep study analyses using the AASM classification criteria.
Collapse
Affiliation(s)
- Hanrong Cheng
- Department of Sleep Medicine, Institute of Respiratory Diseases, Shenzhen People's Hospital, The Second Clinical Medical College of Jinan University, The First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, 518020, China.
| | - Yifei Yang
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Jingshu Shi
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Zhangbo Li
- Shenzhen Gianta Information Technology Co., LTD, Shenzhen, 518048, China
| | - Yang Feng
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Xingjun Wang
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China.
| |
Collapse
|
2
|
Zhang X, Landsness EC, Miao H, Chen W, Tang MJ, Brier LM, Culver JP, Lee JM, Anastasio MA. Attention-based CNN-BiLSTM for sleep state classification of spatiotemporal wide-field calcium imaging data. J Neurosci Methods 2024; 411:110250. [PMID: 39151658 DOI: 10.1016/j.jneumeth.2024.110250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Revised: 08/03/2024] [Accepted: 08/13/2024] [Indexed: 08/19/2024]
Abstract
BACKGROUND Wide-field calcium imaging (WFCI) with genetically encoded calcium indicators allows for spatiotemporal recordings of neuronal activity in mice. When applied to the study of sleep, WFCI data are manually scored into the sleep states of wakefulness, non-REM (NREM) and REM by use of adjunct EEG and EMG recordings. However, this process is time-consuming, invasive and often suffers from low inter- and intra-rater reliability. Therefore, an automated sleep state classification method that operates on spatiotemporal WFCI data is desired. NEW METHOD A hybrid network architecture consisting of a convolutional neural network (CNN) to extract spatial features of image frames and a bidirectional long short-term memory network (BiLSTM) with attention mechanism to identify temporal dependencies among different time points was proposed to classify WFCI data into states of wakefulness, NREM and REM sleep. RESULTS Sleep states were classified with an accuracy of 84 % and Cohen's κ of 0.64. Gradient-weighted class activation maps revealed that the frontal region of the cortex carries more importance when classifying WFCI data into NREM sleep while posterior area contributes most to the identification of wakefulness. The attention scores indicated that the proposed network focuses on short- and long-range temporal dependency in a state-specific manner. COMPARISON WITH EXISTING METHOD On a held out, repeated 3-hour WFCI recording, the CNN-BiLSTM achieved a κ of 0.67, comparable to a κ of 0.65 corresponding to the human EEG/EMG-based scoring. CONCLUSIONS The CNN-BiLSTM effectively classifies sleep states from spatiotemporal WFCI data and will enable broader application of WFCI in sleep research.
Collapse
Affiliation(s)
- Xiaohui Zhang
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA
| | - Eric C Landsness
- Department of Neurology, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - Hanyang Miao
- Department of Neurology, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - Wei Chen
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA
| | - Michelle J Tang
- Department of Neurology, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - Lindsey M Brier
- Department of Radiology, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - Joseph P Culver
- Department of Radiology, Washington University School of Medicine, St. Louis, MO 63110, USA; Department of Biomedical Engineering, Washington University School of Engineering, St. Louis, MO 63130, USA; Department of Electrical and Systems Engineering, Washington University School of Engineering, St. Louis, MO 63130, USA; Department of Physics, Washington University School of Arts and Science, St. Louis, Mo 63130, USA
| | - Jin-Moo Lee
- Department of Neurology, Washington University School of Medicine, St. Louis, MO 63110, USA; Department of Radiology, Washington University School of Medicine, St. Louis, MO 63110, USA; Department of Biomedical Engineering, Washington University School of Engineering, St. Louis, MO 63130, USA
| | - Mark A Anastasio
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA.
| |
Collapse
|
3
|
Chen J, Peng G, Sun B. Alzheimer's disease and sleep disorders: A bidirectional relationship. Neuroscience 2024; 557:12-23. [PMID: 39137870 DOI: 10.1016/j.neuroscience.2024.08.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2024] [Revised: 07/30/2024] [Accepted: 08/07/2024] [Indexed: 08/15/2024]
Abstract
Alzheimer's disease (AD) is the most prevalent dementia, pathologically featuring abnormal accumulation of amyloid-β (Aβ) and hyperphosphorylated tau, while sleep, divided into rapid eye movement sleep (REM) and nonrapid eye movement sleep (NREM), plays a key role in consolidating social and spatial memory. Emerging evidence has revealed that sleep disorders such as circadian disturbances and disruption of neuronal rhythm activity are considered as both candidate risks and consequence of AD, suggesting a bidirectional relationship between sleep and AD. This review will firstly grasp basic knowledge of AD pathogenesis, then highlight macrostructural and microstructural alteration of sleep along with AD progression, explain the interaction between accumulation of Aβ and hyperphosphorylated tau, which are two critical neuropathological processes of AD, as well as neuroinflammation and sleep, and finally introduce several methods of sleep enhancement as strategies to reduce AD-associated neuropathology. Although theories about the bidirectional relationship and relevant therapeutic methods in mice have been well developed in recent years, the knowledge in human is still limited. More studies on how to effectively ameliorate AD pathology in patients by sleep enhancement and what specific roles of sleep play in AD are needed.
Collapse
Affiliation(s)
- Junhua Chen
- Chu Kochen Honors College of Zhejiang University, Hangzhou, Zhejiang 310058, China.
| | - Guoping Peng
- Department of Neurology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, Zhejiang 310003, China.
| | - Binggui Sun
- Department of Anesthesiology of the Children's Hospital and School of Brain Science and Brain Medicine, Zhejiang University School of Medicine and National Clinical Research Center for Child Health, Zhejiang University, Hangzhou, Zhejiang Province 310058, China; NHC and CAMS Key Laboratory of Medical Neurobiology, School of Brain Science and Brain Medicine, Zhejiang University Hangzhou, Zhejiang Province 310058, China.
| |
Collapse
|
4
|
de Sena S, Häggman M, Ranta J, Roienko O, Ilén E, Acosta N, Salama J, Kirjavainen T, Stevenson N, Airaksinen M, Vanhatalo S. NAPping PAnts (NAPPA): An open wearable solution for monitoring Infant's sleeping rhythms, respiration and posture. Heliyon 2024; 10:e33295. [PMID: 39027497 PMCID: PMC11255670 DOI: 10.1016/j.heliyon.2024.e33295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 05/13/2024] [Accepted: 06/18/2024] [Indexed: 07/20/2024] Open
Abstract
Study objectives To develop a non-invasive and practical wearable method for long-term tracking of infants' sleep. Methods An infant wearable, NAPping PAnts (NAPPA), was constructed by combining a diaper cover and a movement sensor (triaxial accelerometer and gyroscope), allowing either real-time data streaming to mobile devices or offline feature computation stored in the sensor memory. A sleep state classifier (wake, N1/REM, N2/N3) was trained and tested for NAPPA recordings (N = 16649 epochs of 30 s), using hypnograms from co-registered polysomnography (PSG) as a training target in 33 infants (age 2 weeks to 18 months; Mean = 4). User experience was assessed from an additional group of 16 parents. Results Overnight NAPPA recordings were successfully performed in all infants. The sleep state classifier showed good overall accuracy (78 %; Range 74-83 %) when using a combination of five features related to movement and respiration. Sleep depth trends were generated from the classifier outputs to visualise sleep state fluctuations, which closely aligned with PSG-derived hypnograms in all infants. Consistently positive parental feedback affirmed the effectiveness of the NAPPA-design. Conclusions NAPPA offers a practical and feasible method for out-of-hospital assessment of infants' sleep behaviour. It can directly support large-scale quantitative studies and development of new paradigms in scientific research and infant healthcare. Moreover, NAPPA provides accurate and informative computational measures for body positions, respiration rates, and activity levels, each with their respective clinical and behavioural value.
Collapse
Affiliation(s)
- Sofie de Sena
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
- Department of Physiology, University of Helsinki, Helsinki, Finland
| | - Matias Häggman
- School of Science, Department of Mathematics and Systems Analysis, Aalto University, Espoo, Finland
| | - Jukka Ranta
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | - Oleksii Roienko
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | - Elina Ilén
- Department of Materials Science and Engineering, Universitat Politècnica de Catalunya, BarcelonaTech, Barcelona, Spain
| | - Natalia Acosta
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | - Jonna Salama
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | - Turkka Kirjavainen
- Department of Paediatrics, Children's Hospital, Helsinki University Hospital, Helsinki, Finland
| | - Nathan Stevenson
- Brain Modelling Group, QIMR Berghofer Medical Research Institute, Brisbane, Australia
| | - Manu Airaksinen
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
- Department of Physiology, University of Helsinki, Helsinki, Finland
| | - Sampsa Vanhatalo
- BABA Center, Department of Clinical Neurophysiology, Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
- Department of Physiology, University of Helsinki, Helsinki, Finland
| |
Collapse
|
5
|
Jones AM, Itti L, Sheth BR. Expert-level sleep staging using an electrocardiography-only feed-forward neural network. Comput Biol Med 2024; 176:108545. [PMID: 38749325 DOI: 10.1016/j.compbiomed.2024.108545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 04/05/2024] [Accepted: 04/28/2024] [Indexed: 05/31/2024]
Abstract
Reliable classification of sleep stages is crucial in sleep medicine and neuroscience research for providing valuable insights, diagnoses, and understanding of brain states. The current gold standard method for sleep stage classification is polysomnography (PSG). Unfortunately, PSG is an expensive and cumbersome process involving numerous electrodes, often conducted in an unfamiliar clinic and annotated by a professional. Although commercial devices like smartwatches track sleep, their performance is well below PSG. To address these disadvantages, we present a feed-forward neural network that achieves gold-standard levels of agreement using only a single lead of electrocardiography (ECG) data. Specifically, the median five-stage Cohen's kappa is 0.725 on a large, diverse dataset of 5 to 90-year-old subjects. Comparisons with a comprehensive meta-analysis of between-human inter-rater agreement confirm the non-inferior performance of our model. Finally, we developed a novel loss function to align the training objective with Cohen's kappa. Our method offers an inexpensive, automated, and convenient alternative for sleep stage classification-further enhanced by a real-time scoring option. Cardiosomnography, or a sleep study conducted with ECG only, could take expert-level sleep studies outside the confines of clinics and laboratories and into realistic settings. This advancement democratizes access to high-quality sleep studies, considerably enhancing the field of sleep medicine and neuroscience. It makes less-expensive, higher-quality studies accessible to a broader community, enabling improved sleep research and more personalized, accessible sleep-related healthcare interventions.
Collapse
Affiliation(s)
- Adam M Jones
- Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, USA.
| | - Laurent Itti
- Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, USA
| | - Bhavin R Sheth
- Department of Electrical & Computer Engineering, University of Houston, Houston, TX, USA; Center for NeuroEngineering and Cognitive Systems, University of Houston, Houston, TX, USA
| |
Collapse
|
6
|
Bechny M, Monachino G, Fiorillo L, van der Meer J, Schmidt MH, Bassetti CLA, Tzovara A, Faraci FD. Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review. Nat Sci Sleep 2024; 16:555-572. [PMID: 38827394 PMCID: PMC11143488 DOI: 10.2147/nss.s455649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Accepted: 04/18/2024] [Indexed: 06/04/2024] Open
Abstract
Purpose This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain (ID) and out-of-domain (OOD) data, and considering subjects' diagnoses. Patients and Methods A total of 19,578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of an additional 8832 PSGs, covering a full spectrum of ages (0-91 years) and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician. Results U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve κ ≥ 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians' workload, and facilitating near-perfect agreement. Conclusion Inter-scorer variability limits the accuracy of the scoring algorithms to ~80%. By integrating an uncertainty estimation with U-Sleep, we enhance the review of predicted hypnograms, to align with the scoring taste of a responsible physician. Validated across ID and OOD data and various sleep-disorders, our approach offers a strategy to boost automated scoring tools' usability in clinical settings.
Collapse
Affiliation(s)
- Michal Bechny
- Institute of Computer Science, University of Bern, Bern, Switzerland
- Institute of Digital Technologies for Personalized Healthcare (Meditech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland
| | - Giuliana Monachino
- Institute of Computer Science, University of Bern, Bern, Switzerland
- Institute of Digital Technologies for Personalized Healthcare (Meditech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland
| | - Luigi Fiorillo
- Institute of Digital Technologies for Personalized Healthcare (Meditech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland
| | | | - Markus H Schmidt
- Department of Neurology, University Hospital of Bern, Bern, Switzerland
- Ohio Sleep Medicine Institute, Dublin, OH, USA
| | | | - Athina Tzovara
- Institute of Computer Science, University of Bern, Bern, Switzerland
- Department of Neurology, University Hospital of Bern, Bern, Switzerland
| | - Francesca D Faraci
- Institute of Digital Technologies for Personalized Healthcare (Meditech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland
| |
Collapse
|
7
|
Proost R, Heremans E, Lagae L, Van Paesschen W, De Vos M, Jansen K. Automated sleep staging on reduced channels in children with epilepsy. Front Neurol 2024; 15:1390465. [PMID: 38798709 PMCID: PMC11116721 DOI: 10.3389/fneur.2024.1390465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Accepted: 04/15/2024] [Indexed: 05/29/2024] Open
Abstract
Objectives This study aimed to validate a sleep staging algorithm using in-hospital video-electroencephalogram (EEG) in children without epilepsy, with well-controlled epilepsy (WCE), and with drug-resistant epilepsy (DRE). Methods Overnight video-EEG, along with electrooculogram (EOG) and chin electromyogram (EMG), was recorded in children between 4 and 18 years of age. Classical sleep staging was performed manually as a ground truth. An end-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging (SeqSleepNet) was used to perform automated sleep staging using three channels: C4-A1, EOG, and chin EMG. Results In 176 children sleep stages were manually scored: 47 children without epilepsy, 74 with WCE, and 55 with DRE. The 5-class sleep staging accuracy of the automatic sleep staging algorithm was 84.7% for the children without epilepsy, 83.5% for those with WCE, and 80.8% for those with DRE (Kappa of 0.79, 0.77, and 0.73 respectively). Performance per sleep stage was assessed with an F1 score of 0.91 for wake, 0.50 for N1, 0.83 for N2, 0.84 for N3, and 0.86 for rapid eye movement (REM) sleep. Conclusion We concluded that the tested algorithm has a high accuracy in children without epilepsy and with WCE. Performance in children with DRE was acceptable, but significantly lower, which could be explained by a tendency of more time spent in N1, and by abundant interictal epileptiform discharges and intellectual disability leading to less recognizable sleep stages. REM sleep time, however, significantly affected in children with DRE, can be detected reliably by the algorithm.Clinical trial registration: ClinicalTrials.gov, identifier NCT04584385.
Collapse
Affiliation(s)
- Renee Proost
- Pediatric Neurology Department, University Hospitals Leuven, KU Leuven, Leuven, Belgium
| | - Elisabeth Heremans
- Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
| | - Lieven Lagae
- Pediatric Neurology Department, University Hospitals Leuven, KU Leuven, Leuven, Belgium
| | - Wim Van Paesschen
- Neurology Department, University Hospitals Leuven, KU Leuven, Leuven, Belgium
| | - Maarten De Vos
- Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
| | - Katrien Jansen
- Pediatric Neurology Department, University Hospitals Leuven, KU Leuven, Leuven, Belgium
| |
Collapse
|
8
|
Waters SH, Clifford GD. Physics-Informed Transfer Learning to Enhance Sleep Staging. IEEE Trans Biomed Eng 2024; 71:1599-1606. [PMID: 38133969 DOI: 10.1109/tbme.2023.3345888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2023]
Abstract
OBJECTIVE At-home sleep staging using wearable medical sensors poses a viable alternative to in-hospital polysomnography due to its lower cost and lower disruption to the daily lives of patients, especially in the case of long-term monitoring. Machine learning with wearables however is difficult due to the paucity of data from wearable sensors, making automation a challenge. Transfer learning from hospital polysomnograms can boost performance, but is still hindered by differences between wearable and in-hospital EEG resulting in part from differing electrode placement. We improve transfer learning performance by using electrophysiological models of a human head to generate synthetic EEG resembling EEG from a wearable sensor. METHODS The data generation method utilizes Low-Resolution Electromagnetic Tomography Analysis (LORETA). Real EEG from standard in- hospital recordings is first mapped to point currents within the brain using LORETA, after which the point currents are used to estimate EEG that would have been recorded using a wearable sensor at any given point on the head. RESULTS Augmenting source datasets with synthetic data statistically significantly boosted accuracy on a wearable sleep staging task from 80.8% to 81.3% on average, depending on the transfer learning parameters and data sources. CONCLUSION Machine learning performance can be improved using data synthesized using physical models. SIGNIFICANCE Our approach represents a new form of transfer learning and demonstrates that incorporating domain knowledge of electrophysiological modeling can improve machine learning results for sleep staging tasks. We expect this approach to be particularly useful for EEG data which is hard to collect, or which is obtained using unusual electrode configurations.
Collapse
|
9
|
Ogg M, Coon WG. Self-Supervised Transformer Model Training for a Sleep-EEG Foundation Model. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.18.576245. [PMID: 38293234 PMCID: PMC10827180 DOI: 10.1101/2024.01.18.576245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2024]
Abstract
The American Academy of Sleep Medicine (AASM) recognizes five sleep/wake states (Wake, N1, N2, N3, REM), yet this classification schema provides only a high-level summary of sleep and likely overlooks important neurological or health information. New, data-driven approaches are needed to more deeply probe the information content of sleep signals. Here we present a self-supervised approach that learns the structure embedded in large quantities of neurophysiological sleep data. This masked transformer training procedure is inspired by high performing self-supervised methods developed for speech transcription. We show that self-supervised pre-training matches or outperforms supervised sleep stage classification, especially when labeled data or compute-power is limited. Perhaps more importantly, we also show that our pretrained model is flexible and can be fine-tuned to perform well on new tasks including distinguishing individuals and quantifying "brain age" (a potential health biomarker). This suggests that modern methods can automatically learn information that is potentially overlooked by the 5-class sleep staging schema, laying the groundwork for new schemas and further data-driven exploration of sleep.
Collapse
|
10
|
Radke FA, da Silva Souto CF, Pätzold W, Wolf KI. Transfer Learning for Automatic Sleep Staging Using a Pre-Gelled Electrode Grid. Diagnostics (Basel) 2024; 14:909. [PMID: 38732323 PMCID: PMC11083934 DOI: 10.3390/diagnostics14090909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 04/18/2024] [Accepted: 04/24/2024] [Indexed: 05/13/2024] Open
Abstract
Novel sensor solutions for sleep monitoring at home could alleviate bottlenecks in sleep medical care as well as enable selective or continuous observation over long periods of time and contribute to new insights in sleep medicine and beyond. Since especially in the latter case the sensor data differ strongly in signal, number and extent of sensors from the classical polysomnography (PSG) sensor technology, an automatic evaluation is essential for the application. However, the training of an automatic algorithm is complicated by the fact that the development phase of the new sensor technology, extensive comparative measurements with standardized reference systems, is often not possible and therefore only small datasets are available. In order to circumvent high system-specific training data requirements, we employ pre-training on large datasets with finetuning on small datasets of new sensor technology to enable automatic sleep phase detection for small test series. By pre-training on publicly available PSG datasets and finetuning on 12 nights recorded with new sensor technology based on a pre-gelled electrode grid to capture electroencephalography (EEG), electrooculography (EOG) and electromyography (EMG), an F1 score across all sleep phases of 0.81 is achieved (wake 0.84, N1 0.62, N2 0.81, N3 0.87, REM 0.88), using only EEG and EOG. The analysis additionally considers the spatial distribution of the channels and an approach to approximate classical electrode positions based on specific linear combinations of the new sensor grid channels.
Collapse
Affiliation(s)
- Fabian A. Radke
- Fraunhofer Institute for Digital Media Technology IDMT, Oldenburg Branch for Hearing, Speech and Audio Technology HSA, 26129 Oldenburg, Germany
| | | | | | | |
Collapse
|
11
|
Schyvens AM, Van Oost NC, Aerts JM, Masci F, Peters B, Neven A, Dirix H, Wets G, Ross V, Verbraecken J. Accuracy of Fitbit Charge 4, Garmin Vivosmart 4, and WHOOP Versus Polysomnography: Systematic Review. JMIR Mhealth Uhealth 2024; 12:e52192. [PMID: 38557808 PMCID: PMC11004611 DOI: 10.2196/52192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 02/01/2024] [Accepted: 02/01/2024] [Indexed: 04/04/2024] Open
Abstract
Background Despite being the gold-standard method for objectively assessing sleep, polysomnography (PSG) faces several limitations as it is expensive, time-consuming, and labor-intensive; requires various equipment and technical expertise; and is impractical for long-term or in-home use. Consumer wrist-worn wearables are able to monitor sleep parameters and thus could be used as an alternative for PSG. Consequently, wearables gained immense popularity over the past few years, but their accuracy has been a major concern. Objective A systematic review of the literature was conducted to appraise the performance of 3 recent-generation wearable devices (Fitbit Charge 4, Garmin Vivosmart 4, and WHOOP) in determining sleep parameters and sleep stages. Methods Per the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement, a comprehensive search was conducted using the PubMed, Web of Science, Google Scholar, Scopus, and Embase databases. Eligible publications were those that (1) involved the validity of sleep data of any marketed model of the candidate wearables and (2) used PSG or an ambulatory electroencephalogram monitor as a reference sleep monitoring device. Exclusion criteria were as follows: (1) incorporated a sleep diary or survey method as a reference, (2) review paper, (3) children as participants, and (4) duplicate publication of the same data and findings. Results The search yielded 504 candidate articles. After eliminating duplicates and applying the eligibility criteria, 8 articles were included. WHOOP showed the least disagreement relative to PSG and Sleep Profiler for total sleep time (-1.4 min), light sleep (-9.6 min), and deep sleep (-9.3 min) but showed the largest disagreement for rapid eye movement (REM) sleep (21.0 min). Fitbit Charge 4 and Garmin Vivosmart 4 both showed moderate accuracy in assessing sleep stages and total sleep time compared to PSG. Fitbit Charge 4 showed the least disagreement for REM sleep (4.0 min) relative to PSG. Additionally, Fitbit Charge 4 showed higher sensitivities to deep sleep (75%) and REM sleep (86.5%) compared to Garmin Vivosmart 4 and WHOOP. Conclusions The findings of this systematic literature review indicate that the devices with higher relative agreement and sensitivities to multistate sleep (ie, Fitbit Charge 4 and WHOOP) seem appropriate for deriving suitable estimates of sleep parameters. However, analyses regarding the multistate categorization of sleep indicate that all devices can benefit from further improvement in the assessment of specific sleep stages. Although providers are continuously developing new versions and variants of wearables, the scientific research on these wearables remains considerably limited. This scarcity in literature not only reduces our ability to draw definitive conclusions but also highlights the need for more targeted research in this domain. Additionally, future research endeavors should strive for standardized protocols including larger sample sizes to enhance the comparability and power of the results across studies.
Collapse
Affiliation(s)
- An-Marie Schyvens
- Multidisciplinary Sleep Disorders Centre, Antwerp University Hospital, Edegem, Belgium
- Laboratory of Experimental Medicine and Pediatrics, University of Antwerp, Wilrijk, Belgium
| | | | | | | | - Brent Peters
- Transportation Research Institute (IMOB), School of Transportation Sciences, UHasselt, Hasselt, Belgium
| | - An Neven
- Transportation Research Institute (IMOB), School of Transportation Sciences, UHasselt, Hasselt, Belgium
| | - Hélène Dirix
- Transportation Research Institute (IMOB), School of Transportation Sciences, UHasselt, Hasselt, Belgium
| | - Geert Wets
- Transportation Research Institute (IMOB), School of Transportation Sciences, UHasselt, Hasselt, Belgium
| | - Veerle Ross
- Transportation Research Institute (IMOB), School of Transportation Sciences, UHasselt, Hasselt, Belgium
- Faresa, Evidence-Based Psychological Centre, Hasselt, Belgium
| | - Johan Verbraecken
- Multidisciplinary Sleep Disorders Centre, Antwerp University Hospital, Edegem, Belgium
- Laboratory of Experimental Medicine and Pediatrics, University of Antwerp, Wilrijk, Belgium
| |
Collapse
|
12
|
Pevernagie DA, Arnardottir ES. Looking for clues in the hypnogram-the human eye and the machine. Sleep 2024; 47:zsae011. [PMID: 38219052 DOI: 10.1093/sleep/zsae011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Indexed: 01/15/2024] Open
Affiliation(s)
- Dirk A Pevernagie
- Department Respiratory Diseases and Sleep Disorders Centre, AZ Delta, Rumbeke, Belgium
- Department Internal Medicine and Paediatrics, Faculty of Medicine and Health Sciences, Ghent University, Gent, Belgium
| | - Erna S Arnardottir
- Reykjavik University Sleep Institute, School of Technology, Reykjavik University, Reykjavik, Iceland
- Internal Medicine Services, Landspitali - The National University Hospital of Iceland, Reykjavik, Iceland
| |
Collapse
|
13
|
Le TQ, Huynh P, Tomaselli L. Navigating the night: evaluating and accessing wearable sleep trackers for clinical use. Sleep 2024; 47:zsad319. [PMID: 38097380 PMCID: PMC10925943 DOI: 10.1093/sleep/zsad319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2024] Open
Affiliation(s)
- Trung Q Le
- Department of Industrial and Management Systems Engineering, University of South Florida, Tampa, FL, USA
- Department of Medical Engineering, University of South Florida, Tampa, FL, USA
- James A. Haley Veterans’ Hospital, Tampa VA Healthcare System, Tampa, FL, USA and
| | - Phat Huynh
- Department of Industrial and Management Systems Engineering, University of South Florida, Tampa, FL, USA
| | - Lennon Tomaselli
- Department of Health Sciences, University of South Florida, Tampa, FL, USA
| |
Collapse
|
14
|
Korkalainen H, Kainulainen S, Islind AS, Óskarsdóttir M, Strassberger C, Nikkonen S, Töyräs J, Kulkas A, Grote L, Hedner J, Sund R, Hrubos-Strom H, Saavedra JM, Ólafsdóttir KA, Ágústsson JS, Terrill PI, McNicholas WT, Arnardóttir ES, Leppänen T. Review and perspective on sleep-disordered breathing research and translation to clinics. Sleep Med Rev 2024; 73:101874. [PMID: 38091850 DOI: 10.1016/j.smrv.2023.101874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 09/18/2023] [Accepted: 11/09/2023] [Indexed: 01/23/2024]
Abstract
Sleep-disordered breathing, ranging from habitual snoring to severe obstructive sleep apnea, is a prevalent public health issue. Despite rising interest in sleep and awareness of sleep disorders, sleep research and diagnostic practices still rely on outdated metrics and laborious methods reducing the diagnostic capacity and preventing timely diagnosis and treatment. Consequently, a significant portion of individuals affected by sleep-disordered breathing remain undiagnosed or are misdiagnosed. Taking advantage of state-of-the-art scientific, technological, and computational advances could be an effective way to optimize the diagnostic and treatment pathways. We discuss state-of-the-art multidisciplinary research, review the shortcomings in the current practices of SDB diagnosis and management in adult populations, and provide possible future directions. We critically review the opportunities for modern data analysis methods and machine learning to combine multimodal information, provide a perspective on the pitfalls of big data analysis, and discuss approaches for developing analysis strategies that overcome current limitations. We argue that large-scale and multidisciplinary collaborative efforts based on clinical, scientific, and technical knowledge and rigorous clinical validation and implementation of the outcomes in practice are needed to move the research of sleep-disordered breathing forward, thus increasing the quality of diagnostics and treatment.
Collapse
Affiliation(s)
- Henri Korkalainen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland.
| | - Samu Kainulainen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland
| | - Anna Sigridur Islind
- Department of Computer Science, Reykjavik University, Reykjavik, Iceland; Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland
| | - María Óskarsdóttir
- Department of Computer Science, Reykjavik University, Reykjavik, Iceland
| | - Christian Strassberger
- Centre for Sleep and Wake Disorders, Sahlgrenska Academy, Gothenburg University, Gothenburg, Sweden
| | - Sami Nikkonen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland
| | - Juha Töyräs
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Australia; Science Service Center, Kuopio University Hospital, Kuopio, Finland
| | - Antti Kulkas
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; Department of Clinical Neurophysiology, Seinäjoki Central Hospital, Seinäjoki, Finland
| | - Ludger Grote
- Centre for Sleep and Wake Disorders, Sahlgrenska Academy, Gothenburg University, Gothenburg, Sweden; Sleep Disorders Centre, Pulmonary Medicine, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Jan Hedner
- Centre for Sleep and Wake Disorders, Sahlgrenska Academy, Gothenburg University, Gothenburg, Sweden; Sleep Disorders Centre, Pulmonary Medicine, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Reijo Sund
- School of Medicine, Institute of Clinical Medicine, University of Eastern Finland, Kuopio, Finland
| | - Harald Hrubos-Strom
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway; Department of Ear, Nose and Throat Surgery, Akershus University Hospital, Lørenskog, Norway
| | - Jose M Saavedra
- Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland; Physical Activity, Physical Education, Sport and Health (PAPESH) Research Group, Department of Sports Science, Reykjavik University, Reykjavik, Iceland
| | | | | | - Philip I Terrill
- School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Australia
| | - Walter T McNicholas
- School of Medicine, University College Dublin, and Department of Respiratory and Sleep Medicine, St Vincent's Hospital Group, Dublin Ireland
| | - Erna Sif Arnardóttir
- Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland; Landspitali - The National University Hospital of Iceland, Reykjavik, Iceland
| | - Timo Leppänen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland; Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland; School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Australia
| |
Collapse
|
15
|
Nikkonen S, Somaskandhan P, Korkalainen H, Kainulainen S, Terrill PI, Gretarsdottir H, Sigurdardottir S, Olafsdottir KA, Islind AS, Óskarsdóttir M, Arnardóttir ES, Leppänen T. Multicentre sleep-stage scoring agreement in the Sleep Revolution project. J Sleep Res 2024; 33:e13956. [PMID: 37309714 PMCID: PMC10909532 DOI: 10.1111/jsr.13956] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 05/04/2023] [Accepted: 05/11/2023] [Indexed: 06/14/2023]
Abstract
Determining sleep stages accurately is an important part of the diagnostic process for numerous sleep disorders. However, as the sleep stage scoring is done manually following visual scoring rules there can be considerable variation in the sleep staging between different scorers. Thus, this study aimed to comprehensively evaluate the inter-rater agreement in sleep staging. A total of 50 polysomnography recordings were manually scored by 10 independent scorers from seven different sleep centres. We used the 10 scorings to calculate a majority score by taking the sleep stage that was the most scored stage for each epoch. The overall agreement for sleep staging was κ = 0.71 and the mean agreement with the majority score was 0.86. The scorers were in perfect agreement in 48% of all scored epochs. The agreement was highest in rapid eye movement sleep (κ = 0.86) and lowest in N1 sleep (κ = 0.41). The agreement with the majority scoring varied between the scorers from 81% to 91%, with large variations between the scorers in sleep stage-specific agreements. Scorers from the same sleep centres had the highest pairwise agreements at κ = 0.79, κ = 0.85, and κ = 0.78, while the lowest pairwise agreement between the scorers was κ = 0.58. We also found a moderate negative correlation between sleep staging agreement and the apnea-hypopnea index, as well as the rate of sleep stage transitions. In conclusion, although the overall agreement was high, several areas of low agreement were also found, mainly between non-rapid eye movement stages.
Collapse
Affiliation(s)
- Sami Nikkonen
- Department of Technical PhysicsUniversity of Eastern FinlandKuopioFinland
- Diagnostic Imaging CenterKuopio University HospitalKuopioFinland
| | - Pranavan Somaskandhan
- School of Information Technology and Electrical EngineeringThe University of QueenslandBrisbaneQueenslandAustralia
| | - Henri Korkalainen
- Department of Technical PhysicsUniversity of Eastern FinlandKuopioFinland
- Diagnostic Imaging CenterKuopio University HospitalKuopioFinland
| | - Samu Kainulainen
- Department of Technical PhysicsUniversity of Eastern FinlandKuopioFinland
- Diagnostic Imaging CenterKuopio University HospitalKuopioFinland
| | - Philip I. Terrill
- School of Information Technology and Electrical EngineeringThe University of QueenslandBrisbaneQueenslandAustralia
| | - Heidur Gretarsdottir
- Reykjavik University Sleep Institute, School of TechnologyReykjavik UniversityReykjavikIceland
| | - Sigridur Sigurdardottir
- Reykjavik University Sleep Institute, School of TechnologyReykjavik UniversityReykjavikIceland
| | | | - Anna Sigridur Islind
- Reykjavik University Sleep Institute, School of TechnologyReykjavik UniversityReykjavikIceland
- Department of Computer ScienceReykjavík UniversityReykajvíkIceland
| | - María Óskarsdóttir
- Reykjavik University Sleep Institute, School of TechnologyReykjavik UniversityReykjavikIceland
- Department of Computer ScienceReykjavík UniversityReykajvíkIceland
| | - Erna Sif Arnardóttir
- Reykjavik University Sleep Institute, School of TechnologyReykjavik UniversityReykjavikIceland
| | - Timo Leppänen
- Department of Technical PhysicsUniversity of Eastern FinlandKuopioFinland
- Diagnostic Imaging CenterKuopio University HospitalKuopioFinland
- School of Information Technology and Electrical EngineeringThe University of QueenslandBrisbaneQueenslandAustralia
| |
Collapse
|
16
|
Ong JL, Golkashani HA, Ghorbani S, Wong KF, Chee NIYN, Willoughby AR, Chee MWL. Selecting a sleep tracker from EEG-based, iteratively improved, low-cost multisensor, and actigraphy-only devices. Sleep Health 2024; 10:9-23. [PMID: 38087674 DOI: 10.1016/j.sleh.2023.11.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Revised: 11/01/2023] [Accepted: 11/11/2023] [Indexed: 03/01/2024]
Abstract
AIMS Evaluate the performance of 6 wearable sleep trackers across 4 classes (EEG-based headband, research-grade actigraphy, iteratively improved consumer tracker, low-cost consumer tracker). FOCUS TECHNOLOGY Dreem 3 headband, Actigraph GT9X, Oura Ring Gen3, Fitbit Sense, Xiaomi Mi Band 7, Axtro Fit3. REFERENCE TECHNOLOGY In-lab polysomnography with 3-reader, consensus sleep scoring. SAMPLE Sixty participants (26 males) across 3 age groups (18-30, 31-50, and 51-70years). DESIGN Overnight in a sleep laboratory from habitual sleep time to wake time. CORE ANALYTICS Discrepancy and epoch-by-epoch analyses for sleep/wake (2-stage) and sleep-stage (4-stage; wake/light/deep/rapid eye movement) classification (devices vs. polysomnography). CORE OUTCOMES EEG-based Dreem performed the best (2-stage kappa=0.76, 4-stage kappa=0.76-0.86) with the lowest total sleep time, sleep efficiency, sleep onset latency, and wake after sleep onset discrepancies vs. polysomnography. This was followed by the iteratively improved consumer trackers: Oura (2-stage kappa=0.64, 4-stage kappa=0.55-0.70) and Fitbit (2-stage kappa=0.58, 4-stage kappa=0.45-0.60) which had comparable total sleep time and sleep efficiency discrepancies that outperformed accelerometry-only Actigraph (2-stage kappa=0.47). The low-cost consumer trackers had poorest overall performance (2-stage kappa<0.31, 4-stage kappa<0.33). IMPORTANT ADDITIONAL OUTCOMES Proportional biases were driven by nights with poorer sleep (longer sleep onset latencies and/or wake after sleep onset). CORE CONCLUSION EEG-based Dreem is recommended when evaluating poor quality sleep or when highest accuracy sleep-staging is required. Iteratively improved non-EEG sleep trackers (Oura, Fitbit) balance classification accuracy with well-tolerated, and economic deployment at-scale, and are recommended for studies involving mostly healthy sleepers. The low-cost trackers, can log time in bed but are not recommended for research use.
Collapse
Affiliation(s)
- Ju Lynn Ong
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
| | - Hosein Aghayan Golkashani
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Shohreh Ghorbani
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Kian F Wong
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Nicholas I Y N Chee
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Adrian R Willoughby
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Michael W L Chee
- Sleep and Cognition Laboratory, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
| |
Collapse
|
17
|
Liao YS, Wu MC, Li CX, Lin WK, Lin CY, Liang SF. Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement. J Clin Sleep Med 2024; 20:271-278. [PMID: 37811900 PMCID: PMC10835767 DOI: 10.5664/jcsm.10852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 09/29/2023] [Accepted: 10/05/2023] [Indexed: 10/10/2023]
Abstract
STUDY OBJECTIVES To efficiently improve the scoring competency of scorers with varying levels of experience across regions in Taiwan, we developed a training program with a cloud-based polysomnography scoring platform to evaluate and improve interscorer agreement. METHODS A total of 70 scorers from 34 sleep centers in Taiwan (job tenure: 0.5-39.0 years) completed a scoring test. All scorers scored a 742-epoch (30 s/epoch) overnight polysomnography recording of a patient with a moderate apnea-hypopnea index. Subsequently, 8 scoring experts delivered 8 interactive online lectures (each lasting 30 minutes). The training program included identifying scoring weaknesses, highlighting the latest scoring rules, and providing physicians' perspectives. Afterward, the scorers completed the second scoring test on the same participant. Changes in agreement from the first to second scoring test were identified. Sleep staging, sleep parameters, and respiratory events were considered for evaluating scoring agreement. RESULTS The scorers' agreement in overall sleep stage scoring significantly increased from 74.6 to 82.3% (median score). The proportion of scorers with an agreement of ≥ 80% increased from 20.0% (14/70) to 58.6% (41/70) after the online training program. In addition, the scorers' agreement in overall respiratory-event scoring increased to 88.8% (median score) after training. The scorers with a job tenure of 2.0-4.9 years exhibited the highest level of improvement in overall sleep staging (their median agreement increased from 72.8 to 84.9%; P < .001). CONCLUSIONS Our interactive online training program efficiently targeted the scorers' scoring weaknesses identified in the first scoring test, leading to substantial improvements in scoring proficiency. CITATION Liao Y-S, Wu M-C, Li C-X, Lin W-K, Lin C-Y, Liang S-F. Polysomnography scoring-related training and quantitative assessment for improving interscorer agreement. J Clin Sleep Med. 2024;20(2):271-278.
Collapse
Affiliation(s)
- Ying-Siou Liao
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan
| | - Meng-Chun Wu
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan
| | - Cheng-Xue Li
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan
| | - Wen-Kuei Lin
- Sleep Medicine Center, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan
| | - Cheng-Yu Lin
- Sleep Medicine Center, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan
- Department of Otolaryngology, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan
| | - Sheng-Fu Liang
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan
- Institute of Medical Informatics, National Cheng Kung University, Tainan, Taiwan
| |
Collapse
|
18
|
Van Der Aar JF, Van Den Ende DA, Fonseca P, Van Meulen FB, Overeem S, Van Gilst MM, Peri E. Deep transfer learning for automated single-lead EEG sleep staging with channel and population mismatches. Front Physiol 2024; 14:1287342. [PMID: 38250654 PMCID: PMC10796543 DOI: 10.3389/fphys.2023.1287342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Accepted: 12/08/2023] [Indexed: 01/23/2024] Open
Abstract
Introduction: Automated sleep staging using deep learning models typically requires training on hundreds of sleep recordings, and pre-training on public databases is therefore common practice. However, suboptimal sleep stage performance may occur from mismatches between source and target datasets, such as differences in population characteristics (e.g., an unrepresented sleep disorder) or sensors (e.g., alternative channel locations for wearable EEG). Methods: We investigated three strategies for training an automated single-channel EEG sleep stager: pre-training (i.e., training on the original source dataset), training-from-scratch (i.e., training on the new target dataset), and fine-tuning (i.e., training on the original source dataset, fine-tuning on the new target dataset). As source dataset, we used the F3-M2 channel of healthy subjects (N = 94). Performance of the different training strategies was evaluated using Cohen's Kappa (κ) in eight smaller target datasets consisting of healthy subjects (N = 60), patients with obstructive sleep apnea (OSA, N = 60), insomnia (N = 60), and REM sleep behavioral disorder (RBD, N = 22), combined with two EEG channels, F3-M2 and F3-F4. Results: No differences in performance between the training strategies was observed in the age-matched F3-M2 datasets, with an average performance across strategies of κ = .83 in healthy, κ = .77 in insomnia, and κ = .74 in OSA subjects. However, in the RBD set, where data availability was limited, fine-tuning was the preferred method (κ = .67), with an average increase in κ of .15 to pre-training and training-from-scratch. In the presence of channel mismatches, targeted training is required, either through training-from-scratch or fine-tuning, increasing performance with κ = .17 on average. Discussion: We found that, when channel and/or population mismatches cause suboptimal sleep staging performance, a fine-tuning approach can yield similar to superior performance compared to building a model from scratch, while requiring a smaller sample size. In contrast to insomnia and OSA, RBD data contains characteristics, either inherent to the pathology or age-related, which apparently demand targeted training.
Collapse
Affiliation(s)
- Jaap F. Van Der Aar
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
- Philips Research, Eindhoven, Netherlands
| | | | - Pedro Fonseca
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
- Philips Research, Eindhoven, Netherlands
| | - Fokke B. Van Meulen
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
- Kempenhaeghe Center for Sleep Medicine, Heeze, Netherlands
| | - Sebastiaan Overeem
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
- Kempenhaeghe Center for Sleep Medicine, Heeze, Netherlands
| | - Merel M. Van Gilst
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
- Kempenhaeghe Center for Sleep Medicine, Heeze, Netherlands
| | - Elisabetta Peri
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| |
Collapse
|
19
|
Brodersen PJN, Alfonsa H, Krone LB, Blanco-Duque C, Fisk AS, Flaherty SJ, Guillaumin MCC, Huang YG, Kahn MC, McKillop LE, Milinski L, Taylor L, Thomas CW, Yamagata T, Foster RG, Vyazovskiy VV, Akerman CJ. Somnotate: A probabilistic sleep stage classifier for studying vigilance state transitions. PLoS Comput Biol 2024; 20:e1011793. [PMID: 38232122 PMCID: PMC10824458 DOI: 10.1371/journal.pcbi.1011793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 01/29/2024] [Accepted: 01/02/2024] [Indexed: 01/19/2024] Open
Abstract
Electrophysiological recordings from freely behaving animals are a widespread and powerful mode of investigation in sleep research. These recordings generate large amounts of data that require sleep stage annotation (polysomnography), in which the data is parcellated according to three vigilance states: awake, rapid eye movement (REM) sleep, and non-REM (NREM) sleep. Manual and current computational annotation methods ignore intermediate states because the classification features become ambiguous, even though intermediate states contain important information regarding vigilance state dynamics. To address this problem, we have developed "Somnotate"-a probabilistic classifier based on a combination of linear discriminant analysis (LDA) with a hidden Markov model (HMM). First we demonstrate that Somnotate sets new standards in polysomnography, exhibiting annotation accuracies that exceed human experts on mouse electrophysiological data, remarkable robustness to errors in the training data, compatibility with different recording configurations, and an ability to maintain high accuracy during experimental interventions. However, the key feature of Somnotate is that it quantifies and reports the certainty of its annotations. We leverage this feature to reveal that many intermediate vigilance states cluster around state transitions, whereas others correspond to failed attempts to transition. This enables us to show for the first time that the success rates of different types of transition are differentially affected by experimental manipulations and can explain previously observed sleep patterns. Somnotate is open-source and has the potential to both facilitate the study of sleep stage transitions and offer new insights into the mechanisms underlying sleep-wake dynamics.
Collapse
Affiliation(s)
- Paul J. N. Brodersen
- Department of Pharmacology, University of Oxford; Mansfield Road, Oxford, United Kingdom
| | - Hannah Alfonsa
- Department of Pharmacology, University of Oxford; Mansfield Road, Oxford, United Kingdom
| | - Lukas B. Krone
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Cristina Blanco-Duque
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Angus S. Fisk
- Nuffield Department of Clinical Neurosciences, University of Oxford; John Radcliffe Hospital, Oxford, United Kingdom
| | - Sarah J. Flaherty
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Mathilde C. C. Guillaumin
- Nuffield Department of Clinical Neurosciences, University of Oxford; John Radcliffe Hospital, Oxford, United Kingdom
- Sleep and Circadian Neuroscience Institute, University of Oxford; Oxford, United Kingdom
- Institute for Neuroscience, Department of Health Sciences and Technology, ETH Zurich; Schwerzenbach, Switzerland
| | - Yi-Ge Huang
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Martin C. Kahn
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Laura E. McKillop
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Linus Milinski
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Lewis Taylor
- Nuffield Department of Clinical Neurosciences, University of Oxford; John Radcliffe Hospital, Oxford, United Kingdom
| | - Christopher W. Thomas
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Tomoko Yamagata
- Nuffield Department of Clinical Neurosciences, University of Oxford; John Radcliffe Hospital, Oxford, United Kingdom
| | - Russell G. Foster
- Sleep and Circadian Neuroscience Institute, University of Oxford; Oxford, United Kingdom
| | - Vladyslav V. Vyazovskiy
- Department of Physiology, Anatomy and Genetics, University of Oxford; Parks Road, United Kingdom
| | - Colin J. Akerman
- Department of Pharmacology, University of Oxford; Mansfield Road, Oxford, United Kingdom
| |
Collapse
|
20
|
Jeong J, Yoon W, Lee JG, Kim D, Woo Y, Kim DK, Shin HW. Standardized image-based polysomnography database and deep learning algorithm for sleep-stage classification. Sleep 2023; 46:zsad242. [PMID: 37703391 DOI: 10.1093/sleep/zsad242] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2022] [Revised: 08/11/2023] [Indexed: 09/15/2023] Open
Abstract
STUDY OBJECTIVES Polysomnography (PSG) scoring is labor-intensive, subjective, and often ambiguous. Recently several deep learning (DL) models for automated sleep scoring have been developed, they are tied to a fixed amount of input channels and resolution. In this study, we constructed a standardized image-based PSG dataset in order to overcome the heterogeneity of raw signal data obtained from various PSG devices and various sleep laboratory environments. METHODS All individually exported European data format files containing raw signals were converted into images with an annotation file, which contained the demographics, diagnoses, and sleep statistics. An image-based DL model for automatic sleep staging was developed, compared with a signal-based model, and validated in an external dataset. RESULTS We constructed 10253 image-based PSG datasets using a standardized format. Among these, 7745 diagnostic PSG data were used to develop our DL model. The DL model using the image dataset showed similar performance to the signal-based dataset for the same subject. The overall DL accuracy was greater than 80%, even with severe obstructive sleep apnea. Moreover, for the first time, we showed explainable DL in the field of sleep medicine as visualized key inference regions using Eigen-class activation maps. Furthermore, when a DL model for sleep scoring performs external validation, we achieved a relatively good performance. CONCLUSIONS Our main contribution demonstrates the availability of a standardized image-based dataset, and highlights that changing the data sampling rate or number of sensors may not require retraining, although performance decreases slightly as the number of sensors decreases.
Collapse
Affiliation(s)
- Jaemin Jeong
- Department of Computer Engineering, School of Software, Hallym University, Chuncheon, Republic of Korea
| | | | - Jeong-Gun Lee
- Department of Computer Engineering, School of Software, Hallym University, Chuncheon, Republic of Korea
| | - Dongyoung Kim
- Department of Computer Engineering, School of Software, Hallym University, Chuncheon, Republic of Korea
| | - Yunhee Woo
- Institute of New Frontier Research, Division of Big Data and Artificial Intelligence, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon, Republic of Korea
| | - Dong-Kyu Kim
- OUaR LaB, Inc, Seoul, Republic of Korea
- Institute of New Frontier Research, Division of Big Data and Artificial Intelligence, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon, Republic of Korea
- Department of Otorhinolaryngology-Head and Neck Surgery, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon, Republic of Korea¸
| | - Hyun-Woo Shin
- OUaR LaB, Inc, Seoul, Republic of Korea
- Obstructive Upper Airway Research (OUaR) Laboratory, Department of Pharmacology, Seoul National University College of Medicine, Seoul, Republic of Korea
- Department of Biomedical Sciences, Seoul National University Graduate School, Seoul, Republic of Korea
- Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
- Sensory Organ Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Hospital, Seoul, Republic of Korea
| |
Collapse
|
21
|
Grassi M, Daccò S, Caldirola D, Perna G, Schruers K, Defillo A. Enhanced sleep staging with artificial intelligence: a validation study of new software for sleep scoring. Front Artif Intell 2023; 6:1278593. [PMID: 38145233 PMCID: PMC10739507 DOI: 10.3389/frai.2023.1278593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 11/14/2023] [Indexed: 12/26/2023] Open
Abstract
Manual sleep staging (MSS) using polysomnography is a time-consuming task, requires significant training, and can lead to significant variability among scorers. STAGER is a software program based on machine learning algorithms that has been developed by Medibio Limited (Savage, MN, USA) to perform automatic sleep staging using only EEG signals from polysomnography. This study aimed to extensively investigate its agreement with MSS performed during clinical practice and by three additional expert sleep technicians. Forty consecutive polysomnographic recordings of patients referred to three US sleep clinics for sleep evaluation were retrospectively collected and analyzed. Three experienced technicians independently staged the recording using the electroencephalography, electromyography, and electrooculography signals according to the American Academy of Sleep Medicine guidelines. The staging initially performed during clinical practice was also considered. Several agreement statistics between the automatic sleep staging (ASS) and MSS, among the different MSSs, and their differences were calculated. Bootstrap resampling was used to calculate 95% confidence intervals and the statistical significance of the differences. STAGER's ASS was most comparable with, or statistically significantly better than the MSS, except for a partial reduction in the positive percent agreement in the wake stage. These promising results indicate that STAGER software can perform ASS of inpatient polysomnographic recordings accurately in comparison with MSS.
Collapse
Affiliation(s)
- Massimiliano Grassi
- Medibio Limited, Savage, MN, United States
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Department of Clinical Neurosciences, Villa San Benedetto Menni Hospital, Hermanas Hospitalarias, Albese con Cassano, Italy
| | - Silvia Daccò
- Medibio Limited, Savage, MN, United States
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Department of Clinical Neurosciences, Villa San Benedetto Menni Hospital, Hermanas Hospitalarias, Albese con Cassano, Italy
- Humanitas San Pio X, Personalized Medicine Center for Anxiety and Panic Disorders, Milan, Italy
| | - Daniela Caldirola
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Department of Clinical Neurosciences, Villa San Benedetto Menni Hospital, Hermanas Hospitalarias, Albese con Cassano, Italy
- Humanitas San Pio X, Personalized Medicine Center for Anxiety and Panic Disorders, Milan, Italy
| | - Giampaolo Perna
- Medibio Limited, Savage, MN, United States
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Department of Clinical Neurosciences, Villa San Benedetto Menni Hospital, Hermanas Hospitalarias, Albese con Cassano, Italy
- Humanitas San Pio X, Personalized Medicine Center for Anxiety and Panic Disorders, Milan, Italy
- Department of Psychiatry and Neuropsychology, Faculty of Health, Medicine, and Life Sciences, Research Institute of Mental Health and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Koen Schruers
- Department of Psychiatry and Neuropsychology, Faculty of Health, Medicine, and Life Sciences, Research Institute of Mental Health and Neuroscience, Maastricht University, Maastricht, Netherlands
| | | |
Collapse
|
22
|
Morssinkhof MWL, van der Werf YD, van den Heuvel OA, van den Ende DA, van der Tuuk K, den Heijer M, Broekman BFP. Influence of sex hormone use on sleep architecture in a transgender cohort. Sleep 2023; 46:zsad249. [PMID: 37715990 PMCID: PMC10636253 DOI: 10.1093/sleep/zsad249] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 09/05/2023] [Indexed: 09/18/2023] Open
Abstract
STUDY OBJECTIVES Sex differences in sleep architecture are well-documented, with females experiencing longer total sleep time, more slow wave sleep (SWS), and shorter Rapid Eye Movement (REM) sleep duration than males. Although studies imply that sex hormones could affect sleep, research on exogenous sex hormones on sleep architecture is still inconclusive. This study examined sleep architecture changes in transgender individuals after 3 months of gender-affirming hormone therapy (GAHT). METHODS We assessed sleep architecture in 73 transgender individuals: 38 transmasculine participants who started using testosterone and 35 transfeminine participants who started using estrogens and antiandrogens. Sleep architecture was measured before GAHT and after 3 months of GAHT for 7 nights using an ambulatory single-electrode sleep EEG device. Changes in sleep architecture were analyzed using linear mixed models, and non-normally distributed outcomes were log-transformed and reported as percentages. RESULTS In transmasculine participants, SWS decreased by 7 minutes (95% CI: -12; -3) and 1.7% (95% CI: -3%; -0.5%), REM sleep latency decreased by 39% (95% CI: -52%; -22%) and REM sleep duration increased by 17 minutes (95% CI: 7; 26) after 3 months of GAHT. In transfeminine participants, sleep architecture showed no significant changes after 3 months of GAHT. CONCLUSIONS Sleep architecture changes after 3 months of masculinizing GAHT in line with sleep in cisgender males, while it shows no changes after feminizing GAHT. The sex-specific nature of these changes raises new questions about sex hormones and sleep. Future research should focus on studying possible underlying neural mechanisms and clinical consequences of these changes.
Collapse
Affiliation(s)
- Margot W L Morssinkhof
- Department of Psychiatry, Amsterdam UMC, Location Vrije Universiteit Amsterdam, The Netherlands
- Department of Endocrinology and Metabolism, Amsterdam UMC, Location Vrije Universiteit Amsterdam, The Netherlands
- Department of Psychiatry and Medical Psychology, OLVG, Amsterdam, The Netherlands
- Center of Expertise on Gender Dysphoria, Amsterdam UMC, Vrije Universiteit Amsterdam, The Netherlands
| | - Ysbrand D van der Werf
- Department of Anatomy and Neurosciences, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Amsterdam Neuroscience, Compulsivity Impulsivity and Attention, Amsterdam, The Netherlands
| | - Odile A van den Heuvel
- Department of Psychiatry, Amsterdam UMC, Location Vrije Universiteit Amsterdam, The Netherlands
- Department of Anatomy and Neurosciences, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Amsterdam Neuroscience, Compulsivity Impulsivity and Attention, Amsterdam, The Netherlands
| | - Daan A van den Ende
- Remote Patient Monitoring & Chronic Care, Philips, Eindhoven, The Netherlands
| | - Karin van der Tuuk
- Department of Obstetrics and Gynecology, University Medical Centre Groningen, Groningen, The Netherlands
| | - Martin den Heijer
- Department of Endocrinology and Metabolism, Amsterdam UMC, Location Vrije Universiteit Amsterdam, The Netherlands
- Center of Expertise on Gender Dysphoria, Amsterdam UMC, Vrije Universiteit Amsterdam, The Netherlands
| | - Birit F P Broekman
- Department of Psychiatry, Amsterdam UMC, Location Vrije Universiteit Amsterdam, The Netherlands
- Department of Psychiatry and Medical Psychology, OLVG, Amsterdam, The Netherlands
- Amsterdam Public Health, Mental Health Program, Amsterdam, The Netherlands
| |
Collapse
|
23
|
Anido-Alonso A, Alvarez-Estevez D. Decentralized Data-Privacy Preserving Deep-Learning Approaches for Enhancing Inter-Database Generalization in Automatic Sleep Staging. IEEE J Biomed Health Inform 2023; 27:5610-5621. [PMID: 37651482 DOI: 10.1109/jbhi.2023.3310869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Abstract
Automatic sleep staging has been an active field of development. Despite multiple efforts, the area remains a focus of research interest. Indeed, while promising results have reported in past literature, uptake of automatic sleep scoring in the clinical setting remains low. One of the current issues regards the difficulty to generalization performance results beyond the local testing scenario, i.e. across data from different clinics. Issues derived from data-privacy restrictions, that generally apply in the medical domain, pose additional difficulties in the successful development of these methods. We propose the use of several decentralized deep-learning approaches, namely ensemble models and federated learning, for robust inter-database performance generalization and data-privacy preservation in automatic sleep staging scenario. Specifically, we explore four ensemble combination strategies (max-voting, output averaging, size-proportional weighting, and Nelder-Mead) and present a new federated learning algorithm, so-called sub-sampled federated stochastic gradient descent (ssFedSGD). To evaluate generalization capabilities of such approaches, experimental procedures are carried out using a leaving-one-database-out direct-transfer scenario on six independent and heterogeneous public sleep staging databases. The resulting performance is compared with respect to two baseline approaches involving single-database and centralized multiple-database derived models. Our results show that proposed decentralized learning methods outperform baseline local approaches, and provide similar generalization results to centralized database-combined approaches. We conclude that these methods are more preferable choices, as they come with additional advantages concerning improved scalability, flexible design, and data-privacy preservation.
Collapse
|
24
|
van Gorp H, van Gilst MM, Fonseca P, Overeem S, van Sloun RJG. Modeling the Impact of Inter-Rater Disagreement on Sleep Statistics Using Deep Generative Learning. IEEE J Biomed Health Inform 2023; 27:5599-5609. [PMID: 37561616 DOI: 10.1109/jbhi.2023.3304010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2023]
Abstract
Sleep staging is the process by which an overnight polysomnographic measurement is segmented into epochs of 30 seconds, each of which is annotated as belonging to one of five discrete sleep stages. The resulting scoring is graphically depicted as a hypnogram, and several overnight sleep statistics are derived, such as total sleep time and sleep onset latency. Gold standard sleep staging as performed by human technicians is time-consuming, costly, and comes with imperfect inter-scorer agreement, which also results in inter-scorer disagreement about the overnight statistics. Deep learning algorithms have shown promise in automating sleep scoring, but struggle to model inter-scorer disagreement in sleep statistics. To that end, we introduce a novel technique using conditional generative models based on Normalizing Flows that permits the modeling of the inter-rater disagreement of overnight sleep statistics, termed U-Flow. We compare U-Flow to other automatic scoring methods on a hold-out test set of 70 subjects, each scored by six independent scorers. The proposed method achieves similar sleep staging performance in terms of accuracy and Cohen's kappa on the majority-voted hypnograms. At the same time, U-Flow outperforms the other methods in terms of modeling the inter-rater disagreement of overnight sleep statistics. The consequences of inter-rater disagreement about overnight sleep statistics may be great, and the disagreement potentially carries diagnostic and scientifically relevant information about sleep structure. U-Flow is able to model this disagreement efficiently and can support further investigations into the impact inter-rater disagreement has on sleep medicine and basic sleep research.
Collapse
|
25
|
Huang X, Schmelter F, Irshad MT, Piet A, Nisar MA, Sina C, Grzegorzek M. Optimizing sleep staging on multimodal time series: Leveraging borderline synthetic minority oversampling technique and supervised convolutional contrastive learning. Comput Biol Med 2023; 166:107501. [PMID: 37742416 DOI: 10.1016/j.compbiomed.2023.107501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Revised: 08/15/2023] [Accepted: 09/15/2023] [Indexed: 09/26/2023]
Abstract
Sleep is an important research area in nutritional medicine that plays a crucial role in human physical and mental health restoration. It can influence diet, metabolism, and hormone regulation, which can affect overall health and well-being. As an essential tool in the sleep study, the sleep stage classification provides a parsing of sleep architecture and a comprehensive understanding of sleep patterns to identify sleep disorders and facilitate the formulation of targeted sleep interventions. However, the class imbalance issue is typically salient in sleep datasets, which severely affects classification performances. To address this issue and to extract optimal multimodal features of EEG, EOG, and EMG that can improve the accuracy of sleep stage classification, a Borderline Synthetic Minority Oversampling Technique (B-SMOTE)-Based Supervised Convolutional Contrastive Learning (BST-SCCL) is proposed, which can avoid the risk of data mismatch between various sleep knowledge domains (varying health conditions and annotation rules) and strengthening learning characteristics of the N1 stage from the pair-wise segments comparison strategy. The lightweight residual network architecture with a novel truncated cross-entropy loss function is designed to accommodate multimodal time series and boost the training speed and performance stability. The proposed model has been validated on four well-known public sleep datasets (Sleep-EDF-20, Sleep-EDF-78, ISRUC-1, and ISRUC-3) and its superior performance (overall accuracy of 91.31-92.34%, MF1 of 88.21-90.08%, and Cohen's Kappa coefficient k of 0.87-0.89) has further demonstrated its effectiveness. It shows the great potential of contrastive learning for cross-domain knowledge interaction in precision medicine.
Collapse
Affiliation(s)
- Xinyu Huang
- Institute of Medical Informatics, University of Lübeck, Germany.
| | - Franziska Schmelter
- Institute of Nutritional Medicine, University of Lübeck and University Medical Center Schleswig-Holstein, Lübeck, Germany.
| | | | - Artur Piet
- Institute of Medical Informatics, University of Lübeck, Germany.
| | | | - Christian Sina
- Institute of Nutritional Medicine, University of Lübeck and University Medical Center Schleswig-Holstein, Lübeck, Germany; Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering (IMTE), Lübeck, Germany.
| | - Marcin Grzegorzek
- Institute of Medical Informatics, University of Lübeck, Germany; Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering (IMTE), Lübeck, Germany.
| |
Collapse
|
26
|
Li W, Gao J. Automatic sleep staging by a hybrid model based on deep 1D-ResNet-SE and LSTM with single-channel raw EEG signals. PeerJ Comput Sci 2023; 9:e1561. [PMID: 37810362 PMCID: PMC10557479 DOI: 10.7717/peerj-cs.1561] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 08/10/2023] [Indexed: 10/10/2023]
Abstract
Sleep staging is crucial for assessing sleep quality and diagnosing sleep disorders. Recent advances in deep learning methods with electroencephalogram (EEG) signals have shown remarkable success in automatic sleep staging. However, the use of deeper neural networks may lead to the issues of gradient disappearance and explosion, while the non-stationary nature and low signal-to-noise ratio of EEG signals can negatively impact feature representation. To overcome these challenges, we proposed a novel lightweight sequence-to-sequence deep learning model, 1D-ResNet-SE-LSTM, to classify sleep stages into five classes using single-channel raw EEG signals. Our proposed model consists of two main components: a one-dimensional residual convolutional neural network with a squeeze-and-excitation module to extract and reweight features from EEG signals, and a long short-term memory network to capture the transition rules among sleep stages. In addition, we applied the weighted cross-entropy loss function to alleviate the class imbalance problem. We evaluated the performance of our model on two publicly available datasets; Sleep-EDF Expanded consists of 153 overnight PSG recordings collected from 78 healthy subjects and ISRUC-Sleep includes 100 PSG recordings collected from 100 subjects diagnosed with various sleep disorders, and obtained an overall accuracy rate of 86.39% and 81.97%, respectively, along with corresponding macro average F1-scores of 81.95% and 79.94%. Our model outperforms existing sleep staging models in terms of overall performance metrics and per-class F1-scores for several sleep stages, particularly for the N1 stage, where it achieves F1-scores of 59.00% and 55.53%. The kappa coefficient is 0.812 and 0.766 for the Sleep-EDF Expanded and ISRUC-Sleep datasets, respectively, indicating strong agreement with certified sleep experts. We also investigated the effect of different weight coefficient combinations and sequence lengths of EEG epochs used as input to the model on its performance. Furthermore, the ablation study was conducted to evaluate the contribution of each component to the model's performance. The results demonstrate the effectiveness and robustness of the proposed model in classifying sleep stages, and highlights its potential to reduce human clinicians' workload, making sleep assessment and diagnosis more effective. However, the proposed model is subject to several limitations. Firstly, the model is a sequence-to-sequence network, which requires input sequences of EEG epochs. Secondly, the weight coefficients in the loss function could be further optimized to balance the classification performance of each sleep stage. Finally, apart from the channel attention mechanism, incorporating more advanced attention mechanisms could enhance the model's effectiveness.
Collapse
Affiliation(s)
- Weiming Li
- Shanghai Nuanhe Brain Technology Co. Ltd., Shanghai, China
| | - Junhui Gao
- Shanghai Nuanhe Brain Technology Co. Ltd., Shanghai, China
| |
Collapse
|
27
|
Muto V, Berthomier C. Looking for a balance between visual and automatic sleep scoring. NPJ Digit Med 2023; 6:165. [PMID: 37670135 PMCID: PMC10480143 DOI: 10.1038/s41746-023-00915-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 08/22/2023] [Indexed: 09/07/2023] Open
Affiliation(s)
- Vincenzo Muto
- GIGA CRC In Vivo Imaging, Université de Liège, Liège, Belgium.
| | | |
Collapse
|
28
|
Abstract
Automatic polysomnography analysis can be leveraged to shorten scoring times, reduce associated costs, and ultimately improve the overall diagnosis of sleep disorders. Multiple and diverse strategies have been attempted for implementation of this technology at scale in the routine workflow of sleep centers. The field, however, is complex and presents unsolved challenges in a number of areas. Recent developments in computer science and artificial intelligence are nevertheless closing the gap. Technological advances are also opening new pathways for expanding our current understanding of the domain and its analysis.
Collapse
Affiliation(s)
- Diego Alvarez-Estevez
- Center for Information and Communications Technology Research (CITIC), Universidade da Coruña, 15071 A Coruña, Spain.
| |
Collapse
|
29
|
Zahid AN, Jennum P, Mignot E, Sorensen HBD. MSED: A Multi-Modal Sleep Event Detection Model for Clinical Sleep Analysis. IEEE Trans Biomed Eng 2023; 70:2508-2518. [PMID: 37028083 DOI: 10.1109/tbme.2023.3252368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2023]
Abstract
Clinical sleep analysis require manual analysis of sleep patterns for correct diagnosis of sleep disorders. However, several studies have shown significant variability in manual scoring of clinically relevant discrete sleep events, such as arousals, leg movements, and sleep disordered breathing (apneas and hypopneas). We investigated whether an automatic method could be used for event detection and if a model trained on all events (joint model) performed better than corresponding event-specific models (single-event models). We trained a deep neural network event detection model on 1653 individual recordings and tested the optimized model on 1000 separate hold-out recordings. F1 scores for the optimized joint detection model were 0.70, 0.63, and 0.62 for arousals, leg movements, and sleep disordered breathing, respectively, compared to 0.65, 0.61, and 0.60 for the optimized single-event models. Index values computed from detected events correlated positively with manual annotations (r2 = 0.73, r2 = 0.77, r2 = 0.78, respectively). We furthermore quantified model accuracy based on temporal difference metrics, which improved overall by using the joint model compared to single-event models. Our automatic model jointly detects arousals, leg movements and sleep disordered breathing events with high correlation with human annotations. Finally, we benchmark against previous state-of-the-art multi-event detection models and found an overall increase in F1 score with our proposed model despite a 97.5% reduction in model size.
Collapse
|
30
|
Ma YJX, Zschocke J, Glos M, Kluge M, Penzel T, Kantelhardt JW, Bartsch RP. Automatic sleep-stage classification of heart rate and actigraphy data using deep and transfer learning approaches. Comput Biol Med 2023; 163:107193. [PMID: 37421734 DOI: 10.1016/j.compbiomed.2023.107193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 06/01/2023] [Accepted: 06/19/2023] [Indexed: 07/10/2023]
Abstract
Manual sleep-stage scoring based on full-night polysomnography data recorded in a sleep lab has been the gold standard of clinical sleep medicine. This costly and time-consuming approach is unfit for long-term studies as well as assessment of sleep on a population level. With the vast amount of physiological data becoming available from wrist-worn devices, deep learning techniques provide an opportunity for fast and reliable automatic sleep-stage classification tasks. However, training a deep neural network requires large annotated sleep databases, which are not available for long-term epidemiological studies. In this paper, we introduce an end-to-end temporal convolutional neural network able to automatically score sleep stages from raw heartbeat RR interval (RRI) and wrist actigraphy data. Moreover, a transfer learning approach enables the training of the network on a large public database (Sleep Heart Health Study, SHHS) and its subsequent application to a much smaller database recorded by a wristband device. The transfer learning significantly shortens training time and improves sleep-scoring accuracy from 68.9% to 73.8% and inter-rater reliability (Cohen's kappa) from 0.51 to 0.59. We also found that for the SHHS database, automatic sleep-scoring accuracy using deep learning shows a logarithmic relationship with the training size. Although deep learning approaches for automatic sleep scoring are not yet comparable to the inter-rater reliability among sleep technicians, performance is expected to significantly improve in the near future when more large public databases become available. We anticipate those deep learning techniques, when combined with our transfer learning approach, will leverage automatic sleep scoring of physiological data from wearable devices and enable the investigation of sleep in large cohort studies.
Collapse
Affiliation(s)
- Yaopeng J X Ma
- Department of Physics, Bar-Ilan University, Ramat Gan, Israel.
| | - Johannes Zschocke
- Institute of Medical Epidemiology, Biometrics and Informatics (IMEBI), Interdisciplinary Center for Health Sciences, Martin-Luther-University Halle-Wittenberg, Halle (Saale), Germany; Institute of Physics, Martin-Luther University Halle-Wittenberg, Halle (Saale), Germany
| | - Martin Glos
- Interdisciplinary Sleep Medicine Center, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Maria Kluge
- Interdisciplinary Sleep Medicine Center, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Thomas Penzel
- Interdisciplinary Sleep Medicine Center, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Jan W Kantelhardt
- Institute of Physics, Martin-Luther University Halle-Wittenberg, Halle (Saale), Germany
| | - Ronny P Bartsch
- Department of Physics, Bar-Ilan University, Ramat Gan, Israel.
| |
Collapse
|
31
|
Lepage KQ, Jain S, Kvavilashvili A, Witcher M, Vijayan S. Unsupervised Multitaper Spectral Method for Identifying REM Sleep in Intracranial EEG Recordings Lacking EOG/EMG Data. Bioengineering (Basel) 2023; 10:1009. [PMID: 37760111 PMCID: PMC10525760 DOI: 10.3390/bioengineering10091009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 08/10/2023] [Accepted: 08/15/2023] [Indexed: 09/29/2023] Open
Abstract
A large number of human intracranial EEG (iEEG) recordings have been collected for clinical purposes, in institutions all over the world, but the vast majority of these are unaccompanied by EOG and EMG recordings which are required to separate Wake episodes from REM sleep using accepted methods. In order to make full use of this extremely valuable data, an accurate method of classifying sleep from iEEG recordings alone is required. Existing methods of sleep scoring using only iEEG recordings accurately classify all stages of sleep, with the exception that wake (W) and rapid-eye movement (REM) sleep are not well distinguished. A novel multitaper (Wake vs. REM) alpha-rhythm classifier is developed by generalizing K-means clustering for use with multitaper spectral eigencoefficients. The performance of this unsupervised method is assessed on eight subjects exhibiting normal sleep architecture in a hold-out analysis and is compared against a classical power detector. The proposed multitaper classifier correctly identifies 36±6 min of REM in one night of recorded sleep, while incorrectly labeling less than 10% of all labeled 30 s epochs for all but one subject (human rater reliability is estimated to be near 80%), and outperforms the equivalent statistical-power classical test. Hold-out analysis indicates that when using one night's worth of data, an accurate generalization of the method on new data is likely. For the purpose of studying sleep, the introduced multitaper alpha-rhythm classifier further paves the way to making available a large quantity of otherwise unusable IEEG data.
Collapse
Affiliation(s)
- Kyle Q. Lepage
- School of Neuroscience, Sandy Hall, Virginia Tech, 210 Drillfield Drive, Blacksburg, VA 24060, USA; (A.K.); (S.V.)
| | - Sparsh Jain
- Department of Biomedical Engineering and Mechanics, Virginia Tech, 325 Stanger St., Blacksburg, VA 24061, USA;
| | - Andrew Kvavilashvili
- School of Neuroscience, Sandy Hall, Virginia Tech, 210 Drillfield Drive, Blacksburg, VA 24060, USA; (A.K.); (S.V.)
| | - Mark Witcher
- Section of Neurosurgery, Carilion Clinic, Carilion Roanoke Memorial Hospital, 1906 Belleview Ave SE, Roanoke, VA 24014, USA;
| | - Sujith Vijayan
- School of Neuroscience, Sandy Hall, Virginia Tech, 210 Drillfield Drive, Blacksburg, VA 24060, USA; (A.K.); (S.V.)
| |
Collapse
|
32
|
Skorucak J, Bölsterli BK, Storz S, Leach S, Schmitt B, Ramantani G, Huber R. Automated analysis of a large-scale paediatric dataset illustrates the interdependent relationship between epilepsy and sleep. Sci Rep 2023; 13:12882. [PMID: 37553387 PMCID: PMC10409812 DOI: 10.1038/s41598-023-39984-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 08/03/2023] [Indexed: 08/10/2023] Open
Abstract
Slow waves are an electrophysiological characteristic of non-rapid eye movement sleep and a marker of the restorative function of sleep. In certain pathological conditions, such as different types of epilepsy, slow-wave sleep is affected by epileptiform discharges forming so-called "spike-waves". Previous evidence shows that the overnight change in slope of slow waves during sleep is impaired under these conditions. However, these past studies were performed in a small number of patients, considering only short segments of the recording night. Here, we screened a clinical data set of 39'179 pediatric EEG recordings acquired in the past 25 years (1994-2019) at the University Children's Hospital Zurich and identified 413 recordings of interest. We applied an automated approach based on machine learning to investigate the relationship between sleep and epileptic spikes in this large-scale data set. Our findings show that the overnight change in the slope of slow waves was correlated with the spike-wave index, indicating that the impairment of the net reduction in synaptic strength during sleep is spike dependent.
Collapse
Affiliation(s)
- Jelena Skorucak
- Child Development Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Bigna K Bölsterli
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Department of Pediatric Neurology, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Department of Pediatric Neurology, Children's Hospital of Eastern Switzerland, St. Gallen, Switzerland
| | - Sarah Storz
- Child Development Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Department of Pediatric Neurology, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Sven Leach
- Child Development Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Bernhard Schmitt
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Department of Pediatric Neurology, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Georgia Ramantani
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
- Department of Pediatric Neurology, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Reto Huber
- Child Development Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland.
- Children's Research Center, University Children's Hospital Zurich, University of Zurich, Zurich, Switzerland.
- Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric Hospital, University of Zurich, Zurich, Switzerland.
| |
Collapse
|
33
|
Gaiduk M, Serrano Alarcón Á, Seepold R, Martínez Madrid N. Current status and prospects of automatic sleep stages scoring: Review. Biomed Eng Lett 2023; 13:247-272. [PMID: 37519865 PMCID: PMC10382458 DOI: 10.1007/s13534-023-00299-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 06/07/2023] [Accepted: 06/18/2023] [Indexed: 08/01/2023] Open
Abstract
The scoring of sleep stages is one of the essential tasks in sleep analysis. Since a manual procedure requires considerable human and financial resources, and incorporates some subjectivity, an automated approach could result in several advantages. There have been many developments in this area, and in order to provide a comprehensive overview, it is essential to review relevant recent works and summarise the characteristics of the approaches, which is the main aim of this article. To achieve it, we examined articles published between 2018 and 2022 that dealt with the automated scoring of sleep stages. In the final selection for in-depth analysis, 125 articles were included after reviewing a total of 515 publications. The results revealed that automatic scoring demonstrates good quality (with Cohen's kappa up to over 0.80 and accuracy up to over 90%) in analysing EEG/EEG + EOG + EMG signals. At the same time, it should be noted that there has been no breakthrough in the quality of results using these signals in recent years. Systems involving other signals that could potentially be acquired more conveniently for the user (e.g. respiratory, cardiac or movement signals) remain more challenging in the implementation with a high level of reliability but have considerable innovation capability. In general, automatic sleep stage scoring has excellent potential to assist medical professionals while providing an objective assessment.
Collapse
Affiliation(s)
- Maksym Gaiduk
- HTWG Konstanz – University of Applied Sciences, Alfred-Wachtel-Str.8, 78462 Konstanz, Germany
| | | | - Ralf Seepold
- HTWG Konstanz – University of Applied Sciences, Alfred-Wachtel-Str.8, 78462 Konstanz, Germany
| | | |
Collapse
|
34
|
West LC, Summers M, Tang S, Hirt L, Halpern CH, Maroni D, Das R, Gliske SV, Abosch A, Kushida CA, Thompson JA. Evaluation of consensus sleep stage scoring of dysregulated sleep in Parkinson's disease. Sleep Med 2023; 107:236-242. [PMID: 37257366 PMCID: PMC10344673 DOI: 10.1016/j.sleep.2023.04.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Revised: 04/24/2023] [Accepted: 04/30/2023] [Indexed: 06/02/2023]
Abstract
OBJECTIVE Sleep dysregulation in Parkinson's disease (PD) has been hypothesized to occur, in part, from dysfunction in the basal ganglia-cortical circuit. Assessment of this relationship requires accurate sleep stage determination, a known challenge in this clinical population. Our objective was to optimize the consensus on the sleep staging process and reduce interrater variability in a cohort of advanced PD subjects. METHODS Fifteen PD subjects were enrolled from three sites in a clinical trial that involved recordings from subthalamic nucleus (STN) deep brain stimulation (DBS) leads (NCT04620551). Video polysomnography (vPSG) data for a total of 45 nights were analyzed. Four experienced scorers independently scored data on initial review. Epochs with less than 75% consensus were flagged for secondary review. In secondary review of discordant epochs, two of the original scorers re-assessed epochs, from which the final consensus stage was derived. RESULTS Sleep stage classification agreement averaged 83.10% across all sleep stages on initial scoring (IS), and on secondary consensus scoring (CS) review, agreement reached 96.58%. Greatest disagreement was noted in determination of awake epochs (33.6% of discordant epochs) and non-rapid-eye-movement stage 2 (N2) epochs (31.8% of discordant epochs). Scoring discrepancy was resolved with direct measurement of cortical frequency and amplitudes, physiologic context of the epoch, and video review. CONCLUSION Our method of multi-level initial and then secondary consensus review scoring resulted in consensus scoring agreement superior to conventional standards. This work features a custom-engineered vPSG software and review platform for integration of consensus sleep stage scoring in a multi-site clinical trial.
Collapse
Affiliation(s)
- Leslie C West
- University of California San Francisco, Department of Neurology, USA.
| | - Michael Summers
- University of Nebraska Medical Center, Nebraska Medicine Sleep Center, Internal Medicine, Division of Pulmonary, Critical Care & Sleep Medicine, USA
| | - Siqun Tang
- Stanford University, Sleep Medicine Division, Department of Psychiatry and Behavioral Science, USA
| | - Lisa Hirt
- University of Colorado School of Medicine, Department of Neurosurgery, USA
| | - Casey H Halpern
- University of Pennsylvania School of Medicine, Department of Neurosurgery, USA; Department of Surgery, Corporal Michael J. Crescenz Veterans Affairs Medical Center, USA
| | - Dulce Maroni
- University of Nebraska Medical Center, Department of Neurosurgery, USA
| | - Rig Das
- University of Nebraska Medical Center, Department of Neurosurgery, USA
| | - Stephen V Gliske
- University of Nebraska Medical Center, Department of Neurosurgery, USA
| | - Aviva Abosch
- University of Nebraska Medical Center, Department of Neurosurgery, USA
| | - Clete A Kushida
- Stanford University, Sleep Medicine Division, Department of Psychiatry and Behavioral Science, USA
| | - John A Thompson
- University of Colorado School of Medicine, Department of Neurosurgery, USA; University of Colorado School of Medicine, Department of Neurology, USA
| |
Collapse
|
35
|
Lambert I, Peter-Derex L. Spotlight on Sleep Stage Classification Based on EEG. Nat Sci Sleep 2023; 15:479-490. [PMID: 37405208 PMCID: PMC10317531 DOI: 10.2147/nss.s401270] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 06/21/2023] [Indexed: 07/06/2023] Open
Abstract
The recommendations for identifying sleep stages based on the interpretation of electrophysiological signals (electroencephalography [EEG], electro-oculography [EOG], and electromyography [EMG]), derived from the Rechtschaffen and Kales manual, were published in 2007 at the initiative of the American Academy of Sleep Medicine, and regularly updated over years. They offer an important tool to assess objective markers in different types of sleep/wake subjective complaints. With the aims and advantages of simplicity, reproducibility and standardization of practices in research and, most of all, in sleep medicine, they have overall changed little in the way they describe sleep. However, our knowledge on sleep/wake physiology and sleep disorders has evolved since then. High-density electroencephalography and intracranial electroencephalography studies have highlighted local regulation of sleep mechanisms, with spatio-temporal heterogeneity in vigilance states. Progress in the understanding of sleep disorders has allowed the identification of electrophysiological biomarkers better correlated with clinical symptoms and outcomes than standard sleep parameters. Finally, the huge development of sleep medicine, with a demand for explorations far exceeding the supply, has led to the development of alternative studies, which can be carried out at home, based on a smaller number of electrophysiological signals and on their automatic analysis. In this perspective article, we aim to examine how our description of sleep has been constructed, has evolved, and may still be reshaped in the light of advances in knowledge of sleep physiology and the development of technical recording and analysis tools. After presenting the strengths and limitations of the classification of sleep stages, we propose to challenge the "EEG-EOG-EMG" paradigm by discussing the physiological signals required for sleep stages identification, provide an overview of new tools and automatic analysis methods and propose avenues for the development of new approaches to describe and understand sleep/wake states.
Collapse
Affiliation(s)
- Isabelle Lambert
- APHM, Timone Hospital, Sleep Unit, Epileptology and Cerebral Rhythmology, Marseille, France
- Aix Marseille University, INSERM, Institut de Neuroscience des Systemes, Marseille, France
| | - Laure Peter-Derex
- Center for Sleep Medicine and Respiratory Diseases, Croix-Rousse Hospital, Hospices Civils de Lyon, Lyon 1 University, Lyon, France
- Lyon Neuroscience Research Center, PAM Team, INSERM U1028, CNRS UMR 5292, Lyon, France
| |
Collapse
|
36
|
Zhang D, Sun J, She Y, Cui Y, Zeng X, Lu L, Tang C, Xu N, Chen B, Qin W. A two-branch trade-off neural network for balanced scoring sleep stages on multiple cohorts. Front Neurosci 2023; 17:1176551. [PMID: 37424992 PMCID: PMC10326279 DOI: 10.3389/fnins.2023.1176551] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Accepted: 05/16/2023] [Indexed: 07/11/2023] Open
Abstract
Introduction Automatic sleep staging is a classification process with severe class imbalance and suffers from instability of scoring stage N1. Decreased accuracy in classifying stage N1 significantly impacts the staging of individuals with sleep disorders. We aim to achieve automatic sleep staging with expert-level performance in both N1 stage and overall scoring. Methods A neural network model combines an attention-based convolutional neural network and a classifier with two branches is developed. A transitive training strategy is employed to balance universal feature learning and contextual referencing. Parameter optimization and benchmark comparisons are conducted using a large-scale dataset, followed by evaluation on seven datasets in five cohorts. Results The proposed model achieves an accuracy of 88.16%, Cohen's kappa of 0.836, and MF1 score of 0.818 on the SHHS1 test set, also with comparable performance to human scorers in scoring stage N1. Incorporating multiple cohort data improves its performance. Notably, the model maintains high performance when applied to unseen datasets and patients with neurological or psychiatric disorders. Discussion The proposed algorithm demonstrates strong performance and generalizablility, and its direct transferability is noteworthy among similar studies on automated sleep staging. It is publicly available, which is conducive to expanding access to sleep-related analysis, especially those associated with neurological or psychiatric disorders.
Collapse
Affiliation(s)
- Di Zhang
- Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an, China
- Intelligent Non-invasive Neuromodulation Technology and Transformation Joint Laboratory, Xidian University, Xi'an, China
| | - Jinbo Sun
- Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an, China
- Intelligent Non-invasive Neuromodulation Technology and Transformation Joint Laboratory, Xidian University, Xi'an, China
| | - Yichong She
- Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an, China
- Intelligent Non-invasive Neuromodulation Technology and Transformation Joint Laboratory, Xidian University, Xi'an, China
| | - Yapeng Cui
- Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an, China
- Intelligent Non-invasive Neuromodulation Technology and Transformation Joint Laboratory, Xidian University, Xi'an, China
| | - Xiao Zeng
- Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an, China
- Intelligent Non-invasive Neuromodulation Technology and Transformation Joint Laboratory, Xidian University, Xi'an, China
| | - Liming Lu
- South China Research Center for Acupuncture and Moxibustion, Medical College of Acu-Moxi and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Chunzhi Tang
- South China Research Center for Acupuncture and Moxibustion, Medical College of Acu-Moxi and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Nenggui Xu
- South China Research Center for Acupuncture and Moxibustion, Medical College of Acu-Moxi and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Badong Chen
- College of Artificial Intelligence, Xian Jiaotong University, Xian, Shaanxi, China
| | - Wei Qin
- Engineering Research Center of Molecular and Neuro Imaging of the Ministry of Education, School of Life Science and Technology, Xidian University, Xi’an, China
- Intelligent Non-invasive Neuromodulation Technology and Transformation Joint Laboratory, Xidian University, Xi'an, China
| |
Collapse
|
37
|
Fraigne JJ, Wang J, Lee H, Luke R, Pintwala SK, Peever JH. A novel machine learning system for identifying sleep-wake states in mice. Sleep 2023; 46:zsad101. [PMID: 37021715 PMCID: PMC10262194 DOI: 10.1093/sleep/zsad101] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2022] [Revised: 03/23/2023] [Indexed: 04/07/2023] Open
Abstract
Research into sleep-wake behaviors relies on scoring sleep states, normally done by manual inspection of electroencephalogram (EEG) and electromyogram (EMG) recordings. This is a highly time-consuming process prone to inter-rater variability. When studying relationships between sleep and motor function, analyzing arousal states under a four-state system of active wake (AW), quiet wake (QW), nonrapid-eye-movement (NREM) sleep, and rapid-eye-movement (REM) sleep provides greater precision in behavioral analysis but is a more complex model for classification than the traditional three-state identification (wake, NREM, and REM sleep) usually used in rodent models. Characteristic features between sleep-wake states provide potential for the use of machine learning to automate classification. Here, we devised SleepEns, which uses a novel ensemble architecture, the time-series ensemble. SleepEns achieved 90% accuracy to the source expert, which was statistically similar to the performance of two other human experts. Considering the capacity for classification disagreements that are still physiologically reasonable, SleepEns had an acceptable performance of 99% accuracy, as determined blindly by the source expert. Classifications given by SleepEns also maintained similar sleep-wake characteristics compared to expert classifications, some of which were essential for sleep-wake identification. Hence, our approach achieves results comparable to human ability in a fraction of the time. This new machine-learning ensemble will significantly impact the ability of sleep researcher to detect and study sleep-wake behaviors in mice and potentially in humans.
Collapse
Affiliation(s)
- Jimmy J Fraigne
- Department of Cell & Systems Biology, University of Toronto, Toronto, ON, Canada
| | - Jeffrey Wang
- Department of Cell & Systems Biology, University of Toronto, Toronto, ON, Canada
| | - Hanhee Lee
- Department of Cell & Systems Biology, University of Toronto, Toronto, ON, Canada
| | - Russell Luke
- Department of Cell & Systems Biology, University of Toronto, Toronto, ON, Canada
| | - Sara K Pintwala
- Department of Cell & Systems Biology, University of Toronto, Toronto, ON, Canada
| | - John H Peever
- Department of Cell & Systems Biology, University of Toronto, Toronto, ON, Canada
- Department of Physiology, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
38
|
Kwon S, Kim HS, Kwon K, Kim H, Kim YS, Lee SH, Kwon YT, Jeong JW, Trotti LM, Duarte A, Yeo WH. At-home wireless sleep monitoring patches for the clinical assessment of sleep quality and sleep apnea. SCIENCE ADVANCES 2023; 9:eadg9671. [PMID: 37224243 DOI: 10.1126/sciadv.adg9671] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 04/17/2023] [Indexed: 05/26/2023]
Abstract
Although many people suffer from sleep disorders, most are undiagnosed, leading to impairments in health. The existing polysomnography method is not easily accessible; it's costly, burdensome to patients, and requires specialized facilities and personnel. Here, we report an at-home portable system that includes wireless sleep sensors and wearable electronics with embedded machine learning. We also show its application for assessing sleep quality and detecting sleep apnea with multiple patients. Unlike the conventional system using numerous bulky sensors, the soft, all-integrated wearable platform offers natural sleep wherever the user prefers. In a clinical study, the face-mounted patches that detect brain, eye, and muscle signals show comparable performance with polysomnography. When comparing healthy controls to sleep apnea patients, the wearable system can detect obstructive sleep apnea with an accuracy of 88.5%. Furthermore, deep learning offers automated sleep scoring, demonstrating portability, and point-of-care usability. At-home wearable electronics could ensure a promising future supporting portable sleep monitoring and home healthcare.
Collapse
Affiliation(s)
- Shinjae Kwon
- IEN Center for Human-Centric Interfaces and Engineering at the Institute for Electronics and Nanotechnology, Georgia Institute of Technology, Atlanta, GA 30332, USA
- George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - Hyeon Seok Kim
- IEN Center for Human-Centric Interfaces and Engineering at the Institute for Electronics and Nanotechnology, Georgia Institute of Technology, Atlanta, GA 30332, USA
- George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - Kangkyu Kwon
- IEN Center for Human-Centric Interfaces and Engineering at the Institute for Electronics and Nanotechnology, Georgia Institute of Technology, Atlanta, GA 30332, USA
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - Hodam Kim
- IEN Center for Human-Centric Interfaces and Engineering at the Institute for Electronics and Nanotechnology, Georgia Institute of Technology, Atlanta, GA 30332, USA
- George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - Yun Soung Kim
- Department of Radiology, Icahn School of Medicine at Mount Sinai, BioMedical Engineering and Imaging Institute, New York, NY 10029, USA
| | - Sung Hoon Lee
- IEN Center for Human-Centric Interfaces and Engineering at the Institute for Electronics and Nanotechnology, Georgia Institute of Technology, Atlanta, GA 30332, USA
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - Young-Tae Kwon
- Metal Powder Department, Korea Institute of Materials Science, Changwon 51508, Republic of Korea
| | - Jae-Woong Jeong
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea
| | - Lynn Marie Trotti
- Emory Sleep Center and Department of Neurology, Emory University School of Medicine, Atlanta, GA 30329, USA
| | - Audrey Duarte
- Department of Psychology, University of Texas at Austin, Austin, TX 78712, USA
| | - Woon-Hong Yeo
- IEN Center for Human-Centric Interfaces and Engineering at the Institute for Electronics and Nanotechnology, Georgia Institute of Technology, Atlanta, GA 30332, USA
- George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
- Wallace H. Coulter Department of Biomedical Engineering, Georgia Tech and Emory University, Atlanta, GA 30332, USA
- Parker H. Petit Institute for Bioengineering and Biosciences, Institute for Materials, Neural Engineering Center, Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA 30332, USA
| |
Collapse
|
39
|
Huang X, Shirahama K, Irshad MT, Nisar MA, Piet A, Grzegorzek M. Sleep Stage Classification in Children Using Self-Attention and Gaussian Noise Data Augmentation. SENSORS (BASEL, SWITZERLAND) 2023; 23:3446. [PMID: 37050506 PMCID: PMC10098613 DOI: 10.3390/s23073446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 03/20/2023] [Accepted: 03/22/2023] [Indexed: 06/19/2023]
Abstract
The analysis of sleep stages for children plays an important role in early diagnosis and treatment. This paper introduces our sleep stage classification method addressing the following two challenges: the first is the data imbalance problem, i.e., the highly skewed class distribution with underrepresented minority classes. For this, a Gaussian Noise Data Augmentation (GNDA) algorithm was applied to polysomnography recordings to seek the balance of data sizes for different sleep stages. The second challenge is the difficulty in identifying a minority class of sleep stages, given their short sleep duration and similarities to other stages in terms of EEG characteristics. To overcome this, we developed a DeConvolution- and Self-Attention-based Model (DCSAM) which can inverse the feature map of a hidden layer to the input space to extract local features and extract the correlations between all possible pairs of features to distinguish sleep stages. The results on our dataset show that DCSAM based on GNDA obtains an accuracy of 90.26% and a macro F1-score of 86.51% which are higher than those of our previous method. We also tested DCSAM on a well-known public dataset-Sleep-EDFX-to prove whether it is applicable to sleep data from adults. It achieves a comparable performance to state-of-the-art methods, especially accuracies of 91.77%, 92.54%, 94.73%, and 95.30% for six-stage, five-stage, four-stage, and three-stage classification, respectively. These results imply that our DCSAM based on GNDA has a great potential to offer performance improvements in various medical domains by considering the data imbalance problems and correlations among features in time series data.
Collapse
Affiliation(s)
- Xinyu Huang
- Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Kimiaki Shirahama
- Department of Informatics, Kindai University, 3-4-1 Kowakae, Higashiosaka City 577-8502, Osaka, Japan
| | - Muhammad Tausif Irshad
- Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Department of IT, University of the Punjab, Lahore 54000, Pakistan
| | | | - Artur Piet
- Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Marcin Grzegorzek
- Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Department of Knowledge Engineering, University of Economics, Bogucicka 3, 40287 Katowice, Poland
| |
Collapse
|
40
|
Haghayegh S, Hu K, Stone K, Redline S, Schernhammer E. Automated Sleep Stages Classification Using Convolutional Neural Network From Raw and Time-Frequency Electroencephalogram Signals: Systematic Evaluation Study. J Med Internet Res 2023; 25:e40211. [PMID: 36763454 PMCID: PMC9960035 DOI: 10.2196/40211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 09/09/2022] [Accepted: 01/09/2023] [Indexed: 02/11/2023] Open
Abstract
BACKGROUND Most existing automated sleep staging methods rely on multimodal data, and scoring a specific epoch requires not only the current epoch but also a sequence of consecutive epochs that precede and follow the epoch. OBJECTIVE We proposed and tested a convolutional neural network called SleepInceptionNet, which allows sleep classification of a single epoch using a single-channel electroencephalogram (EEG). METHODS SleepInceptionNet is based on our systematic evaluation of the effects of different EEG preprocessing methods, EEG channels, and convolutional neural networks on automatic sleep staging performance. The evaluation was performed using polysomnography data of 883 participants (937,975 thirty-second epochs). Raw data of individual EEG channels (ie, frontal, central, and occipital) and 3 specific transformations of the data, including power spectral density, continuous wavelet transform, and short-time Fourier transform, were used separately as the inputs of the convolutional neural network models. To classify sleep stages, 7 sequential deep neural networks were tested for the 1D data (ie, raw EEG and power spectral density), and 16 image classifier convolutional neural networks were tested for the 2D data (ie, continuous wavelet transform and short-time Fourier transform time-frequency images). RESULTS The best model, SleepInceptionNet, which uses time-frequency images developed by the continuous wavelet transform method from central single-channel EEG data as input to the InceptionV3 image classifier algorithm, achieved a Cohen κ agreement of 0.705 (SD 0.077) in reference to the gold standard polysomnography. CONCLUSIONS SleepInceptionNet may allow real-time automated sleep staging in free-living conditions using a single-channel EEG, which may be useful for on-demand intervention or treatment during specific sleep stages.
Collapse
Affiliation(s)
- Shahab Haghayegh
- Harvard Medical School, Boston, MA, United States
- Brigham and Women's Hospital, Boston, MA, United States
| | - Kun Hu
- Harvard Medical School, Boston, MA, United States
- Brigham and Women's Hospital, Boston, MA, United States
| | - Katie Stone
- California Pacific Medical Center Research Institute, San Francisco, CA, United States
| | - Susan Redline
- Harvard Medical School, Boston, MA, United States
- Brigham and Women's Hospital, Boston, MA, United States
| | - Eva Schernhammer
- Harvard Medical School, Boston, MA, United States
- Brigham and Women's Hospital, Boston, MA, United States
- Medical University of Vienna, Vienna, Austria
| |
Collapse
|
41
|
Bakker JP, Ross M, Cerny A, Vasko R, Shaw E, Kuna S, Magalang UJ, Punjabi NM, Anderer P. Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring. Sleep 2023; 46:6628222. [PMID: 35780449 PMCID: PMC9905781 DOI: 10.1093/sleep/zsac154] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 06/22/2022] [Indexed: 11/12/2022] Open
Abstract
STUDY OBJECTIVES To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers. METHODS We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6-12 scorers, to compare sleep stage probabilities (hypnodensity; i.e. the probability of each sleep stage being assigned to a given epoch) as the primary output, as well as a single sleep stage per epoch assigned by hierarchical majority rule. RESULTS The percentage of epochs with 100% agreement across scorers was 46 ± 9%, 38 ± 10% and 32 ± 9% for the datasets with 6, 9, and 12 scorers, respectively. The mean intra-class correlation coefficient between sleep stage probabilities from auto- and manual-scoring was 0.91, representing excellent reliability. Within each dataset, agreement between auto-scoring and consensus manual-scoring was significantly higher than agreement between manual-scoring and consensus manual-scoring (0.78 vs. 0.69; 0.74 vs. 0.67; and 0.75 vs. 0.67; all p < 0.01). CONCLUSIONS Analysis of scoring performed by multiple scorers reveals that sleep stage ambiguity is the rule rather than the exception. Probabilities of the sleep stages determined by artificial intelligence auto-scoring provide an excellent estimate of this ambiguity. Compared to consensus manual-scoring, sleep staging derived from auto-scoring is for each individual PSG noninferior to manual-scoring meaning that auto-scoring output is ready for interpretation without the need for manual adjustment.
Collapse
Affiliation(s)
| | - Marco Ross
- Philips Sleep and Respiratory Care, Vienna, Austria
| | | | - Ray Vasko
- Philips Sleep and Respiratory Care, Pittsburgh, PA,USA
| | - Edmund Shaw
- Philips Sleep and Respiratory Care, Pittsburgh, PA,USA
| | - Samuel Kuna
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA,USA.,Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA,USA
| | - Ulysses J Magalang
- Division of Pulmonary, Critical Care, and Sleep Medicine, Ohio State University Wexner Medical Center, Columbus, OH, USA
| | - Naresh M Punjabi
- Division of Pulmonary, Critical Care, and Sleep Medicine, University of Miami, Miami FL, USA
| | | |
Collapse
|
42
|
Ganglberger W, Krishnamurthy PV, Quadri SA, Tesh RA, Bucklin AA, Adra N, Da Silva Cardoso M, Leone MJ, Hemmige A, Rajan S, Panneerselvam E, Paixao L, Higgins J, Ayub MA, Shao YP, Coughlin B, Sun H, Ye EM, Cash SS, Thompson BT, Akeju O, Kuller D, Thomas RJ, Westover MB. Sleep staging in the ICU with heart rate variability and breathing signals. An exploratory cross-sectional study using deep neural networks. FRONTIERS IN NETWORK PHYSIOLOGY 2023; 3:1120390. [PMID: 36926545 PMCID: PMC10013021 DOI: 10.3389/fnetp.2023.1120390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 02/13/2023] [Indexed: 03/02/2023]
Abstract
Introduction: To measure sleep in the intensive care unit (ICU), full polysomnography is impractical, while activity monitoring and subjective assessments are severely confounded. However, sleep is an intensely networked state, and reflected in numerous signals. Here, we explore the feasibility of estimating conventional sleep indices in the ICU with heart rate variability (HRV) and respiration signals using artificial intelligence methods Methods: We used deep learning models to stage sleep with HRV (through electrocardiogram) and respiratory effort (through a wearable belt) signals in critically ill adult patients admitted to surgical and medical ICUs, and in age and sex-matched sleep laboratory patients Results: We studied 102 adult patients in the ICU across multiple days and nights, and 220 patients in a clinical sleep laboratory. We found that sleep stages predicted by HRV- and breathing-based models showed agreement in 60% of the ICU data and in 81% of the sleep laboratory data. In the ICU, deep NREM (N2 + N3) proportion of total sleep duration was reduced (ICU 39%, sleep laboratory 57%, p < 0.01), REM proportion showed heavy-tailed distribution, and the number of wake transitions per hour of sleep (median 3.6) was comparable to sleep laboratory patients with sleep-disordered breathing (median 3.9). Sleep in the ICU was also fragmented, with 38% of sleep occurring during daytime hours. Finally, patients in the ICU showed faster and less variable breathing patterns compared to sleep laboratory patients Conclusion: The cardiovascular and respiratory networks encode sleep state information, which, together with artificial intelligence methods, can be utilized to measure sleep state in the ICU.
Collapse
Affiliation(s)
- Wolfgang Ganglberger
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States.,Sleep and Health Zurich, University of Zurich, Zurich, Switzerland.,Henry and Allison McCance Center for Brain Health, Massachusetts General Hospital, Boston, MA, United States
| | - Parimala Velpula Krishnamurthy
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Syed A Quadri
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Ryan A Tesh
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Abigail A Bucklin
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Noor Adra
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Madalena Da Silva Cardoso
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Michael J Leone
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Aashritha Hemmige
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Subapriya Rajan
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Ezhil Panneerselvam
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Luis Paixao
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Jasmine Higgins
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Muhammad Abubakar Ayub
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Yu-Ping Shao
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Brian Coughlin
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States
| | - Haoqi Sun
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States.,Harvard Medical School, Boston, MA, United States.,Henry and Allison McCance Center for Brain Health, Massachusetts General Hospital, Boston, MA, United States
| | - Elissa M Ye
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States
| | - Sydney S Cash
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States.,Harvard Medical School, Boston, MA, United States
| | - B Taylor Thompson
- Department of Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Oluwaseun Akeju
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA, United States.,Henry and Allison McCance Center for Brain Health, Massachusetts General Hospital, Boston, MA, United States
| | | | - Robert J Thomas
- Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States.,Harvard Medical School, Boston, MA, United States.,Beth Israel Deaconess Medical Center, Department of Medicine, Division of Pulmonary, Critical Care and Sleep, Boston, MA, United States
| | - M Brandon Westover
- Department of Neurology, Massachusetts General Hospital, MGH, Boston, MA, United States.,Clinical Data Animation Center (CDAC), Massachusetts General Hospital, Boston, MA, United States.,Harvard Medical School, Boston, MA, United States.,Henry and Allison McCance Center for Brain Health, Massachusetts General Hospital, Boston, MA, United States
| |
Collapse
|
43
|
Stanley N. The Future of Sleep Staging, Revisited. Nat Sci Sleep 2023; 15:313-322. [PMID: 37159812 PMCID: PMC10163901 DOI: 10.2147/nss.s405663] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 04/19/2023] [Indexed: 05/11/2023] Open
Abstract
In 1996, I published a paper entitled "The Future of Sleep Staging". At this time, paper and ink records were the standard way of recording sleep records. Computerised systems had only recently become commercially available. The original article was a response to those initial computer-based systems, pointing out the potential limitations of the systems. Now, digital sleep recording is ubiquitous and software and hardware capabilities have improved immeasurably. However, I will argue that despite 50 years of progress, there has not been an increase in the accuracy of sleep staging. I will propose that this is due to the limitations of the task that we have set the automatic analysis methods.
Collapse
Affiliation(s)
- Neil Stanley
- Independent Sleep Expert, Farnborough, Hampshire, UK
- Correspondence: Neil Stanley, Email
| |
Collapse
|
44
|
Somaskandhan P, Leppänen T, Terrill PI, Sigurdardottir S, Arnardottir ES, Ólafsdóttir KA, Serwatko M, Sigurðardóttir SÞ, Clausen M, Töyräs J, Korkalainen H. Deep learning-based algorithm accurately classifies sleep stages in preadolescent children with sleep-disordered breathing symptoms and age-matched controls. Front Neurol 2023; 14:1162998. [PMID: 37122306 PMCID: PMC10140398 DOI: 10.3389/fneur.2023.1162998] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Accepted: 03/23/2023] [Indexed: 05/02/2023] Open
Abstract
Introduction Visual sleep scoring has several shortcomings, including inter-scorer inconsistency, which may adversely affect diagnostic decision-making. Although automatic sleep staging in adults has been extensively studied, it is uncertain whether such sophisticated algorithms generalize well to different pediatric age groups due to distinctive EEG characteristics. The preadolescent age group (10-13-year-olds) is relatively understudied, and thus, we aimed to develop an automatic deep learning-based sleep stage classifier specifically targeting this cohort. Methods A dataset (n = 115) containing polysomnographic recordings of Icelandic preadolescent children with sleep-disordered breathing (SDB) symptoms, and age and sex-matched controls was utilized. We developed a combined convolutional and long short-term memory neural network architecture relying on electroencephalography (F4-M1), electrooculography (E1-M2), and chin electromyography signals. Performance relative to human scoring was further evaluated by analyzing intra- and inter-rater agreements in a subset (n = 10) of data with repeat scoring from two manual scorers. Results The deep learning-based model achieved an overall cross-validated accuracy of 84.1% (Cohen's kappa κ = 0.78). There was no meaningful performance difference between SDB-symptomatic (n = 53) and control subgroups (n = 52) [83.9% (κ = 0.78) vs. 84.2% (κ = 0.78)]. The inter-rater reliability between manual scorers was 84.6% (κ = 0.78), and the automatic method reached similar agreements with scorers, 83.4% (κ = 0.76) and 82.7% (κ = 0.75). Conclusion The developed algorithm achieved high classification accuracy and substantial agreements with two manual scorers; the performance metrics compared favorably with typical inter-rater reliability between manual scorers and performance reported in previous studies. These suggest that our algorithm may facilitate less labor-intensive and reliable automatic sleep scoring in preadolescent children.
Collapse
Affiliation(s)
- Pranavan Somaskandhan
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, QLD, Australia
- *Correspondence: Pranavan Somaskandhan,
| | - Timo Leppänen
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, QLD, Australia
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland
- Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland
| | - Philip I. Terrill
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, QLD, Australia
| | - Sigridur Sigurdardottir
- Reykjavik University Sleep Institute, School of Technology, Reykjavik University, Reykjavik, Iceland
| | - Erna Sif Arnardottir
- Reykjavik University Sleep Institute, School of Technology, Reykjavik University, Reykjavik, Iceland
- Internal Medicine Services, Landspitali–The National University Hospital of Iceland, Reykjavik, Iceland
| | - Kristín A. Ólafsdóttir
- Reykjavik University Sleep Institute, School of Technology, Reykjavik University, Reykjavik, Iceland
| | - Marta Serwatko
- Department of Clinical Engineering, Landspitali University Hospital, Reykjavik, Iceland
| | - Sigurveig Þ. Sigurðardóttir
- Department of Immunology, Landspitali University Hospital, Reykjavik, Iceland
- Faculty of Medicine, University of Iceland, Reykjavik, Iceland
| | - Michael Clausen
- Department of Allergy, Landspitali University Hospital, Reykjavik, Iceland
- Children's Hospital Reykjavik, Reykjavik, Iceland
| | - Juha Töyräs
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, QLD, Australia
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland
- Science Service Center, Kuopio University Hospital, Kuopio, Finland
| | - Henri Korkalainen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland
- Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland
| |
Collapse
|
45
|
Relational local electroencephalography representations for sleep scoring. Neural Netw 2022; 154:310-322. [DOI: 10.1016/j.neunet.2022.07.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2021] [Revised: 05/12/2022] [Accepted: 07/13/2022] [Indexed: 11/22/2022]
|
46
|
Alvarez-Estevez D, Rijsman RM. Computer-assisted analysis of polysomnographic recordings improves inter-scorer associated agreement and scoring times. PLoS One 2022; 17:e0275530. [PMID: 36174095 PMCID: PMC9522290 DOI: 10.1371/journal.pone.0275530] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 09/19/2022] [Indexed: 11/18/2022] Open
Abstract
Study objectives To investigate inter-scorer agreement and scoring time differences associated with visual and computer-assisted analysis of polysomnographic (PSG) recordings. Methods A group of 12 expert scorers reviewed 5 PSGs that were independently selected in the context of each of the following tasks: (i) sleep staging, (ii) scoring of leg movements, (iii) detection of respiratory (apneic-related) events, and (iv) of electroencephalographic (EEG) arousals. All scorers independently reviewed the same recordings, hence resulting in 20 scoring exercises per scorer from an equal amount of different subjects. The procedure was repeated, separately, using the classical visual manual approach and a computer-assisted (semi-automatic) procedure. Resulting inter-scorer agreement and scoring times were examined and compared among the two methods. Results Computer-assisted sleep scoring showed a consistent and statistically relevant effect toward less time required for the completion of each of the PSG scoring tasks. Gain factors ranged from 1.26 (EEG arousals) to 2.41 (leg movements). Inter-scorer kappa agreement was also consistently increased with the use of supervised semi-automatic scoring. Specifically, agreement increased from Κ = 0.76 to K = 0.80 (sleep stages), Κ = 0.72 to K = 0.91 (leg movements), Κ = 0.55 to K = 0.66 (respiratory events), and Κ = 0.58 to Κ = 0.65 (EEG arousals). Inter-scorer agreement on the examined set of diagnostic indices did also show a trend toward higher Interclass Correlation Coefficient scores when using the semi-automatic scoring approach. Conclusions Computer-assisted analysis can improve inter-scorer agreement and scoring times associated with the review of PSG studies resulting in higher efficiency and overall quality in the diagnosis sleep disorders.
Collapse
Affiliation(s)
- Diego Alvarez-Estevez
- Center for Information and Communications Technology Research (CITIC), Universidade da Coruña, A Coruña, Spain
- * E-mail:
| | - Roselyne M. Rijsman
- Sleep Center and Clinical Neurophysiology Department, Haaglanden Medisch Centrum, The Hague, The Netherlands
| |
Collapse
|
47
|
Waters SH, Clifford GD. Comparison of deep transfer learning algorithms and transferability measures for wearable sleep staging. Biomed Eng Online 2022; 21:66. [PMID: 36096868 PMCID: PMC9465946 DOI: 10.1186/s12938-022-01033-3] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 08/18/2022] [Indexed: 11/10/2022] Open
Abstract
Background Obtaining medical data using wearable sensors is a potential replacement for in-hospital monitoring, but the lack of data for such sensors poses a challenge for development. One solution is using in-hospital recordings to boost performance via transfer learning. While there are many possible transfer learning algorithms, few have been tested in the domain of EEG-based sleep staging. Furthermore, there are few ways for determining which transfer learning method will work best besides exhaustive testing. Measures of transferability do exist, but are typically used for selection of pre-trained models rather than algorithms and few have been tested on medical signals. We tested several supervised transfer learning algorithms on a sleep staging task using a single channel of EEG (AF7-Fpz) captured from an in-home commercial system. Results Two neural networks—one bespoke and another state-of-art open-source architecture—were pre-trained on one of six source datasets comprising 11,561 subjects undergoing clinical polysomnograms (PSGs), then re-trained on a target dataset of 75 full-night recordings from 24 subjects. Several transferability measures were then tested to determine which is most effective for assessing performance on unseen target data. Performance on the target dataset was improved using transfer learning, with re-training the head layers being the most effective in the majority of cases (up to 63.9% of cases). Transferability measures generally provided significant correlations with accuracy (up to \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$r = -0.53$$\end{document}r=-0.53). Conclusion Re-training the head layers provided the largest performance boost. Transferability measures are useful indicators of transfer learning effectiveness.
Collapse
Affiliation(s)
- Samuel H Waters
- Department of Bioengineering, Georgia Institute of Technology, Atlanta, United States.
| | - Gari D Clifford
- Department of Biomedical Informatics, Emory University, Atlanta, United States
| |
Collapse
|
48
|
Affiliation(s)
- Thomas Penzel
- Corresponding author. Thomas Penzel, Interdisciplinary Sleep Medicine Center, Charite center for Pneumology CC12, Charite University Hospital, Chariteplatz 1, 10117 Berlin, Germany.
| |
Collapse
|
49
|
van Gorp H, Huijben IAM, Fonseca P, van Sloun RJG, Overeem S, van Gilst MM. Certainty about uncertainty in sleep staging: a theoretical framework. Sleep 2022; 45:6604464. [DOI: 10.1093/sleep/zsac134] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Revised: 05/12/2022] [Indexed: 11/14/2022] Open
Abstract
Abstract
Sleep stage classification is an important tool for the diagnosis of sleep disorders. Because sleep staging has such a high impact on clinical outcome, it is important that it is done reliably. However, it is known that uncertainty exists in both expert scorers and automated models. On average, the agreement between human scorers is only 82.6%. In this study, we provide a theoretical framework to facilitate discussion and further analyses of uncertainty in sleep staging. To this end, we introduce two variants of uncertainty, known from statistics and the machine learning community: aleatoric and epistemic uncertainty. We discuss what these types of uncertainties are, why the distinction is useful, where they arise from in sleep staging, and provide recommendations on how this framework can improve sleep staging in the future.
Collapse
Affiliation(s)
- Hans van Gorp
- Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven , the Netherlands
- Personal Health, Philips Research , Eindhoven , the Netherlands
| | - Iris A M Huijben
- Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven , the Netherlands
- Onera Health , Eindhoven , the Netherlands
| | - Pedro Fonseca
- Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven , the Netherlands
- Personal Health, Philips Research , Eindhoven , the Netherlands
| | - Ruud J G van Sloun
- Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven , the Netherlands
- Personal Health, Philips Research , Eindhoven , the Netherlands
| | - Sebastiaan Overeem
- Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven , the Netherlands
- Sleep Medicine Centre, Kempenhaeghe Foundation , Eindhoven , the Netherlands
| | - Merel M van Gilst
- Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven , the Netherlands
- Sleep Medicine Centre, Kempenhaeghe Foundation , Eindhoven , the Netherlands
| |
Collapse
|
50
|
External proficiency testing improves inter-scorer reliability of polysomnography scoring. Sleep Breath 2022; 27:923-932. [DOI: 10.1007/s11325-022-02673-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Revised: 06/06/2022] [Accepted: 06/21/2022] [Indexed: 10/16/2022]
|