1
|
Mostafaei SH, Tanha J, Sharafkhaneh A. A novel deep learning model based on transformer and cross modality attention for classification of sleep stages. J Biomed Inform 2024; 157:104689. [PMID: 39029770 DOI: 10.1016/j.jbi.2024.104689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 06/13/2024] [Accepted: 07/10/2024] [Indexed: 07/21/2024]
Abstract
The classification of sleep stages is crucial for gaining insights into an individual's sleep patterns and identifying potential health issues. Employing several important physiological channels in different views, each providing a distinct perspective on sleep patterns, can have a great impact on the efficiency of the classification models. In the context of neural networks and deep learning models, transformers are very effective, especially when dealing with time series data, and have shown remarkable compatibility with sequential data analysis as physiological channels. On the other hand, cross-modality attention by integrating information from multiple views of the data enables to capture relationships among different modalities, allowing models to selectively focus on relevant information from each modality. In this paper, we introduce a novel deep-learning model based on transformer encoder-decoder and cross-modal attention for sleep stage classification. The proposed model processes information from various physiological channels with different modalities using the Sleep Heart Health Study Dataset (SHHS) data and leverages transformer encoders for feature extraction and cross-modal attention for effective integration to feed into the transformer decoder. The combination of these elements increased the accuracy of the model up to 91.33% in classifying five classes of sleep stages. Empirical evaluations demonstrated the model's superior performance compared to standalone approaches and other state-of-the-art techniques, showcasing the potential of combining transformer and cross-modal attention for improved sleep stage classification.
Collapse
Affiliation(s)
| | - Jafar Tanha
- Faculty of Electrical and Computer Engineering, University of Tabriz, P.O. Box 51666-16471, Tabriz, Iran.
| | - Amir Sharafkhaneh
- Professor of Medicine, Section of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, Baylor College of Medicine, Houston, TX, USA.
| |
Collapse
|
2
|
Yue H, Chen Z, Guo W, Sun L, Dai Y, Wang Y, Ma W, Fan X, Wen W, Lei W. Research and application of deep learning-based sleep staging: Data, modeling, validation, and clinical practice. Sleep Med Rev 2024; 74:101897. [PMID: 38306788 DOI: 10.1016/j.smrv.2024.101897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 12/30/2023] [Accepted: 01/04/2024] [Indexed: 02/04/2024]
Abstract
Over the past few decades, researchers have attempted to simplify and accelerate the process of sleep stage classification through various approaches; however, only a few such approaches have gained widespread acceptance. Artificial intelligence technology, particularly deep learning, is promising for earning the trust of the sleep medicine community in automated sleep-staging systems, thus facilitating its application in clinical practice and integration into daily life. We aimed to comprehensively review the latest methods that are applying deep learning for enhancing sleep staging efficiency and accuracy. Starting from the requisite "data" for constructing deep learning algorithms, we elucidated the current landscape of this domain and summarized the fundamental modeling process, encompassing signal selection, data pre-processing, model architecture, classification tasks, and performance metrics. Furthermore, we reviewed the applications of automated sleep staging in scenarios such as sleep-disorder screening, diagnostic procedures, and health monitoring and management. Finally, we conducted an in-depth analysis and discussion of the challenges and future in intelligent sleep staging, particularly focusing on large-scale sleep datasets, interdisciplinary collaborations, and human-computer interactions.
Collapse
Affiliation(s)
- Huijun Yue
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China
| | - Zhuqi Chen
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China
| | - Wenbin Guo
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China
| | - Lin Sun
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China
| | - Yidan Dai
- School of Computer Science, South China Normal University, Guangzhou, People's Republic of China
| | - Yiming Wang
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China
| | - Wenjun Ma
- School of Computer Science, South China Normal University, Guangzhou, People's Republic of China
| | - Xiaomao Fan
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen, People's Republic of China
| | - Weiping Wen
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China; Department of Otolaryngology, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China.
| | - Wenbin Lei
- Otorhinolaryngology Hospital, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, People's Republic of China.
| |
Collapse
|
3
|
Huang X, Schmelter F, Irshad MT, Piet A, Nisar MA, Sina C, Grzegorzek M. Optimizing sleep staging on multimodal time series: Leveraging borderline synthetic minority oversampling technique and supervised convolutional contrastive learning. Comput Biol Med 2023; 166:107501. [PMID: 37742416 DOI: 10.1016/j.compbiomed.2023.107501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Revised: 08/15/2023] [Accepted: 09/15/2023] [Indexed: 09/26/2023]
Abstract
Sleep is an important research area in nutritional medicine that plays a crucial role in human physical and mental health restoration. It can influence diet, metabolism, and hormone regulation, which can affect overall health and well-being. As an essential tool in the sleep study, the sleep stage classification provides a parsing of sleep architecture and a comprehensive understanding of sleep patterns to identify sleep disorders and facilitate the formulation of targeted sleep interventions. However, the class imbalance issue is typically salient in sleep datasets, which severely affects classification performances. To address this issue and to extract optimal multimodal features of EEG, EOG, and EMG that can improve the accuracy of sleep stage classification, a Borderline Synthetic Minority Oversampling Technique (B-SMOTE)-Based Supervised Convolutional Contrastive Learning (BST-SCCL) is proposed, which can avoid the risk of data mismatch between various sleep knowledge domains (varying health conditions and annotation rules) and strengthening learning characteristics of the N1 stage from the pair-wise segments comparison strategy. The lightweight residual network architecture with a novel truncated cross-entropy loss function is designed to accommodate multimodal time series and boost the training speed and performance stability. The proposed model has been validated on four well-known public sleep datasets (Sleep-EDF-20, Sleep-EDF-78, ISRUC-1, and ISRUC-3) and its superior performance (overall accuracy of 91.31-92.34%, MF1 of 88.21-90.08%, and Cohen's Kappa coefficient k of 0.87-0.89) has further demonstrated its effectiveness. It shows the great potential of contrastive learning for cross-domain knowledge interaction in precision medicine.
Collapse
Affiliation(s)
- Xinyu Huang
- Institute of Medical Informatics, University of Lübeck, Germany.
| | - Franziska Schmelter
- Institute of Nutritional Medicine, University of Lübeck and University Medical Center Schleswig-Holstein, Lübeck, Germany.
| | | | - Artur Piet
- Institute of Medical Informatics, University of Lübeck, Germany.
| | | | - Christian Sina
- Institute of Nutritional Medicine, University of Lübeck and University Medical Center Schleswig-Holstein, Lübeck, Germany; Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering (IMTE), Lübeck, Germany.
| | - Marcin Grzegorzek
- Institute of Medical Informatics, University of Lübeck, Germany; Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering (IMTE), Lübeck, Germany.
| |
Collapse
|
4
|
Li W, Gao J. Automatic sleep staging by a hybrid model based on deep 1D-ResNet-SE and LSTM with single-channel raw EEG signals. PeerJ Comput Sci 2023; 9:e1561. [PMID: 37810362 PMCID: PMC10557479 DOI: 10.7717/peerj-cs.1561] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 08/10/2023] [Indexed: 10/10/2023]
Abstract
Sleep staging is crucial for assessing sleep quality and diagnosing sleep disorders. Recent advances in deep learning methods with electroencephalogram (EEG) signals have shown remarkable success in automatic sleep staging. However, the use of deeper neural networks may lead to the issues of gradient disappearance and explosion, while the non-stationary nature and low signal-to-noise ratio of EEG signals can negatively impact feature representation. To overcome these challenges, we proposed a novel lightweight sequence-to-sequence deep learning model, 1D-ResNet-SE-LSTM, to classify sleep stages into five classes using single-channel raw EEG signals. Our proposed model consists of two main components: a one-dimensional residual convolutional neural network with a squeeze-and-excitation module to extract and reweight features from EEG signals, and a long short-term memory network to capture the transition rules among sleep stages. In addition, we applied the weighted cross-entropy loss function to alleviate the class imbalance problem. We evaluated the performance of our model on two publicly available datasets; Sleep-EDF Expanded consists of 153 overnight PSG recordings collected from 78 healthy subjects and ISRUC-Sleep includes 100 PSG recordings collected from 100 subjects diagnosed with various sleep disorders, and obtained an overall accuracy rate of 86.39% and 81.97%, respectively, along with corresponding macro average F1-scores of 81.95% and 79.94%. Our model outperforms existing sleep staging models in terms of overall performance metrics and per-class F1-scores for several sleep stages, particularly for the N1 stage, where it achieves F1-scores of 59.00% and 55.53%. The kappa coefficient is 0.812 and 0.766 for the Sleep-EDF Expanded and ISRUC-Sleep datasets, respectively, indicating strong agreement with certified sleep experts. We also investigated the effect of different weight coefficient combinations and sequence lengths of EEG epochs used as input to the model on its performance. Furthermore, the ablation study was conducted to evaluate the contribution of each component to the model's performance. The results demonstrate the effectiveness and robustness of the proposed model in classifying sleep stages, and highlights its potential to reduce human clinicians' workload, making sleep assessment and diagnosis more effective. However, the proposed model is subject to several limitations. Firstly, the model is a sequence-to-sequence network, which requires input sequences of EEG epochs. Secondly, the weight coefficients in the loss function could be further optimized to balance the classification performance of each sleep stage. Finally, apart from the channel attention mechanism, incorporating more advanced attention mechanisms could enhance the model's effectiveness.
Collapse
Affiliation(s)
- Weiming Li
- Shanghai Nuanhe Brain Technology Co. Ltd., Shanghai, China
| | - Junhui Gao
- Shanghai Nuanhe Brain Technology Co. Ltd., Shanghai, China
| |
Collapse
|
5
|
Gaiduk M, Serrano Alarcón Á, Seepold R, Martínez Madrid N. Current status and prospects of automatic sleep stages scoring: Review. Biomed Eng Lett 2023; 13:247-272. [PMID: 37519865 PMCID: PMC10382458 DOI: 10.1007/s13534-023-00299-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 06/07/2023] [Accepted: 06/18/2023] [Indexed: 08/01/2023] Open
Abstract
The scoring of sleep stages is one of the essential tasks in sleep analysis. Since a manual procedure requires considerable human and financial resources, and incorporates some subjectivity, an automated approach could result in several advantages. There have been many developments in this area, and in order to provide a comprehensive overview, it is essential to review relevant recent works and summarise the characteristics of the approaches, which is the main aim of this article. To achieve it, we examined articles published between 2018 and 2022 that dealt with the automated scoring of sleep stages. In the final selection for in-depth analysis, 125 articles were included after reviewing a total of 515 publications. The results revealed that automatic scoring demonstrates good quality (with Cohen's kappa up to over 0.80 and accuracy up to over 90%) in analysing EEG/EEG + EOG + EMG signals. At the same time, it should be noted that there has been no breakthrough in the quality of results using these signals in recent years. Systems involving other signals that could potentially be acquired more conveniently for the user (e.g. respiratory, cardiac or movement signals) remain more challenging in the implementation with a high level of reliability but have considerable innovation capability. In general, automatic sleep stage scoring has excellent potential to assist medical professionals while providing an objective assessment.
Collapse
Affiliation(s)
- Maksym Gaiduk
- HTWG Konstanz – University of Applied Sciences, Alfred-Wachtel-Str.8, 78462 Konstanz, Germany
| | | | - Ralf Seepold
- HTWG Konstanz – University of Applied Sciences, Alfred-Wachtel-Str.8, 78462 Konstanz, Germany
| | | |
Collapse
|
6
|
Power fingerprint identification based on the improved V-I trajectory with color encoding and transferred CBAM-ResNet. PLoS One 2023; 18:e0281482. [PMID: 36757938 PMCID: PMC9910749 DOI: 10.1371/journal.pone.0281482] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 01/24/2023] [Indexed: 02/10/2023] Open
Abstract
In power fingerprint identification, feature information is insufficient when using a single feature to identify equipment, and small load data of specific customers, difficult to meet the refined equipment classification needs. A power fingerprint identification based on the improved voltage-current(V-I) trajectory with color encoding and transferred CBAM-ResNet34 is proposed. First, the current, instantaneous power, and trajectory momentum information are added to the original V-I trajectory image using color coding to obtain a color V-I trajectory image. Then, the ResNet34 model was pre-trained using the ImageNet dataset and a new fully-connected layer meeting the device classification goal was used to replace the fully-connected layer of ResNet34. The Convolutional Block Attention Module (CBAM) was added to each residual structure module of ResNet34. Finally, Class-Balanced (CB) loss is introduced to reweight the Softmax cross-entropy (SM-CE) loss function to solve the problem of data imbalance in V-I trajectory identification. All parameters are retrained to extract features from the color V-I trajectory images for device classification. The experimental results on the imbalanced PLAID dataset verify that the method in this paper has better classification capability in small sample imbalanced datasets. The experimental results show that the method effectively improves the identification accuracy by 4.4% and reduces the training time of the model by 14 minutes compared with the existing methods, which meets the accuracy requirements of fine-grained power fingerprint identification.
Collapse
|