Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification

Performance Investigation of Somfit Sleep Staging Algorithm

Purpose

To investigate accuracy of the sleep staging algorithm in a new miniaturized home sleep monitoring device - Compumedics® Somfit. Somfit is attached to patient's forehead and combines channels specified for a pulse arterial tonometry (PAT)-based home sleep apnea testing (HSAT) device with the neurological signals. Somfit sleep staging deep learning algorithm is based on convolutional neural network architecture.

Patients and Methods

One hundred and ten participants referred for sleep investigation with suspected or preexisting obstructive sleep apnea (OSA) in need of a review were enrolled into the study involving simultaneous recording of full overnight polysomnography (PSG) and Somfit data. The recordings were conducted at three centers in Australia. The reported statistics include standard measures of agreement between Somfit automatic hypnogram and consensus PSG hypnogram.

Results

Overall percent agreement across five sleep stages (N1, N2, N3, REM, and wake) between Somfit automatic and consensus PSG hypnograms was 76.14 (SE: 0.79). The percent agreements between different pairs of sleep technologists' PSG hypnograms varied from 74.36 (1.93) to 85.50 (0.64), with interscorer agreement being greater for scorers from the same sleep laboratory. The estimate of kappa between Somfit and consensus PSG was 0.672 (0.002). Percent agreement for sleep/wake discrimination was 89.30 (0.37). The accuracy of Somfit sleep staging algorithm varied with increasing OSA severity - percent agreement was 79.67 (1.87) for the normal subjects, 77.38 (1.06) for mild OSA, 74.83 (1.79) for moderate OSA and 72.93 (1.68) for severe OSA.

Conclusion

Agreement between Somfit and PSG hypnograms was non-inferior to PSG interscorer agreement for a number of scorers, thus confirming acceptability of electrode placement at the center of the forehead. The directions for algorithm improvement include additional arousal detection, integration of motion and oximetry signals and separate inference models for individual sleep stages.

A Review on Automated Sleep Study

In recent years, research on automated sleep analysis has witnessed significant growth, reflecting advancements in understanding sleep patterns and their impact on overall health. This review synthesizes findings from an exhaustive analysis of 87 papers, systematically retrieved from prominent databases such as Google Scholar, PubMed, IEEE Xplore, and ScienceDirect. The selection criteria prioritized studies focusing on methods employed, signal modalities utilized, and machine learning algorithms applied in automated sleep analysis. The overarching goal was to critically evaluate the strengths and weaknesses of the proposed methods, shedding light on the current landscape and future directions in sleep research. An in-depth exploration of the reviewed literature revealed a diverse range of methodologies and machine learning approaches employed in automated sleep studies. Notably, K-Nearest Neighbors (KNN), Ensemble Learning Methods, and Support Vector Machine (SVM) emerged as versatile and potent classifiers, exhibiting high accuracies in various applications. However, challenges such as performance variability and computational demands were observed, necessitating judicious classifier selection based on dataset intricacies. In addition, the integration of traditional feature extraction methods with deep structures and the combination of different deep neural networks were identified as promising strategies to enhance diagnostic accuracy in sleep-related studies. The reviewed literature emphasized the need for adaptive classifiers, cross-modality integration, and collaborative efforts to drive the field toward more accurate, robust, and accessible sleep-related diagnostic solutions. This comprehensive review serves as a solid foundation for researchers and practitioners, providing an organized synthesis of the current state of knowledge in automated sleep analysis. By highlighting the strengths and challenges of various methodologies, this review aims to guide future research toward more effective and nuanced approaches to sleep diagnostics.

Insights from the 2nd China intelligent sleep staging competition

STUDY OBJECTIVES

Artificial intelligence (AI) is quickly advancing in the field of sleep medicine, which bodes well for the potential of actual clinical use. In this study, an analysis of the 2nd China Intelligent Sleep Staging Competition was conducted to gain insights into the general level and constraints of AI-assisted sleep staging in China.

METHODS

The outcomes of 10 teams from the children's track and 13 teams from the adult track were investigated in this study. The analysis included overall performance, differences between five different sleep stages, variations across subjects, and performance during stage transitions.

RESULTS

The adult track's accuracy peaked at 80.46%, while the children's track's accuracy peaked at 88.96%. On average, accuracy rates stood at 71.43% for children and 68.40% for adults. All results were produced within a mere 5-min timeframe. The N1 stage was prone to misclassification as W, N2, and R stages. In the adult track, significant differences were apparent among subjects (p < 0.05), whereas in the children's track, such differences were not observed. Nonetheless, both tracks experienced a performance decline during stage transitions.

CONCLUSIONS

The computational speed of AI is remarkably fast, simultaneously holding the potential to surpass the accuracy of physicians. Improving the machine learning model's classification of the N1 stage and transitional periods between stages, along with bolstering its robustness to individual subject variations, is imperative for maximizing its ability in assisting clinical scoring.

MVF-SleepNet: Multi-View Fusion Network for Sleep Stage Classification

Sleep stage classification is of great importance in human health monitoring and disease diagnosing. Clinically, visual-inspected classifying sleep into different stages is quite time consuming and highly relies on the expertise of sleep specialists. Many automated models for sleep stage classification have been proposed in previous studies but their performances still exist a gap to the real clinical application. In this work, we propose a novel multi-view fusion network named MVF-SleepNet based on multi-modal physiological signals of electroencephalography (EEG), electrocardiography (ECG), electrooculography (EOG), and electromyography (EMG). To capture the relationship representation among multi-modal physiological signals, we construct two views of Time-frequency images (TF images) and Graph-learned graphs (GL graphs). To learn the spectral-temporal representation from sequentially timed TF images, the combination of VGG-16 and GRU networks is utilized. To learn the spatial-temporal representation from sequentially timed GL graphs, the combination of Chebyshev graph convolution and temporal convolution networks is employed. Fusing the spectral-temporal representation and spatial-temporal representation can further boost the performance of sleep stage classification. A large number of experiment results on the publicly available datasets of ISRUC-S1 and ISRUC-S3 show that the MVF-SleepNet achieves overall accuracy of 0.821, F1 score of 0.802 and Kappa of 0.768 on ISRUC-S1 dataset, and accuracy of 0.841, F1 score of 0.828 and Kappa of 0.795 on ISRUC-S3 dataset. The MVF-SleepNet achieves competitive results on both datasets of ISRUC-S1 and ISRUC-S3 for sleep stage classification compared to the state-of-the-art baselines. The source code of MVF-SleepNet is available on Github (https://github.com/YJPai65/MVF-SleepNet).

Amplitude-Time Dual-View Fused EEG Temporal Feature Learning for Automatic Sleep Staging

Electroencephalogram (EEG) plays an important role in studying brain function and human cognitive performance, and the recognition of EEG signals is vital to develop an automatic sleep staging system. However, due to the complex nonstationary characteristics and the individual difference between subjects, how to obtain the effective signal features of the EEG for practical application is still a challenging task. In this article, we investigate the EEG feature learning problem and propose a novel temporal feature learning method based on amplitude-time dual-view fusion for automatic sleep staging. First, we explore the feature extraction ability of convolutional neural networks for the EEG signal from the perspective of interpretability and construct two new representation signals for the raw EEG from the views of amplitude and time. Then, we extract the amplitude-time signal features that reflect the transformation between different sleep stages from the obtained representation signals by using conventional 1-D CNNs. Furthermore, a hybrid dilation convolution module is used to learn the long-term temporal dependency features of EEG signals, which can overcome the shortcoming that the small-scale convolution kernel can only learn the local signal variation information. Finally, we conduct attention-based feature fusion for the learned dual-view signal features to further improve sleep staging performance. To evaluate the performance of the proposed method, we test 30-s-epoch EEG signal samples for healthy subjects and subjects with mild sleep disorders. The experimental results from the most commonly used datasets show that the proposed method has better sleep staging performance and has the potential for the development and application of an EEG-based automatic sleep staging system.

ZleepAnlystNet: a novel deep learning model for automatic sleep stage scoring based on single-channel raw EEG data using separating training

Numerous models for sleep stage scoring utilizing single-channel raw EEG signal have typically employed CNN and BiLSTM architectures. While these models, incorporating temporal information for sequence classification, demonstrate superior overall performance, they often exhibit low per-class performance for N1-stage, necessitating an adjustment of loss function. However, the efficacy of such adjustment is constrained by the training process. In this study, a pioneering training approach called separating training is introduced, alongside a novel model, to enhance performance. The developed model comprises 15 CNN models with varying loss function weights for feature extraction and 1 BiLSTM for sequence classification. Due to its architecture, this model cannot be trained using an end-to-end approach, necessitating separate training for each component using the Sleep-EDF dataset. Achieving an overall accuracy of 87.02%, MF1 of 82.09%, Kappa of 0.8221, and per-class F1-socres (W 90.34%, N1 54.23%, N2 89.53%, N3 88.96%, and REM 87.40%), our model demonstrates promising performance. Comparison with sleep technicians reveals a Kappa of 0.7015, indicating alignment with reference sleep stags. Additionally, cross-dataset validation and adaptation through training with the SHHS dataset yield an overall accuracy of 84.40%, MF1 of 74.96% and Kappa of 0.7785 when tested with the Sleep-EDF-13 dataset. These findings underscore the generalization potential in model architecture design facilitated by our novel training approach.

Multi-layer CNN-LSTM network with self-attention mechanism for robust estimation of nonlinear uncertain systems

Introduction

With the help of robot technology, intelligent rehabilitation of patients with lower limb motor dysfunction caused by stroke can be realized. A key factor constraining the clinical application of rehabilitation robots is how to realize pattern recognition of human movement intentions by using the surface electromyography (sEMG) sensors to ensure unhindered human-robot interaction.

Methods

A multilayer CNN-LSTM prediction network incorporating the self-attention mechanism (SAM) is proposed, in this paper, which can extract and learn the periodic and trend characteristics of the sEMG signals, and realize the accurate autoregressive prediction of the human motion information. Firstly, the multilayer CNN-LSTM network utilizes the CNN layer for initial feature extraction of data, and the LSTM network is used to improve the enhancement of the historical time-series features. Then, the SAM is used to improve the global feature extraction performance and parallel computation speed of the network.

Results

In comparison with existing test is carried out using actual data from five healthy subjects as well as a clinical hemiplegic patient to verify the superiority and practicality of the proposed algorithm. The results show that most of the model's prediction R > 0.9 for different motion states of healthy subjects; in the experiments oriented to the motion characteristics of patient subjects, the angle prediction results of R > 0.99 for the untrained data on the affected side, which proves that our proposed model also has a better effect on the angle prediction of the affected side.

Discussion

The main contribution of this paper is to realize continuous motion estimation of ankle joint for healthy and hemiplegic individuals under non-ideal conditions (weak sEMG signals, muscle fatigue, high muscle tension, etc.), which improves the pattern recognition accuracy and robustness of the sEMG sensor-based system.

Research and application of deep learning-based sleep staging: Data, modeling, validation, and clinical practice

Over the past few decades, researchers have attempted to simplify and accelerate the process of sleep stage classification through various approaches; however, only a few such approaches have gained widespread acceptance. Artificial intelligence technology, particularly deep learning, is promising for earning the trust of the sleep medicine community in automated sleep-staging systems, thus facilitating its application in clinical practice and integration into daily life. We aimed to comprehensively review the latest methods that are applying deep learning for enhancing sleep staging efficiency and accuracy. Starting from the requisite "data" for constructing deep learning algorithms, we elucidated the current landscape of this domain and summarized the fundamental modeling process, encompassing signal selection, data pre-processing, model architecture, classification tasks, and performance metrics. Furthermore, we reviewed the applications of automated sleep staging in scenarios such as sleep-disorder screening, diagnostic procedures, and health monitoring and management. Finally, we conducted an in-depth analysis and discussion of the challenges and future in intelligent sleep staging, particularly focusing on large-scale sleep datasets, interdisciplinary collaborations, and human-computer interactions.

A Sequential End-to-End Neonatal Sleep Staging Model with Squeeze and Excitation Blocks and Sequential Multi-Scale Convolution Neural Networks

Automatic sleep staging offers a quick and objective assessment for quantitatively interpreting sleep stages in neonates. However, most of the existing studies either do not encompass any temporal information, or simply apply neural networks to exploit temporal information at the expense of high computational overhead and modeling ambiguity. This limits the application of these methods to multiple scenarios. In this paper, a sequential end-to-end sleep staging model, SeqEESleepNet, which is competent for parallelly processing sequential epochs and has a fast training rate to adapt to different scenarios, is proposed. SeqEESleepNet consists of a sequence epoch generation (SEG) module, a sequential multi-scale convolution neural network (SMSCNN) and squeeze and excitation (SE) blocks. The SEG module expands independent epochs into sequential signals, enabling the model to learn the temporal information between sleep stages. SMSCNN is a multi-scale convolution neural network that can extract both multi-scale features and temporal information from the signal. Subsequently, the followed SE block can reassign the weights of features through mapping and pooling. Experimental results exhibit that in a clinical dataset, the proposed method outperforms the state-of-the-art approaches, achieving an overall accuracy, F1-score, and Kappa coefficient of 71.8%, 71.8%, and 0.684 on a three-class classification task with a single channel EEG signal. Based on our overall results, we believe the proposed method could pave the way for convenient multi-scenario neonatal sleep staging methods.

An automatic method using MFCC features for sleep stage classification

Sleep stage classification is a necessary step for diagnosing sleep disorders. Generally, experts use traditional methods based on every 30 seconds (s) of the biological signals, such as electrooculograms (EOGs), electrocardiograms (ECGs), electromyograms (EMGs), and electroencephalograms (EEGs), to classify sleep stages. Recently, various state-of-the-art approaches based on a deep learning model have been demonstrated to have efficient and accurate outcomes in sleep stage classification. In this paper, a novel deep convolutional neural network (CNN) combined with a long short-time memory (LSTM) model is proposed for sleep scoring tasks. A key frequency domain feature named Mel-frequency Cepstral Coefficient (MFCC) is extracted from EEG and EMG signals. The proposed method can learn features from frequency domains on different bio-signal channels. It firstly extracts the MFCC features from multi-channel signals, and then inputs them to several convolutional layers and an LSTM layer. Secondly, the learned representations are fed to a fully connected layer and a softmax classifier for sleep stage classification. The experiments are conducted on two widely used sleep datasets, Sleep Heart Health Study (SHHS) and Vincent's University Hospital/University College Dublin Sleep Apnoea (UCDDB) to test the effectiveness of the method. The results of this study indicate that the model can perform well in the classification of sleep stages using the features of the 2-dimensional (2D) MFCC feature. The advantage of using the feature is that it can be used to input a two-dimensional data stream, which can be used to retain information about each sleep stage. Using 2D data streams can reduce the time it takes to retrieve the data from the one-dimensional stream. Another advantage of this method is that it eliminates the need for deep layers, which can help improve the performance of the model. For instance, by reducing the number of layers, our seven layers of the model structure takes around 400 s to train and test 100 subjects in the SHHS1 dataset. Its best accuracy and Cohen's kappa are 82.35% and 0.75 for the SHHS dataset, and 73.07% and 0.63 for the UCDDB dataset, respectively.

Automatic Sleep Staging Based on Contextual Scalograms and Attention Convolution Neural Network Using Single-Channel EEG

Single-channel EEG based sleep staging is of interest to researchers due to its broad application prospect in daily sleep monitoring recently. We proposed using contextual scalograms as input and developed a convolutional neural network with attention modules named Co-ScaleNet for sleep staging. The contextual scalograms were obtained by combining the same color channels of three original RGB scalograms from consecutive epochs, and a simple and efficient data augmentation was designed according to their various forms. The Co-ScaleNet consists of two main parts. Firstly, three parallel convolutional branches with attention modules correspondingly extract and fuse features from contextual scalograms at the top layers. The remaining part is a stack of lightweight blocks. We achieved an overall accuracy of 87.0% for healthy individuals, 84.7% for depressed patients. And we obtained comparable performance on the public Sleep-EDFx (82.8%), ISRUC (84.6%) and SHHS datasets (87.7%), including a high recall of N1. The contextual scalograms of R channel as input achieved the best performance, which conform to the features of interest in visual scoring. The attention modules improved the recall of N1 and N3. Overall, the contextual scalograms provided a novel scheme for both contextual information extraction and data augmentation. Our study successfully expanded its application to depression datasets, as well as patients with sleep apnea, demonstrating its wide applicability.

Modality-Specific Feature Selection, Data Augmentation and Temporal Context for Improved Performance in Sleep Staging

This work attempts to design an effective sleep staging system, making the best use of the available signals, strategies, and features in the literature. It must not only perform well on different datasets comprising healthy and clinical populations but also achieve good accuracy in cross-dataset experiments. Toward this end, we propose a model comprising multiple binary classifiers in a hierarchical fashion, where, at each level, one or more of EEG, EOG, and EMG are selected to best differentiate between two sleep stages. The best set of 100 features is chosen out of all the features derived from selected signals. The class imbalance in data is addressed by random undersampling and boosting techniques with decision trees as weak learners. Temporal context and data augmentation are used to improve the performance. We also evaluate the performance of our model by training and testing on different datasets. We compare the results of five approaches: using only EEG, EEG+EOG, EEG+EMG+EOG, EEG+EMG, and selective modality with a specific combination of EEG, EMG, and/or EOG at each level. The best results are obtained by considering features from EEG+EMG+EOG at each hierarchical level. The proposed model achieves average accuracies of 83.1%, 90.0%, 84.4%, 82.1%, 81.5%, 79.9%, and 73.7% on Sleep-EDF, Exp Sleep-EDF, ISRUC-S1, S2 and S3, DRMS-SUB, and DRMS-PAT datasets, respectively. For all the datasets except DRMS-SUB, the proposed method outperforms all the state-of-the-art approaches. Cross-dataset performance exceeds 80% for all datasets except DRMS-PAT; independent of whether the test data is from normal subjects or patients.

Review and perspective on sleep-disordered breathing research and translation to clinics

Sleep-disordered breathing, ranging from habitual snoring to severe obstructive sleep apnea, is a prevalent public health issue. Despite rising interest in sleep and awareness of sleep disorders, sleep research and diagnostic practices still rely on outdated metrics and laborious methods reducing the diagnostic capacity and preventing timely diagnosis and treatment. Consequently, a significant portion of individuals affected by sleep-disordered breathing remain undiagnosed or are misdiagnosed. Taking advantage of state-of-the-art scientific, technological, and computational advances could be an effective way to optimize the diagnostic and treatment pathways. We discuss state-of-the-art multidisciplinary research, review the shortcomings in the current practices of SDB diagnosis and management in adult populations, and provide possible future directions. We critically review the opportunities for modern data analysis methods and machine learning to combine multimodal information, provide a perspective on the pitfalls of big data analysis, and discuss approaches for developing analysis strategies that overcome current limitations. We argue that large-scale and multidisciplinary collaborative efforts based on clinical, scientific, and technical knowledge and rigorous clinical validation and implementation of the outcomes in practice are needed to move the research of sleep-disordered breathing forward, thus increasing the quality of diagnostics and treatment.

Automatic classification of sleep stages using EEG signals and convolutional neural networks

Sleep stages classification is one of the new topics in studying human life quality because it plays a crucial role in getting a healthy lifestyle. Abnormal changes or absence of normal sleep may lead to different diseases such as heart-related diseases, diabetes, and obesity. In general, sleep staging analysis can be performed using electroencephalography (EEG) signals. This study proposes a convolutional neural network (CNN) based methodology for sleep stage classification using EEG signals taken by six channels and transformed into time-frequency analysis images. The proposed methodology consists of three major steps: (i) segment the EEG signal into epochs with 30 seconds in length, (ii) convert epochs into 2D representation using time-frequency analysis, and (iii) feed the 2D time-frequency analysis to the 2D CNN. The results showed that the proposed methodology is robust and achieved a very high accuracy of 99.39% for channel C4-A1. All other channels have accuracy values above 98.5%, which indicates that any channel can be used for sleep stage classification with high accuracy. The proposed methodology outperformed the methods in the literature in terms of overall accuracy or single channel accuracy. It is expected to provide a great benefit for physicians, especially neurologists; by providing them with a new powerful tool to support the clinical diagnosis of sleep-related diseases.

Somnotate: A probabilistic sleep stage classifier for studying vigilance state transitions

Electrophysiological recordings from freely behaving animals are a widespread and powerful mode of investigation in sleep research. These recordings generate large amounts of data that require sleep stage annotation (polysomnography), in which the data is parcellated according to three vigilance states: awake, rapid eye movement (REM) sleep, and non-REM (NREM) sleep. Manual and current computational annotation methods ignore intermediate states because the classification features become ambiguous, even though intermediate states contain important information regarding vigilance state dynamics. To address this problem, we have developed "Somnotate"-a probabilistic classifier based on a combination of linear discriminant analysis (LDA) with a hidden Markov model (HMM). First we demonstrate that Somnotate sets new standards in polysomnography, exhibiting annotation accuracies that exceed human experts on mouse electrophysiological data, remarkable robustness to errors in the training data, compatibility with different recording configurations, and an ability to maintain high accuracy during experimental interventions. However, the key feature of Somnotate is that it quantifies and reports the certainty of its annotations. We leverage this feature to reveal that many intermediate vigilance states cluster around state transitions, whereas others correspond to failed attempts to transition. This enables us to show for the first time that the success rates of different types of transition are differentially affected by experimental manipulations and can explain previously observed sleep patterns. Somnotate is open-source and has the potential to both facilitate the study of sleep stage transitions and offer new insights into the mechanisms underlying sleep-wake dynamics.

Few-shot EEG sleep staging based on transductive prototype optimization network

Electroencephalography (EEG) is a commonly used technology for monitoring brain activities and diagnosing sleep disorders. Clinically, doctors need to manually stage sleep based on EEG signals, which is a time-consuming and laborious task. In this study, we propose a few-shot EEG sleep staging termed transductive prototype optimization network (TPON) method, which aims to improve the performance of EEG sleep staging. Compared with traditional deep learning methods, TPON uses a meta-learning algorithm, which generalizes the classifier to new classes that are not visible in the training set, and only have a few examples for each new class. We learn the prototypes of existing objects through meta-training, and capture the sleep features of new objects through the "learn to learn" method of meta-learning. The prototype distribution of the class is optimized and captured by using support set and unlabeled high confidence samples to increase the authenticity of the prototype. Compared with traditional prototype networks, TPON can effectively solve too few samples in few-shot learning and improve the matching degree of prototypes in prototype network. The experimental results on the public SleepEDF-2013 dataset show that the proposed algorithm outperform than most advanced algorithms in the overall performance. In addition, we experimentally demonstrate the feasibility of cross-channel recognition, which indicates that there are many similar sleep EEG features between different channels. In future research, we can further explore the common features among different channels and investigate the combination of universal features in sleep EEG. Overall, our method achieves high accuracy in sleep stage classification, demonstrating the effectiveness of this approach and its potential applications in other medical fields.

Automated Sleep Stage Classification in Home Environments: An Evaluation of Seven Deep Neural Network Architectures

Sleep is an essential human physiological need that has garnered increasing scientific attention due to the burgeoning prevalence of sleep-related disorders and their impact on public health. Among contemporary challenges, the demand for authentic sleep monitoring outside the confines of specialized laboratories, ideally within the home environment, has arisen. Addressing this, we explore the development of pragmatic approaches that facilitate implementation within domestic settings. Such approaches necessitate the deployment of streamlined, computationally efficient automated classifiers. In pursuit of a sleep stage classifier tailored for home use, this study rigorously assessed seven conventional neural network architectures prominent in deep learning (LeNet, ResNet, VGG, MLP, LSTM-CNN, LSTM, BLSTM). Leveraging sleep recordings from a cohort of 20 subjects, we elucidate that LeNet, VGG, and ResNet exhibit superior performance compared to recent advancements reported in the literature. Furthermore, a comprehensive architectural analysis was conducted, illuminating the strengths and limitations of each in the context of home-based sleep monitoring. Our findings distinctly identify LeNet as the most-amenable architecture for this purpose, with LSTM and BLSTM demonstrating relatively lesser compatibility. Ultimately, this research substantiates the feasibility of automating sleep stage classification employing lightweight neural networks, thereby accommodating scenarios with constrained computational resources. This advancement aims at revolutionizing the field of sleep monitoring, making it more accessible and reliable for individuals in their homes.

DenSleepNet: DenseNet based model for sleep staging with two-frequency feature fusion and coordinate attention

Sleep staging is often applied to assess the quality of sleep and also be used to prevent and monitor psychiatric disorders caused by sleep. However, it remains a challenge to extract the discriminative features of salient waveforms in sleep EEG and enable the network to effectively classify sleep stages by emphasizing these crucial features, thus achieving higher accuracy. In this study, an end-to-end deep learning model based on DenseNet for automatic sleep staging is designed and constructed. In the framework, two convolutional branches are devised to extract the underlying features (Two-Frequency Feature) at various frequencies, which are then fused and input into the DenseNet module to extract salient waveform features. After that, the Coordinate Attention mechanism is employed to enhance the localization of salient waveform features by emphasizing the position of salient waveforms and the spatial relationship across the entire frequency spectrum. Finally, the obtained features are accessed to the fully connected for sleep staging. The model was validated with a 20-fold cross-validation procedure on two public available datasets, and the overall accuracy, kappa coefficient, and MF1 score reached 92.9%, 78.7, 0.86 and 90.0%, 75.8, 0.80 on Sleep-EDF-20 and Sleep-EDFx, respectively. Experimental results show that the proposed model achieves competitive performance for sleep staging compared with the reported approaches under the identical conditions.

Optimizing sleep staging on multimodal time series: Leveraging borderline synthetic minority oversampling technique and supervised convolutional contrastive learning

Sleep is an important research area in nutritional medicine that plays a crucial role in human physical and mental health restoration. It can influence diet, metabolism, and hormone regulation, which can affect overall health and well-being. As an essential tool in the sleep study, the sleep stage classification provides a parsing of sleep architecture and a comprehensive understanding of sleep patterns to identify sleep disorders and facilitate the formulation of targeted sleep interventions. However, the class imbalance issue is typically salient in sleep datasets, which severely affects classification performances. To address this issue and to extract optimal multimodal features of EEG, EOG, and EMG that can improve the accuracy of sleep stage classification, a Borderline Synthetic Minority Oversampling Technique (B-SMOTE)-Based Supervised Convolutional Contrastive Learning (BST-SCCL) is proposed, which can avoid the risk of data mismatch between various sleep knowledge domains (varying health conditions and annotation rules) and strengthening learning characteristics of the N1 stage from the pair-wise segments comparison strategy. The lightweight residual network architecture with a novel truncated cross-entropy loss function is designed to accommodate multimodal time series and boost the training speed and performance stability. The proposed model has been validated on four well-known public sleep datasets (Sleep-EDF-20, Sleep-EDF-78, ISRUC-1, and ISRUC-3) and its superior performance (overall accuracy of 91.31-92.34%, MF1 of 88.21-90.08%, and Cohen's Kappa coefficient k of 0.87-0.89) has further demonstrated its effectiveness. It shows the great potential of contrastive learning for cross-domain knowledge interaction in precision medicine.

A Novel Sleep Staging Method Based on EEG and ECG Multimodal Features Combination

Accurate sleep staging evaluates the quality of sleep, supporting the clinical diagnosis and intervention of sleep disorders and related diseases. Although previous attempts to classify sleep stages have achieved high classification performance, little attention has been paid to integrating the rich information in brain and heart dynamics during sleep for sleep staging. In this study, we propose a generalized EEG and ECG multimodal feature combination to classify sleep stages with high efficiency and accuracy. Briefly, a hybrid features combination in terms of multiscale entropy and intrinsic mode function are used to reflect nonlinear dynamics in multichannel EEGs, along with heart rate variability measures over time/frequency domains, and sample entropy across scales are applied for ECGs. For both the max-relevance and min-redundancy method and principal component analysis were used for dimensionality reduction. The selected features were classified by four traditional machine learning classifiers. Macro-F1 score, macro-geometric mean, and Cohen kappa value are adopted to evaluate the classification performance of each class in an imbalanced dataset. Experimental results show that EEG features contribute more to wake stage classification while ECG features contribute more to deep sleep stages. The proposed combination achieves the highest accuracy of 84.3% and the highest kappa value of 0.794 on the support vector machine in the ISRUC-S3 dataset, suggesting the proposed multimodal features combination is promising in accuracy and efficiency compared to other state-of-the-art methods.

Multi-task learning for arousal and sleep stage detection using fully convolutional networks

Objective.Sleep is a critical physiological process that plays a vital role in maintaining physical and mental health. Accurate detection of arousals and sleep stages is essential for the diagnosis of sleep disorders, as frequent and excessive occurrences of arousals disrupt sleep stage patterns and lead to poor sleep quality, negatively impacting physical and mental health. Polysomnography is a traditional method for arousal and sleep stage detection that is time-consuming and prone to high variability among experts.Approach. In this paper, we propose a novel multi-task learning approach for arousal and sleep stage detection using fully convolutional neural networks. Our model, FullSleepNet, accepts a full-night single-channel EEG signal as input and produces segmentation masks for arousal and sleep stage labels. FullSleepNet comprises four modules: a convolutional module to extract local features, a recurrent module to capture long-range dependencies, an attention mechanism to focus on relevant parts of the input, and a segmentation module to output final predictions.Main results.By unifying the two interrelated tasks as segmentation problems and employing a multi-task learning approach, FullSleepNet achieves state-of-the-art performance for arousal detection with an area under the precision-recall curve of 0.70 on Sleep Heart Health Study and Multi-Ethnic Study of Atherosclerosis datasets. For sleep stage classification, FullSleepNet obtains comparable performance on both datasets, achieving an accuracy of 0.88 and an F1-score of 0.80 on the former and an accuracy of 0.83 and an F1-score of 0.76 on the latter.Significance. Our results demonstrate that FullSleepNet offers improved practicality, efficiency, and accuracy for the detection of arousal and classification of sleep stages using raw EEG signals as input.

GAC-SleepNet: A dual-structured sleep staging method based on graph structure and Euclidean structure

Sleep staging is a precondition for the diagnosis and treatment of sleep disorders. However, how to fully exploit the relationship between spatial features of the brain and sleep stages is an important task. Many current classical algorithms only extract the characteristic information of the brain in the Euclidean space without considering other spatial structures. In this study, a sleep staging network named GAC-SleepNet is designed. GAC-SleepNet uses the characteristic information in the dual structure of the graph structure and the Euclidean structure for the classification of sleep stages. In the graph structure, this study uses a graph convolutional neural network to learn the deep features of each sleep stage and converts the features in the topological structure into feature vectors by a multilayer perceptron. In the Euclidean structure, this study uses convolutional neural networks to learn the temporal features of sleep information and combine attention mechanism to portray the connection between different sleep periods and EEG signals, while enhancing the description of global features to avoid local optima. In this study, the performance of the proposed network is evaluated on two public datasets. The experimental results show that the dual spatial structure captures more adequate and comprehensive information about sleep features and shows advancement in terms of different evaluation metrics.

A Review of Recurrent Neural Network-Based Methods in Computational Physiology

Artificial intelligence and machine learning techniques have progressed dramatically and become powerful tools required to solve complicated tasks, such as computer vision, speech recognition, and natural language processing. Since these techniques have provided promising and evident results in these fields, they emerged as valuable methods for applications in human physiology and healthcare. General physiological recordings are time-related expressions of bodily processes associated with health or morbidity. Sequence classification, anomaly detection, decision making, and future status prediction drive the learning algorithms to focus on the temporal pattern and model the nonstationary dynamics of the human body. These practical requirements give birth to the use of recurrent neural networks (RNNs), which offer a tractable solution in dealing with physiological time series and provide a way to understand complex time variations and dependencies. The primary objective of this article is to provide an overview of current applications of RNNs in the area of human physiology for automated prediction and diagnosis within different fields. Finally, we highlight some pathways of future RNN developments for human physiology.

Automatic sleep staging by a hybrid model based on deep 1D-ResNet-SE and LSTM with single-channel raw EEG signals

Sleep staging is crucial for assessing sleep quality and diagnosing sleep disorders. Recent advances in deep learning methods with electroencephalogram (EEG) signals have shown remarkable success in automatic sleep staging. However, the use of deeper neural networks may lead to the issues of gradient disappearance and explosion, while the non-stationary nature and low signal-to-noise ratio of EEG signals can negatively impact feature representation. To overcome these challenges, we proposed a novel lightweight sequence-to-sequence deep learning model, 1D-ResNet-SE-LSTM, to classify sleep stages into five classes using single-channel raw EEG signals. Our proposed model consists of two main components: a one-dimensional residual convolutional neural network with a squeeze-and-excitation module to extract and reweight features from EEG signals, and a long short-term memory network to capture the transition rules among sleep stages. In addition, we applied the weighted cross-entropy loss function to alleviate the class imbalance problem. We evaluated the performance of our model on two publicly available datasets; Sleep-EDF Expanded consists of 153 overnight PSG recordings collected from 78 healthy subjects and ISRUC-Sleep includes 100 PSG recordings collected from 100 subjects diagnosed with various sleep disorders, and obtained an overall accuracy rate of 86.39% and 81.97%, respectively, along with corresponding macro average F1-scores of 81.95% and 79.94%. Our model outperforms existing sleep staging models in terms of overall performance metrics and per-class F1-scores for several sleep stages, particularly for the N1 stage, where it achieves F1-scores of 59.00% and 55.53%. The kappa coefficient is 0.812 and 0.766 for the Sleep-EDF Expanded and ISRUC-Sleep datasets, respectively, indicating strong agreement with certified sleep experts. We also investigated the effect of different weight coefficient combinations and sequence lengths of EEG epochs used as input to the model on its performance. Furthermore, the ablation study was conducted to evaluate the contribution of each component to the model's performance. The results demonstrate the effectiveness and robustness of the proposed model in classifying sleep stages, and highlights its potential to reduce human clinicians' workload, making sleep assessment and diagnosis more effective. However, the proposed model is subject to several limitations. Firstly, the model is a sequence-to-sequence network, which requires input sequences of EEG epochs. Secondly, the weight coefficients in the loss function could be further optimized to balance the classification performance of each sleep stage. Finally, apart from the channel attention mechanism, incorporating more advanced attention mechanisms could enhance the model's effectiveness.

MultiChannelSleepNet: A Transformer-Based Model for Automatic Sleep Stage Classification With PSG

Automatic sleep stage classification plays an essential role in sleep quality measurement and sleep disorder diagnosis. Although many approaches have been developed, most use only single-channel electroencephalogram signals for classification. Polysomnography (PSG) provides multiple channels of signal recording, enabling the use of the appropriate method to extract and integrate the information from different channels to achieve higher sleep staging performance. We present a transformer encoder-based model, MultiChannelSleepNet, for automatic sleep stage classification with multichannel PSG data, whose architecture is implemented based on the transformer encoder for single-channel feature extraction and multichannel feature fusion. In a single-channel feature extraction block, transformer encoders extract features from time-frequency images of each channel independently. Based on our integration strategy, the feature maps extracted from each channel are fused in the multichannel feature fusion block. Another set of transformer encoders further capture joint features, and a residual connection preserves the original information from each channel in this block. Experimental results on three publicly available datasets demonstrate that our method achieves higher classification performance than state-of-the-art techniques. MultiChannelSleepNet is an efficient method to extract and integrate the information from multichannel PSG data, which facilitates precision sleep staging in clinical applications.

A temporal multi-scale hybrid attention network for sleep stage classification

Sleep is crucial for human health. Automatic sleep stage classification based on polysomnogram (PSG) is meaningful for the diagnosis of sleep disorders, which has attracted extensive attention in recent years. Most existing methods could not fully consider the different transitions of sleep stages and fit the visual inspection of sleep experts simultaneously. To this end, we propose a temporal multi-scale hybrid attention network, namely TMHAN, to automatically achieve sleep staging. The temporal multi-scale mechanism incorporates short-term abrupt and long-term periodic transitions of the successive PSG epochs. Furthermore, the hybrid attention mechanism includes 1-D local attention, 2-D global attention, and 2-D contextual sparse multi-head self-attention for three kinds of sequence-level representations. The concatenated representation is subsequently fed into a softmax layer to train an end-to-end model. Experimental results on two benchmark sleep datasets show that TMHAN obtains the best performance compared with several baselines, demonstrating the effectiveness of our model. In general, our work not only provides good classification performance, but also fits the actual sleep staging processes, which makes contribution for the combination of deep learning and sleep medicine.

MSED: A Multi-Modal Sleep Event Detection Model for Clinical Sleep Analysis

Clinical sleep analysis require manual analysis of sleep patterns for correct diagnosis of sleep disorders. However, several studies have shown significant variability in manual scoring of clinically relevant discrete sleep events, such as arousals, leg movements, and sleep disordered breathing (apneas and hypopneas). We investigated whether an automatic method could be used for event detection and if a model trained on all events (joint model) performed better than corresponding event-specific models (single-event models). We trained a deep neural network event detection model on 1653 individual recordings and tested the optimized model on 1000 separate hold-out recordings. F1 scores for the optimized joint detection model were 0.70, 0.63, and 0.62 for arousals, leg movements, and sleep disordered breathing, respectively, compared to 0.65, 0.61, and 0.60 for the optimized single-event models. Index values computed from detected events correlated positively with manual annotations (r2 = 0.73, r2 = 0.77, r2 = 0.78, respectively). We furthermore quantified model accuracy based on temporal difference metrics, which improved overall by using the joint model compared to single-event models. Our automatic model jointly detects arousals, leg movements and sleep disordered breathing events with high correlation with human annotations. Finally, we benchmark against previous state-of-the-art multi-event detection models and found an overall increase in F1 score with our proposed model despite a 97.5% reduction in model size.

SeriesSleepNet: an EEG time series model with partial data augmentation for automatic sleep stage scoring

Introduction: We propose an automatic sleep stage scoring model, referred to as SeriesSleepNet, based on convolutional neural network (CNN) and bidirectional long short-term memory (bi-LSTM) with partial data augmentation. We used single-channel raw electroencephalography signals for automatic sleep stage scoring. Methods: Our framework was focused on time series information, so we applied partial data augmentation to learn the connected time information in small series. In specific, the CNN module learns the time information of one epoch (intra-epoch) whereas the bi-LSTM trains the sequential information between the adjacent epochs (inter-epoch). Note that the input of the bi-LSTM is the augmented CNN output. Moreover, the proposed loss function was used to fine-tune the model by providing additional weights. To validate the proposed framework, we conducted two experiments using the Sleep-EDF and SHHS datasets. Results and Discussion: The results achieved an overall accuracy of 0.87 and 0.84 and overall F1-score of 0.80 and 0.78 and kappa value of 0.81 and 0.78 for five-class classification, respectively. We showed that the SeriesSleepNet was superior to the baselines based on each component in the proposed framework. Our architecture also outperformed the state-of-the-art methods with overall F1-score, accuracy, and kappa value. Our framework could provide information on sleep disorders or quality of sleep to automatically classify sleep stages with high performance.

A Spatial-Temporal Transformer Architecture Using Multi-Channel Signals for Sleep Stage Classification

Sleep stage classification is a fundamental task in diagnosing and monitoring sleep diseases. There are 2 challenges that remain open: (1) Since most methods only rely on input from a single channel, the spatial-temporal relationship of sleep signals has not been fully explored. (2) Lack of sleep data makes models hard to train from scratch. Here, we propose a vision Transformer-based architecture to process multi-channel polysomnogram signals. The method is an end-to-end framework that consists of a spatial encoder, a temporal encoder, and an MLP head classifier. The spatial encoder using a pre-trained Vision Transformer captures spatial information from multiple PSG channels. The temporal encoder utilizing the self-attention mechanism understands transitions between nearby epochs. In addition, we introduce a tailored image generation method to extract features within multi-channel and reshape them for transfer learning. We validate our method on 3 datasets and outperform the state-of-the-art algorithms. Our method fully explores the spatial-temporal relationship among different brain regions and addresses the problem of data insufficiency in clinical environments. Benefiting from reformulating the problem as image classification, the method could be applied to other 1D-signal problems in the future.

Current status and prospects of automatic sleep stages scoring: Review

The scoring of sleep stages is one of the essential tasks in sleep analysis. Since a manual procedure requires considerable human and financial resources, and incorporates some subjectivity, an automated approach could result in several advantages. There have been many developments in this area, and in order to provide a comprehensive overview, it is essential to review relevant recent works and summarise the characteristics of the approaches, which is the main aim of this article. To achieve it, we examined articles published between 2018 and 2022 that dealt with the automated scoring of sleep stages. In the final selection for in-depth analysis, 125 articles were included after reviewing a total of 515 publications. The results revealed that automatic scoring demonstrates good quality (with Cohen's kappa up to over 0.80 and accuracy up to over 90%) in analysing EEG/EEG + EOG + EMG signals. At the same time, it should be noted that there has been no breakthrough in the quality of results using these signals in recent years. Systems involving other signals that could potentially be acquired more conveniently for the user (e.g. respiratory, cardiac or movement signals) remain more challenging in the implementation with a high level of reliability but have considerable innovation capability. In general, automatic sleep stage scoring has excellent potential to assist medical professionals while providing an objective assessment.

Micro SleepNet: efficient deep learning model for mobile terminal real-time sleep staging

The real-time sleep staging algorithm that can perform inference on mobile devices without burden is a prerequisite for closed-loop sleep modulation. However, current deep learning sleep staging models have poor real-time efficiency and redundant parameters. We propose a lightweight and high-performance sleep staging model named Micro SleepNet, which takes a 30-s electroencephalography (EEG) epoch as input, without relying on contextual signals. The model features a one-dimensional group convolution with a kernel size of 1 × 3 and an Efficient Channel and Spatial Attention (ECSA) module for feature extraction and adaptive recalibration. Moreover, the model efficiently performs feature fusion using dilated convolution module and replaces the conventional fully connected layer with Global Average Pooling (GAP). These design choices significantly reduce the total number of model parameters to 48,226, with only approximately 48.95 Million Floating-point Operations per Second (MFLOPs) computation. The proposed model is conducted subject-independent cross-validation on three publicly available datasets, achieving an overall accuracy of up to 83.3%, and the Cohen Kappa is 0.77. Additionally, we introduce Class Activation Mapping (CAM) to visualize the model's attention to EEG waveforms, which demonstrate the model's ability to accurately capture feature waveforms of EEG at different sleep stages. This provides a strong interpretability foundation for practical applications. Furthermore, the Micro SleepNet model occupies approximately 100 KB of memory on the Android smartphone and takes only 2.8 ms to infer one EEG epoch, meeting the real-time requirements of sleep staging tasks on mobile devices. Consequently, our proposed model has the potential to serve as a foundation for accurate closed-loop sleep modulation.

[Automatic sleep staging model based on single channel electroencephalogram signal]

Sleep staging is the basis for solving sleep problems. There's an upper limit for the classification accuracy of sleep staging models based on single-channel electroencephalogram (EEG) data and features. To address this problem, this paper proposed an automatic sleep staging model that mixes deep convolutional neural network (DCNN) and bi-directional long short-term memory network (BiLSTM). The model used DCNN to automatically learn the time-frequency domain features of EEG signals, and used BiLSTM to extract the temporal features between the data, fully exploiting the feature information contained in the data to improve the accuracy of automatic sleep staging. At the same time, noise reduction techniques and adaptive synthetic sampling were used to reduce the impact of signal noise and unbalanced data sets on model performance. In this paper, experiments were conducted using the Sleep-European Data Format Database Expanded and the Shanghai Mental Health Center Sleep Database, and achieved an overall accuracy rate of 86.9% and 88.9% respectively. When compared with the basic network model, all the experimental results outperformed the basic network, further demonstrating the validity of this paper's model, which can provide a reference for the construction of a home sleep monitoring system based on single-channel EEG signals.

An End-to-End Multi-Channel Convolutional Bi-LSTM Network for Automatic Sleep Stage Detection

Sleep stage detection from polysomnography (PSG) recordings is a widely used method of monitoring sleep quality. Despite significant progress in the development of machine-learning (ML)-based and deep-learning (DL)-based automatic sleep stage detection schemes focusing on single-channel PSG data, such as single-channel electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG), developing a standard model is still an active subject of research. Often, the use of a single source of information suffers from data inefficiency and data-skewed problems. Instead, a multi-channel input-based classifier can mitigate the aforementioned challenges and achieve better performance. However, it requires extensive computational resources to train the model, and, hence, a tradeoff between performance and computational resources cannot be ignored. In this article, we aim to introduce a multi-channel, more specifically a four-channel, convolutional bidirectional long short-term memory (Bi-LSTM) network that can effectively exploit spatiotemporal features of data collected from multiple channels of the PSG recording (e.g., EEG Fpz-Cz, EEG Pz-Oz, EOG, and EMG) for automatic sleep stage detection. First, a dual-channel convolutional Bi-LSTM network module has been designed and pre-trained utilizing data from every two distinct channels of the PSG recording. Subsequently, we have leveraged the concept of transfer learning circuitously and have fused two dual-channel convolutional Bi-LSTM network modules to detect sleep stages. In the dual-channel convolutional Bi-LSTM module, a two-layer convolutional neural network has been utilized to extract spatial features from two channels of the PSG recordings. These extracted spatial features are subsequently coupled and given as input at every level of the Bi-LSTM network to extract and learn rich temporal correlated features. Both Sleep EDF-20 and Sleep EDF-78 (expanded version of Sleep EDF-20) datasets are used in this study to evaluate the result. The model that includes an EEG Fpz-Cz + EOG module and an EEG Fpz-Cz + EMG module can classify sleep stage with the highest value of accuracy (ACC), Kappa (Kp), and F1 score (e.g., 91.44%, 0.89, and 88.69%, respectively) on the Sleep EDF-20 dataset. On the other hand, the model consisting of an EEG Fpz-Cz + EMG module and an EEG Pz-Oz + EOG module shows the best performance (e.g., the value of ACC, Kp, and F1 score are 90.21%, 0.86, and 87.02%, respectively) compared to other combinations for the Sleep EDF-78 dataset. In addition, a comparative study with respect to other existing literature has been provided and discussed in order to exhibit the efficacy of our proposed model.

DynamicSleepNet: a multi-exit neural network with adaptive inference time for sleep stage classification

Sleep is an essential human physiological behavior, and the quality of sleep directly affects a person's physical and mental state. In clinical medicine, sleep stage is an important basis for doctors to diagnose and treat sleep disorders. The traditional method of classifying sleep stages requires sleep experts to classify them manually, and the whole process is time-consuming and laborious. In recent years, with the help of deep learning, automatic sleep stage classification has made great progress, especially networks using multi-modal electrophysiological signals, which have greatly improved in terms of accuracy. However, we found that the existing multimodal networks have a large number of redundant calculations in the process of using multiple electrophysiological signals, and the networks become heavier due to the use of multiple signals, and difficult to be used in small devices. To solve these two problems, this paper proposes DynamicSleepNet, a network that can maximize the use of multiple electrophysiological signals and can dynamically adjust between accuracy and efficiency. DynamicSleepNet consists of three effective feature extraction modules (EFEMs) and three classifier modules, each EFEM is connected to a classifier. Each EFEM is able to extract signal features while making the effective features more prominent and the invalid features are suppressed. The samples processed by the EFEM are given to the corresponding classifier for classification, and if the classifier considers the uncertainty of the sample to be below the threshold we set, the sample can be output early without going through the whole network. We validated our model on four datasets. The results show that the highest accuracy of our model outperforms all baselines. With accuracy close to baselines, our model is faster than the baselines by a factor of several to several tens, and the number of parameters of the model is lower or close. The implementation code is available at: https://github.com/Quinella7291/A-Multi-exit-Neural-Network-with-Adaptive-Inference-Time-for-Sleep-Stage-Classification/.

A novel semi-supervised meta learning method for subject-transfer brain-computer interface

The brain-computer interface (BCI) provides a direct communication pathway between the human brain and external devices. However, the models trained for existing subjects perform poorly on new subjects, which is termed the subject calibration problem. In this paper, we propose a semi-supervised meta learning (SSML) method for subject-transfer calibration. The proposed SSML learns a model-agnostic meta learner with existing subjects and then fine-tunes the meta learner in a semi-supervised learning manner, i.e. using a few labelled samples and many unlabelled samples of the target subject for calibration. It is significant for BCI applications in which labelled data are scarce or expensive while unlabelled data are readily available. Three different BCI paradigms are tested: event-related potential detection, emotion recognition and sleep staging. The SSML achieved classification accuracies of 0.95, 0.89 and 0.83 in the benchmark datasets of three paradigms. The runtime complexity of SSML grows linearly as the number of samples of target subject increases so that is possible to apply it in real-time systems. This study is the first attempt to apply semi-supervised model-agnostic meta learning methodology for subject calibration. The experimental results demonstrated the effectiveness and potential of the SSML method for subject-transfer BCI applications.

Simultaneous Sleep Stage and Sleep Disorder Detection from Multimodal Sensors Using Deep Learning

Sleep scoring involves the inspection of multimodal recordings of sleep data to detect potential sleep disorders. Given that symptoms of sleep disorders may be correlated with specific sleep stages, the diagnosis is typically supported by the simultaneous identification of a sleep stage and a sleep disorder. This paper investigates the automatic recognition of sleep stages and disorders from multimodal sensory data (EEG, ECG, and EMG). We propose a new distributed multimodal and multilabel decision-making system (MML-DMS). It comprises several interconnected classifier modules, including deep convolutional neural networks (CNNs) and shallow perceptron neural networks (NNs). Each module works with a different data modality and data label. The flow of information between the MML-DMS modules provides the final identification of the sleep stage and sleep disorder. We show that the fused multilabel and multimodal method improves the diagnostic performance compared to single-label and single-modality approaches. We tested the proposed MML-DMS on the PhysioNet CAP Sleep Database, with VGG16 CNN structures, achieving an average classification accuracy of 94.34% and F1 score of 0.92 for sleep stage detection (six stages) and an average classification accuracy of 99.09% and F1 score of 0.99 for sleep disorder detection (eight disorders). A comparison with related studies indicates that the proposed approach significantly improves upon the existing state-of-the-art approaches.

Sleep Stage Classification in Children Using Self-Attention and Gaussian Noise Data Augmentation

The analysis of sleep stages for children plays an important role in early diagnosis and treatment. This paper introduces our sleep stage classification method addressing the following two challenges: the first is the data imbalance problem, i.e., the highly skewed class distribution with underrepresented minority classes. For this, a Gaussian Noise Data Augmentation (GNDA) algorithm was applied to polysomnography recordings to seek the balance of data sizes for different sleep stages. The second challenge is the difficulty in identifying a minority class of sleep stages, given their short sleep duration and similarities to other stages in terms of EEG characteristics. To overcome this, we developed a DeConvolution- and Self-Attention-based Model (DCSAM) which can inverse the feature map of a hidden layer to the input space to extract local features and extract the correlations between all possible pairs of features to distinguish sleep stages. The results on our dataset show that DCSAM based on GNDA obtains an accuracy of 90.26% and a macro F1-score of 86.51% which are higher than those of our previous method. We also tested DCSAM on a well-known public dataset-Sleep-EDFX-to prove whether it is applicable to sleep data from adults. It achieves a comparable performance to state-of-the-art methods, especially accuracies of 91.77%, 92.54%, 94.73%, and 95.30% for six-stage, five-stage, four-stage, and three-stage classification, respectively. These results imply that our DCSAM based on GNDA has a great potential to offer performance improvements in various medical domains by considering the data imbalance problems and correlations among features in time series data.

Novel methods for elucidating modality importance in multimodal electrophysiology classifiers

Introduction

Multimodal classification is increasingly common in electrophysiology studies. Many studies use deep learning classifiers with raw time-series data, which makes explainability difficult, and has resulted in relatively few studies applying explainability methods. This is concerning because explainability is vital to the development and implementation of clinical classifiers. As such, new multimodal explainability methods are needed.

Methods

In this study, we train a convolutional neural network for automated sleep stage classification with electroencephalogram (EEG), electrooculogram, and electromyogram data. We then present a global explainability approach that is uniquely adapted for electrophysiology analysis and compare it to an existing approach. We present the first two local multimodal explainability approaches. We look for subject-level differences in the local explanations that are obscured by global methods and look for relationships between the explanations and clinical and demographic variables in a novel analysis.

Results

We find a high level of agreement between methods. We find that EEG is globally the most important modality for most sleep stages and that subject-level differences in importance arise in local explanations that are not captured in global explanations. We further show that sex, followed by medication and age, had significant effects upon the patterns learned by the classifier.

Discussion

Our novel methods enhance explainability for the growing field of multimodal electrophysiology classification, provide avenues for the advancement of personalized medicine, yield unique insights into the effects of demographic and clinical variables upon classifiers, and help pave the way for the implementation of multimodal electrophysiology clinical classifiers.

A novel automated robust dual-channel EEG-based sleep scoring system using optimal half-band pair linear-phase biorthogonal wavelet filter bank

Nowadays, the hectic work life of people has led to sleep deprivation. This may further result in sleep-related disorders and adverse physiological conditions. Therefore, sleep study has become an active research area. Sleep scoring is crucial for detecting sleep-related disorders like sleep apnea, insomnia, narcolepsy, periodic leg movement (PLM), and restless leg syndrome (RLS). Sleep is conventionally monitored in a sleep laboratory using polysomnography (PSG) which is the recording of various physiological signals. The traditional sleep stage scoring (SSG) done by professional sleep scorers is a tedious, strenuous, and time-consuming process as it is manual. Hence, developing a machine-learning model for automatic SSG is essential. In this study, we propose an automated SSG approach based on the biorthogonal wavelet filter bank's (BWFB) novel least squares (LS) design. We have utilized a huge Wisconsin sleep cohort (WSC) database in this study. The proposed study is a pioneering work on automatic sleep stage classification using the WSC database, which includes good sleepers and patients suffering from various sleep-related disorders, including apnea, insomnia, hypertension, diabetes, and asthma. To investigate the generalization of the proposed system, we evaluated the proposed model with the following publicly available databases: cyclic alternating pattern (CAP), sleep EDF, ISRUC, MIT-BIH, and the sleep apnea database from St. Vincent's University. This study uses only two unipolar EEG channels, namely O1-M2 and C3-M2, for the scoring. The Hjorth parameters (HP) are extracted from the wavelet subbands (SBS) that are obtained from the optimal BWFB. To classify sleep stages, the HP features are fed to several supervised machine learning classifiers. 12 different datasets have been created to develop a robust model. A total of 12 classification tasks (CT) have been conducted employing various classification algorithms. Our developed model achieved the best accuracy of 83.2% and Cohen's Kappa of 0.7345 to reliably distinguish five sleep stages, using an ensemble bagged tree classifier with 10-fold cross-validation using WSC data. We also observed that our system is either better or competitive with existing state-of-art systems when we tested with the above-mentioned five databases other than WSC. This method yielded promising results using only two EEG channels using a huge WSC database. Our approach is simple and hence, the developed model can be installed in home-based clinical systems and wearable devices for sleep scoring.

Analysis of the relationship among land surface temperature (LST), land use land cover (LULC), and normalized difference vegetation index (NDVI) with topographic elements in the lower Himalayan region

Land Surface Temperature (LST) affects exchange of energy between earth surface and atmosphere which is important for studying environmental changes. However, research on the relationship between LST, Land Use Land Cover (LULC), and Normalized Difference Vegetation Index (NDVI) with topographic elements in the lower Himalayan region has not been done. Therefore, the present study explored the relationship between LST and NDVI, and LULC types with topographic elements in the lower Himalayan region of Pakistan. The study area was divided into North-South, West-East, North-West to South-East and North-East to South-East directions using ArcMap 3D analysis. The current study used Landsat 8 (OLI/TIRS) data from May 2021 for LULC and LST analysis in the study area. The LST data was obtained from the thermal band of Landsat 8 (TIRS), while the LULC of the study areas was classified using the Maximum Likelihood Classification (MLC) method utilizing Landsat 8 (OLI) data. TIRS collects data for two narrow spectral bands (B10 and B11) with spectral wavelength of 10.6 μm-12.51 μm in the thermal region formerly covered by one wide spectral band (B6) on Landsat 4-7. With 12-bit data products, TIRS data is available in radiometric, geometric, and terrain-corrected file format. The effect of elevation on LST was assessed using LST and elevation data obtained from the USGS website. The LST across LULC types with sunny and shady slopes was analyzed to assess the influence of slope directions. The relationship of LST with elevation and NDVI was examined using correlation analysis. The results indicated that LST decreased from North-South and South-East, while increasing from North-East and South-West directions. The correlation coefficient between LST and elevation was negative, with an R-value of -0.51. The NDVI findings with elevation showed that NDVI increases with an increase in elevation. Zonal analysis of LST for different LULC types showed that built-up and bare soil had the highest mean LST, which was 35.76 °C and 28.08 °C, respectively, followed by agriculture, vegetation, and water bodies. The mean LST difference between sunny and shady slopes was 1.02 °C. The correlation between NDVI and LST was negative for all LULC types except the water body. This study findings can be used to ensure sustainable urban development and minimize urban heat island effects by providing effective guidelines for urban planners, policymakers, and respective authorities in the Lower Himalayan region. The current thermal remote sensing findings can be used to model energy fluxes and surface processes in the study area.

A Siamese Network-Based Method for Improving the Performance of Sleep Staging with Single-Channel EEG

Sleep staging is of critical significance to the diagnosis of sleep disorders, and the electroencephalogram (EEG), which is used for monitoring brain activity, is commonly employed in sleep staging. In this paper, we propose a novel method for improving the performance of sleep staging models based on Siamese networks, based on single-channel EEG. Our proposed method consists of a Siamese network architecture and a redesigned loss with distance metrics. Two encoders are used in the Siamese network to generate latent features of the EEG epochs, and the contrastive loss, which is also a distance metric, is used to compare the similarity or differences between EEG epochs from the same or different sleep stages. We evaluated our method on single-channel EEGs from different channels (Fpz-Cz and F4-EOG (left)) from two public datasets SleepEDF and MASS-SS3 and achieved the overall accuracies MF1 and Cohen's kappa coefficient of 85.2%, 78.3% and 0.79 on SleepEDF and 87.2%, 82.1% and 0.81 on MASS-SS3. The results show that our method can significantly improve the performance of sleep staging models and outperform the state-of-the-art sleep staging methods. The performance of our method also confirms that the features captured by Siamese networks and distance metrics are useful for sleep staging.

Sleep Staging Framework with Physiologically Harmonized Sub-Networks

Sleep screening is an important tool for both healthcare and neuroscientific research. Automatic sleep scoring is an alternative to the time-consuming gold-standard manual scoring procedure. Recently there have seen promising results on automatic stage scoring by extracting spatio-temporal features via deep neural networks from electroencephalogram (EEG). However, such methods fail to consistently yield good performance due to a missing piece in data representation: the medical criterion of the sleep scoring task on top of EEG features. We argue that capturing stage-specific features that satisfy the criterion of sleep medicine is non-trivial for automatic sleep scoring. This paper considers two criteria: Transient stage marker and Overall profile of EEG features, then we propose a physiologically meaningful framework for sleep stage scoring via mixed deep neural networks. The framework consists of two sub-networks: feature extraction networks, constructed in consideration of the physiological characteristics of sleep, and an attention-based scoring decision network. Moreover, we quantize the framework for potential use under an IoT setting. For proof-of-concept, the performance of the proposed framework is demonstrated by introducing multiple sleep datasets with the largest comprising 42,560 h recorded from 5,793 subjects. From the experiment results, the proposed method achieves a competitive stage scoring performance, especially for Wake, N2, and N3, with higher F1 scores of 0.92, 0.86, and 0.88, respectively. Moreover, the feasibility analysis of framework quantization provides a potential for future implementation in the edge computing field and clinical settings.

Deep learning-based algorithm accurately classifies sleep stages in preadolescent children with sleep-disordered breathing symptoms and age-matched controls

Introduction

Visual sleep scoring has several shortcomings, including inter-scorer inconsistency, which may adversely affect diagnostic decision-making. Although automatic sleep staging in adults has been extensively studied, it is uncertain whether such sophisticated algorithms generalize well to different pediatric age groups due to distinctive EEG characteristics. The preadolescent age group (10-13-year-olds) is relatively understudied, and thus, we aimed to develop an automatic deep learning-based sleep stage classifier specifically targeting this cohort.

Methods

A dataset (n = 115) containing polysomnographic recordings of Icelandic preadolescent children with sleep-disordered breathing (SDB) symptoms, and age and sex-matched controls was utilized. We developed a combined convolutional and long short-term memory neural network architecture relying on electroencephalography (F4-M1), electrooculography (E1-M2), and chin electromyography signals. Performance relative to human scoring was further evaluated by analyzing intra- and inter-rater agreements in a subset (n = 10) of data with repeat scoring from two manual scorers.

Results

The deep learning-based model achieved an overall cross-validated accuracy of 84.1% (Cohen's kappa κ = 0.78). There was no meaningful performance difference between SDB-symptomatic (n = 53) and control subgroups (n = 52) [83.9% (κ = 0.78) vs. 84.2% (κ = 0.78)]. The inter-rater reliability between manual scorers was 84.6% (κ = 0.78), and the automatic method reached similar agreements with scorers, 83.4% (κ = 0.76) and 82.7% (κ = 0.75).

Conclusion

The developed algorithm achieved high classification accuracy and substantial agreements with two manual scorers; the performance metrics compared favorably with typical inter-rater reliability between manual scorers and performance reported in previous studies. These suggest that our algorithm may facilitate less labor-intensive and reliable automatic sleep scoring in preadolescent children.

SleepXAI: An explainable deep learning approach for multi-class sleep stage identification

Extensive research has been conducted on the automatic classification of sleep stages utilizing deep neural networks and other neurophysiological markers. However, for sleep specialists to employ models as an assistive solution, it is necessary to comprehend how the models arrive at a particular outcome, necessitating the explainability of these models. This work proposes an explainable unified CNN-CRF approach (SleepXAI) for multi-class sleep stage classification designed explicitly for univariate time-series signals using modified gradient-weighted class activation mapping (Grad-CAM). The proposed approach significantly increases the overall accuracy of sleep stage classification while demonstrating the explainability of the multi-class labeling of univariate EEG signals, highlighting the parts of the signals emphasized most in predicting sleep stages. We extensively evaluated our approach to the sleep-EDF dataset, and it demonstrates the highest overall accuracy of 86.8% in identifying five sleep stage classes. More importantly, we achieved the highest accuracy when classifying the crucial sleep stage N1 with the lowest number of instances, outperforming the state-of-the-art machine learning approaches by 16.3%. These results motivate us to adopt the proposed approach in clinical practice as an aid to sleep experts.