A deep learning algorithm for sleep stage scoring in mice based on a multimodal network with fine-tuning technique

Sleep-Deep-Learner is taught sleep-wake scoring by the end-user to complete each record in their style

Sleep-wake scoring is a time-consuming, tedious but essential component of clinical and preclinical sleep research. Sleep scoring is even more laborious and challenging in rodents due to the smaller EEG amplitude differences between states and the rapid state transitions which necessitate scoring in shorter epochs. Although many automated rodent sleep scoring methods exist, they do not perform as well when scoring new datasets, especially those which involve changes in the EEG/EMG profile. Thus, manual scoring by expert scorers remains the gold standard. Here we take a different approach to this problem by using a neural network to accelerate the scoring of expert scorers. Sleep-Deep-Learner creates a bespoke deep convolution neural network model for individual electroencephalographic or local-field-potential (LFP) records via transfer learning of GoogLeNet, by learning from a small subset of manual scores of each EEG/LFP record as provided by the end-user. Sleep-Deep-Learner then automates scoring of the remainder of the EEG/LFP record. A novel REM sleep scoring correction procedure further enhanced accuracy. Sleep-Deep-Learner reliably scores EEG and LFP data and retains sleep-wake architecture in wild-type mice, in sleep induced by the hypnotic zolpidem, in a mouse model of Alzheimer's disease and in a genetic knock-down study, when compared to manual scoring. Sleep-Deep-Learner reduced manual scoring time to 1/12. Since Sleep-Deep-Learner uses transfer learning on each independent recording, it is not biased by previously scored existing datasets. Thus, we find Sleep-Deep-Learner performs well when used on signals altered by a drug, disease model, or genetic modification.

Sleep-Deep-Net learns sleep wake scoring from the end-user and completes each record in their style

Sleep-wake scoring is a time-consuming, tedious but essential component of clinical and pre-clinical sleep research. Sleep scoring is even more laborious and challenging in rodents due to the smaller EEG amplitude differences between states and the rapid state transitions which necessitate scoring in shorter epochs. Although many automated rodent sleep scoring methods exist, they do not perform as well when scoring new data sets, especially those which involve changes in the EEG/EMG profile. Thus, manual scoring by expert scorers remains the gold-standard. Here we take a different approach to this problem by using a neural network to accelerate the scoring of expert scorers. Sleep-Deep-Net (SDN) creates a bespoke deep convolution neural network model for individual electroencephalographic or local-field-potential records via transfer learning of GoogleNet, by learning from a small subset of manual scores of each EEG/LFP record as provided by the end-user. SDN then automates scoring of the remainder of the EEG/LFP record. A novel REM scoring correction procedure further enhanced accuracy. SDN reliably scores EEG and LFP data and retains sleep-wake architecture in wild-type mice, in sleep induced by the hypnotic zolpidem, in a mouse model of Alzheimer's disease and in a genetic knock-down study, when compared to manual scoring. SDN reduced manual scoring time to 1/12. Since SDN uses transfer learning on each independent recording, it is not biased by previously scored existing data sets. Thus, we find SDN performs well when used on signals altered by a drug, disease model or genetic modification.

A novel machine learning system for identifying sleep-wake states in mice

Research into sleep-wake behaviors relies on scoring sleep states, normally done by manual inspection of electroencephalogram (EEG) and electromyogram (EMG) recordings. This is a highly time-consuming process prone to inter-rater variability. When studying relationships between sleep and motor function, analyzing arousal states under a four-state system of active wake (AW), quiet wake (QW), nonrapid-eye-movement (NREM) sleep, and rapid-eye-movement (REM) sleep provides greater precision in behavioral analysis but is a more complex model for classification than the traditional three-state identification (wake, NREM, and REM sleep) usually used in rodent models. Characteristic features between sleep-wake states provide potential for the use of machine learning to automate classification. Here, we devised SleepEns, which uses a novel ensemble architecture, the time-series ensemble. SleepEns achieved 90% accuracy to the source expert, which was statistically similar to the performance of two other human experts. Considering the capacity for classification disagreements that are still physiologically reasonable, SleepEns had an acceptable performance of 99% accuracy, as determined blindly by the source expert. Classifications given by SleepEns also maintained similar sleep-wake characteristics compared to expert classifications, some of which were essential for sleep-wake identification. Hence, our approach achieves results comparable to human ability in a fraction of the time. This new machine-learning ensemble will significantly impact the ability of sleep researcher to detect and study sleep-wake behaviors in mice and potentially in humans.

IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm

Machine learning has been applied in recent years to categorize sleep stages (NREM, REM, and wake) using electroencephalogram (EEG) recordings; however, a well-validated sleep scoring automatic pipeline in rodent research is still not publicly available. Here, we present IntelliSleepScorer, a software package with a graphic user interface to score sleep stages automatically in mice. IntelliSleepScorer uses the light gradient boosting machine (LightGBM) to score sleep stages for each epoch of recordings. We developed LightGBM models using a large cohort of data, which consisted of 5776 h of sleep EEG and electromyogram (EMG) signals across 519 unique recordings from 124 mice. The LightGBM model achieved an overall accuracy of 95.2% and a Cohen's kappa of 0.91, which outperforms the baseline models such as the logistic regression model (accuracy = 93.3%, kappa = 0.88) and the random forest model (accuracy = 94.3%, kappa = 0.89). The overall performance of the LightGBM model as well as the performance across different sleep stages are on par with that of the human experts. Most importantly, we validated the generalizability of the LightGBM models: (1) The LightGBM model performed well on two publicly available, independent datasets (kappa >  = 0.80), which have different sampling frequency and epoch lengths; (2) The LightGBM model performed well on data recorded at a lower sampling frequency (kappa = 0.90); (3) The performance of the LightGBM model is not affected by the light/dark cycle; and (4) A modified LightGBM model performed well on data containing only one EEG and one EMG electrode (kappa >  = 0.89). Taken together, the LightGBM models offer state-of-the-art performance for automatic sleep stage scoring in mice. Last, we implemented the IntelliSleepScorer software package based on the validated model to provide an out-of-box solution to sleep researchers (available for download at https://sites.broadinstitute.org/pan-lab/resources ).

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.

Optimization of real-time analysis of sleep-wake cycle in mice

Studying the biology of sleep requires accurate and efficient assessment of the sleep stages. However, analysis of sleep-wake cycles in mice and other laboratory animals remains a time-consuming and laborious process. In this study, we developed a Python script and a process for the streamlined analysis of sleep data that includes real-time processing of electroencephalogram (EEG) and electromyogram (EMG) signals that is compatible with commercial sleep-recording software that supports user datagram protocol (UDP) communication. The process consists of EEG/EMG data acquisition, automated threshold calculation for real-time determination of sleep stages, sleep staging and EEG power spectrum analysis. It also allows data storage in the format that facilitates further analysis of the sleep pattern in mice. The described method is aimed at increasing efficiency of sleep stage scoring and analysis in mice thus facilitating sleep research. • A process of EEG/EMG recording and streamline analysis of sleep-wake cycle in real time in mice. • The compatibility with commercial sleep-recording software that can generate a UDP stream. • The capability of further analysis of recorded data by an open-source software.