Sleep-stage scoring in the rat using a support vector machine

Automatic sleep-wake classification and Parkinson's disease recognition using multifeature fusion with support vector machine

AIMS

Sleep disturbance is a prevalent nonmotor symptom of Parkinson's disease (PD), however, assessing sleep conditions is always time-consuming and labor-intensive. In this study, we performed an automatic sleep-wake state classification and early diagnosis of PD by analyzing the electrocorticography (ECoG) and electromyogram (EMG) signals of both normal and PD rats.

METHODS

The study utilized ECoG power, EMG amplitude, and corticomuscular coherence values extracted from normal and PD rats to construct sleep-wake scoring models based on the support vector machine algorithm. Subsequently, we incorporated feature values that could act as diagnostic markers for PD and then retrained the models, which could encompass the identification of vigilance states and the diagnosis of PD.

RESULTS

Features extracted from occipital ECoG signals were more suitable for constructing sleep-wake scoring models than those from frontal ECoG (average Cohen's kappa: 0.73 vs. 0.71). Additionally, after retraining, the new models demonstrated increased sensitivity to PD and accurately determined the sleep-wake states of rats (average Cohen's kappa: 0.79).

CONCLUSION

This study accomplished the precise detection of substantia nigra lesions and the monitoring of sleep-wake states. The integration of circadian rhythm monitoring and disease state assessment has the potential to improve the efficacy of therapeutic strategies considerably.

Sleep scoring in rodents: Criteria, automatic approaches and outstanding issues

There is nothing we spend as much time on in our lives as we do sleeping, which makes it even more surprising that we currently do not know why we need to sleep. Most of the research addressing this question is performed in rodents to allow for invasive, mechanistic approaches. However, in contrast to human sleep, we currently do not have shared and agreed upon standards on sleep states in rodents. In this article, we present an overview on sleep stages in humans and rodents and a historical perspective on the development of automatic sleep scoring systems in rodents. Further, we highlight specific issues in rodent sleep that also call into question some of the standards used in human sleep research.

Somnotate: A probabilistic sleep stage classifier for studying vigilance state transitions

Electrophysiological recordings from freely behaving animals are a widespread and powerful mode of investigation in sleep research. These recordings generate large amounts of data that require sleep stage annotation (polysomnography), in which the data is parcellated according to three vigilance states: awake, rapid eye movement (REM) sleep, and non-REM (NREM) sleep. Manual and current computational annotation methods ignore intermediate states because the classification features become ambiguous, even though intermediate states contain important information regarding vigilance state dynamics. To address this problem, we have developed "Somnotate"-a probabilistic classifier based on a combination of linear discriminant analysis (LDA) with a hidden Markov model (HMM). First we demonstrate that Somnotate sets new standards in polysomnography, exhibiting annotation accuracies that exceed human experts on mouse electrophysiological data, remarkable robustness to errors in the training data, compatibility with different recording configurations, and an ability to maintain high accuracy during experimental interventions. However, the key feature of Somnotate is that it quantifies and reports the certainty of its annotations. We leverage this feature to reveal that many intermediate vigilance states cluster around state transitions, whereas others correspond to failed attempts to transition. This enables us to show for the first time that the success rates of different types of transition are differentially affected by experimental manipulations and can explain previously observed sleep patterns. Somnotate is open-source and has the potential to both facilitate the study of sleep stage transitions and offer new insights into the mechanisms underlying sleep-wake dynamics.

A novel machine learning system for identifying sleep-wake states in mice

Research into sleep-wake behaviors relies on scoring sleep states, normally done by manual inspection of electroencephalogram (EEG) and electromyogram (EMG) recordings. This is a highly time-consuming process prone to inter-rater variability. When studying relationships between sleep and motor function, analyzing arousal states under a four-state system of active wake (AW), quiet wake (QW), nonrapid-eye-movement (NREM) sleep, and rapid-eye-movement (REM) sleep provides greater precision in behavioral analysis but is a more complex model for classification than the traditional three-state identification (wake, NREM, and REM sleep) usually used in rodent models. Characteristic features between sleep-wake states provide potential for the use of machine learning to automate classification. Here, we devised SleepEns, which uses a novel ensemble architecture, the time-series ensemble. SleepEns achieved 90% accuracy to the source expert, which was statistically similar to the performance of two other human experts. Considering the capacity for classification disagreements that are still physiologically reasonable, SleepEns had an acceptable performance of 99% accuracy, as determined blindly by the source expert. Classifications given by SleepEns also maintained similar sleep-wake characteristics compared to expert classifications, some of which were essential for sleep-wake identification. Hence, our approach achieves results comparable to human ability in a fraction of the time. This new machine-learning ensemble will significantly impact the ability of sleep researcher to detect and study sleep-wake behaviors in mice and potentially in humans.

An artificial neural network for automated behavioral state classification in rats

Accurate behavioral state classification is critical for many research applications. Researchers typically rely upon manual identification of behavioral state through visual inspection of electrophysiological signals, but this approach is time intensive and subject to low inter-rater reliability. To overcome these limitations, a diverse set of algorithmic approaches have been put forth to automate the classification process. Recently, novel machine learning approaches have been detailed that produce rapid and highly accurate classifications. These approaches however, are often computationally expensive, require significant expertise to implement, and/or require proprietary software that limits broader adoption. Here we detail a novel artificial neural network that uses electrophysiological features to automatically classify behavioral state in rats with high accuracy, sensitivity, and specificity. Common parameters of interest to sleep scientists, including state-dependent power spectra and homeostatic non-REM slow wave activity, did not significantly differ when using this automated classifier as compared to manual scoring. Flexible options enable researchers to further increase classification accuracy through manual rescoring of a small subset of time intervals with low model prediction certainty or further decrease researcher time by generalizing trained networks across multiple recording days. The algorithm is fully open-source and coded within a popular, and freely available, software platform to increase access to this research tool and provide additional flexibility for future researchers. In sum, we have developed a readily implementable, efficient, and effective approach for automated behavioral state classification in rats.

Machine learning prediction of sleep stages in dairy cows from heart rate and muscle activity measures

Sleep is important for cow health and shows promise as a tool for assessing welfare, but methods to accurately distinguish between important sleep stages are difficult and impractical to use with cattle in typical farm environments. The objective of this study was to determine if data from more easily applied non-invasive devices assessing neck muscle activity and heart rate (HR) alone could be used to differentiate between sleep stages. We developed, trained, and compared two machine learning models using neural networks and random forest algorithms to predict sleep stages from 15 variables (features) of the muscle activity and HR data collected from 12 cows in two environments. Using k-fold cross validation we compared the success of the models to the gold standard, Polysomnography (PSG). Overall, both models learned from the data and were able to accurately predict sleep stages from HR and muscle activity alone with classification accuracy in the range of similar human models. Further research is required to validate the models with a larger sample size, but the proposed methodology appears to give an accurate representation of sleep stages in cattle and could consequentially enable future sleep research into conditions affecting cow sleep and welfare.

Identification of the sleep-wake states in rats using the high-frequency activity of the electroencephalogram

The electroencephalographic signal constitutes the main sign classically used for the identification of states of alertness. However, activities in the high frequency (>100 Hz) range have not been properly studied despite their high potential for sleep scoring in rodents. In the present study, we designed a method for the identification of the sleep-wake states in rats by exclusively using high-frequency activities of the electroencephalogram. By calculating the ratio between the amplitude of the electroencephalographic signal from 110 to 200 Hz and from 110 to 300 Hz, we obtained an index that had values that were low during wakefulness, intermediate during non-REM sleep and high during REM sleep. This high-frequency index (HiFI) allowed the identification of each state without the need to study other signs such as muscle activity or eye movements. To evaluate the performance of the index, we compared it with the conventional scoring of the sleep-wake cycle based upon the study of the electromyogram and delta (0.5-4 Hz), theta (6-9 Hz) and sigma (10-14 Hz) bands of the electroencephalogram. The index had an accuracy of 90.43 ± 1.91% (Cohen's kappa value of 0.82), confirming that the study of the high-frequency activities of the electroencephalogram was sufficient to reliably identify alertness states in the rat. Compared to other sleep-scoring methods, the HiFI has several advantages. It only requires one electroencephalography electrode, thus reducing the severity of the surgical preparation of the experimental animal, and its calculation is very simple, so it can be easily implemented online to classify sleep-wake states in real time.

Robust, automated sleep scoring by a compact neural network with distributional shift correction

Studying the biology of sleep requires the accurate assessment of the state of experimental subjects, and manual analysis of relevant data is a major bottleneck. Recently, deep learning applied to electroencephalogram and electromyogram data has shown great promise as a sleep scoring method, approaching the limits of inter-rater reliability. As with any machine learning algorithm, the inputs to a sleep scoring classifier are typically standardized in order to remove distributional shift caused by variability in the signal collection process. However, in scientific data, experimental manipulations introduce variability that should not be removed. For example, in sleep scoring, the fraction of time spent in each arousal state can vary between control and experimental subjects. We introduce a standardization method, mixture z-scoring, that preserves this crucial form of distributional shift. Using both a simulated experiment and mouse in vivo data, we demonstrate that a common standardization method used by state-of-the-art sleep scoring algorithms introduces systematic bias, but that mixture z-scoring does not. We present a free, open-source user interface that uses a compact neural network and mixture z-scoring to allow for rapid sleep scoring with accuracy that compares well to contemporary methods. This work provides a set of computational tools for the robust automation of sleep scoring.

Development of a rule-based automatic five-sleep-stage scoring method for rats

Background

Sleep problem or disturbance often exists in pain or neurological/psychiatric diseases. However, sleep scoring is a time-consuming tedious labor. Very few studies discuss the 5-stage (wake/NREM1/NREM2/transition sleep/REM) automatic fine analysis of wake–sleep stages in rodent models. The present study aimed to develop and validate an automatic rule-based classification of 5-stage wake–sleep pattern in acid-induced widespread hyperalgesia model of the rat.

Results

The overall agreement between two experts’ consensus and automatic scoring in the 5-stage and 3-stage analyses were 92.32% (κ = 0.88) and 94.97% (κ = 0.91), respectively. Standard deviation of the accuracy among all rats was only 2.93%. Both frontal–occipital EEG and parietal EEG data showed comparable accuracies. The results demonstrated the performance of the proposed method with high accuracy and reliability. Subtle changes exhibited in the 5-stage wake–sleep analysis but not in the 3-stage analysis during hyperalgesia development of the acid-induced pain model. Compared with existing methods, our method can automatically classify vigilance states into 5-stage or 3-stage wake–sleep pattern with a promising high agreement with sleep experts.

Conclusions

In this study, we have performed and validated a reliable automated sleep scoring system in rats. The classification algorithm is less computation power, a high robustness, and consistency of results. The algorithm can be implanted into a versatile wireless portable monitoring system for real-time analysis in the future.

Validation of 'Somnivore', a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data

Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, Somnivore^TM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.

Harnessing olfactory bulb oscillations to perform fully brain-based sleep-scoring and real-time monitoring of anaesthesia depth

Real-time tracking of vigilance states related to both sleep or anaesthesia has been a goal for over a century. However, sleep scoring cannot currently be performed with brain signals alone, despite the deep neuromodulatory transformations that accompany sleep state changes. Therefore, at heart, the operational distinction between sleep and wake is that of immobility and movement, despite numerous situations in which this one-to-one mapping fails. Here we demonstrate, using local field potential (LFP) recordings in freely moving mice, that gamma (50–70 Hz) power in the olfactory bulb (OB) allows for clear classification of sleep and wake, thus providing a brain-based criterion to distinguish these two vigilance states without relying on motor activity. Coupled with hippocampal theta activity, it allows the elaboration of a sleep scoring algorithm that relies on brain activity alone. This method reaches over 90% homology with classical methods based on muscular activity (electromyography [EMG]) and video tracking. Moreover, contrary to EMG, OB gamma power allows correct discrimination between sleep and immobility in ambiguous situations such as fear-related freezing. We use the instantaneous power of hippocampal theta oscillation and OB gamma oscillation to construct a 2D phase space that is highly robust throughout time, across individual mice and mouse strains, and under classical drug treatment. Dynamic analysis of trajectories within this space yields a novel characterisation of sleep/wake transitions: whereas waking up is a fast and direct transition that can be modelled by a ballistic trajectory, falling asleep is best described as a stochastic and gradual state change. Finally, we demonstrate that OB oscillations also allow us to track other vigilance states. Non-REM (NREM) and rapid eye movement (REM) sleep can be distinguished with high accuracy based on beta (10–15 Hz) power. More importantly, we show that depth of anaesthesia can be tracked in real time using OB gamma power. Indeed, the gamma power predicts and anticipates the motor response to stimulation both in the steady state under constant anaesthetic and dynamically during the recovery period. Altogether, this methodology opens the avenue for multi-timescale characterisation of brain states and provides an unprecedented window onto levels of vigilance.

Real-time tracking of vigilance states related to wake, sleep, and anaesthesia has been a goal for over a century. However identification of wakefulness and different sleep states cannot currently be performed routinely with brain signals and instead relies on motor activity. Here we demonstrate that 50–70 Hz electrical oscillations in the olfactory bulb (OB) of mice are a reliable indicator for global brain states. Recording this activity with an implanted electrode allows for clear classification of sleep and wake, without the need for motor activity monitoring. We construct a fully automatic sleep scoring algorithm that relies on brain activity alone and is robust throughout time, between animals, and after drug administration. Our method also tracks in real time the depth of anaesthesia both in the steady state under constant anaesthetic and dynamically during the recovery period from anaesthesia. Furthermore, this index predicts responsiveness to noxious stimulation under anaesthesia. Altogether, this methodology opens the avenue for characterisation of vigilance states based on OB recordings.

Integrating artificial intelligence with real-time intracranial EEG monitoring to automate interictal identification of seizure onset zones in focal epilepsy

OBJECTIVE

An ability to map seizure-generating brain tissue, i.e. the seizure onset zone (SOZ), without recording actual seizures could reduce the duration of invasive EEG monitoring for patients with drug-resistant epilepsy. A widely-adopted practice in the literature is to compare the incidence (events/time) of putative pathological electrophysiological biomarkers associated with epileptic brain tissue with the SOZ determined from spontaneous seizures recorded with intracranial EEG, primarily using a single biomarker. Clinical translation of the previous efforts suffers from their inability to generalize across multiple patients because of (a) the inter-patient variability and (b) the temporal variability in the epileptogenic activity.

APPROACH

Here, we report an artificial intelligence-based approach for combining multiple interictal electrophysiological biomarkers and their temporal characteristics as a way of accounting for the above barriers and show that it can reliably identify seizure onset zones in a study cohort of 82 patients who underwent evaluation for drug-resistant epilepsy.

MAIN RESULTS

Our investigation provides evidence that utilizing the complementary information provided by multiple electrophysiological biomarkers and their temporal characteristics can significantly improve the localization potential compared to previously published single-biomarker incidence-based approaches, resulting in an average area under ROC curve (AUC) value of 0.73 in a cohort of 82 patients. Our results also suggest that recording durations between 90 min and 2 h are sufficient to localize SOZs with accuracies that may prove clinically relevant.

SIGNIFICANCE

The successful validation of our approach on a large cohort of 82 patients warrants future investigation on the feasibility of utilizing intra-operative EEG monitoring and artificial intelligence to localize epileptogenic brain tissue. Broadly, our study demonstrates the use of artificial intelligence coupled with careful feature engineering in augmenting clinical decision making.

Tracking wakefulness as it fades: Micro-measures of alertness

A major problem in psychology and physiology experiments is drowsiness: around a third of participants show decreased wakefulness despite being instructed to stay alert. In some non-visual experiments participants keep their eyes closed throughout the task, thus promoting the occurrence of such periods of varying alertness. These wakefulness changes contribute to systematic noise in data and measures of interest. To account for this omnipresent problem in data acquisition we defined criteria and code to allow researchers to detect and control for varying alertness in electroencephalography (EEG) experiments under eyes-closed settings. We first revise a visual-scoring method developed for detection and characterization of the sleep-onset process, and adapt the same for detection of alertness levels. Furthermore, we show the major issues preventing the practical use of this method, and overcome these issues by developing an automated method (micro-measures algorithm) based on frequency and sleep graphoelements, which are capable of detecting micro variations in alertness. The validity of the micro-measures algorithm was verified by training and testing using a dataset where participants are known to fall asleep. In addition, we tested generalisability by independent validation on another dataset. The methods developed constitute a unique tool to assess micro variations in levels of alertness and control trial-by-trial retrospectively or prospectively in every experiment performed with EEG in cognitive neuroscience under eyes-closed settings.

Sleep-stage scoring in mice: The influence of data pre-processing on a system's performance

Sleep-stage analysis in mice and rats has received growing attention in recent years, due to the fact that mice display electrical activity during sleep which has underlying similarities with that of human sleep. Both conventional manual and automatic sleep-wakefulness scoring are rule based tasks which use brain waves measured by Electroencephalogram (EEG) and activity detected by Electromyography (EMG) of skeletal muscles. Several works have been conducted trying to provide an automatic sleep-scoring system on the basis of machine learning methods. In this study we try to understand the reasons behind the complexity of this problem and we emphasize the importance of normalization procedure that leads to a better stage discrimination comparing different classification methods.

Automatic detection of periods of slow wave sleep based on intracranial depth electrode recordings

BACKGROUND

An automated process for sleep staging based on intracranial EEG data alone is needed to facilitate research into the neural processes occurring during slow wave sleep (SWS). Current manual methods for sleep scoring require a full polysomnography (PSG) set-up, including electrooculography (EOG), electromyography (EMG), and scalp electroencephalography (EEG). This set-up can be technically difficult to place in the presence of intracranial EEG electrodes. There is thus a need for a method for sleep staging based on intracranial recordings alone.

NEW METHOD

Here we show a reliable automated method for the detection of periods of SWS solely based on intracranial EEG recordings. The method utilizes the ratio of spectral power in delta, theta, and spindle frequencies relative to alpha and beta frequencies to classify 30-s segments as SWS or not.

RESULTS

We evaluated this new method by comparing its performance against visually scored patients (n=9), in which we also recorded EOG and EMG simultaneously. Our method had a mean positive predictive value of 64% across all nights. Also, an ROC analysis of the performance of our algorithm compared to manually labeled nights revealed a mean average area under the curve of 0.91 across all nights.

COMPARISON WITH EXISTING METHOD

Our method had an average kappa score of 0.72 when compared to visual sleep scoring by an independent blinded sleep scorer.

CONCLUSION

This shows that this simple method is capable of differentiating between SWS and non-SWS epochs reliably based solely on intracranial EEG recordings.

Electronic Sleep Stage Classifiers: A Survey and VLSI Design Methodology

First, existing sleep stage classifier sensors and algorithms are reviewed and compared in terms of classification accuracy, level of automation, implementation complexity, invasiveness, and targeted application. Next, the implementation of a miniature microsystem for low-latency automatic sleep stage classification in rodents is presented. The classification algorithm uses one EMG (electromyogram) and two EEG (electroencephalogram) signals as inputs in order to detect REM (rapid eye movement) sleep, and is optimized for low complexity and low power consumption. It is implemented in an on-board low-power FPGA connected to a multi-channel neural recording IC, to achieve low-latency (order of 1 ms or less) classification. Off-line experimental results using pre-recorded signals from nine mice show REM detection sensitivity and specificity of 81.69% and 93.86%, respectively, with the maximum latency of 39 [Formula: see text]. The device is designed to be used in a non-disruptive closed-loop REM sleep suppression microsystem, for future studies of the effects of REM sleep deprivation on memory consolidation.

Accurate discrimination of the wake-sleep states of mice using non-invasive whole-body plethysmography

A major limitation in the study of sleep breathing disorders in mouse models of pathology is the need to combine whole-body plethysmography (WBP) to measure respiration with electroencephalography/electromyography (EEG/EMG) to discriminate wake-sleep states. However, murine wake-sleep states may be discriminated from breathing and body movements registered by the WBP signal alone. Our goal was to compare the EEG/EMG-based and the WBP-based scoring of wake-sleep states of mice, and provide formal guidelines for the latter. EEG, EMG, blood pressure and WBP signals were simultaneously recorded from 20 mice. Wake-sleep states were scored based either on EEG/EMG or on WBP signals and sleep-dependent respiratory and cardiovascular estimates were calculated. We found that the overall agreement between the 2 methods was 90%, with a high Cohen's Kappa index (0.82). The inter-rater agreement between 2 experts and between 1 expert and 1 naïve sleep investigators gave similar results. Sleep-dependent respiratory and cardiovascular estimates did not depend on the scoring method. We show that non-invasive discrimination of the wake-sleep states of mice based on visual inspection of the WBP signal is accurate, reliable and reproducible. This work may set the stage for non-invasive high-throughput experiments evaluating sleep and breathing patterns on mouse models of pathophysiology.

Predicting postoperative vomiting among orthopedic patients receiving patient-controlled epidural analgesia using SVM and LR

Patient-controlled epidural analgesia (PCEA) has been applied to reduce postoperative pain in orthopedic surgical patients. Unfortunately, PCEA is occasionally accompanied by nausea and vomiting. The logistic regression (LR) model is widely used to predict vomiting, and recently support vector machines (SVM), a supervised machine learning method, has been used for classification and prediction. Unlike our previous work which compared Artificial Neural Networks (ANNs) with LR, this study uses a SVM-based predictive model to identify patients with high risk of vomiting during PCEA and comparing results with those derived from the LR-based model. From January to March 2007, data from 195 patients undergoing PCEA following orthopedic surgery were applied to develop two predictive models. 75% of the data were randomly selected for training, while the remainder was used for testing to validate predictive performance. The area under curve (AUC) was measured using the Receiver Operating Characteristic curve (ROC). The area under ROC curves of LR and SVM models were 0.734 and 0.929, respectively. A computer-based predictive model can be used to identify those who are at high risk for vomiting after PCEA, allowing for patient-specific therapeutic intervention or the use of alternative analgesic methods.

Multiple classifier systems for automatic sleep scoring in mice

BACKGROUND

Electroencephalogram (EEG) and electromyogram (EMG) recordings are often used in rodents to study sleep architecture and sleep-associated neural activity. These recordings must be scored to designate what sleep/wake state the animal is in at each time point. Manual sleep-scoring is very time-consuming, so machine-learning classifier algorithms have been used to automate scoring.

NEW METHOD

Instead of using single classifiers, we implement a multiple classifier system. The multiple classifier is built from six base classifiers: decision tree, k-nearest neighbors, naïve Bayes, support vector machine, neural net, and linear discriminant analysis. Decision tree and k-nearest neighbors were improved into ensemble classifiers by using bagging and random subspace. Confidence scores from each classifier were combined to determine the final classification. Ambiguous epochs can be rejected and left for a human to classify.

RESULTS

Support vector machine was the most accurate base classifier, and had error rate of 0.054. The multiple classifier system reduced the error rate to 0.049, which was not significantly different from a second human scorer. When 10% of epochs were rejected, the remaining epochs' error rate dropped to 0.018.

COMPARISON WITH EXISTING METHOD(S)

Compared with the most accurate single classifier (support vector machine), the multiple classifier reduced errors by 9.4%. The multiple classifier surpassed the accuracy of a second human scorer after rejecting only 2% of epochs.

CONCLUSIONS

Multiple classifier systems are an effective way to increase automated sleep scoring accuracy. Improvements in autoscoring will allow sleep researchers to increase sample sizes and recording lengths, opening new experimental possibilities.

An automated sleep-state classification algorithm for quantifying sleep timing and sleep-dependent dynamics of electroencephalographic and cerebral metabolic parameters

INTRODUCTION

Rodent sleep research uses electroencephalography (EEG) and electromyography (EMG) to determine the sleep state of an animal at any given time. EEG and EMG signals, typically sampled at >100 Hz, are segmented arbitrarily into epochs of equal duration (usually 2-10 seconds), and each epoch is scored as wake, slow-wave sleep (SWS), or rapid-eye-movement sleep (REMS), on the basis of visual inspection. Automated state scoring can minimize the burden associated with state and thereby facilitate the use of shorter epoch durations.

METHODS

We developed a semiautomated state-scoring procedure that uses a combination of principal component analysis and naïve Bayes classification, with the EEG and EMG as inputs. We validated this algorithm against human-scored sleep-state scoring of data from C57BL/6J and BALB/CJ mice. We then applied a general homeostatic model to characterize the state-dependent dynamics of sleep slow-wave activity and cerebral glycolytic flux, measured as lactate concentration.

RESULTS

More than 89% of epochs scored as wake or SWS by the human were scored as the same state by the machine, whether scoring in 2-second or 10-second epochs. The majority of epochs scored as REMS by the human were also scored as REMS by the machine. However, of epochs scored as REMS by the human, more than 10% were scored as SWS by the machine and 18 (10-second epochs) to 28% (2-second epochs) were scored as wake. These biases were not strain-specific, as strain differences in sleep-state timing relative to the light/dark cycle, EEG power spectral profiles, and the homeostatic dynamics of both slow waves and lactate were detected equally effectively with the automated method or the manual scoring method. Error associated with mathematical modeling of temporal dynamics of both EEG slow-wave activity and cerebral lactate either did not differ significantly when state scoring was done with automated versus visual scoring, or was reduced with automated state scoring relative to manual classification.

CONCLUSIONS

Machine scoring is as effective as human scoring in detecting experimental effects in rodent sleep studies. Automated scoring is an efficient alternative to visual inspection in studies of strain differences in sleep and the temporal dynamics of sleep-related physiological parameters.

A machine learning approach using EEG data to predict response to SSRI treatment for major depressive disorder

OBJECTIVE

The problem of identifying, in advance, the most effective treatment agent for various psychiatric conditions remains an elusive goal. To address this challenge, we investigate the performance of the proposed machine learning (ML) methodology (based on the pre-treatment electroencephalogram (EEG)) for prediction of response to treatment with a selective serotonin reuptake inhibitor (SSRI) medication in subjects suffering from major depressive disorder (MDD).

METHODS

A relatively small number of most discriminating features are selected from a large group of candidate features extracted from the subject's pre-treatment EEG, using a machine learning procedure for feature selection. The selected features are fed into a classifier, which was realized as a mixture of factor analysis (MFA) model, whose output is the predicted response in the form of a likelihood value. This likelihood indicates the extent to which the subject belongs to the responder vs. non-responder classes. The overall method was evaluated using a "leave-n-out" randomized permutation cross-validation procedure.

RESULTS

A list of discriminating EEG biomarkers (features) was found. The specificity of the proposed method is 80.9% while sensitivity is 94.9%, for an overall prediction accuracy of 87.9%. There is a 98.76% confidence that the estimated prediction rate is within the interval [75%, 100%].

CONCLUSIONS

These results indicate that the proposed ML method holds considerable promise in predicting the efficacy of SSRI antidepressant therapy for MDD, based on a simple and cost-effective pre-treatment EEG.

SIGNIFICANCE

The proposed approach offers the potential to improve the treatment of major depression and to reduce health care costs.

FASTER: an unsupervised fully automated sleep staging method for mice

Identifying the stages of sleep, or sleep staging, is an unavoidable step in sleep research and typically requires visual inspection of electroencephalography (EEG) and electromyography (EMG) data. Currently, scoring is slow, biased and prone to error by humans and thus is the most important bottleneck for large-scale sleep research in animals. We have developed an unsupervised, fully automated sleep staging method for mice that allows less subjective and high-throughput evaluation of sleep. Fully Automated Sleep sTaging method via EEG/EMG Recordings (FASTER) is based on nonparametric density estimation clustering of comprehensive EEG/EMG power spectra. FASTER can accurately identify sleep patterns in mice that have been perturbed by drugs or by genetic modification of a clock gene. The overall accuracy is over 90% in every group. 24-h data are staged by a laptop computer in 10 min, which is faster than an experienced human rater. Dramatically improving the sleep staging process in both quality and throughput FASTER will open the door to quantitative and comprehensive animal sleep research.

Phase space and power spectral approaches for EEG-based automatic sleep-wake classification in humans: a comparative study using short and standard epoch lengths

Sleep disorders in humans have become a public health issue in recent years. Sleep can be analysed by studying the electroencephalogram (EEG) recorded during a night's sleep. Alternating between sleep-wake stages gives information related to the sleep quality and quantity since this alternating pattern is highly affected during sleep disorders. Spectral composition of EEG signals varies according to sleep stages, alternating phases of high energy associated to low frequency (deep sleep) with periods of low energy associated to high frequency (wake and light sleep). The analysis of sleep in humans is usually made on periods (epochs) of 30-s length according to the original Rechtschaffen and Kales sleep scoring manual. In this work, we propose a new phase space-based (mainly based on Poincaré plot) algorithm for automatic classification of sleep-wake states in humans using EEG data gathered over relatively short-time periods. The effectiveness of our approach is demonstrated through a series of experiments involving EEG data from seven healthy adult female subjects and was tested on epoch lengths ranging from 3-s to 30-s. The performance of our phase space approach was compared to a 2-dimensional state space approach using the power spectral (PS) in two selected human-specific frequency bands. These powers were calculated by dividing integrated spectral amplitudes at selected human-specific frequency bands. The comparison demonstrated that the phase space approach gives better performance in the case of short as well as standard 30-s epoch lengths.

Automatic sleep staging using fMRI functional connectivity data

Recent EEG-fMRI studies have shown that different stages of sleep are associated with changes in both brain activity and functional connectivity. These results raise the concern that lack of vigilance measures in resting state experiments may introduce confounds and contamination due to subjects falling asleep inside the scanner. In this study we present a method to perform automatic sleep staging using only fMRI functional connectivity data, thus providing vigilance information while circumventing the technical demands of simultaneous recording of EEG, the gold standard for sleep scoring. The features to classify are the linear correlation values between 20 cortical regions identified using independent component analysis and two regions in the bilateral thalamus. The method is based on the construction of binary support vector machine classifiers discriminating between all pairs of sleep stages and the subsequent combination of them into multiclass classifiers. Different multiclass schemes and kernels are explored. After parameter optimization through 5-fold cross validation we achieve accuracies over 0.8 in the binary problem with functional connectivities obtained for epochs as short as 60s. The multiclass classifier generalizes well to two independent datasets (accuracies over 0.8 in both sets) and can be efficiently applied to any dataset using a sliding window procedure. Modeling vigilance states in resting state analysis will avoid confounded inferences and facilitate the study of vigilance states themselves. We thus consider the method introduced in this study a novel and practical contribution for monitoring vigilance levels inside an MRI scanner without the need of extra recordings other than fMRI BOLD signals.

Automated determination of wakefulness and sleep in rats based on non-invasively acquired measures of movement and respiratory activity

The current standard for monitoring sleep in rats requires labor intensive surgical procedures and the implantation of chronic electrodes which have the potential to impact behavior and sleep. With the goal of developing a non-invasive method to determine sleep and wakefulness, we constructed a non-contact monitoring system to measure movement and respiratory activity using signals acquired with pulse Doppler radar and from digitized video analysis. A set of 23 frequency and time-domain features were derived from these signals and were calculated in 10s epochs. Based on these features, a classification method for automated scoring of wakefulness, non-rapid eye movement sleep (NREM) and REM in rats was developed using a support vector machine (SVM). We then assessed the utility of the automated scoring system in discriminating wakefulness and sleep by comparing the results to standard scoring of wakefulness and sleep based on concurrently recorded EEG and EMG. Agreement between SVM automated scoring based on selected features and visual scores based on EEG and EMG were approximately 91% for wakefulness, 84% for NREM and 70% for REM. The results indicate that automated scoring based on non-invasively acquired movement and respiratory activity will be useful for studies requiring discrimination of wakefulness and sleep. However, additional information or signals will be needed to improve discrimination of NREM and REM episodes within sleep.

A hidden Markov model to assess drug-induced sleep fragmentation in the telemetered rat

Drug-induced sleep fragmentation can cause sleep disturbances either via their intended pharmacological action or as a side effect. Examples of disturbances include excessive daytime sleepiness, insomnia and nightmares. Developing drugs without these side effects requires insight into the mechanisms leading to sleep disturbance. The characterization of the circadian sleep pattern by EEG following drug exposure has improved our understanding of these mechanisms and their translatability across species. The EEG shows frequent transitions between specific sleep states leading to multiple correlated sojourns in these states. We have developed a Markov model to consider the high correlation in the data and quantitatively compared sleep disturbance in telemetered rats induced by methylphenidate, which is known to disturb sleep, and of a new chemical entity (NCE). It was assumed that these drugs could either accelerate or decelerate the transitions between the sleep states. The difference in sleep disturbance of methylphenidate and the NCE were quantitated and different mechanisms of action on rebound sleep were identified. The estimated effect showed that both compounds induce sleep fragmentation with methylphenidate being fivefold more potent compared to the NCE.

Automated sleep scoring in rats and mice using the naive Bayes classifier

We describe a new simple MATLAB-based method for automated scoring of rat and mouse sleep using the naive Bayes classifier. This method is highly sensitive resulting in overall auto-rater agreement of 93%, comparable to an inter-rater agreement between two human scorers (92%), with high sensitivity and specificity values for wake (94% and 96%), NREM sleep (94% and 97%) and REM sleep (89% and 97%) states. In addition to baseline sleep-wake conditions, the performance of the naive Bayes classifier was assessed in sleep deprivation and drug infusion experiments, as well as in aged and transgenic animals using multiple EEG derivations. 24-h recordings from 30 different animals were used, with approximately 5% of the data manually scored as training data for the classification algorithm.

Manual rat sleep classification in principal component space

A simple method is described for using principal component analysis (PCA) to score rat sleep recordings as awake, rapid-eye-movement (REM) sleep, or non-REM (NREM) sleep. PCA was used to reduce the dimensionality of the features extracted from each epoch to three, and the projections were then graphed in a scatterplot where the clusters were visually apparent. The clusters were then directly manually selected, classifying the entire recording at once. The method was tested in a set of ten 24-h rat sleep electroencephalogram (EEG) and electromyogram (EMG) recordings. Classifications by two human raters performing traditional epoch-by-epoch scoring were blindly compared with classifications by another two human raters using the new PCA method. Overall inter-rater median percent agreements ranged between 93.7% and 94.9%. Median Cohen's kappa coefficient ranged from 0.890 to 0.909. The PCA method on average required about 5 min for classification of each 24-h recording. The combination of good accuracy and reduced time compared to traditional sleep scoring suggests that the method may be useful for sleep research.

Automated analysis of sleep-wake state in rats

A fully automated computer-based sleep scoring system is described and validated for use in rats. The system was designed to emulate visual sleep scoring by using the same basic features of the electroencephalogram (EEG) and electromyogram (EMG), and a similar set of decision-making rules. State indices are calculated for each 5s epoch by combination of amplitudes (microVrms) of 6 filtered EEG frequency bands (EEGlo, d.c.-1.5Hz; delta, 1.5-6Hz; theta, 6-9Hz; alpha, 10.5-15Hz; beta, 22-30Hz; gamma, 35-45Hz; Sigma(EEG)=delta+theta+alpha+beta+gamma) and EMG (10-100Hz) yielding dimensionless ratios: WAKE-index=(EMGxgamma)/theta; NREM-index=(deltaxalpha)/gamma(2); REM-index=theta(3)/(deltaxalphaxEMG); artifact-index=[(2xEEG(lo))+beta]*(gamma/Sigma(EEG)). The index values are re-scaled and normalized, thereby dispensing with the need for animal-specific threshold values. The system was validated by direct comparison with visually scored data in 9 rats. Overall, the computer and visual scores were 96% concordant, which is similar to inter-rater agreement in visual scoring. False-positive error rate was <5%. A re-test protocol in 7 rats confirmed the long-term stability of the system in studies lasting 5 weeks. The system was implemented and further validated in a study of sleep architecture in 7 rats under a 12:12h LD cycle.

Open-source logic-based automated sleep scoring software using electrophysiological recordings in rats

Manual state scoring of physiological recordings in sleep studies is time-consuming, resulting in a data backlog, research delays and increased personnel costs. We developed MATLAB-based software to automate scoring of sleep/waking states in rats, potentially extendable to other animals, from a variety of recording systems. The software contains two programs, Sleep Scorer and Auto-Scorer, for manual and automated scoring. Auto-Scorer is a logic-based program that displays power spectral densities of an electromyographic (EMG) signal and sigma, delta, and theta frequency bands of an electroencephalographic (EEG) signal, along with the delta/theta ratio and sigmaxtheta, for every epoch. The user defines thresholds from the training file state definitions which the Auto-Scorer uses with logic to discriminate the state of every epoch in the file. Auto-Scorer was evaluated by comparing its output to manually scored files from 6 rats under 2 experimental conditions by 3 users. Each user generated a training file, set thresholds, and auto-scored the 12 files into 4 states (waking, non-REM, transition-to-REM, and REM sleep) in 1/4 the time required to manually score the file. Overall performance comparisons between Auto-Scorer and manual scoring resulted in a mean agreement of 80.24+/-7.87%, comparable to the average agreement among 3 manual scorers (83.03+/-4.00%). There was no significant difference between user-user and user-Auto-Scorer agreement ratios. These results support the use of our open-source Auto-Scorer, coupled with user review, to rapidly and accurately score sleep/waking states from rat recordings.

Remote effects of focal hippocampal seizures on the rat neocortex

Seizures have both local and remote effects on nervous system function. Whereas propagated seizures are known to disrupt cerebral activity, little work has been done on remote network effects of seizures that do not propagate. Human focal temporal lobe seizures demonstrate remote changes including slow waves on electroencephalography (EEG) and decreased cerebral blood flow (CBF) in the neocortex. Ictal neocortical slow waves have been interpreted as seizure propagation; however, we hypothesize that they reflect a depressed cortical state resembling sleep or coma. To investigate this hypothesis, we performed multimodal studies of partial and secondarily generalized limbic seizures in rats. Video/EEG monitoring of spontaneous seizures revealed slow waves in the frontal cortex during behaviorally mild partial seizures, contrasted with fast polyspike activity during convulsive generalized seizures. Seizures induced by hippocampal stimulation produced a similar pattern, and were used to perform functional magnetic resonance imaging weighted for blood oxygenation and blood volume, demonstrating increased signals in hippocampus, thalamus and septum, but decreases in orbitofrontal, cingulate, and retrosplenial cortex during partial seizures, and increases in all of these regions during propagated seizures. Combining these results with neuronal recordings and CBF measurements, we related neocortical slow waves to reduced neuronal activity and cerebral metabolism during partial seizures, but found increased neuronal activity and metabolism during propagated seizures. These findings suggest that ictal neocortical slow waves represent an altered cortical state of depressed function, not propagated seizure activity. This remote effect of partial seizures may cause impaired cerebral functions, including loss of consciousness.