Pattern Recognition of Momentary Mental Workload Based on Multi-Channel Electrophysiological Data and Ensemble Convolutional Neural Networks

Multi-source deep domain adaptation ensemble framework for cross-dataset motor imagery EEG transfer learning

Objective. Electroencephalography (EEG) is an important kind of bioelectric signal for measuring physiological activities of the brain, and motor imagery (MI) EEG has significant clinical application prospects. Convolutional neural network has become a mainstream algorithm for MI EEG classification, however lack of subject-specific data considerably restricts its decoding accuracy and generalization performance. To address this challenge, a novel transfer learning (TL) framework using auxiliary dataset to improve the MI EEG classification performance of target subject is proposed in this paper.Approach. We developed a multi-source deep domain adaptation ensemble framework (MSDDAEF) for cross-dataset MI EEG decoding. The proposed MSDDAEF comprises three main components: model pre-training, deep domain adaptation, and multi-source ensemble. Moreover, for each component, different designs were examined to verify the robustness of MSDDAEF.Main results. Bidirectional validation experiments were performed on two large public MI EEG datasets (openBMI and GIST). The highest average classification accuracy of MSDDAEF reaches 74.28% when openBMI serves as target dataset and GIST serves as source dataset. While the highest average classification accuracy of MSDDAEF is 69.85% when GIST serves as target dataset and openBMI serves as source dataset. In addition, the classification performance of MSDDAEF surpasses several well-established studies and state-of-the-art algorithms.Significance. The results of this study show that cross-dataset TL is feasible for left/right-hand MI EEG decoding, and further indicate that MSDDAEF is a promising solution for addressing MI EEG cross-dataset variability.

The Hybrid Deep Learning Model for Identification of Attention-Deficit/Hyperactivity Disorder Using EEG

Common misbehavior among children that prevents them from paying attention to tasks and interacting with their surroundings appropriately is attention-deficit/hyperactivity disorder (ADHD). Studies of children's behavior presently face a significant problem in the early and timely diagnosis of this disease. To diagnose this disease, doctors often use the patient's description and questionnaires, psychological tests, and the patient's behavior in which reliability is questionable. Convolutional neural network (CNN) is one deep learning technique that has been used for the diagnosis of ADHD. CNN, however, does not account for how signals change over time, which leads to low classification performances and ambiguous findings. In this study, the authors designed a hybrid deep learning model that combines long-short-term memory (LSTM) and CNN to simultaneously extract and learn the spatial features and long-term dependencies of the electroencephalography (EEG) data. The effectiveness of the proposed hybrid deep learning model was assessed using 2 publicly available EEG datasets. The suggested model achieves a classification accuracy of 98.86% on the ADHD dataset and 98.28% on the FOCUS dataset, respectively. The experimental findings show that the proposed hybrid CNN-LSTM model outperforms the state-of-the-art methods to diagnose ADHD using EEG. Hence, the proposed hybrid CNN-LSTM model could therefore be utilized to help with the clinical diagnosis of ADHD patients.

Mental workload level assessment based on compounded hysteresis effect

In the domain of neuroergonomics, cognitive workload estimation has taken a significant concern among the researchers. This is because the knowledge gathered from its estimation is useful for distributing tasks among the operators, understanding human capability and intervening operators at times of havoc. Brain signals give a promising prospective for understanding cognitive workload. For this, electroencephalography (EEG) is by far the most efficient modality in interpreting the covert information arising in the brain. The present work explores the feasibility of EEG rhythms for monitoring continuous change occurring in a person's cognitive workload. This continuous monitoring is achieved by graphicallyinterpreting the cumulative effect of changes in EEG rhythms observed in the current instance and the former instance based on the hysteresis effect. In this work, classification is done to predict the data class label using an artificial neural network (ANN) architecture. The proposed model gives a classification accuracy of 98.66%.

Fused CNN-LSTM deep learning emotion recognition model using electroencephalography signals

Introduction: The traditional machine learning-based emotion recognition models have shown effective performance for classifying Electroencephalography (EEG) based emotions.Methods: The different machine learning algorithms outperform the various EEG based emotion models for valence and arousal. But the downside is to devote numerous efforts to designing features from the given noisy signals which are also a very time-consuming process. The Deep Learning analysis overcomes the hand-engineered feature extraction and selection problems.Results: In this study, the Database of Emotion analysis using Physiological signals (DEAP) has been visualized to classify High-Arousal- Low-Arousal (HALA), High-Valence-Low-Valence (HVLV), familiarity, Dominance and Liking emotions. The fusion of deep learning models, namely CNN and LSTM-RNN seems to perform better for the analysis of emotions using EEG signals. The average accuracies analyzed by the fused deep learning classification model for DEAP are 97.39%, 97.41%, 98.21%, 97.68%, and 97.89% for HALA, HVLV, familiarity, dominance and liking respectively. The model has been evaluated over the SJTU Emotion EEG Dataset (SEED) dataset too for the detection of positive and negative emotions, which results with an average accuracy of 93.74%.Conclusion: The results show that the developed model can classify the inner emotions of different EEG based emotion databases.

Signal2Image Modules in Deep Neural Networks for EEG Classification

Deep learning has revolutionized computer vision utilizing the increased availability of big data and the power of parallel computational units such as graphical processing units. The vast majority of deep learning research is conducted using images as training data, however the biomedical domain is rich in physiological signals that are used for diagnosis and prediction problems. It is still an open research question how to best utilize signals to train deep neural networks.In this paper we define the term Signal2Image (S2Is) as trainable or non-trainable prefix modules that convert signals, such as Electroencephalography (EEG), to image-like representations making them suitable for training image-based deep neural networks defined as `base models'. We compare the accuracy and time performance of four S2Is (`signal as image', spectrogram, one and two layer Convolutional Neural Networks (CNNs)) combined with a set of `base models' (LeNet, AlexNet, VGGnet, ResNet, DenseNet) along with the depth-wise and 1D variations of the latter. We also provide empirical evidence that the one layer CNN S2I performs better in eleven out of fifteen tested models than non-trainable S2Is for classifying EEG signals and present visual comparisons of the outputs of some of the S2Is.

Deep learning-based electroencephalography analysis: a systematic review

CONTEXT

Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question.

OBJECTIVE

In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain-computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations.

METHODS

Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends.

RESULTS

Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About [Formula: see text] of the studies used convolutional neural networks (CNNs), while [Formula: see text] used recurrent neural networks (RNNs), most often with a total of 3-10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was [Formula: see text] across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code.

SIGNIFICANCE

To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

Deep Learning and Big Data in Healthcare: A Double Review for Critical Beginners

In the last few years, there has been a growing expectation created about the analysis of large amounts of data often available in organizations, which has been both scrutinized by the academic world and successfully exploited by industry. Nowadays, two of the most common terms heard in scientific circles are Big Data and Deep Learning. In this double review, we aim to shed some light on the current state of these different, yet somehow related branches of Data Science, in order to understand the current state and future evolution within the healthcare area. We start by giving a simple description of the technical elements of Big Data technologies, as well as an overview of the elements of Deep Learning techniques, according to their usual description in scientific literature. Then, we pay attention to the application fields that can be said to have delivered relevant real-world success stories, with emphasis on examples from large technology companies and financial institutions, among others. The academic effort that has been put into bringing these technologies to the healthcare sector are then summarized and analyzed from a twofold view as follows: first, the landscape of application examples is globally scrutinized according to the varying nature of medical data, including the data forms in electronic health recordings, medical time signals, and medical images; second, a specific application field is given special attention, in particular the electrocardiographic signal analysis, where a number of works have been published in the last two years. A set of toy application examples are provided with the publicly-available MIMIC dataset, aiming to help the beginners start with some principled, basic, and structured material and available code. Critical discussion is provided for current and forthcoming challenges on the use of both sets of techniques in our future healthcare.

PSO optimized 1-D CNN-SVM architecture for real-time detection and classification applications

In this paper, we propose a novel Particle Swarm Optimized (PSO) One-Dimensional Convolutional Neural Network with Support Vector Machine (1-D CNN-SVM) architecture for real-time detection and classification of diseases. The performance of the proposed architecture is validated with a novel hardware model for detecting Chronic Kidney Disease (CKD) from saliva samples. For detecting CKD, the urea concentration in the saliva sample is monitored by converting it into ammonia. The urea on hydrolysis in the presence of urease enzyme produces ammonia. This ammonia is then measured using a semiconductor gas sensor. The sensor response is given to the proposed architecture for feature extraction and classification. The performance of the architecture is optimized by regulating the parameter values using a PSO algorithm. The proposed architecture outperforms current conventional methods, as this approach is a combination of strong feature extraction and classification techniques. Optimal features are extracted directly from the raw signal, aiming to reduce the computational time and complexity. The proposed architecture has achieved an accuracy of 98.25%.

EEG classification of driver mental states by deep learning

Driver fatigue is attracting more and more attention, as it is the main cause of traffic accidents, which bring great harm to society and families. This paper proposes to use deep convolutional neural networks, and deep residual learning, to predict the mental states of drivers from electroencephalography (EEG) signals. Accordingly we have developed two mental state classification models called EEG-Conv and EEG-Conv-R. Tested on intra- and inter-subject, our results show that both models outperform the traditional LSTM- and SVM-based classifiers. Our major findings include (1) Both EEG-Conv and EEG-Conv-R yield very good classification performance for mental state prediction; (2) EEG-Conv-R is more suitable for inter-subject mental state prediction; (3) EEG-Conv-R converges more quickly than EEG-Conv. In summary, our proposed classifiers have better predictive power and are promising for application in practical brain-computer interaction .

A CNN-SVM combined model for pattern recognition of knee motion using mechanomyography signals

The commonly used classifiers for pattern recognition of human motion, like backpropagation neural network (BPNN) and support vector machine (SVM), usually implement the classification by extracting some hand-crafted features from the human biological signals. These features generally require the domain knowledge for researchers to be designed and take a long time to be tested and selected for high classification performance. In contrast, convolutional neural network (CNN), which has been widely applied to computer vision, can learn to automatically extract features from the training data by means of convolution and subsampling, but CNN training usually requires large sample data and has the overfitting problem. On the other hand, SVM has good generalization ability and can solve the small sample problem. Therefore, we proposed a CNN-SVM combined model to make use of their advantages. In this paper, we detected 4-channel mechanomyography (MMG) signals from the thigh muscles and fed them in the form of time series signals to the CNN-SVM combined model for the pattern recognition of knee motion. Compared with the common classifier performing the classification with hand-crafted features, the CNN-SVM combined model could automatically extract features using CNN, and better improved the generalization ability of CNN and the classification accuracy by means of combining the SVM. This study would provide reference for human motion recognition using other time series signals and further expand the application fields of CNN.

Deep learning for healthcare applications based on physiological signals: A review

BACKGROUND AND OBJECTIVE

We have cast the net into the ocean of knowledge to retrieve the latest scientific research on deep learning methods for physiological signals. We found 53 research papers on this topic, published from 01.01.2008 to 31.12.2017.

METHODS

An initial bibliometric analysis shows that the reviewed papers focused on Electromyogram(EMG), Electroencephalogram(EEG), Electrocardiogram(ECG), and Electrooculogram(EOG). These four categories were used to structure the subsequent content review.

RESULTS

During the content review, we understood that deep learning performs better for big and varied datasets than classic analysis and machine classification methods. Deep learning algorithms try to develop the model by using all the available input.

CONCLUSIONS

This review paper depicts the application of various deep learning algorithms used till recently, but in future it will be used for more healthcare areas to improve the quality of diagnosis.