Cross-subject recognition of operator functional states via EEG and switching deep belief networks with adaptive weights

Harnessing Few-Shot Learning for EEG signal classification: a survey of state-of-the-art techniques and future directions

This paper presents a systematic literature review, providing a comprehensive taxonomy of Data Augmentation (DA), Transfer Learning (TL), and Self-Supervised Learning (SSL) techniques within the context of Few-Shot Learning (FSL) for EEG signal classification. EEG signals have shown significant potential in various paradigms, including Motor Imagery, Emotion Recognition, Visual Evoked Potentials, Steady-State Visually Evoked Potentials, Rapid Serial Visual Presentation, Event-Related Potentials, and Mental Workload. However, challenges such as limited labeled data, noise, and inter/intra-subject variability have impeded the effectiveness of traditional machine learning (ML) and deep learning (DL) models. This review methodically explores how FSL approaches, incorporating DA, TL, and SSL, can address these challenges and enhance classification performance in specific EEG paradigms. It also delves into the open research challenges related to these techniques in EEG signal classification. Specifically, the review examines the identification of DA strategies tailored to various EEG paradigms, the creation of TL architectures for efficient knowledge transfer, and the formulation of SSL methods for unsupervised representation learning from EEG data. Addressing these challenges is crucial for enhancing the efficacy and robustness of FSL-based EEG signal classification. By presenting a structured taxonomy of FSL techniques and discussing the associated research challenges, this systematic review offers valuable insights for future investigations in EEG signal classification. The findings aim to guide and inspire researchers, promoting advancements in applying FSL methodologies for improved EEG signal analysis and classification in real-world settings.

Translation directionality and the Inhibitory Control Model: a machine learning approach to an eye-tracking study

Introduction

Based on such physiological data as pupillometry collected in an eye-tracking experiment, the study has further confirmed the effect of directionality on cognitive loads during L1 and L2 textual translations by novice translators, a phenomenon called "translation asymmetry" suggested by the Inhibitory Control Model, while revealing that machine learning-based approaches can be usefully applied to the field of Cognitive Translation and Interpreting Studies.

Methods

Directionality was the only factor guiding the eye-tracking experiment where 14 novice translators with the language combination of Chinese and English were recruited to conduct L1 and L2 translations while their pupillometry were recorded. They also filled out a Language and Translation Questionnaire with which categorical data on their demographics were obtained.

Results

A nonparametric related-samples Wilcoxon signed rank test on pupillometry verified the effect of directionality, suggested by the model, during bilateral translations, verifying "translation asymmetry" at a textual level. Further, using the pupillometric data, together with the categorical information, the XGBoost machine-learning algorithm yielded a model that could reliably and effectively predict translation directions.

Conclusion

The study has shown that translation asymmetry suggested by the model was valid at a textual level, and that machine learning-based approaches can be gainfully applied to Cognitive Translation and Interpreting Studies.

Estimating distribution shifts for predicting cross-subject generalization in electroencephalography-based mental workload assessment

Assessment of mental workload in real-world conditions is key to ensuring the performance of workers executing tasks that demand sustained attention. Previous literature has employed electroencephalography (EEG) to this end despite having observed that EEG correlates of mental workload vary across subjects and physical strain, thus making it difficult to devise models capable of simultaneously presenting reliable performance across users. Domain adaptation consists of a set of strategies that aim at allowing for improving machine learning systems performance on unseen data at training time. Such methods, however, might rely on assumptions over the considered data distributions, which typically do not hold for applications of EEG data. Motivated by this observation, in this work we propose a strategy to estimate two types of discrepancies between multiple data distributions, namely marginal and conditional shifts, observed on data collected from different subjects. Besides shedding light on the assumptions that hold for a particular dataset, the estimates of statistical shifts obtained with the proposed approach can be used for investigating other aspects of a machine learning pipeline, such as quantitatively assessing the effectiveness of domain adaptation strategies. In particular, we consider EEG data collected from individuals performing mental tasks while running on a treadmill and pedaling on a stationary bike and explore the effects of different normalization strategies commonly used to mitigate cross-subject variability. We show the effects that different normalization schemes have on statistical shifts and their relationship with the accuracy of mental workload prediction as assessed on unseen participants at training time.

Vowel speech recognition from rat electroencephalography using long short-term memory neural network

Over the years, considerable research has been conducted to investigate the mechanisms of speech perception and recognition. Electroencephalography (EEG) is a powerful tool for identifying brain activity; therefore, it has been widely used to determine the neural basis of speech recognition. In particular, for the classification of speech recognition, deep learning-based approaches are in the spotlight because they can automatically learn and extract representative features through end-to-end learning. This study aimed to identify particular components that are potentially related to phoneme representation in the rat brain and to discriminate brain activity for each vowel stimulus on a single-trial basis using a bidirectional long short-term memory (BiLSTM) network and classical machine learning methods. Nineteen male Sprague-Dawley rats subjected to microelectrode implantation surgery to record EEG signals from the bilateral anterior auditory fields were used. Five different vowel speech stimuli were chosen, /a/, /e/, /i/, /o/, and /u/, which have highly different formant frequencies. EEG recorded under randomly given vowel stimuli was minimally preprocessed and normalized by a z-score transformation to be used as input for the classification of speech recognition. The BiLSTM network showed the best performance among the classifiers by achieving an overall accuracy, f1-score, and Cohen’s κ values of 75.18%, 0.75, and 0.68, respectively, using a 10-fold cross-validation approach. These results indicate that LSTM layers can effectively model sequential data, such as EEG; hence, informative features can be derived through BiLSTM trained with end-to-end learning without any additional hand-crafted feature extraction methods.

Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review

Classification of electroencephalogram (EEG) is a key approach to measure the rhythmic oscillations of neural activity, which is one of the core technologies of brain-computer interface systems (BCIs). However, extraction of the features from non-linear and non-stationary EEG signals is still a challenging task in current algorithms. With the development of artificial intelligence, various advanced algorithms have been proposed for signal classification in recent years. Among them, deep neural networks (DNNs) have become the most attractive type of method due to their end-to-end structure and powerful ability of automatic feature extraction. However, it is difficult to collect large-scale datasets in practical applications of BCIs, which may lead to overfitting or weak generalizability of the classifier. To address these issues, a promising technique has been proposed to improve the performance of the decoding model based on data augmentation (DA). In this article, we investigate recent studies and development of various DA strategies for EEG classification based on DNNs. The review consists of three parts: what kind of paradigms of EEG-based on BCIs are used, what types of DA methods are adopted to improve the DNN models, and what kind of accuracy can be obtained. Our survey summarizes the current practices and performance outcomes that aim to promote or guide the deployment of DA to EEG classification in future research and development.

A Multi-Scale Activity Transition Network for Data Translation in EEG Signals Decoding

Electroencephalogram (EEG) is a non-invasive collection method for brain signals. It has broad prospects in brain-computer interface (BCI) applications. Recent advances have shown the effectiveness of the widely used convolutional neural network (CNN) in EEG decoding. However, some studies reveal that a slight disturbance to the inputs, e.g., data translation, can change CNN's outputs. Such instability is dangerous for EEG-based BCI applications because signals in practice are different from training data. In this study, we propose a multi-scale activity transition network (MSATNet) to alleviate the influence of the translation problem in convolution-based models. MSATNet provides an activity state pyramid consisting of multi-scale recurrent neural networks to capture the relationship between brain activities, which is a translation-invariant feature. In the experiment, Kullback-Leibler divergence is applied to measure the degree of translation. The comprehensive results demonstrate that our method surpasses the AUC of 0.0080, 0.0254, 0.0393 in 1, 5, and 10 KL divergence compared to competitors with various convolution structures.

Data Augmentation: Using Channel-Level Recombination to Improve Classification Performance for Motor Imagery EEG

The quality and quantity of training data are crucial to the performance of a deep-learning-based brain-computer interface (BCI) system. However, it is not practical to record EEG data over several long calibration sessions. A promising time- and cost-efficient solution is artificial data generation or data augmentation (DA). Here, we proposed a DA method for the motor imagery (MI) EEG signal called brain-area-recombination (BAR). For the BAR, each sample was first separated into two ones (named half-sample) by left/right brain channels, and the artificial samples were generated by recombining the half-samples. We then designed two schemas (intra- and adaptive-subject schema) corresponding to the single- and multi-subject scenarios. Extensive experiments using the classifier of EEGnet were conducted on two public datasets under various training set sizes. In both schemas, the BAR method can make the EEGnet have a better performance of classification (p < 0.01). To make a comparative investigation, we selected two common DA methods (noise-added and flipping), and the BAR method beat them (p < 0.05). Further, using the proposed BAR for augmentation, EEGnet achieved up to 8.3% improvement than a typical decoding algorithm CSP-SVM (p < 0.01), note that both the models were trained on the augmented dataset. This study shows that BAR usage can significantly improve the classification ability of deep learning to MI-EEG signals. To a certain extent, it may promote the development of deep learning technology in the field of BCI.

Recent Advancements and Future Prospects on E-Nose Sensors Technology and Machine Learning Approaches for Non-Invasive Diabetes Diagnosis: A Review

Diabetes mellitus, commonly measured through an invasive process which although is accurate, has manifold drawbacks especially when multiple reading are required at regular intervals. Accordingly, there is a need to develop a dependable non-invasive diabetes detection technique. Recent studies have observed that other human serums such as tears, saliva, urine and breath indicate the presence of glucose in them. These parameters open quite a few ways for non-invasive blood glucose level prediction. The analysis of a persons breath poses as a good non-invasive technique to monitor the glucose levels. It is seen that in breath, there are many bio-markers and monitoring the levels of these bio-markers indicate the possibility of various chronic diseases. Among these bio-markers, acetone a volatile organic compound found in breath has shown a good correlation to the glucose levels present in blood. Therefore, by evaluating the acetone levels in breath samples it is possible to monitor diabetes non-invasively. This paper reviews the various approaches and sensory techniques used to monitor diabetes though human breath samples.

A LightGBM-Based EEG Analysis Method for Driver Mental States Classification

Fatigue driving can easily lead to road traffic accidents and bring great harm to individuals and families. Recently, electroencephalography- (EEG-) based physiological and brain activities for fatigue detection have been increasingly investigated. However, how to find an effective method or model to timely and efficiently detect the mental states of drivers still remains a challenge. In this paper, we combine common spatial pattern (CSP) and propose a light-weighted classifier, LightFD, which is based on gradient boosting framework for EEG mental states identification. The comparable results with traditional classifiers, such as support vector machine (SVM), convolutional neural network (CNN), gated recurrent unit (GRU), and large margin nearest neighbor (LMNN), show that the proposed model could achieve better classification performance, as well as the decision efficiency. Furthermore, we also test and validate that LightFD has better transfer learning performance in EEG classification of driver mental states. In summary, our proposed LightFD classifier has better performance in real-time EEG mental state prediction, and it is expected to have broad application prospects in practical brain-computer interaction (BCI).

Deep learning-based electroencephalography analysis: a systematic review

CONTEXT

Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question.

OBJECTIVE

In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain-computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations.

METHODS

Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends.

RESULTS

Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About [Formula: see text] of the studies used convolutional neural networks (CNNs), while [Formula: see text] used recurrent neural networks (RNNs), most often with a total of 3-10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was [Formula: see text] across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code.

SIGNIFICANCE

To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

Multiple Transferable Recursive Feature Elimination Technique for Emotion Recognition Based on EEG Signals

Feature selection plays a crucial role in analyzing huge-volume, high-dimensional EEG signals in human-centered automation systems. However, classical feature selection methods pay little attention to transferring cross-subject information for emotions. To perform cross-subject emotion recognition, a classifier able to utilize EEG data to train a general model suitable for different subjects is needed. However, existing methods are imprecise due to the fact that the effective feelings of individuals are personalized. In this work, the cross-subject emotion recognition model on both binary and multi affective states are developed based on the newly designed multiple transferable recursive feature elimination (M-TRFE). M-TRFE manages not only a stricter feature selection of all subjects to discover the most robust features but also a unique subject selection to decide the most trusted subjects for certain emotions. Via a least square support vector machine (LSSVM), the overall multi (joy, peace, anger and depression) classification accuracy of the proposed M-TRFE reaches 0.6513, outperforming all other methods used or referenced in this paper.

Deep learning for electroencephalogram (EEG) classification tasks: a review

OBJECTIVE

Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks?

APPROACH

A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture.

MAIN RESULTS

For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review.

SIGNIFICANCE

This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.