EEG-Based Emotion Recognition Using a 2D CNN with Different Kernels

Cross-Modal Guiding Neural Network for Multimodal Emotion Recognition From EEG and Eye Movement Signals

Multimodal emotion recognition research is gaining attention because of the emerging trend of integrating information from different sensory modalities to improve performance. Electroencephalogram (EEG) signals are considered objective indicators of emotions and provide precise insights despite their complex data collection. In contrast, eye movement signals are more susceptible to environmental and individual differences but offer convenient data collection. Conventional emotion recognition methods typically use separate models for different modalities, potentially overlooking their inherent connections. This study introduces a cross-modal guiding neural network designed to fully leverage the strengths of both modalities. The network includes a dual-branch feature extraction module that simultaneously extracts features from EEG and eye movement signals. In addition, the network includes a feature guidance module that uses EEG features to direct eye movement feature extraction, reducing the impact of subjective factors. This study also introduces a feature reweighting module to explore emotion-related features within eye movement signals, thereby improving emotion classification accuracy. The empirical findings from both the SEED-IV dataset and our collected dataset substantiate the commendable performance of the model, thereby confirming its efficacy.

OxcarNet: sinc convolutional network with temporal and channel attention for prediction of oxcarbazepine monotherapy responses in patients with newly diagnosed epilepsy

Objective.Monotherapy with antiepileptic drugs (AEDs) is the preferred strategy for the initial treatment of epilepsy. However, an inadequate response to the initially prescribed AED is a significant indicator of a poor long-term prognosis, emphasizing the importance of precise prediction of treatment outcomes with the initial AED regimen in patients with epilepsy.Approach. We introduce OxcarNet, an end-to-end neural network framework developed to predict treatment outcomes in patients undergoing oxcarbazepine monotherapy. The proposed predictive model adopts a Sinc Module in its initial layers for adaptive identification of discriminative frequency bands. The derived feature maps are then processed through a Spatial Module, which characterizes the scalp distribution patterns of the electroencephalography (EEG) signals. Subsequently, these features are fed into an attention-enhanced Temporal Module to capture temporal dynamics and discrepancies. A channel module with an attention mechanism is employed to reveal inter-channel dependencies within the output of the Temporal Module, ultimately achieving response prediction. OxcarNet was rigorously evaluated using a proprietary dataset of retrospectively collected EEG data from newly diagnosed epilepsy patients at Nanjing Drum Tower Hospital. This dataset included patients who underwent long-term EEG monitoring in a clinical inpatient setting.Main results.OxcarNet demonstrated exceptional accuracy in predicting treatment outcomes for patients undergoing Oxcarbazepine monotherapy. In the ten-fold cross-validation, the model achieved an accuracy of 97.27%, and in the validation involving unseen patient data, it maintained an accuracy of 89.17%, outperforming six conventional machine learning methods and three generic neural decoding networks. These findings underscore the model's effectiveness in accurately predicting the treatment responses in patients with newly diagnosed epilepsy. The analysis of features extracted by the Sinc filters revealed a predominant concentration of predictive frequencies in the high-frequency range of the gamma band.Significance. The findings of our study offer substantial support and new insights into tailoring early AED selection, enhancing the prediction accuracy for the responses of AEDs.

Exploring Convolutional Neural Network Architectures for EEG Feature Extraction

The main purpose of this paper is to provide information on how to create a convolutional neural network (CNN) for extracting features from EEG signals. Our task was to understand the primary aspects of creating and fine-tuning CNNs for various application scenarios. We considered the characteristics of EEG signals, coupled with an exploration of various signal processing and data preparation techniques. These techniques include noise reduction, filtering, encoding, decoding, and dimension reduction, among others. In addition, we conduct an in-depth analysis of well-known CNN architectures, categorizing them into four distinct groups: standard implementation, recurrent convolutional, decoder architecture, and combined architecture. This paper further offers a comprehensive evaluation of these architectures, covering accuracy metrics, hyperparameters, and an appendix that contains a table outlining the parameters of commonly used CNN architectures for feature extraction from EEG signals.

Emotion recognition in EEG signals using deep learning methods: A review

Emotions are a critical aspect of daily life and serve a crucial role in human decision-making, planning, reasoning, and other mental states. As a result, they are considered a significant factor in human interactions. Human emotions can be identified through various sources, such as facial expressions, speech, behavior (gesture/position), or physiological signals. The use of physiological signals can enhance the objectivity and reliability of emotion detection. Compared with peripheral physiological signals, electroencephalogram (EEG) recordings are directly generated by the central nervous system and are closely related to human emotions. EEG signals have the great spatial resolution that facilitates the evaluation of brain functions, making them a popular modality in emotion recognition studies. Emotion recognition using EEG signals presents several challenges, including signal variability due to electrode positioning, individual differences in signal morphology, and lack of a universal standard for EEG signal processing. Moreover, identifying the appropriate features for emotion recognition from EEG data requires further research. Finally, there is a need to develop more robust artificial intelligence (AI) including conventional machine learning (ML) and deep learning (DL) methods to handle the complex and diverse EEG signals associated with emotional states. This paper examines the application of DL techniques in emotion recognition from EEG signals and provides a detailed discussion of relevant articles. The paper explores the significant challenges in emotion recognition using EEG signals, highlights the potential of DL techniques in addressing these challenges, and suggests the scope for future research in emotion recognition using DL techniques. The paper concludes with a summary of its findings.

Exploiting Information in Event-Related Brain Potentials from Average Temporal Waveform, Time-Frequency Representation, and Phase Dynamics

Characterizing the brain's dynamic pattern of response to an input in electroencephalography (EEG) is not a trivial task due to the entanglement of the complex spontaneous brain activity. In this context, the brain's response can be defined as (1) the additional neural activity components generated after the input or (2) the changes in the ongoing spontaneous activities induced by the input. Moreover, the response can be manifested in multiple features. Three commonly studied examples of features are (1) transient temporal waveform, (2) time-frequency representation, and (3) phase dynamics. The most extensively used method of average event-related potentials (ERPs) captures the first one, while the latter two and other more complex features are attracting increasing attention. However, there has not been much work providing a systematic illustration and guidance for how to effectively exploit multifaceted features in neural cognitive research. Based on a visual oddball ERPs dataset with 200 participants, this work demonstrates how the information from the above-mentioned features are complementary to each other and how they can be integrated based on stereotypical neural-network-based machine learning approaches to better exploit neural dynamic information in basic and applied cognitive research.

Electroencephalograph Emotion Classification Using a Novel Adaptive Ensemble Classifier Considering Personality Traits

Introduction

The study explores the use of Electroencephalograph (EEG) signals as a means to uncover various states of the human brain, with a specific focus on emotion classification. Despite the potential of EEG signals in this domain, existing methods face challenges. Features extracted from EEG signals may not accurately represent an individual's emotional patterns due to interference from time-varying factors and noise. Additionally, higher-level cognitive factors, such as personality, mood, and past experiences, further complicate emotion recognition. The dynamic nature of EEG data in terms of time series introduces variability in feature distribution and interclass discrimination across different time stages.

Methods

To address these challenges, the paper proposes a novel adaptive ensemble classification method. The study introduces a new method for providing emotional stimuli, categorizing them into three groups (sadness, neutral, and happiness) based on their valence-arousal (VA) scores. The experiment involved 60 participants aged 19-30 years, and the proposed method aimed to mitigate the limitations associated with conventional classifiers.

Results

The results demonstrate a significant improvement in the performance of emotion classifiers compared to conventional methods. The classification accuracy achieved by the proposed adaptive ensemble classification method is reported at 87.96%. This suggests a promising advancement in the ability to accurately classify emotions using EEG signals, overcoming the limitations outlined in the introduction.

Conclusion

In conclusion, the paper introduces an innovative approach to emotion classification based on EEG signals, addressing key challenges associated with existing methods. By employing a new adaptive ensemble classification method and refining the process of providing emotional stimuli, the study achieves a noteworthy improvement in classification accuracy. This advancement is crucial for enhancing our understanding of the complexities of emotion recognition through EEG signals, paving the way for more effective applications in fields such as neuroinformatics and affective computing.

Personality-Based Emotion Recognition Using EEG Signals with a CNN-LSTM Network

The accurate detection of emotions has significant implications in healthcare, psychology, and human-computer interaction. Integrating personality information into emotion recognition can enhance its utility in various applications. The present study introduces a novel deep learning approach to emotion recognition, which utilizes electroencephalography (EEG) signals and the Big Five personality traits. The study recruited 60 participants and recorded their EEG data while they viewed unique sequence stimuli designed to effectively capture the dynamic nature of human emotions and personality traits. A pre-trained convolutional neural network (CNN) was used to extract emotion-related features from the raw EEG data. Additionally, a long short-term memory (LSTM) network was used to extract features related to the Big Five personality traits. The network was able to accurately predict personality traits from EEG data. The extracted features were subsequently used in a novel network to predict emotional states within the arousal and valence dimensions. The experimental results showed that the proposed classifier outperformed common classifiers, with a high accuracy of 93.97%. The findings suggest that incorporating personality traits as features in the designed network, for emotion recognition, leads to higher accuracy, highlighting the significance of examining these traits in the analysis of emotions.

Deep learning-based EEG emotion recognition: Current trends and future perspectives

Automatic electroencephalogram (EEG) emotion recognition is a challenging component of human-computer interaction (HCI). Inspired by the powerful feature learning ability of recently-emerged deep learning techniques, various advanced deep learning models have been employed increasingly to learn high-level feature representations for EEG emotion recognition. This paper aims to provide an up-to-date and comprehensive survey of EEG emotion recognition, especially for various deep learning techniques in this area. We provide the preliminaries and basic knowledge in the literature. We review EEG emotion recognition benchmark data sets briefly. We review deep learning techniques in details, including deep belief networks, convolutional neural networks, and recurrent neural networks. We describe the state-of-the-art applications of deep learning techniques for EEG emotion recognition in detail. We analyze the challenges and opportunities in this field and point out its future directions.

EEG Channel Selection Techniques in Motor Imagery Applications: A Review and New Perspectives

Communication, neuro-prosthetics, and environmental control are just a few applications for disabled persons who use robots and manipulators that use brain-computer interface (BCI) systems. The brain's motor imagery (MI) signal is an essential input for a brain-related task in BCI applications. Due to their noninvasive, portability, and cost-effectiveness, electroencephalography (EEG) signals are the most widely used input in BCI systems. The EEG data are often collected from more than 100 different locations in the brain; channel selection techniques are critical for selecting the optimum channels for a given application. However, when analyzing EEG data, the principal purpose of channel selection is to reduce computational complexity, improve classification accuracy by avoiding overfitting, and reduce setup time. Several channel selection assessment algorithms, both with and without classification-based methods, extracted appropriate channel subsets using defined criteria. Therefore, based on the exhaustive analysis of the EEG channel selection, this manuscript analyses several existing studies to reduce the number of noisy channels and improve system performance. We review several existing works to find the most promising MI-based EEG channel selection algorithms and associated classification methodologies on various datasets. Moreover, we focus on channel selection methods that choose fewer channels with great precision. Finally, our main finding is that a smaller channel set, typically 10-30% of total channels, provided excellent performance compared to other existing studies.