EEG-Based Emotion Recognition Using Logistic Regression with Gaussian Kernel and Laplacian Prior and Investigation of Critical Frequency Bands

Epileptic EEG patterns recognition through machine learning techniques and relevant time-frequency features

OBJECTIVES

The present study is designed to explore the process of epileptic patterns' automatic detection, specifically, epileptic spikes and high-frequency oscillations (HFOs), via a selection of machine learning (ML) techniques. The primary motivation for conducting such a research lies mainly in the need to investigate the long-term electroencephalography (EEG) recordings' visual examination process, often considered as a time-consuming and potentially error-prone procedure, requiring a great deal of mental focus and highly experimented neurologists. On attempting to resolve such a challenge, a number of state-of-the-art ML algorithms have been evaluated and compare in terms of performance, to pinpoint the most effective algorithm fit for accurately extracting epileptic EEG patterns.

CONTENT

Based on intracranial as well as simulated EEG data, the attained findings turn out to reveal that the randomforest (RF) method proved to be the most consistently effective approach, significantly outperforming the entirety of examined methods in terms of EEG recordings epileptic-pattern identification. Indeed, the RF classifier appeared to record an average balanced classification rate (BCR) of 92.38 % in regard to spikes recognition process, and 78.77 % in terms of HFOs detection.

SUMMARY

Compared to other approaches, our results provide valuable insights into the RF classifier's effectiveness as a powerful ML technique, fit for detecting EEG signals born epileptic bursts.

OUTLOOK

As a potential future work, we envisage to further validate and sustain our major reached findings through incorporating a larger EEG dataset. We also aim to explore the generative adversarial networks (GANs) application so as to generate synthetic EEG signals or combine signal generation techniques with deep learning approaches. Through this new vein of thought, we actually preconize to enhance and boost the automated detection methods' performance even more, thereby, noticeably enhancing the epileptic EEG pattern recognition area.

Characterizing social and cognitive EEG-ERP through multiple kernel learning

EEG-ERP social-cognitive studies with healthy populations commonly fail to provide significant evidence due to low-quality data and the inherent similarity between groups. We propose a multiple kernel learning-based approach to enhance classification accuracy while keeping the traceability of the features (frequency bands or regions of interest) as a linear combination of kernels. These weights determine the relevance of each source of information, which is crucial for specialists. As a case study, we classify healthy ex-combatants of the Colombian armed conflict and civilians through a cognitive valence recognition task. Although previous works have shown accuracies below 80% with these groups, our proposal achieved an F1 score of 98%, revealing the most relevant bands and brain regions, which are the base for socio-cognitive trainings. With this methodology, we aim to contribute to standardizing EEG analyses and enhancing their statistics.

Emotion recognition with residual network driven by spatial-frequency characteristics of EEG recorded from hearing-impaired adults in response to video clips

In recent years, emotion recognition based on electroencephalography (EEG) signals has attracted plenty of attention. Most of the existing works focused on normal or depressed people. Due to the lack of hearing ability, it is difficult for hearing-impaired people to express their emotions through language in their social activities. In this work, we collected the EEG signals of hearing-impaired subjects when they were watching six kinds of emotional video clips (happiness, inspiration, neutral, anger, fear, and sadness) for emotion recognition. The biharmonic spline interpolation method was utilized to convert the traditional frequency domain features, Differential Entropy (DE), Power Spectral Density (PSD), and Wavelet Entropy (WE) into the spatial domain. The patch embedding (PE) method was used to segment the feature map into the same patch to obtain the differences in the distribution of emotional information among brain regions. For feature classification, a compact residual network with Depthwise convolution (DC) and Pointwise convolution (PC) is proposed to separate spatial and channel mixing dimensions to better extract information between channels. Dependent subject experiments based on 70% training sets and 30% testing sets were performed. The results showed that the average classification accuracies by PE (DE), PE (PSD), and PE (WE) were 91.75%, 85.53%, and 75.68%, respectively which were improved by 11.77%, 23.54%, and 16.61% compared with DE, PSD, and WE. Moreover, the comparison experiments were carried out on the SEED and DEAP datasets with PE (DE), which achieved average accuracies of 90.04% (positive, neutral, and negative) and 88.75% (high valence and low valence). By exploring the emotional brain regions, we found that the frontal, parietal, and temporal lobes of hearing-impaired people were associated with emotional activity compared to normal people whose main emotional brain area was the frontal lobe.

A Review of AI Cloud and Edge Sensors, Methods, and Applications for the Recognition of Emotional, Affective and Physiological States

Affective, emotional, and physiological states (AFFECT) detection and recognition by capturing human signals is a fast-growing area, which has been applied across numerous domains. The research aim is to review publications on how techniques that use brain and biometric sensors can be used for AFFECT recognition, consolidate the findings, provide a rationale for the current methods, compare the effectiveness of existing methods, and quantify how likely they are to address the issues/challenges in the field. In efforts to achieve the key goals of Society 5.0, Industry 5.0, and human-centered design better, the recognition of emotional, affective, and physiological states is progressively becoming an important matter and offers tremendous growth of knowledge and progress in these and other related fields. In this research, a review of AFFECT recognition brain and biometric sensors, methods, and applications was performed, based on Plutchik's wheel of emotions. Due to the immense variety of existing sensors and sensing systems, this study aimed to provide an analysis of the available sensors that can be used to define human AFFECT, and to classify them based on the type of sensing area and their efficiency in real implementations. Based on statistical and multiple criteria analysis across 169 nations, our outcomes introduce a connection between a nation's success, its number of Web of Science articles published, and its frequency of citation on AFFECT recognition. The principal conclusions present how this research contributes to the big picture in the field under analysis and explore forthcoming study trends.

Classifying oscillatory brain activity associated with Indian Rasas using network metrics

Neural signatures for the western classification of emotions have been widely discussed in the literature. The ancient Indian treatise on performing arts known as Natyashastra categorizes emotions into nine classes, known as Rasas. Rasa-as opposed to a pure emotion-is defined as a superposition of certain transitory, dominant, and temperamental emotional states. Although Rasas have been widely discussed in the text, dedicated brain imaging studies have not been conducted in their research. Our study examines the neural oscillations, recorded through electroencephalography (EEG) imaging, that are elicited while experiencing emotional states corresponding to Rasas. We identify differences among them using network-based functional connectivity metrics in five different frequency bands. Further, Random Forest models are trained on the extracted network features, and we present our findings based on classifier predictions. We observe slow (delta) and fast brain waves (beta and gamma) exhibited the maximum discriminating features between Rasas, whereas alpha and theta bands showed fewer distinguishable pairs. Out of nine Rasas, Sringaram (love), Bibhatsam (odious), and Bhayanakam (terror) were distinguishable from other Rasas the most across frequency bands. On the scale of most network metrics, Raudram (rage) and Sringaram are on the extremes, which also resulted in their good classification accuracy of 95%. This is reminiscent of the circumplex model where anger and contentment/happiness are on extremes on the pleasant scale. Interestingly, our results are consistent with the previous studies which highlight the significant role of higher frequency oscillations in the classification of emotions, in contrast to the alpha band that has shows non-significant differences across emotions. This research contributes to one of the first attempts to investigate the neural correlates of Rasas. Therefore, the results of this study can potentially guide the explorations into the entrainment of brain oscillations between performers and viewers, which can further lead to better performances and viewer experience.

The Effect of Time Window Length on EEG-Based Emotion Recognition

Various lengths of time window have been used in feature extraction for electroencephalogram (EEG) signal processing in previous studies. However, the effect of time window length on feature extraction for the downstream tasks such as emotion recognition has not been well examined. To this end, we investigate the effect of different time window (TW) lengths on human emotion recognition to find the optimal TW length for extracting electroencephalogram (EEG) emotion signals. Both power spectral density (PSD) features and differential entropy (DE) features are used to evaluate the effectiveness of different TW lengths based on the SJTU emotion EEG dataset (SEED). Different lengths of TW are then processed with an EEG feature-processing approach, namely experiment-level batch normalization (ELBN). The processed features are used to perform emotion recognition tasks in the six classifiers, the results of which are then compared with the results without ELBN. The recognition accuracies indicate that a 2-s TW length has the best performance on emotion recognition and is the most suitable to be used in EEG feature extraction for emotion recognition. The deployment of ELBN in the 2-s TW can further improve the emotion recognition performances by 21.63% and 5.04% when using an SVM based on PSD and DE features, respectively. These results provide a solid reference for the selection of TW length in analyzing EEG signals for applications in intelligent systems.

Comparison of cross-subject EEG emotion recognition algorithms in the BCI Controlled Robot Contest in World Robot Contest 2021

Electroencephalogram (EEG) data depict various emotional states and reflect brain activity. There has been increasing interest in EEG emotion recognition in brain–computer interface systems (BCIs). In the World Robot Contest (WRC), the BCI Controlled Robot Contest successfully staged an emotion recognition technology competition. Three types of emotions (happy, sad, and neutral) are modeled using EEG signals. In this study, 5 methods employed by different teams are compared. The results reveal that classical machine learning approaches and deep learning methods perform similarly in offline recognition, whereas deep learning methods perform better in online cross-subject decoding.

Entropy and the Emotional Brain: Overview of a Research Field

During the last years, there has been a notable increase in the number of studies focused on the assessment of brain dynamics for the recognition of emotional states by means of nonlinear methodologies. More precisely, different entropy metrics have been applied for the analysis of electroencephalographic recordings for the detection of emotions. In this sense, regularity-based entropy metrics, symbolic predictability-based entropy indices, and different multiscale and multilag variants of the aforementioned methods have been successfully tested in a series of studies for emotion recognition from the EEG recording. This chapter aims to unify all those contributions to this scientific area, summarizing the main discoverings recently achieved in this research field.

A Novel Psychotherapy Effect Detector of Public Art Based on ResNet and EEG Imaging

Methods

The EEG emotion dataset seed is used for feature extraction with DE, and the emotion is recognized by ResNet. Adam optimizer is used to classify the extracted DE through ResNet50 model. Each batch is set as 5 groups of data and is trained for 50 rounds, then the model is optimized, and the accuracy rate is 76.47%, which output the probability of good emotion through the model. We put the model optimized by ResNet into the intelligent module and visualize it with numerical value.

Results

The detector designed by EEG data and ResNet50 optimization model has high accuracy. The results show that the error between the detector data and the questionnaire interview data is small, the average error is 2.77, and the accuracy is 97%. The closer the subject's emotion before the test is to neutral emotion, the closer the questionnaire result is to the test result of the tester, and the smaller the error is. The difference between the tester data and the survey questionnaire data is 0.2, which is in good agreement and has small error. It can be seen that the detector has high accuracy.

Conclusion

Our proposed public art psychotherapy effect detector has good accuracy in detecting users' emotions. It can detect the group psychotherapy effect of public art and can classify and screen a large number of public arts in the city by quantitative methods. It provides support for further summarizing the practical utility of public art and provides a new way for the optimal design and follow-up evaluation of public art design.

Automated Feature Extraction on AsMap for Emotion Classification Using EEG

Emotion recognition using EEG has been widely studied to address the challenges associated with affective computing. Using manual feature extraction methods on EEG signals results in sub-optimal performance by the learning models. With the advancements in deep learning as a tool for automated feature engineering, in this work, a hybrid of manual and automatic feature extraction methods has been proposed. The asymmetry in different brain regions is captured in a 2D vector, termed the AsMap, from the differential entropy features of EEG signals. These AsMaps are then used to extract features automatically using a convolutional neural network model. The proposed feature extraction method has been compared with differential entropy and other feature extraction methods such as relative asymmetry, differential asymmetry and differential caudality. Experiments are conducted using the SJTU emotion EEG dataset and the DEAP dataset on different classification problems based on the number of classes. Results obtained indicate that the proposed method of feature extraction results in higher classification accuracy, outperforming the other feature extraction methods. The highest classification accuracy of 97.10% is achieved on a three-class classification problem using the SJTU emotion EEG dataset. Further, this work has also assessed the impact of window size on classification accuracy.

Automated emotion recognition: Current trends and future perspectives

BACKGROUND

Human emotions greatly affect the actions of a person. The automated emotion recognition has applications in multiple domains such as health care, e-learning, surveillance, etc. The development of computer-aided diagnosis (CAD) tools has led to the automated recognition of human emotions.

OBJECTIVE

This review paper provides an insight into various methods employed using electroencephalogram (EEG), facial, and speech signals coupled with multi-modal emotion recognition techniques. In this work, we have reviewed most of the state-of-the-art papers published on this topic.

METHOD

This study was carried out by considering the various emotion recognition (ER) models proposed between 2016 and 2021. The papers were analysed based on methods employed, classifier used and performance obtained.

RESULTS

There is a significant rise in the application of deep learning techniques for ER. They have been widely applied for EEG, speech, facial expression, and multimodal features to develop an accurate ER model.

CONCLUSION

Our study reveals that most of the proposed machine and deep learning-based systems have yielded good performances for automated ER in a controlled environment. However, there is a need to obtain high performance for ER even in an uncontrolled environment.

Strict-Feedback Backstepping Digital Twin and Machine Learning Solution in AE Signals for Bearing Crack Identification

Bearings are nonlinear systems that can be used in several industrial applications. In this study, the combination of a strict-feedback backstepping digital twin and machine learning algorithm was developed for bearing crack type/size diagnosis. Acoustic emission sensors were used to collect normal and abnormal data for various crack sizes and motor speeds. The proposed method has three main steps. In the first step, the strict-feedback backstepping digital twin is designed for acoustic emission signal modeling and estimation. After that, the acoustic emission residual signal is generated. Finally, a support vector machine is recommended for crack type/size classification. The proposed digital twin is presented in two steps, (a) AE signal modeling and (b) AE signal estimation. The AE signal in normal conditions is modeled using an autoregressive technique, the Laguerre algorithm, a support vector regression technique and a Gaussian process regression procedure. To design the proposed digital twin, a strict-feedback backstepping observer, an integral term, a support vector regression and a fuzzy logic algorithm are suggested for AE signal estimation. The Ulsan Industrial Artificial Intelligence (UIAI) Lab’s bearing dataset was used to test the efficiency of the combined strict-feedback backstepping digital twin and machine learning technique for bearing crack type/size diagnosis. The average accuracies of the crack type diagnosis and crack size diagnosis of acoustic emission signals for the bearings used in the proposed algorithm were 97.13% and 96.9%, respectively.

Real-Time Emotion Classification Using EEG Data Stream in E-Learning Contexts

In face-to-face and online learning, emotions and emotional intelligence have an influence and play an essential role. Learners’ emotions are crucial for e-learning system because they promote or restrain the learning. Many researchers have investigated the impacts of emotions in enhancing and maximizing e-learning outcomes. Several machine learning and deep learning approaches have also been proposed to achieve this goal. All such approaches are suitable for an offline mode, where the data for emotion classification are stored and can be accessed infinitely. However, these offline mode approaches are inappropriate for real-time emotion classification when the data are coming in a continuous stream and data can be seen to the model at once only. We also need real-time responses according to the emotional state. For this, we propose a real-time emotion classification system (RECS)-based Logistic Regression (LR) trained in an online fashion using the Stochastic Gradient Descent (SGD) algorithm. The proposed RECS is capable of classifying emotions in real-time by training the model in an online fashion using an EEG signal stream. To validate the performance of RECS, we have used the DEAP data set, which is the most widely used benchmark data set for emotion classification. The results show that the proposed approach can effectively classify emotions in real-time from the EEG data stream, which achieved a better accuracy and F1-score than other offline and online approaches. The developed real-time emotion classification system is analyzed in an e-learning context scenario.

Hybrid Method of Automated EEG Signals' Selection Using Reversed Correlation Algorithm for Improved Classification of Emotions

Based on the growing interest in encephalography to enhance human-computer interaction (HCI) and develop brain-computer interfaces (BCIs) for control and monitoring applications, efficient information retrieval from EEG sensors is of great importance. It is difficult due to noise from the internal and external artifacts and physiological interferences. The enhancement of the EEG-based emotion recognition processes can be achieved by selecting features that should be taken into account in further analysis. Therefore, the automatic feature selection of EEG signals is an important research area. We propose a multistep hybrid approach incorporating the Reversed Correlation Algorithm for automated frequency band-electrode combinations selection. Our method is simple to use and significantly reduces the number of sensors to only three channels. The proposed method has been verified by experiments performed on the DEAP dataset. The obtained effects have been evaluated regarding the accuracy of two emotions-valence and arousal. In comparison to other research studies, our method achieved classification results that were 4.20-8.44% greater. Moreover, it can be perceived as a universal EEG signal classification technique, as it belongs to unsupervised methods.

The Role of Features Types and Personalized Assessment in Detecting Affective State Using Dry Electrode EEG

Assessing the human affective state using electroencephalography (EEG) have shown good potential but failed to demonstrate reliable performance in real-life applications. Especially if one applies a setup that might impact affective processing and relies on generalized models of affect. Additionally, using subjective assessment of ones affect as ground truth has often been disputed. To shed the light on the former challenge we explored the use of a convenient EEG system with 20 participants to capture their reaction to affective movie clips in a naturalistic setting. Employing state-of-the-art machine learning approach demonstrated that the highest performance is reached when combining linear features, namely symmetry features and single-channel features, with nonlinear ones derived by a multiscale entropy approach. Nevertheless, the best performance, reflected in the highest F1-score achieved in a binary classification task for valence was 0.71 and for arousal 0.62. The performance was 10–20% better compared to using ratings provided by 13 independent raters. We argue that affective self-assessment might be underrated and it is crucial to account for personal differences in both perception and physiological response to affective cues.

A Discriminative Multi-Output Gaussian Processes Scheme for Brain Electrical Activity Analysis

The study of brain electrical activity (BEA) from different cognitive conditions has attracted a lot of interest in the last decade due to the high number of possible applications that could be generated from it. In this work, a discriminative framework for BEA via electroencephalography (EEG) is proposed based on multi-output Gaussian Processes (MOGPs) with a specialized spectral kernel. First, a signal segmentation stage is executed, and the channels from the EEG are used as the model outputs. Then, a novel covariance function within the MOGP known as the multispectral mixture kernel (MOSM) allows us to find and quantify the relationships between different channels. Several MOGPs are trained from different conditions grouped in bi-class problems, and the discrimination is performed based on the likelihood score of the test signals against all the models. Finally, the mean likelihood is computed to predict the correspondence of new inputs with each class’s existing models. Results show that this framework allows us to model the EEG signals adequately using generative models and allows analyzing the relationships between channels of the EEG for a particular condition. At the same time, the set of trained MOGPs is well suited to discriminate new input data.