Discrimination of emotional states from scalp- and intracranial EEG using multiscale Rényi entropy

The Role of Features Types and Personalized Assessment in Detecting Affective State Using Dry Electrode EEG

Assessing the human affective state using electroencephalography (EEG) have shown good potential but failed to demonstrate reliable performance in real-life applications. Especially if one applies a setup that might impact affective processing and relies on generalized models of affect. Additionally, using subjective assessment of ones affect as ground truth has often been disputed. To shed the light on the former challenge we explored the use of a convenient EEG system with 20 participants to capture their reaction to affective movie clips in a naturalistic setting. Employing state-of-the-art machine learning approach demonstrated that the highest performance is reached when combining linear features, namely symmetry features and single-channel features, with nonlinear ones derived by a multiscale entropy approach. Nevertheless, the best performance, reflected in the highest F1-score achieved in a binary classification task for valence was 0.71 and for arousal 0.62. The performance was 10–20% better compared to using ratings provided by 13 independent raters. We argue that affective self-assessment might be underrated and it is crucial to account for personal differences in both perception and physiological response to affective cues.

Localization of deep brain activity with scalp and subdural EEG

To what extent electrocorticography (ECoG) and electroencephalography (scalp EEG) differ in their capability to locate sources of deep brain activity is far from evident. Compared to EEG, the spatial resolution and signal-to-noise ratio of ECoG is superior but its spatial coverage is more restricted, as is arguably the volume of tissue activity effectively measured from. Moreover, scalp EEG studies are providing evidence of locating activity from deep sources such as the hippocampus using high-density setups during quiet wakefulness. To address this question, we recorded a multimodal dataset from 4 patients with refractory epilepsy during quiet wakefulness. This data comprises simultaneous scalp, subdural and depth EEG electrode recordings. The latter was located in the hippocampus or insula and provided us with our "ground truth" for source localization of deep activity. We applied independent component analysis (ICA) for the purpose of separating the independent sources in theta, alpha and beta frequency band activity. In all patients subdural- and scalp EEG components were observed which had a significant zero-lag correlation with one or more contacts of the depth electrodes. Subsequent dipole modeling of the correlating components revealed dipole locations that were significantly closer to the depth electrodes compared to the dipole location of non-correlating components. These findings support the idea that components found in both recording modalities originate from neural activity in close proximity to the depth electrodes. Sources localized with subdural electrodes were ~70% closer to the depth electrode than sources localized with EEG with an absolute improvement of around ~2cm. In our opinion, this is not a considerable improvement in source localization accuracy given that, for clinical purposes, ECoG electrodes were implanted in close proximity to the depth electrodes. Furthermore, the ECoG grid attenuates the scalp EEG, due to the electrically isolating silastic sheets in which the ECoG electrodes are embedded. Our results on dipole modeling show that the deep source localization accuracy of scalp EEG is comparable to that of ECoG. SIGNIFICANCE STATEMENT: Deep and subcortical regions play an important role in brain function. However, as joint recordings at multiple spatial scales to study brain function in humans are still scarce, it is still unresolved to what extent ECoG and EEG differ in their capability to locate sources of deep brain activity. To the best of our knowledge, this is the first study presenting a dataset of simultaneously recorded EEG, ECoG and depth electrodes in the hippocampus or insula, with a focus on non-epileptiform activity (quiet wakefulness). Furthermore, we are the first study to provide experimental findings on the comparison of source localization of deep cortical structures between invasive and non-invasive brain activity measured from the cortical surface.

Real-Time Multi-Modal Estimation of Dynamically Evoked Emotions Using EEG, Heart Rate and Galvanic Skin Response

Emotion estimation systems based on brain and physiological signals such as electro encephalography (EEG), blood-volume pressure (BVP), and galvanic skin response (GSR) are gaining special attention in recent years due to the possibilities they offer. The field of human–robot interactions (HRIs) could benefit from a broadened understanding of the brain and physiological emotion encoding, together with the use of lightweight software and cheap wearable devices, and thus improve the capabilities of robots to fully engage with the users emotional reactions. In this paper, a previously developed methodology for real-time emotion estimation aimed for its use in the field of HRI is tested under realistic circumstances using a self-generated database created using dynamically evoked emotions. Other state-of-the-art, real-time approaches address emotion estimation using constant stimuli to facilitate the analysis of the evoked responses, remaining far from real scenarios since emotions are dynamically evoked. The proposed approach studies the feasibility of the emotion estimation methodology previously developed, under an experimentation paradigm that imitates a more realistic scenario involving dynamically evoked emotions by using a dramatic film as the experimental paradigm. The emotion estimation methodology has proved to perform on real-time constraints while maintaining high accuracy on emotion estimation when using the self-produced dynamically evoked emotions multi-signal database.

Predicting Premature Video Skipping and Viewer Interest from EEG Recordings

Brain–computer interfacing has enjoyed growing attention, not only due to the stunning demonstrations with severely disabled patients, but also the advent of economically viable solutions in areas such as neuromarketing, mental state monitoring, and future human–machine interaction. An interesting case, at least for neuromarketers, is to monitor the customer’s mental state in response to watching a commercial. In this paper, as a novelty, we propose a method to predict from electroencephalography (EEG) recordings whether individuals decide to skip watching a video trailer. Based on multiscale sample entropy and signal power, indices were computed that gauge the viewer’s engagement and emotional affect. We then trained a support vector machine (SVM), a k-nearest neighbor (kNN), and a random forest (RF) classifier to predict whether the viewer declares interest in watching the video and whether he/she decides to skip it prematurely. Our model achieved an average single-subject classification accuracy of 75.803% for skipping and 73.3% for viewer interest for the SVM, 82.223% for skipping and 78.333% for viewer interest for the kNN, and 80.003% for skipping and 75.555% for interest for the RF. We conclude that EEG can provide indications of viewer interest and skipping behavior and provide directions for future research.

Investigating EEG Patterns for Dual-Stimuli Induced Human Fear Emotional State

Most electroencephalography (EEG) based emotion recognition systems make use of videos and images as stimuli. Few used sounds, and even fewer studies were found involving self-induced emotions. Furthermore, most of the studies rely on single stimuli to evoke emotions. The question of "whether different stimuli for same emotion elicitation generate any subject-independent correlations" remains unanswered. This paper introduces a dual modality based emotion elicitation paradigm to investigate if emotions can be classified induced with different stimuli. A method has been proposed based on common spatial pattern (CSP) and linear discriminant analysis (LDA) to analyze human brain signals for fear emotions evoked with two different stimuli. Self-induced emotional imagery is one of the considered stimuli, while audio/video clips are used as the other stimuli. The method extracts features from the CSP algorithm and LDA performs classification. To investigate associated EEG correlations, a spectral analysis was performed. To further improve the performance, CSP was compared with other regularized techniques. Critical EEG channels are identified based on spatial filter weights. To the best of our knowledge, our work provides the first contribution for the assessment of EEG correlations in the case of self versus video induced emotions captured with a commercial grade EEG device.

A novel approach to emotion recognition using local subset feature selection and modified Dempster-Shafer theory

Background

Emotion recognition is an increasingly important field of research in brain computer interactions.

Introduction

With the advance of technology, automatic emotion recognition systems no longer seem far-fetched. Be that as it may, detecting neural correlates of emotion has remained a substantial bottleneck. Settling this issue will be a breakthrough of significance in the literature.

Methods

The current study aims to identify the correlations between different emotions and brain regions with the help of suitable electrodes. Initially, independent component analysis algorithm is employed to remove artifacts and extract the independent components. The informative channels are then selected based on the thresholded average activity value for obtained components. Afterwards, effective features are extracted from selected channels common between all emotion classes. Features are reduced using the local subset feature selection method and then fed to a new classification model using modified Dempster-Shafer theory of evidence.

Results

The presented method is employed to DEAP dataset and the results are compared to those of previous studies, which highlights the significant ability of this method to recognize emotions through electroencephalography, by the accuracy of about 91%. Finally, the obtained results are discussed and new aspects are introduced.

Conclusions

The present study addresses the long-standing challenge of finding neural correlates between human emotions and the activated brain regions. Also, we managed to solve uncertainty problem in emotion classification which is one of the most challenging issues in this field. The proposed method could be employed in other practical applications in future.