The dynamics of EEG gamma responses to unpleasant visual stimuli: from local activity to functional connectivity

Theta oscillations underlie the interplay between emotional processing and empathy

Emotional reactions to salient stimuli are well documented in psychophysiological research. However, some individual variables that can influence how people process emotions (i.e., empathy traits) have received little consideration. The present study investigated the relationship between emotions and empathy. Forty participants completed the Interpersonal Reactivity Index, a questionnaire that measure general and specific empathy dimensions. Then, emotional (erotic and mutilation) and non-emotional pictures were presented, during electroencephalographic recording. Valence and arousal were evaluated for each stimulus. Behavioral results revealed a positive correlation between the arousal induced by mutilation pictures and personal distress (i.e., feeling discomfort in emergency situations). At the electrophysiological level, theta activity elicited by positive and negative emotion processing in the superior frontal gyrus was associated with personal distress. Moreover, erotic-related theta in the middle frontal gyrus was associated with subjective judgement of erotic stimulus valence. Overall, theta activity modulated the interplay between emotions and empathy.

Altered pattern of theta and gamma oscillation to visual stimuli in patients with postconcussion syndrome

Although many patients with mild traumatic brain injury (mTBI) suffer from postconcussional syndrome (PCS) including abnormal emotional responses, most conventional imaging studies fail to detect any causative brain lesion. We hypothesized that event-related electroencephalography (EEG) recordings with time-frequency analysis would show a distinguishable pattern in patients with mTBI with PCS compared with normal healthy controls. EEG signals were collected from a total of 18 subjects: eight patients with mTBI with PCS and 10 healthy control subjects. The signals were recorded while the subjects were presented with affective visual stimuli, including neutral, pleasant, and unpleasant emotional cues. Event-related spectral perturbation analysis was performed to calculate frontal midline theta activity and posterior midline gamma activity, followed by statistical analysis to identify whether patients with mTBI with PCS have distinct patterns of theta or gamma oscillations in response to affective stimuli. Compared with the healthy control group, patients with mTBI with PCS did not show a significant increase in the power of frontal theta activity in response to the pleasant stimuli, indicating less susceptibility toward pleasant cues. Moreover, the patient group showed attenuated gamma oscillatory activity, with no clear alteration in gamma oscillations in response to either pleasant or unpleasant cues. This study demonstrates that patients with mTBI with PCS exhibited altered patterns of oscillatory activities in the theta and gamma bands in response to affective visual stimuli compared with the normal control group. The current finding implicates that these distinguishable patterns of brain oscillation may represent the mechanism behind various psychiatric symptoms in patients with mTBI.NEW & NOTEWORTHY Patients with mild traumatic brain injury (mTBI) with postconcussional syndrome (PCS) exhibited altered patterns of changes in oscillatory activities in the theta and gamma bands in response to visual affective stimuli. Distinguishable patterns of brain oscillation may represent the mechanism behind various psychiatric symptoms in patients with mTBI.

Review of EEG Affective Recognition with a Neuroscience Perspective

Emotions are a series of subconscious, fleeting, and sometimes elusive manifestations of the human innate system. They play crucial roles in everyday life-influencing the way we evaluate ourselves, our surroundings, and how we interact with our world. To date, there has been an abundance of research on the domains of neuroscience and affective computing, with experimental evidence and neural network models, respectively, to elucidate the neural circuitry involved in and neural correlates for emotion recognition. Recent advances in affective computing neural network models often relate closely to evidence and perspectives gathered from neuroscience to explain the models. Specifically, there has been growing interest in the area of EEG-based emotion recognition to adopt models based on the neural underpinnings of the processing, generation, and subsequent collection of EEG data. In this respect, our review focuses on providing neuroscientific evidence and perspectives to discuss how emotions potentially come forth as the product of neural activities occurring at the level of subcortical structures within the brain's emotional circuitry and the association with current affective computing models in recognizing emotions. Furthermore, we discuss whether such biologically inspired modeling is the solution to advance the field in EEG-based emotion recognition and beyond.

Effect of stress on spatial working memory and EEG signal dynamics in the follicular and luteal phases of the menstrual cycle in young single girls

AIM

Women undergo behavioral changes during the menstrual cycle. This study aimed to investigate the effect of estradiol (Es) on stress and effect of stress on spatial working memory (WM) and also to investigate electroencephalogram (EEG) signal's dynamics in the early and late follicular (EF and LF) and luteal (LU) phases of unmarried girls' menstrual cycle.

METHODS

Stress was induced by presentation of a short (3 min) movie clip. Simultaneous with a memory test and stress induction, EEG, serum Es levels, and galvanic skin response (GSR) were assessed.

RESULTS

Serum Es concentrations were decreased in LF, LU, and EF phases. The mean GSR score decreased after stress induction in all three phases, but it increased in the LF and LU phases versus the EF phase. Spatial WM diminished after stress induction in all three phases, but it increased in the LF phase versus the two phases before and after stress induction. Average power spectrum density in all frequency bands increased after stress induction in the frontal and prefrontal channels in the spatial WM test.

CONCLUSION

The results showed that stress led to spatial WM dysfunction; however, Es improved spatial WM performance in the LF phase versus the other two phases.

On the relationship between EEG spectral analysis and pre-sleep cognitive arousal in insomnia disorder: towards an integrated model of cognitive and cortical arousal

According to the hyperarousal model, insomnia is characterised by increased arousal in the cortical, cognitive, and physiological domains. However, the interaction between these arousal domains is poorly understood. The present observational case-control study aimed to investigate cortical arousal during the night, pre-sleep cognitive arousal and the relationship between these two domains. A total of 109 patients with insomnia disorder (ID) and 109 age-and gender matched healthy controls were investigated on two sleep laboratory nights. Electroencephalographic (EEG) spectral power during non-rapid eye movement (NREM) and REM sleep was analysed as a measure of cortical arousal. In addition, patients completed the Pre-Sleep Arousal Scale (PSAS), which consists of two subscales, one for cognitive arousal (PSAS-CA) and one for self-reported somatic arousal (PSAS-SA). The relationship between the subscale scores and EEG spectral power was calculated by multi- and univariate analyses of variance. During NREM and REM sleep, patients with ID showed significantly increased spectral power in the EEG gamma band. In addition, patients with ID showed significantly increased scores on both subscales of the PSAS. The PSAS-CA score was significantly associated with increased NREM and REM gamma power, whereas PSAS-SA was associated with decreases in NREM and REM gamma power. Consistent with our hypothesis, patients with ID showed increased cortical and cognitive arousal. Moreover, there was an association between these two arousal domains, which may indicate that cortical arousal during the night is (at least in part) elicited by pre-sleep worry and rumination.

Enhanced electroencephalography effective connectivity in frontal low-gamma band correlates of emotional regulation after mindfulness training

Practicing mindfulness, focusing attention on the internal and external experiences occurring in the present moment with open and nonjudgement stance, can lead to the development of emotional regulation skills. Yet, the effective connectivity of brain regions during mindfulness has been largely unexplored. Studies have shown that mindfulness practice promotes functional connectivity in practitioners, potentially due to improved emotional regulation abilities and increased connectivity in the lateral prefrontal areas. To examine the changes in effective connectivity due to mindfulness training, we analyzed electroencephalogram (EEG) signals taken before and after mindfulness training, focusing on training-related effective connectivity changes in the frontal area. The mindfulness training group participated in an 8-week mindfulness-based stress reduction (MBSR) program. The control group did not take part. Regardless of the specific mindfulness practice used, low-gamma band effective connectivity increased globally after the mindfulness training. High-beta band effective connectivity increased globally only during Breathing. Moreover, relatively higher outgoing effective connectivity strength was seen during Resting and Breathing and Body-scan. By analyzing the changes in outgoing and incoming connectivity edges, both F7 and F8 exhibited strong parietal connectivity during Resting and Breathing. Multiple regression analysis revealed that the changes in effective connectivity of the right lateral prefrontal area predicted mindfulness and emotional regulation abilities. These results partially support the theory that the lateral prefrontal areas have top-down modulatory control, as these areas have high outflow effective connectivity, implying that mindfulness training cultivates better emotional regulation.

Physiological comfort evaluation under different airflow directions in a heating environment

Background

Indoor airflow and thermal comfort are difficult to assess through subjective evaluations because airflow sensations can differ based on various factors, such as personal characteristics, interests, preferences, and the current state of mind. Thus, subjective evaluations should be combined with objective assessments, such as physiological measurements. This study evaluated airflow and thermal comfort through physiological measurements, including skin temperature, electroencephalography, respiration, and electrocardiography, in addition to subjective evaluations.

Methods

Twenty participants entered a test room at 30 °C after staying in an acclimation room at 18 °C for 20 min. They were exposed to indirect and direct airflow toward their faces and performed four tasks under each condition: resting, counting to 10 s following time alerts, counting to 10 s in the mind, and mental calculation. The mean speed of the air directed to the participants’ faces was 0.123 m/s and 0.225 m/s in the indirect and direct conditions, respectively.

Results

The gamma and beta bands of electroencephalograms taken at the left-temporal (T3) and left-parietal (P7) sites showed significantly lower amplitudes under the indirect condition (gamma, T3: p = 0.034, P7: p = 0.030; beta, T3: p = 0.051, P7: p = 0.028). Similarly, the variability of respiration was lower under the indirect condition (p < 0.010). The amplitudes of gamma and beta waves showed significant correlations with anxiousness levels (gamma, T3: r = 0.41; beta, T3: r = 0.35).

Conclusions

Our results suggest that indirect heating airflow causes lower mental stress and fatigue than those induced by direct flow, which is equivalent to more comfort. The results of this study suggest that physiological measurements can be used for the evaluation of unconscious indoor comfort, which cannot be detected by subjective evaluations alone.

Relations between family cohesion and adolescent-parent's neural synchrony in response to emotional stimulations

BACKGROUND

The interaction between parent and adolescent is more challenging than in other age periods. Family cohesion seriously impacts parent-adolescent emotional interactions. However, the underlying neural mechanism has not been fully examined. This study examined the differences in the neural synchrony in response to emotional film clips between high and low family cohesion adolescent-parent dyads by using the electroencephalograph (EEG) hyperscanning.

RESULTS

Simultaneously electroencephalograph (EEG) was recorded while 15 low family cohesion parent-adolescent dyads (LFCs)and 14 high family cohesion parent-adolescent dyads (HFCs)received different emotional induction when viewing film clips. Interbrain phase-locking-value (PLV) in gamma band was used to calculate parent-adolescent dyads' interbrain synchrony. Results showed that higher gamma interbrain synchrony was observed in the HFCs than the LFCs in the positive conditions. However, there was no significant difference between the HFCs and LFCs in other conditions. Also, the HFCs had significantly higher gamma interbrain synchrony in the positive conditions than in the negative conditions.

CONCLUSION

Interbrain synchrony may represent an underlying neural mechanism of the parent-adolescent emotional bonding, which is the core of family cohesion.

Characterising the spatial and oscillatory unfolding of Theory of Mind in adults using fMRI and MEG

Theory of Mind (ToM) is a core social cognitive skill that refers to the ability to attribute mental states to others. ToM involves understanding that others have beliefs, thoughts and desires that may be different from one's own and from reality. ToM is crucial to predict behaviour and navigate social interactions. This study employed the complementary methodological advantages of both functional MRI (fMRI) and magnetoencephalography (MEG) to examine the neural underpinnings of ToM in adults. Twenty healthy adults were first recruited to rate and describe 28 videos (15s long), each containing three moving shapes designed to depict either social interactions or random motion (control condition). The first sample of adults produced consistent narratives for 6 of those social videos and of those, 4 social videos and 4 control videos were chosen to include in the neuroimaging study. Another sample of twenty-five adults were then recruited to complete the neuroimaging in MEG and fMRI. In fMRI, we found increased activation in frontal-parietal regions in the social compared to the control condition corroborating previous fMRI findings. In MEG, we found recruitment of ToM networks in the social condition in theta, beta and gamma bands. The right supramarginal and angular gyri (right temporal parietal junction), right inferior parietal lobe and right temporal pole were recruited in the first 5s of the videos. Frontal regions such as the superior frontal gyrus were recruited in the second time window (5–10s). Brain regions such as the bilateral amygdalae were also recruited (5–10s), indicating that various social processes were integrated in understanding the social videos. Our study is one of the first to combine multi-modal neuroimaging to examine the neural networks underlying social cognitive processes, combining the strengths of the spatial resolution of fMRI and temporal resolution of MEG. Understanding this information from both modalities helped delineate the mechanism by which ToM processing unfolds over time in healthy adults. This allows us to determine a benchmark against which clinical populations can be compared.

Cross-Subject Emotion Recognition Using Fused Entropy Features of EEG

Emotion recognition based on electroencephalography (EEG) has attracted high interest in fields such as health care, user experience evaluation, and human-computer interaction (HCI), as it plays an important role in human daily life. Although various approaches have been proposed to detect emotion states in previous studies, there is still a need to further study the dynamic changes of EEG in different emotions to detect emotion states accurately. Entropy-based features have been proved to be effective in mining the complexity information in EEG in many areas. However, different entropy features vary in revealing the implicit information of EEG. To improve system reliability, in this paper, we propose a framework for EEG-based cross-subject emotion recognition using fused entropy features and a Bidirectional Long Short-term Memory (BiLSTM) network. Features including approximate entropy (AE), fuzzy entropy (FE), Rényi entropy (RE), differential entropy (DE), and multi-scale entropy (MSE) are first calculated to study dynamic emotional information. Then, we train a BiLSTM classifier with the inputs of entropy features to identify different emotions. Our results show that MSE of EEG is more efficient than other single-entropy features in recognizing emotions. The performance of BiLSTM is further improved with an accuracy of 70.05% using fused entropy features compared with that of single-type feature.

Classifying oscillatory brain activity associated with Indian Rasas using network metrics

Neural signatures for the western classification of emotions have been widely discussed in the literature. The ancient Indian treatise on performing arts known as Natyashastra categorizes emotions into nine classes, known as Rasas. Rasa-as opposed to a pure emotion-is defined as a superposition of certain transitory, dominant, and temperamental emotional states. Although Rasas have been widely discussed in the text, dedicated brain imaging studies have not been conducted in their research. Our study examines the neural oscillations, recorded through electroencephalography (EEG) imaging, that are elicited while experiencing emotional states corresponding to Rasas. We identify differences among them using network-based functional connectivity metrics in five different frequency bands. Further, Random Forest models are trained on the extracted network features, and we present our findings based on classifier predictions. We observe slow (delta) and fast brain waves (beta and gamma) exhibited the maximum discriminating features between Rasas, whereas alpha and theta bands showed fewer distinguishable pairs. Out of nine Rasas, Sringaram (love), Bibhatsam (odious), and Bhayanakam (terror) were distinguishable from other Rasas the most across frequency bands. On the scale of most network metrics, Raudram (rage) and Sringaram are on the extremes, which also resulted in their good classification accuracy of 95%. This is reminiscent of the circumplex model where anger and contentment/happiness are on extremes on the pleasant scale. Interestingly, our results are consistent with the previous studies which highlight the significant role of higher frequency oscillations in the classification of emotions, in contrast to the alpha band that has shows non-significant differences across emotions. This research contributes to one of the first attempts to investigate the neural correlates of Rasas. Therefore, the results of this study can potentially guide the explorations into the entrainment of brain oscillations between performers and viewers, which can further lead to better performances and viewer experience.

Cardiac-Brain Dynamics Depend on Context Familiarity and Their Interaction Predicts Experience of Emotional Arousal

Our brain continuously interacts with the body as we engage with the world. Although we are mostly unaware of internal bodily processes, such as our heartbeats, they may be influenced by and in turn influence our perception and emotional feelings. Although there is a recent focus on understanding cardiac interoceptive activity and interaction with brain activity during emotion processing, the investigation of cardiac-brain interactions with more ecologically valid naturalistic emotional stimuli is still very limited. We also do not understand how an essential aspect of emotions, such as context familiarity, influences affective feelings and is linked to statistical interaction between cardiac and brain activity. Hence, to answer these questions, we designed an exploratory study by recording ECG and EEG signals for the emotional events while participants were watching emotional movie clips. Participants also rated their familiarity with the stimulus on the familiarity scale. Linear mixed effect modelling was performed in which the ECG power and familiarity were considered as predictors of EEG power. We focused on three brain regions, including prefrontal (PF), frontocentral (FC) and parietooccipital (PO). The analyses showed that the interaction between the power of cardiac activity in the mid-frequency range and the power in specific EEG bands is dependent on familiarity, such that the interaction is stronger with high familiarity. In addition, the results indicate that arousal is predicted by cardiac-brain interaction, which also depends on familiarity. The results support emotional theories that emphasize context dependency and interoception. Multimodal studies with more realistic stimuli would further enable us to understand and predict different aspects of emotional experience.

Are there consistent abnormalities in event-related EEG oscillations in patients with Alzheimer's disease compared to other diseases belonging to dementia?

Cerebrospinal and structural-molecular neuroimaging in-vivo biomarkers are recommended for diagnostic purposes in Alzheimer's disease (AD) and other dementias; however, they do not explain the effects of AD neuropathology on neurophysiological mechanisms underpinning cognitive processes. Here, an Expert Panel from the Electrophysiology Professional Interest Area of the Alzheimer's Association reviewed the field literature and reached consensus on the event-related electroencephalographic oscillations (EROs) that show consistent abnormalities in patients with significant cognitive deficits due to Alzheimer's, Parkinson's (PD), Lewy body (LBD), and cerebrovascular diseases. Converging evidence from oddball paradigms showed that, as compared to cognitively unimpaired (CU) older adults, AD patients had lower amplitude in widespread delta (>4 Hz) and theta (4-7 Hz) phase-locked EROs as a function of disease severity. Similar effects were also observed in PD, LBD, and/or cerebrovascular cognitive impairment patients. Non-phase-locked alpha (8-12 Hz) and beta (13-30 Hz) oscillations were abnormally reduced (event-related desynchronization, ERD) in AD patients relative to CU. However, studies on patients with other dementias remain lacking. Delta and theta phase-locked EROs during oddball tasks may be useful neurophysiological biomarkers of cognitive systems at work in heuristic and intervention clinical trials performed in AD patients, but more research is needed regarding their potential role for other dementias.

Evaluation of Brain Functional Connectivity from Electroencephalographic Signals Under Different Emotional States

The identification of the emotional states corresponding to the four quadrants of the valence/arousal space has been widely analyzed in the scientific literature by means of multiple techniques. Nevertheless, most of these methods were based on the assessment of each brain region separately, without considering the possible interactions among different areas. In order to study these interconnections, this study computes for the first time the functional connectivity metric called cross-sample entropy for the analysis of the brain synchronization in four groups of emotions from electroencephalographic signals. Outcomes reported a strong synchronization in the interconnections among central, parietal and occipital areas, while the interactions between left frontal and temporal structures with the rest of brain regions presented the lowest coordination. These differences were statistically significant for the four groups of emotions. All emotions were simultaneously classified with a 95.43% of accuracy, overcoming the results reported in previous studies. Moreover, the differences between high and low levels of valence and arousal, taking into account the state of the counterpart dimension, also provided notable findings about the degree of synchronization in the brain within different emotional conditions and the possible implications of these outcomes from a psychophysiological point of view.

Alpha-to-beta- and gamma-band activity reflect predictive coding in affective visual processing

Processing of negative affective pictures typically leads to desynchronization of alpha-to-beta frequencies (ERD) and synchronization of gamma frequencies (ERS). Given that in predictive coding higher frequencies have been associated with prediction errors, while lower frequencies have been linked to expectations, we tested the hypothesis that alpha-to-beta ERD and gamma ERS induced by aversive pictures are associated with expectations and prediction errors, respectively. We recorded EEG while volunteers were involved in a probabilistically cued affective picture task using three different negative valences to produce expectations and prediction errors. Our data show that alpha-to-beta band activity after stimulus presentation was related to the expected valence of the stimulus as predicted by a cue. The absolute mismatch of the expected and actual valence, which denotes an absolute prediction error was related to increases in alpha, beta and gamma band activity. This demonstrates that top-down predictions and bottom-up prediction errors are represented in typical spectral patterns associated with affective picture processing. This study provides direct experimental evidence that negative affective picture processing can be described by neuronal predictive coding computations.

Atypical development of emotional face processing networks in autism spectrum disorder from childhood through to adulthood

•

MEG connectivity to emotional faces in ASD and typical controls 6–39 years of age was investigated.

•

Distinct age-related changes in connectivity were observed in the groups to happy and angry faces.

•

Age-related between-group differences in functional connectivity were found in gamma band.

•

Emotion-specific age-related between-group differences were seen in beta.

•

Findings highlight specific neurodevelopmental trajectories to emotional faces in ASD vs. TD.

Impairments in social functioning are hallmarks of autism spectrum disorder (ASD) and atypical functional connectivity may underlie these difficulties. Emotion processing networks typically undergo protracted maturational changes, however, those with ASD show either hyper- or hypo-connectivity with little consensus on the functional connectivity underpinning emotion processing. Magnetoencephalography was used to investigate age-related changes in whole-brain functional connectivity of eight regions of interest during happy and angry face processing in 190 children, adolescents and adults (6–39 years) with and without ASD. Findings revealed age-related changes from child- through to mid-adulthood in functional connectivity in controls and in ASD in theta, as well as age-related between-group differences across emotions, with connectivity decreasing in ASD, but increasing for controls, in gamma. Greater connectivity to angry faces was observed across groups in gamma. Emotion-specific age-related between-group differences in beta were also found, that showed opposite trends with age for happy and angry in ASD. Our results establish altered, frequency-specific developmental trajectories of functional connectivity in ASD, across distributed networks and a broad age range, which may finally help explain the heterogeneity in the literature.

Development and Progress in Sensors and Technologies for Human Emotion Recognition

With the advancement of human-computer interaction, robotics, and especially humanoid robots, there is an increasing trend for human-to-human communications over online platforms (e.g., zoom). This has become more significant in recent years due to the Covid-19 pandemic situation. The increased use of online platforms for communication signifies the need to build efficient and more interactive human emotion recognition systems. In a human emotion recognition system, the physiological signals of human beings are collected, analyzed, and processed with the help of dedicated learning techniques and algorithms. With the proliferation of emerging technologies, e.g., the Internet of Things (IoT), future Internet, and artificial intelligence, there is a high demand for building scalable, robust, efficient, and trustworthy human recognition systems. In this paper, we present the development and progress in sensors and technologies to detect human emotions. We review the state-of-the-art sensors used for human emotion recognition and different types of activity monitoring. We present the design challenges and provide practical references of such human emotion recognition systems in the real world. Finally, we discuss the current trends in applications and explore the future research directions to address issues, e.g., scalability, security, trust, privacy, transparency, and decentralization.

Detection of Negative Stress through Spectral Features of Electroencephalographic Recordings and a Convolutional Neural Network

In recent years, electroencephalographic (EEG) signals have been intensively used in the area of emotion recognition, partcularly in distress identification due to its negative impact on physical and mental health. Traditionally, brain activity has been studied from a frequency perspective by computing the power spectral density of the EEG recordings and extracting features from different frequency sub-bands. However, these features are often individually extracted from single EEG channels, such that each brain region is separately evaluated, even when it has been corroborated that mental processes are based on the coordination of different brain areas working simultaneously. To take advantage of the brain’s behaviour as a synchronized network, in the present work, 2-D and 3-D spectral images constructed from common 32 channel EEG signals are evaluated for the first time to discern between emotional states of calm and distress using a well-known deep-learning algorithm, such as AlexNet. The obtained results revealed a significant improvement in the classification performance regarding previous works, reaching an accuracy about 84%. Moreover, no significant differences between the results provided by the diverse approaches considered to reconstruct 2-D and 3-D spectral maps from the original location of the EEG channels over the scalp were noticed, thus suggesting that these kinds of images preserve original spatial brain information.

Mathematical Modeling of Brain Activity under Specific Auditory Stimulation

Understanding the connection between different stimuli and the brain response represents a complex research area. However, the use of mathematical models for this purpose is relatively unexplored. The present study investigates the effects of three different auditory stimuli on cerebral biopotentials by means of mathematical functions. The effects of acoustic stimuli (S1, S2, and S3) on cerebral activity were evaluated by electroencephalographic (EEG) recording on 21 subjects for 20 minutes of stimulation, with a 5-minute period of silence before and after stimulation. For the construction of the mathematical models used for the study of the EEG rhythms, we used the Box-Jenkins methodology. Characteristic mathematical models were obtained for the main frequency bands and were expressed by 2 constant functions, 8 first-degree functions, a second-degree function, a fourth-degree function, 6 recursive functions, and 4 periodic functions. The values obtained for the variance estimator are low, demonstrating that the obtained models are correct. The resulting mathematical models allow us to objectively compare the EEG response to the three stimuli, both between the stimuli itself and between each stimulus and the period before stimulation.

Video‐triggered EEG‐emotion public databases and current methods: A survey

Emotions, formed in the process of perceiving external environment, directly affect human daily life, such as social interaction, work efficiency, physical wellness, and mental health. In recent decades, emotion recognition has become a promising research direction with significant application values. Taking the advantages of electroencephalogram (EEG) signals (i.e., high time resolution) and video‐based external emotion evoking (i.e., rich media information), video‐triggered emotion recognition with EEG signals has been proven as a useful tool to conduct emotion‐related studies in a laboratory environment, which provides constructive technical supports for establishing real‐time emotion interaction systems. In this paper, we will focus on video‐triggered EEG‐based emotion recognition and present a systematical introduction of the current available video‐triggered EEG‐based emotion databases with the corresponding analysis methods. First, current video‐triggered EEG databases for emotion recognition (e.g., DEAP, MAHNOB‐HCI, SEED series databases) will be presented with full details. Then, the commonly used EEG feature extraction, feature selection, and modeling methods in video‐triggered EEG‐based emotion recognition will be systematically summarized and a brief review of current situation about video‐triggered EEG‐based emotion studies will be provided. Finally, the limitations and possible prospects of the existing video‐triggered EEG‐emotion databases will be fully discussed.

Oscillatory EEG Signatures of Affective Processes during Interaction with Adaptive Computer Systems

Affect monitoring is being discussed as a novel strategy to make adaptive systems more user-oriented. Basic knowledge about oscillatory processes and functional connectivity underlying affect during naturalistic human-computer interactions (HCI) is, however, scarce. This study assessed local oscillatory power entrainment and distributed functional connectivity in a close-to-naturalistic HCI-paradigm. Sixteen participants interacted with a simulated assistance system which deliberately evoked positive (supporting goal-achievement) and negative (impeding goal-achievement) affective reactions. Electroencephalography (EEG) was used to examine the reactivity of the cortical system during the interaction by studying both event-related (de-)synchronization (ERD/ERS) and event-related functional coupling of cortical networks towards system-initiated assistance. Significantly higher α-band and β-band ERD in centro-parietal and parieto-occipital regions and β-band ERD in bi-lateral fronto-central regions were observed during impeding system behavior. Supportive system behavior activated significantly higher γ-band ERS in bi-hemispheric parietal-occipital regions. This was accompanied by functional coupling of remote β-band and γ-band activity in the medial frontal, left fronto-central and parietal regions, respectively. Our findings identify oscillatory signatures of positive and negative affective processes as reactions to system-initiated assistance. The findings contribute to the development of EEG-based neuroadaptive assistance loops by suggesting a non-obtrusive method for monitoring affect in HCI.

Brain Activity during Different Throwing Games: EEG Exploratory Study

The purpose of this study is to explore the differences in brain activity in various types of throwing games by making encephalographic records. Three conditions of throwing games were compared looking for significant differences (simple throwing, throwing to a goal, and simultaneous throwing with another player). After signal processing, power spectral densities were compared through variance analysis (p ≤ 0.001). Significant differences were found especially in high-beta oscillations (22-30 Hz). "Goal" and "Simultaneous" throwing conditions show significantly higher values than those shown for throws without opponent. This can be explained by the higher demand for motor control and the higher arousal in competition situations. On the other hand, the high-beta records of the "Goal" condition are significantly higher than those of the "Simultaneous" throwing, which could be understood from the association of the beta waves with decision-making processes. These results support the difference in brain activity during similar games. This has several implications: opening up a path to study the effects of each specific game on brain activity and calling into question the transfer of research findings on animal play to all types of human play.

DE-CNN: An Improved Identity Recognition Algorithm Based on the Emotional Electroencephalography

In the past few decades, identification recognition based on electroencephalography (EEG) has received extensive attention to resolve the security problems of conventional biometric systems. In the present study, a novel EEG-based identification system with different entropy and a continuous convolution neural network (CNN) classifier is proposed. The performance of the proposed method is experimentally evaluated through the emotional EEG data. The conducted experiment shows that the proposed method approaches the stunning accuracy (ACC) of 99.7% on average and can rapidly train and update the DE-CNN model. Then, the effects of different emotions and the impact of different time intervals on the identification performance are investigated. Obtained results show that different emotions affect the identification accuracy, where the negative and neutral mood EEG has a better robustness than positive emotions. For a video signal as the EEG stimulant, it is found that the proposed method with 0–75 Hz is more robust than a single band, while the 15–32 Hz band presents overfitting and reduces the accuracy of the cross-emotion test. It is concluded that time interval reduces the accuracy and the 15–32 Hz band has the best compatibility in terms of the attenuation.

EEG gamma activity to emotional movies in individuals with high traits of primary "successful" psychopathy

This study aimed to investigate emotional alterations in a community sample with primary psychopathic traits. Sixty males selected from a larger sample and divided in two groups, High (HP) and Low (LP) primary Psychopathy, watched 15 validated emotional movies divided in five categories (Erotic, Neutral, Fear, and the new Scenery and Compassion). Subjective responses and cortical activity in the EEG Gamma band (30-49 Hz) were recorded. Concerning self-reports, HP participants felt less anxious and anguished in response to Fear clips and less sad, touched and anguished by Compassion excerpts. Negative clips induced larger EEG Gamma activity in both groups compared to neutral and erotic movies, but Gamma activity to Fear movies was reduced in HP. This group showed also a small cortical response to positive Scenery movies, at the level of the Neutral ones. Source analysis showed in HP participants a reduced cortical activation to Fear in a large brain network, comprising the right prefrontal and temporal cortices and bilateral inferior parietal cortex. Results showed that primary psychopathy, in addition to the impaired response to Fear, was associated with a reduced response also to other specific categories of emotional stimuli, suggesting an altered affect on a broader scale.

Imagery-induced negative affect, social touch and frontal EEG power band activity

Social touch seems to modulate emotions, but its brain correlates are poorly understood. Here, we investigated if frontal power band activity in the electroencephalogram (EEG) during aversive mental imagery is modulated by social touch from one's romantic partner and a stranger. We observed the highest theta and beta power when imaging alone, next so when being touched by a stranger, with lowest theta and beta activity during holding hands with the loved one. Delta power was higher when being alone than with a stranger or a partner, with no difference between the two. Gamma power was highest during the stranger condition and lower both when being alone and with the partner, while alpha power did not change as a function of social touch. Theta power displayed a positive correlation with electrodermal activity supporting its relation to emotional arousal. Attachment style modulated the effect of touch on the EEG as only secure but not insecure partner bonding was associated with theta power reductions. Because theta power was sensitive to the experimental perturbations, mapped onto peripheral physiological arousal and reflected partner attachment style we suggest that frontal theta power might serve as an EEG derived bio-marker for social touch in emotionally significant dyads.

The Temporal and Spatial Dynamics of Cortical Emotion Processing in Different Brain Frequencies as Assessed Using the Cluster-Based Permutation Test: An MEG Study

The processing of emotions in the human brain is an extremely complex process that extends across a large number of brain areas and various temporal processing steps. In the case of magnetoencephalography (MEG) data, various frequency bands also contribute differently. Therefore, in most studies, the analysis of emotional processing has to be limited to specific sub-aspects. Here, we demonstrated that these problems can be overcome by using a nonparametric statistical test called the cluster-based permutation test (CBPT). To the best of our knowledge, our study is the first to apply the CBPT to MEG data of brain responses to emotional stimuli. For this purpose, different emotionally impacting (pleasant and unpleasant) and neutral pictures were presented to 17 healthy subjects. The CBPT was applied to the power spectra of five brain frequencies, comparing responses to emotional versus neutral stimuli over entire MEG channels and time intervals within 1500 ms post-stimulus. Our results showed significant clusters in different frequency bands, and agreed well with many previous emotion studies. However, the use of the CBPT allowed us to easily include large numbers of MEG channels, wide frequency, and long time-ranges in one study, which is a more reliable alternative to other studies that consider only specific sub-aspects.

EEG Functional Connectivity Underlying Emotional Valance and Arousal Using Minimum Spanning Trees

In recent years, traditional methods such as power spectrum and amplitude analysis have been used to research the emotional electroencephalogram (EEG). The brain network method is also used in emotional EEG research, which can better reflect the activity of brains. A minimum spanning tree (MST) represents the key information flow in the weighted brain network, and it provides a sensitive method to capture subtle information in network organization while effectively avoiding the shortcomings of traditional brain networks. The DEAP dataset provides electroencephalogram (EEG) data for four categories of emotions: high arousal and high valence (HAHV), high arousal and low valence (HALV), low arousal and high valence (LAHV), and low arousal and low valence (LALV). Phase lag index (PLI) weighted matrices were calculated in five frequency bands. On this basis, the minimum spanning trees were constructed. At the same valence level in the gamma (γ) band, HAHV and HALV showed significant higher mean PLI (MPLI), maximum degree (Degree_max) and leaf fraction and significant lower diameter and eccentricity than LAHV and LALV. At the same arousal level in the γ band, HALV showed significant higher MPLI, Degree_max and leaf fraction and significant lower diameter and eccentricity than HAHV. These results indicate that the low-arousal showed more line-shaped configurations than the high-arousal. Additionally, in the high-arousal condition, a shift toward more star-shaped trees from high-valence to low-valence supports the trend toward randomness of the brain network with negative emotions and that the brain is more activated when faced with negative emotions. From a brain network perspective, this phenomenon provides a theoretical basis for negative bias.

Evaluation of Features in Detection of Dislike Responses to Audio–Visual Stimuli from EEG Signals

There is a strong correlation between the like/dislike responses to audio–visual stimuli and the emotional arousal and valence reactions of a person. In the present work, our attention is focused on the automated detection of dislike responses based on EEG activity when music videos are used as audio–visual stimuli. Specifically, we investigate the discriminative capacity of the Logarithmic Energy (LogE), Linear Frequency Cepstral Coefficients (LFCC), Power Spectral Density (PSD) and Discrete Wavelet Transform (DWT)-based EEG features, computed with and without segmentation of the EEG signal, on the dislike detection task. We carried out a comparative evaluation with eighteen modifications of the above-mentioned EEG features that cover different frequency bands and use different energy decomposition methods and spectral resolutions. For that purpose, we made use of Naïve Bayes classifier (NB), Classification and regression trees (CART), k-Nearest Neighbors (kNN) classifier, and support vector machines (SVM) classifier with a radial basis function (RBF) kernel trained with the Sequential Minimal Optimization (SMO) method. The experimental evaluation was performed on the well-known and widely used DEAP dataset. A classification accuracy of up to 98.6% was observed for the best performing combination of pre-processing, EEG features and classifier. These results support that the automated detection of like/dislike reactions based on EEG activity is feasible in a personalized setup. This opens opportunities for the incorporation of such functionality in entertainment, healthcare and security applications.

High Gamma Band EEG Closely Related to Emotion: Evidence From Functional Network

High-frequency electroencephalography (EEG) signals play an important role in research on human emotions. However, the different network patterns under different emotional states in the high gamma band (50–80 Hz) remain unclear. In this paper, we investigate different emotional states using functional network analysis on various frequency bands. We constructed multiple functional networks on different frequency bands and performed functional network analysis and time–frequency analysis on these frequency bands to determine the significant features that represent different emotional states. Furthermore, we verified the effectiveness of these features by using them in emotion recognition. Our experimental results revealed that the network connections in the high gamma band with significant differences among the positive, neutral, and negative emotional states were much denser than the network connections in the other frequency bands. The connections mainly occurred in the left prefrontal, left temporal, parietal, and occipital regions. Moreover, long-distance connections with significant differences among the emotional states were observed in the high frequency bands, particularly in the high gamma band. Additionally, high gamma band fusion features derived from the global efficiency, network connections, and differential entropies achieved the highest classification accuracies for both our dataset and the public dataset. These results are consistent with literature and provide further evidence that high gamma band EEG signals are more sensitive and effective than the EEG signals in other frequency bands in studying human affective perception.

Effect of brightness of visual stimuli on EEG signals

The aim of this study was to examine brightness effect, which is the perceptual property of visual stimuli, on brain responses obtained during visual processing of these stimuli. For this purpose, brain responses of the brain to changes in brightness were explored comparatively using different emotional images (pleasant, unpleasant and neutral) with different luminance levels. In the study, electroencephalography recordings from 12 different electrode sites of 31 healthy participants were used. The power spectra obtained from the analysis of the recordings using short time Fourier transform were analyzed, and a statistical analysis was performed on features extracted from these power spectra. Statistical findings were compared with those obtained from behavioral data. The results showed that the brightness of visual stimuli affected the power of brain responses depending on frequency, time and location. According to the statistically verified findings, the increase in the brightness of pleasant and neutral images increased the average power of responses in the parietal and occipital regions whereas the increase in the brightness of unpleasant images decreased the average power of responses in these regions. Moreover, the statistical results obtained for unpleasant images were found to be in accordance with the behavioral data. The results revealed that the brightness of visual stimuli could be represented by changing the activity power of the brain cortex. The findings emphasized that the brightness of visual stimuli should be viewed as an important parameter in studies using emotional image techniques such as image classification, emotion evaluation and neuro-marketing.

Changes in latency of brain rhythms in response to affective information of visual stimuli

It is widely known that emotionally-arousing pictures are perceived more rapidly than non-arousing pictures, although neural underpinnings of this effect remain unclear. Using electroencephalography, we presently measured neural oscillatory rhythms of the human brain in response to various emotional images from the International Affective Picture System. We found that an oscillation frequency in the alpha-to-beta band (8-30 Hz) became higher over the parietal cortex when participants viewed emotionally-arousing than non-arousing pictures. This modulation of neural rhythms was also observed in a valence dimension; emotionally-negative pictures induced faster neural rhythm than emotionally-positive pictures. Those results were consistent with previous studies reporting a speeded perception of high-arousing and negative stimuli (e.g. snakes and spiders) and further provided neural evidence for an adaptive function of emotion to accelerate the processing of potentially-dangerous stimuli.

Affective and cortical EEG gamma responses to emotional movies in women with high vs low traits of empathy

The present study sought to investigate how differences in trait empathy can influence emotional reactivity to a specific set of affective categories. Forty-one female students, divided in High (HE, n = 20) and Low (LE, n = 21) trait empathy, watched eight validated movie clips divided in four emotional categories (Erotic, Fear, Compassion and Neutral) while subjective evaluation of emotion and EEG gamma activity were recorded. Analysis of self-reports revealed that HE compared to LE exhibited an increased arousal level to all emotional clips. Concerning EEG data, the HE group showed a greater cortical gamma to all the emotional categories compared to the Neutral, while the LE group had greater response only to the negative clips. Participants in the HE group also showed a strong positive correlation between subjective arousal and cortical activity in response to Fear and Compassion clips. The greatest correlation was found to Compassion clips and was located in the right inferior parietal lobe (r₍₁₈₎ = 0.63), an important hub for both sensory-emotion integration and empathic sharing of others' emotions. Results suggest that high empathy was associated with enhanced gamma activity and greater self-reported arousal to all emotional stimuli. Furthermore, in this group, scenes with crying characters prompted a distinctive and localized cortical processing.

Lateralization of directional brain-heart information transfer during visual emotional elicitation

Previous studies have characterized the physiological interactions between central nervous system (brain) and peripheral cardiovascular system (heart) during affective elicitation in healthy subjects; however, questions related to the directionality of this functional interplay have been gaining less attention from the scientific community. Here, we explore brain-heart interactions during visual emotional elicitation in healthy subjects using measures of Granger causality (GC), a widely used descriptor of causal influences between two dynamical systems. The proposed approach inferences causality between instantaneous cardiovagal dynamics estimated from inhomogeneous point-process models of the heartbeat and high-density electroencephalogram (EEG) dynamics in 22 healthy subjects who underwent pleasant/unpleasant affective elicitation by watching pictures from the International Affective Picture System database. Particularly, we calculated the GC indexes between the EEG spectrogram in the canonical θ-, α-, β-, and γ-bands and both the instantaneous mean heart rate and its continuous parasympathetic modulations (i.e., the instantaneous HF power). Thus we looked for significant statistical differences among GC values estimated during the resting state, neutral elicitation, and pleasant/unpleasant arousing elicitation. As compared with resting state, coupling strength increases significantly in the left hemisphere during positive stimuli and in the right hemisphere during negative stimuli. Our results further reveal a correlation between emotional valence and lateralization of the dynamical information transfer going from brain-to-heart, mainly localized in the prefrontal, somatosensory, and posterior cortexes, and of the information transfer from heart-to-brain, mainly reflected into the fronto-parietal cortex oscillations in the γ-band (30 −45 Hz).

Affective picture processing: Alpha- and lower beta-band desynchronization reflects emotional arousal

EEG power analysis is firmly established in the cognitive domain. This contrasts with emotional stimulus processing, which thus far has yielded a complex and ambiguous pattern of findings. To further advance understanding, the present study examined emotional stimulus processing in the context of task variations and baseline activity, which included several manipulation checks as well as internal replication of findings across conditions. Participants (N = 16) viewed erotic and romantic pictures, differing in stimulus arousal. Pictures were presented briefly (120 ms), and intertrial interval was systematically varied (~1 vs. ~8 s). In one condition, participants passively viewed the pictures, in the other, they performed an active picture categorization task. The processing of erotic compared to romantic images was associated with a decrease in power in the alpha and lower beta band in posterior and anterior sensor clusters between 600-1,000 ms poststimulus. The finding was robust and confirmed across conditions, different quantifications, and independent from baseline activity. Furthermore, key findings regarding explicit task effects as well as ERPs sensitive to emotional arousal were replicated. Results are discussed with respect to the hypothesis that alpha- and lower beta-band activity may reflect cortical activation associated with emotional stimulus significance.

MAOA variants differ in oscillatory EEG & ECG activities in response to aggression-inducing stimuli

Among the genetic variations in the monoamine oxidase A (MAOA) gene, upstream variable number tandem repeats (uVNTRs) of the promoter have been associated with individual differences in human physiology and aggressive behaviour. However, the evidence for a molecular or neural link between MAOA uVNTRs and aggression remains ambiguous. Additionally, the use of inconsistent promoter constructs in previous studies has added to the confusion. Therefore, it is necessary to demonstrate the genetic function of MAOA uVNTR and its effects on multiple aspects of aggression. Here, we identified three MAOA alleles in Koreans: the predominant 3.5R and 4.5R alleles, as well as the rare 2.5R allele. There was a minor difference in transcriptional efficiency between the 3.5R and 4.5R alleles, with the greatest value for the 2.5R allele, in contrast to existing research. Psychological indices of aggression did not differ among MAOA genotypes. However, our electroencephalogram and electrocardiogram results obtained under aggression-related stimulation revealed oscillatory changes as novel phenotypes that vary with the MAOA genotype. In particular, we observed prominent changes in frontal γ power and heart rate in 4.5R carriers of men. Our findings provide genetic insights into MAOA function and offer a neurobiological basis for various socio-emotional mechanisms in healthy individuals.

Brain Dynamics Induced by Pleasant/Unpleasant Tactile Stimuli Conveyed by Different Fabrics

In this study, we investigated brain dynamics from electroencephalographic (EEG) signals during affective tactile stimulation conveyed by the dynamical contact with different fabrics. Thirty-three healthy subjects (16 females) were enrolled to interact with a haptic device able to mimic caress-like stimuli conveyed by strips of different fabrics moved back and forth at different velocities. Specifically, two velocity levels (i.e., 9.4 and 65 mm/sec) and two kinds of fabric (i.e., burlap and silk) were selected to deliver pleasant and unpleasant affective elicitations, according to subjects' self-assessment. EEG power spectra and functional connectivity were then calculated and analyzed. Experimental results, reported in terms of p-value topographic maps, demonstrated that caresses administered through unpleasant fabrics increased brain activity in the θ (4-8 Hz), α (8-14 Hz), and β (14-30 Hz) bands, whereas the use of pleasant fabrics enhanced functional connections in specific areas (e.g., frontal, occipital, and temporal cortices) depending on the oscillations frequency and caressing velocity. Furthermore, we adopted K-NN algorithms to automatically recognize the pleasantness of the haptic stimulation at a single-subject level using EEG power spectra, achieving a recognition accuracy up to 74.24%. Finally, we showed how brain oscillation power in the α and β bands over contralateral frontal- and central-cortex were the most informative features characterizing the pleasantness of a tactile stimulus on the forearm.

Gender Differences in Aggression-related Responses on EEG and ECG

Gender differences in aggression viewed from an evolutionary and sociocultural perspective have traditionally explained why men engage in more direct and physical aggression, and women engage in more indirect and relational aggression. However, psychological and behavioral studies offer inconsistent support for this theory due to personal or social factors, and little is known about the gender-based neurobiological mechanisms of aggression. This study investigates gender differences in aggression through an analysis of electroencephalography (EEG) and electrocardiography (ECG) based neurobiological responses to commonly encountered stimuli, as well as psychological approaches in healthy Korean youth. Our results from self-reports indicate that overall aggression indices, including physical and reactive/overt aggression, were stronger in men. This agrees with the results of previous studies. Furthermore, our study reveals prominent gender-related patterns in γ signals from the right ventrolateral frontal cortex and changes in heart rate through stimulation by aggressive videos. In particular, gender differences in EEG and ECG responses were observed in response to different scenes, as simple aversion and situation-dependent aggression, respectively. In addition, we discovered decisive gender-distinct EEG signals during stimulation of the situation-dependent aggression regions within the right ventromedial prefrontal and ventrolateral frontal regions. Our findings provide evidence of a psychological propensity for aggression and neurobiological mechanisms of oscillation underlying gender differences in aggression. Further studies of oscillatory responses to aggression and provocation will expand the objective understanding of the different emotional worlds between men and women.

Dynamics of large-scale electrophysiological networks: A technical review

For several years it has been argued that neural synchronisation is crucial for cognition. The idea that synchronised temporal patterns between different neural groups carries information above and beyond the isolated activity of these groups has inspired a shift in focus in the field of functional neuroimaging. Specifically, investigation into the activation elicited within certain regions by some stimulus or task has, in part, given way to analysis of patterns of co-activation or functional connectivity between distal regions. Recently, the functional connectivity community has been looking beyond the assumptions of stationarity that earlier work was based on, and has introduced methods to incorporate temporal dynamics into the analysis of connectivity. In particular, non-invasive electrophysiological data (magnetoencephalography/electroencephalography (MEG/EEG)), which provides direct measurement of whole-brain activity and rich temporal information, offers an exceptional window into such (potentially fast) brain dynamics. In this review, we discuss challenges, solutions, and a collection of analysis tools that have been developed in recent years to facilitate the investigation of dynamic functional connectivity using these imaging modalities. Further, we discuss the applications of these approaches in the study of cognition and neuropsychiatric disorders. Finally, we review some existing developments that, by using realistic computational models, pursue a deeper understanding of the underlying causes of non-stationary connectivity.

EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution

Accurate recognition and understating of human emotions is an essential skill that can improve the collaboration between humans and machines. In this vein, electroencephalogram (EEG)-based emotion recognition is considered an active research field with challenging issues regarding the analyses of the nonstationary EEG signals and the extraction of salient features that can be used to achieve accurate emotion recognition. In this paper, an EEG-based emotion recognition approach with a novel time-frequency feature extraction technique is presented. In particular, a quadratic time-frequency distribution (QTFD) is employed to construct a high resolution time-frequency representation of the EEG signals and capture the spectral variations of the EEG signals over time. To reduce the dimensionality of the constructed QTFD-based representation, a set of 13 time- and frequency-domain features is extended to the joint time-frequency-domain and employed to quantify the QTFD-based time-frequency representation of the EEG signals. Moreover, to describe different emotion classes, we have utilized the 2D arousal-valence plane to develop four emotion labeling schemes of the EEG signals, such that each emotion labeling scheme defines a set of emotion classes. The extracted time-frequency features are used to construct a set of subject-specific support vector machine classifiers to classify the EEG signals of each subject into the different emotion classes that are defined using each of the four emotion labeling schemes. The performance of the proposed approach is evaluated using a publicly available EEG dataset, namely the DEAPdataset. Moreover, we design three performance evaluation analyses, namely the channel-based analysis, feature-based analysis and neutral class exclusion analysis, to quantify the effects of utilizing different groups of EEG channels that cover various regions in the brain, reducing the dimensionality of the extracted time-frequency features and excluding the EEG signals that correspond to the neutral class, on the capability of the proposed approach to discriminate between different emotion classes. The results reported in the current study demonstrate the efficacy of the proposed QTFD-based approach in recognizing different emotion classes. In particular, the average classification accuracies obtained in differentiating between the various emotion classes defined using each of the four emotion labeling schemes are within the range of 73.8%–86.2%. Moreover, the emotion classification accuracies achieved by our proposed approach are higher than the results reported in several existing state-of-the-art EEG-based emotion recognition studies.

Incorporation of Multiple-Days Information to Improve the Generalization of EEG-Based Emotion Recognition Over Time

Current studies have got a series of satisfying accuracies in EEG-based emotion classification, but most of the classifiers used in previous studies are totally time-limited. To produce generalizable results, the emotion classifier should be stable over days, in which the day-to-day variations of EEG should be appropriately handled. To improve the generalization of EEG-based emotion recognition over time by learning multiple-days information which embraces the day-to-day variations, in this paper, 17 subjects were recruited to view several video clips to experience different emotion states, and each subject was required to perform five sessions in 5 days distributed over 1 month. Support vector machine was built to perform a classification, in which the training samples may come from 1, 2, 3, or 4 days' sessions but have a same number, termed learning 1-days information (L1DI), learning 2-days information (L2DI), learning 3-days information (L3DI), and learning 4-days information (L4DI) conditions, respectively. The results revealed that the EEG variability could impair the performance of emotion classifier dramatically, and learning more days' information to construct a classifier could significantly improve the generalization of EEG-based emotion recognition over time. Mean accuracies were 62.78, 67.92, 70.75, and 72.50% at L1DI, L2DI, L3DI, and L4DI conditions, respectively. Features at L4DI condition were ranked by modified RFE, and features providing better contribution were applied to obtain the performances of all conditions, results showed that the performance of SVMs trained and tested with the feature subset were all improved for L1DI, L2DI (^*p < 0.05), L3DI (^**p < 0.01), and L4DI (^*p < 0.05) conditions. It could be a substantial step forward in the development of emotion recognition from EEG signals because it may enable a classifier trained on one time to handle another.

Electroencephalography Based Analysis of Working Memory Load and Affective Valence in an N-back Task with Emotional Stimuli

Most brain-based measures of the electroencephalogram (EEG) are used in highly controlled lab environments and only focus on narrow mental states (e.g., working memory load). However, we assume that outside the lab complex multidimensional mental states are evoked. This could potentially create interference between EEG signatures used for identification of specific mental states. In this study, we aimed to investigate more realistic conditions and therefore induced a combination of working memory load and affective valence to reveal potential interferences in EEG measures. To induce changes in working memory load and affective valence, we used a paradigm which combines an N-back task (for working memory load manipulation) with a standard method to induce affect (affective pictures taken from the International Affective Picture System (IAPS) database). Subjective ratings showed that the experimental task was successful in inducing working memory load as well as affective valence. Additionally, performance measures were analyzed and it was found that behavioral performance decreased with increasing workload as well as negative valence, showing that affective valence can have an effect on cognitive processing. These findings are supported by changes in frontal theta and parietal alpha power, parameters used for measuring of working memory load in the EEG. However, these EEG measures are influenced by the negative valence condition as well and thereby show that detection of working memory load is sensitive to affective contexts. Unexpectedly, we did not find any effects for EEG measures typically used for affective valence detection (Frontal Alpha Asymmetry (FAA)). Therefore we assume that the FAA measure might not be usable if cognitive workload is induced simultaneously. We conclude that future studies should account for potential context-specifity of EEG measures.

Character drawing style in cartoons on empathy induction: an eye-tracking and EEG study

In its most basic form, empathy refers to the ability to understand another person’s feelings and emotions, representing an essential component of human social interaction. Owing to an increase in the use of mass media, which is used to distribute high levels of empathy-inducing content, media plays a key role in individual and social empathy induction. We investigated empathy induction in cartoons using eye movement, EEG and behavioral measures to explore whether empathy factors correlate with character drawing styles. Two different types of empathy-inducing cartoons that consisted of three stages and had the same story plot were used. One had an iconic style, while the other was realistic style. Fifty participants were divided into two groups corresponding to the individual cartoon drawing styles and were presented with only one type of drawing style. We found that there were no significant differences of empathy factors between iconic and realistic style. However, the Induced Empathy Score (IES) had a close relationship with subsequent attentional processing (total fixation length for gaze duration). Furthermore, iconic style suppressed the fronto-central area more than realistic style in the gamma power band. These results suggest that iconic cartoons have the advantage of abstraction during empathy induction, because the iconic cartoons induced the same level of empathy as realistic cartoons while using the same story plot (top-down process), even though lesser time and effort were required by the cartoon artist to draw them. This also means that the top-down process (story plot) is more important than the bottom-up process (drawing style) in empathy induction when viewing cartoons