A review of artificial intelligence methods enabled music-evoked EEG emotion recognition and their applications

Music is an archaic form of emotional expression and arousal that can induce strong emotional experiences in listeners, which has important research and practical value in related fields such as emotion regulation. Among the various emotion recognition methods, the music-evoked emotion recognition method utilizing EEG signals provides real-time and direct brain response data, playing a crucial role in elucidating the neural mechanisms underlying music-induced emotions. Artificial intelligence technology has greatly facilitated the research on the recognition of music-evoked EEG emotions. AI algorithms have ushered in a new era for the extraction of characteristic frequency signals and the identification of novel feature signals. The robust computational capabilities of AI have provided fresh perspectives for the development of innovative quantitative models of emotions, tailored to various emotion recognition paradigms. The discourse surrounding AI algorithms in the context of emotional classification models is gaining momentum, with their applications in music therapy, neuroscience, and social activities increasingly coming under the spotlight. Through an in-depth analysis of the complete process of emotion recognition induced by music through electroencephalography (EEG) signals, we have systematically elucidated the influence of AI on pertinent research issues. This analysis offers a trove of innovative approaches that could pave the way for future research endeavors.

Neural alignment during outgroup intervention predicts future change of affect towards outgroup

While social psychology studies have shown that paradoxical thinking intervention has a moderating effect on negative attitudes toward members from rival social groups (i.e. outgroup), the neural underpinnings of the intervention have not been studied. Here, we investigate this by examining neural alignment across individuals at different phases during the intervention regarding Covid-19 vaccine-supporters' attitudes against vaccine-opposers. We raise two questions: Whether neural alignment varies during the intervention, and whether it predicts a change in outgroup attitudes measured via a survey 2 days after the intervention and compared to baseline. We test the neural alignment using magnetoencephalography-recorded neural oscillations and multiset canonical correlation analysis. We find a build-up of neural alignment which emerges at the final phase of the paradoxical thinking intervention in the precuneus-a hub of mentalizing; there was no such effect in the control conditions. In parallel, we find a behavioral build-up of dissent to the interventional stimuli. These neural and behavioral patterns predict a prosocial future change in affect and actions toward the outgroup. Together, these findings reveal a new operational pattern of mentalizing on the outgroup, which can change the way individuals may feel and behave toward members of that outgroup.

Neural signatures of shared subjective affective engagement and disengagement during movie viewing

When watching a negative emotional movie, we differ from person to person in the ease with which we engage and the difficulty with which we disengage throughout a temporally evolving narrative. We investigated neural responses of emotional processing, by considering inter-individual synchronization in subjective emotional engagement and disengagement. The neural underpinnings of these shared responses are ideally studied in naturalistic scenarios like movie viewing, wherein individuals emotionally engage and disengage at their own time and pace throughout the course of a narrative. Despite the rich data that naturalistic designs can bring to the study, there is a challenge in determining time-resolved behavioral markers of subjective engagement and disengagement and their underlying neural responses. We used a within-subject cross-over design instructing 22 subjects to watch clips of either neutral or sad content while undergoing functional magnetic resonance imaging (fMRI). Participants watched the same movies a second time while continuously annotating the perceived emotional intensity, thus enabling the mapping of brain activity and emotional experience. Our analyses revealed that between-participant similarity in waxing (engagement) and waning (disengagement) of emotional intensity was directly related to the between-participant similarity in spatiotemporal patterns of brain activation during the movie(s). Similar patterns of engagement reflected common activation in the bilateral ventromedial prefrontal cortex, regions often involved in self-referenced evaluation and generation of negative emotions. Similar patterns of disengagement reflected common activation in central executive and default mode network regions often involved in top-down emotion regulation. Together this work helps to better understand cognitive and neural mechanisms underpinning engagement and disengagement from emotionally evocative narratives.

Examining the consistency of continuous affect annotations and psychophysiological measures in response to emotional videos

Despite the growing necessity of understanding the dynamics of emotion by naturalistic stimuli, averaging time-locked responses seems insufficient to capture emotional experiences that change over time. Intersubject correlation (ISC) has been implemented to examine dynamic emotional experiences by quantifying the consistency of responses across individuals. While previous research has shown that enhanced psychophysiological ISC can capture dynamic emotional experiences in response to long-lasting videos that evoke dimensional emotions, it is not yet fully understood how psychophysiological consistency varies during videos that elicit distinct emotions, such as fear. In this study, we re-analyzed publicly available data consisting of continuous affect annotations and psychophysiological signals, namely heart rate (HR), electrodermal activity (EDA), electromyographic signals from zygomaticus major (EMG-z), and corrugator supercilii (EMG-c), in response to categorical emotional videos, namely amusing, boring, relaxing, and fearful. Results showed an overall increase in ISC in multiple measures during fearful videos, indicating that emotional experiences during fearful videos were reliably consistent across participants. The effect of amusing and boring videos on ISC revealed varying results depending on the measurements. In particular, larger ISC in valence rating, EDA, and EMG-z was found for amusing than boring videos, whereas larger ISC in HR and EMG-c was observed for boring than amusing movies. Lastly, decreased ISC for relaxing videos was observed across multiple measurements, showing inconsistent emotional experiences during relaxing videos. This study builds on previous research on physiological consistency during emotional experiences by examining how the consistency of continuous affect annotations and psychophysiological measures differs in response to videos that elicit distinct emotions.

EEG alpha band functional connectivity reveals distinct cortical dynamics for overt and covert emotional face processing

Current knowledge regarding how the focus of our attention during face processing influences neural responses largely comes from neuroimaging studies reporting on regional brain activations. The present study was designed to add novel insights to this research by studying how attention can differentially impact the way cortical regions interact during emotional face processing. High-density electroencephalogram was recorded in a sample of fifty-two healthy participants during an emotional face processing task. The task required participants to either attend to the expressions (i.e., overt processing) or attend to a perceptual distractor, which rendered the expressions task-irrelevant (i.e., covert processing). Functional connectivity in the alpha band was estimated in source space and modeled using graph theory to quantify whole-brain integration and segregation. Results revealed that overt processing of facial expressions is linked to reduced cortical segregation and increased cortical integration, this latter specifically for negative expressions of fear and sadness. Furthermore, we observed increased communication efficiency during overt processing of negative expressions between the core and the extended face processing systems. Overall, these findings reveal that attention makes the interaction among the nodes involved in face processing more efficient, also uncovering a connectivity signature of the prioritized processing mechanism of negative expressions, that is an increased cross-communication within the nodes of the face processing network.

Inter-brain coupling reflects disciplinary differences in real-world classroom learning

The classroom is the primary site for learning. A vital feature of classroom learning is the division of educational content into various disciplines. While disciplinary differences could substantially influence the learning process toward success, little is known about the neural mechanism underlying successful disciplinary learning. In the present study, wearable EEG devices were used to record a group of high school students during their classes of a soft (Chinese) and a hard (Math) discipline throughout one semester. Inter-brain coupling analysis was conducted to characterize students' classroom learning process. The students with higher scores in the Math final exam were found to have stronger inter-brain couplings to the class (i.e., all the other classmates), whereas the students with higher scores in Chinese were found to have stronger inter-brain couplings to the top students in the class. These differences in inter-brain couplings were also reflected in distinct dominant frequencies for the two disciplines. Our results illustrate disciplinary differences in the classroom learning from an inter-brain perspective, suggesting that an individual's inter-brain coupling to the class and to the top students could serve as potential neural correlates for successful learning in hard and soft disciplines correspondingly.

Neural dynamics during emotional video engagement relate to anxiety

Inter-subject correlations (ISCs) of physiological data can reveal common stimulus-driven processing across subjects. ISC has been applied to passive video viewing in small samples to measure common engagement and emotional processing. Here, in a large sample study of healthy adults (N = 163) who watched an emotional film (The Lion Cage by Charlie Chaplin), we recorded electroencephalography (EEG) across participants and measured ISC in theta, alpha and beta frequency bands. Peak ISC on the emotionally engaging video was observed three-quarters into the film clip, during a time period which potentially elicited a positive emotion of relief. Peak ISC in all frequency bands was focused over centro-parietal electrodes localizing to superior parietal cortex. ISC in both alpha and beta frequencies had a significant inverse relationship with anxiety symptoms. Our study suggests that ISC measured during continuous non-event-locked passive viewing may serve as a useful marker for anxious mood.

Are there consistent abnormalities in event-related EEG oscillations in patients with Alzheimer's disease compared to other diseases belonging to dementia?

Cerebrospinal and structural-molecular neuroimaging in-vivo biomarkers are recommended for diagnostic purposes in Alzheimer's disease (AD) and other dementias; however, they do not explain the effects of AD neuropathology on neurophysiological mechanisms underpinning cognitive processes. Here, an Expert Panel from the Electrophysiology Professional Interest Area of the Alzheimer's Association reviewed the field literature and reached consensus on the event-related electroencephalographic oscillations (EROs) that show consistent abnormalities in patients with significant cognitive deficits due to Alzheimer's, Parkinson's (PD), Lewy body (LBD), and cerebrovascular diseases. Converging evidence from oddball paradigms showed that, as compared to cognitively unimpaired (CU) older adults, AD patients had lower amplitude in widespread delta (>4 Hz) and theta (4-7 Hz) phase-locked EROs as a function of disease severity. Similar effects were also observed in PD, LBD, and/or cerebrovascular cognitive impairment patients. Non-phase-locked alpha (8-12 Hz) and beta (13-30 Hz) oscillations were abnormally reduced (event-related desynchronization, ERD) in AD patients relative to CU. However, studies on patients with other dementias remain lacking. Delta and theta phase-locked EROs during oddball tasks may be useful neurophysiological biomarkers of cognitive systems at work in heuristic and intervention clinical trials performed in AD patients, but more research is needed regarding their potential role for other dementias.

Similar brains blend emotion in similar ways: Neural representations of individual difference in emotion profiles

Our daily emotional experience is a complex construct that usually involves multiple emotions blended in a context-dependent manner. However, the co-occurring and context-dependent nature of human emotions was understated in previous studies when addressing the individual difference in emotional experiences. The present study proposed a situated and blended 'profile' perspective to characterize individualized emotional experiences. Eighty participants watched a series of emotional videos with their EEG recorded, and the individual differences in their emotion profiles were measured as the vector distances between their multidimensional emotion ratings for these video stimuli. This measure was found to be a reliable descriptor of individualized emotional experiences and could efficiently predict classical emotional complexity indices. More importantly, inter-subject representational analyses revealed that similar emotion profiles were associated with similar delta-band activities over the prefrontal and temporo-parietal regions and similar theta-band activities over the frontal regions. Furthermore, left- and right-lateralized temporo-parietal representations were observed for positive and negative emotion profiles, respectively. Our findings demonstrate the potential of taking a 'profile' perspective for understanding individual differences in human emotions.

Grounding Adaptive Cognitive Control in the Intrinsic, Functional Brain Organization: An HD-EEG Resting State Investigation

In a recent study, we used the dynamic temporal prediction (DTP) task to demonstrate that the capability to implicitly adapt motor control as a function of task demand is grounded in at least three dissociable neurofunctional mechanisms: expectancy implementation, expectancy violation and response implementation, which are supported by as many distinct cortical networks. In this study, we further investigated if this ability can be predicted by the individual brain's functional organization at rest. To this purpose, we recorded resting-state, high-density electroencephalography (HD-EEG) in healthy volunteers before performing the DTP task. This allowed us to obtain source-reconstructed cortical activity and compute whole-brain resting state functional connectivity at the source level. We then extracted phase locking values from the parceled cortex based on the Destrieux atlas to estimate individual functional connectivity at rest in the three task-related networks. Furthermore, we applied a machine-learning approach (i.e., support vector regression) and were able to predict both behavioral (response speed and accuracy adaptation) and neural (ERP modulation) task-dependent outcome. Finally, by exploiting graph theory nodal measures (i.e., degree, strength, local efficiency and clustering coefficient), we characterized the contribution of each node to the task-related neural and behavioral effects. These results show that the brain's intrinsic functional organization can be potentially used as a predictor of the system capability to adjust motor control in a flexible and implicit way. Additionally, our findings support the theoretical framework in which cognitive control is conceived as an emergent property rooted in bottom-up associative learning processes.

Decreased intersubject synchrony in dynamic valence ratings of sad movie contents in dysphoric individuals

Emotional reactions to movies are typically similar between people. However, depressive symptoms decrease synchrony in brain responses. Less is known about the effect of depressive symptoms on intersubject synchrony in conscious stimulus-related processing. In this study, we presented amusing, sad and fearful movie clips to dysphoric individuals (those with elevated depressive symptoms) and control participants to dynamically rate the clips' valences (positive vs. negative). We analysed both the valence ratings' mean values and intersubject correlation (ISC). We used electrodermal activity (EDA) to complement the measurement in a separate session. There were no group differences in either the EDA or mean valence rating values for each movie type. As expected, the valence ratings' ISC was lower in the dysphoric than the control group, specifically for the sad movie clips. In addition, there was a negative relationship between the valence ratings' ISC and depressive symptoms for sad movie clips in the full sample. The results are discussed in the context of the negative attentional bias in depression. The findings extend previous brain activity results of ISC by showing that depressive symptoms also increase variance in conscious ratings of valence of stimuli in a mood-congruent manner.

Event-related network changes unfold the dynamics of cortical integration during face processing

Face perception arises from a collective activation of brain regions in the occipital, parietal and temporal cortices. Despite the wide acknowledgment that these regions act in an intertwined network, the network behavior itself is poorly understood. Here we present a study in which time-varying connectivity estimated from EEG activity elicited by facial expressions presentation was characterized using graph-theoretical measures of node centrality and global network topology. Results revealed that face perception results from a dynamic reshaping of the network architecture, characterized by the emergence of hubs located in the occipital and temporal regions of the scalp. The importance of these nodes can be observed from the early stages of visual processing and reaches a climax in the same time-window in which the face-sensitive N170 is observed. Furthermore, using Granger causality, we found that the time-evolving centrality of these nodes is associated with ERP amplitude, providing a direct link between the network state and local neural response. Additionally, investigating global network topology by means of small-worldness and modularity, we found that face processing requires a functional network with a strong small-world organization that maximizes integration, at the cost of segregated subdivisions. Interestingly, we found that this architecture is not static, but instead, it is implemented by the network from stimulus onset to ~200 ms. Altogether, this study reveals the event-related changes underlying face processing at the network level, suggesting that a distributed processing mechanism operates through dynamically weighting the contribution of the cortical regions involved.