Single-trial EEG-fMRI coupling of the emotional auditory early posterior negativity

Alpha and theta oscillations during the cognitive reappraisal of aversive pictures: A spatio-temporal qEEG investigation

CONTEXT

Emotion regulation is a set of processes responsible for controlling, evaluating and adjusting reactions to achieve a goal. Results derived from magnetic resonance imaging agreed on the involvement of frontal and limbic structures in this process. Findings using cognition and physiology interactions are still scarce but suggest a role of alpha rhythm in emotional induction and for theta in regulation.

OBJECTIVES AND HYPOTHESES

Our goal was to investigate alpha and theta rhythm during the reappraisal of aversive stimuli. We hypothesized that an implication of alpha rhythm in emotional induction only and an increase in prefrontal theta rhythm positively correlated with successful regulation.

METHOD

Twenty-four healthy participants were recorded with 64 EEG electrodes while asked to watch or reappraise negative pictures passively. Theta and alpha rhythms were compared across maintain, decrease and increase regulation conditions, and a source localization estimated the generators.

RESULTS

Theta activity was consistently higher in the upregulation than in the maintenance condition (p = .04) for the entire control period, but mainly at the beginning of regulation (1-3 s) for low-theta and later (5-7 s) for high-theta. Moreover, our results confirm that a low-theta generator correlated with mainly the middle frontal gyrus and the anterior dorsal cingulate cortex during upregulation. Theta was sensitive to emotion upregulation, whereas the alpha oscillation was non-sensitive to emotion induction and regulation.

CONCLUSION

Theta rhythm was involved explicitly in emotion upregulation processes that occur at a definite time during reappraisal, whereas the alpha rhythm was not altered by emotion induction and regulation.

Early categorization of social affordances during the visual encoding of bodily stimuli

Interpersonal interactions rely on various communication channels, both verbal and non-verbal, through which information regarding one's intentions and emotions are perceived. Here, we investigated the neural correlates underlying the visual processing of hand postures conveying social affordances (i.e., hand-shaking), compared to control stimuli such as hands performing non-social actions (i.e., grasping) or showing no movement at all. Combining univariate and multivariate analysis on electroencephalography (EEG) data, our results indicate that occipito-temporal electrodes show early differential processing of stimuli conveying social information compared to non-social ones. First, the amplitude of the Early Posterior Negativity (EPN, an Event-Related Potential related to the perception of body parts) is modulated differently during the perception of social and non-social content carried by hands. Moreover, our multivariate classification analysis (MultiVariate Pattern Analysis - MVPA) expanded the univariate results by revealing early (<200 ms) categorization of social affordances over occipito-parietal sites. In conclusion, we provide new evidence suggesting that the encoding of socially relevant hand gestures is categorized in the early stages of visual processing.

Listen to my feelings! How prosody and accent drive the empathic relevance of complaining speech

Interpersonal communication often involves sharing our feelings with others; complaining, for example, aims to elicit empathy in listeners by vocally expressing a speaker's suffering. Despite the growing neuroscientific interest in the phenomenon of empathy, few have investigated how it is elicited in real time by vocal signals (prosody), and how this might be affected by interpersonal factors, such as a speaker's cultural background (based on their accent). To investigate the neural processes at play when hearing spoken complaints, twenty-six French participants listened to complaining and neutral utterances produced by in-group French and out-group Québécois (i.e., French-Canadian) speakers. Participants rated how hurt the speaker felt while their cerebral activity was monitored with electroencephalography (EEG). Principal Component Analysis of Event-Related Potentials (ERPs) taken at utterance onset showed culture-dependent time courses of emotive prosody processing. The high motivational relevance of ingroup complaints increased the P200 response compared to all other utterance types; in contrast, outgroup complaints selectively elicited an early posterior negativity in the same time window, followed by an increased N400 (due to ongoing effort to derive affective meaning from outgroup voices). Ingroup neutral utterances evoked a late negativity which may reflect re-analysis of emotively less salient, but culturally relevant ingroup speech. Results highlight the time-course of neurocognitive responses that contribute to emotive speech processing for complaints, establishing the critical role of prosody as well as social-relational factors (i.e., cultural identity) on how listeners are likely to "empathize" with a speaker.

Evaluating the Safety of Simultaneous Intracranial Electroencephalography and Functional Magnetic Resonance Imaging Acquisition Using a 3 Tesla Magnetic Resonance Imaging Scanner

Background

The unsurpassed sensitivity of intracranial electroencephalography (icEEG) and the growing interest in understanding human brain networks and ongoing activities in health and disease have make the simultaneous icEEG and functional magnetic resonance imaging acquisition (icEEG-fMRI) an attractive investigation tool. However, safety remains a crucial consideration, particularly due to the impact of the specific characteristics of icEEG and MRI technologies that were safe when used separately but may risk health when combined. Using a clinical 3-T scanner with body transmit and head-receive coils, we assessed the safety and feasibility of our icEEG-fMRI protocol.

Methods

Using platinum and platinum-iridium grid and depth electrodes implanted in a custom-made acrylic-gel phantom, we assessed safety by focusing on three factors. First, we measured radio frequency (RF)-induced heating of the electrodes during fast spin echo (FSE, as a control) and the three sequences in our icEEG-fMRI protocol. Heating was evaluated with electrodes placed orthogonal or parallel to the static magnetic field. Using the configuration with the greatest heating observed, we then measured the total heating induced in our protocol, which is a continuous 70-min icEEG-fMRI session comprising localizer, echo-planar imaging (EPI), and magnetization-prepared rapid gradient-echo sequences. Second, we measured the gradient switching-induced voltage using configurations mimicking electrode implantation in the frontal and temporal lobes. Third, we assessed the gradient switching-induced electrode movement by direct visual detection and image analyses.

Results

On average, RF-induced local heating on the icEEG electrode contacts tested were greater in the orthogonal than parallel configuration, with a maximum increase of 0.2°C during EPI and 1.9°C during FSE. The total local heating was below the 1°C safety limit across all contacts tested during the 70-min icEEG-fMRI session. The induced voltage was within the 100-mV safety limit regardless of the configuration. No gradient switching-induced electrode displacement was observed.

Conclusion

We provide evidence that the additional health risks associated with heating, neuronal stimulation, or device movement are low when acquiring fMRI at 3 T in the presence of clinical icEEG electrodes under the conditions reported in this study. High specific absorption ratio sequences such as FSE should be avoided to prevent potential inadvertent tissue heating.

Early posterior negativity indicates time dilation by arousal

We investigated whether Early Posterior Negativity (EPN) indicated the subjective dilation of time when judging the duration of arousing stimuli. Participants performed a visual temporal bisection task along with high-level and low-level arousing auditory stimuli, while we simultaneously recorded EEG. In accordance with previous studies, arousing stimuli were temporally overestimated and led to higher EPN amplitude. Yet, we observed that time dilation and EPN amplitude were significantly correlated and this effect cannot be explained by confounds from stimulus valence. We interpret our findings in terms of the pacemaker–accumulator model of human timing, and suggest that EPN indicates an arousal-based increasing of the speed of our mental clock.

Single-Trial EEG-fMRI Reveals the Generation Process of the Mismatch Negativity

Although research on the mismatch negativity (MMN) has been ongoing for 40 years, the generation process of the MMN remains largely unknown. In this study, we used a single-trial electro-encephalography (EEG)-functional magnetic resonance imaging (fMRI) coupling method which can analyze neural activity with both high temporal and high spatial resolution and thus assess the generation process of the MMN. We elicited the MMN with an auditory oddball paradigm while recording simultaneous EEG and fMRI. We divided the MMN into five equal-durational phases. Utilizing the single-trial variability of the MMN, we analyzed the neural generators of the five phases, thereby determining the spatiotemporal generation process of the MMN. We found two distinct bottom-up prediction error propagations: first from the auditory cortex to the motor areas and then from the auditory cortex to the inferior frontal gyrus (IFG). Our results support the regularity-violation hypothesis of MMN generation.

Association of different neural processes during different emotional perceptions of white noise and pure tone auditory stimuli

Sound is a sensory stimulant ubiquitously found throughout our environment. Humans have evolved a system that effectively and automatically converts sound sensory inputs into emotions. Although different emotional responses to sounds with different frequency characteristics are empirically recognized, there is a paucity of studies addressing different emotional responses to these sounds and the underlying neural mechanisms. In this study, we examined effects of pure tone (PT) and white noise (WN) inputs at ordinary loudness levels on emotional responses. We found that WN stimuli produced more aversive responses than PT stimuli. This difference was endorsed by larger late posterior positivity (LPP). In a source localization study, we found increased neural activity in the parietal lobe prior to LPP. These findings show that WN stimuli produce aversive perceptions compared with PT stimuli, at typical loudness levels. In addition, different emotional responses were processed in a similar manner as visual stimulations, as reflected by increased LPP activation. Various emotional effects of WN and PT stimuli, at ordinary loudness levels, could expand our understanding of adverse effects of noise as well as favorable effects associated with music.

Human Novelty Response to Emotional Animal Vocalizations: Effects of Phylogeny and Familiarity

Darwin (1872) postulated that emotional expressions contain universals that are retained across species. We recently showed that human rating responses were strongly affected by a listener's familiarity with vocalization types, whereas evidence for universal cross-taxa emotion recognition was limited. To disentangle the impact of evolutionarily retained mechanisms (phylogeny) and experience-driven cognitive processes (familiarity), we compared the temporal unfolding of event-related potentials (ERPs) in response to agonistic and affiliative vocalizations expressed by humans and three animal species. Using an auditory oddball novelty paradigm, ERPs were recorded in response to task-irrelevant novel sounds, comprising vocalizations varying in their degree of phylogenetic relationship and familiarity to humans. Vocalizations were recorded in affiliative and agonistic contexts. Offline, participants rated the vocalizations for valence, arousal, and familiarity. Correlation analyses revealed a significant correlation between a posteriorly distributed early negativity and arousal ratings. More specifically, a contextual category effect of this negativity was observed for human infant and chimpanzee vocalizations but absent for other species vocalizations. Further, a significant correlation between the later and more posteriorly P3a and P3b responses and familiarity ratings indicates a link between familiarity and attentional processing. A contextual category effect of the P3b was observed for the less familiar chimpanzee and tree shrew vocalizations. Taken together, these findings suggest that early negative ERP responses to agonistic and affiliative vocalizations may be influenced by evolutionary retained mechanisms, whereas the later orienting of attention (positive ERPs) may mainly be modulated by the prior experience.

Auditory attention enhances processing of positive and negative words in inferior and superior prefrontal cortex

Visually presented emotional words are processed preferentially and effects of emotional content are similar to those of explicit attention deployment in that both amplify visual processing. However, auditory processing of emotional words is less well characterized and interactions between emotional content and task-induced attention have not been fully understood. Here, we investigate auditory processing of emotional words, focussing on how auditory attention to positive and negative words impacts their cerebral processing. A Functional magnetic resonance imaging (fMRI) study manipulating word valence and attention allocation was performed. Participants heard negative, positive and neutral words to which they either listened passively or attended by counting negative or positive words, respectively. Regardless of valence, active processing compared to passive listening increased activity in primary auditory cortex, left intraparietal sulcus, and right superior frontal gyrus (SFG). The attended valence elicited stronger activity in left inferior frontal gyrus (IFG) and left SFG, in line with these regions' role in semantic retrieval and evaluative processing. No evidence for valence-specific attentional modulation in auditory regions or distinct valence-specific regional activations (i.e., negative > positive or positive > negative) was obtained. Thus, allocation of auditory attention to positive and negative words can substantially increase their processing in higher-order language and evaluative brain areas without modulating early stages of auditory processing. Inferior and superior frontal brain structures mediate interactions between emotional content, attention, and working memory when prosodically neutral speech is processed.

A multimodal encoding model applied to imaging decision-related neural cascades in the human brain

Perception and cognition in the brain are naturally characterized as spatiotemporal processes. Decision-making, for example, depends on coordinated patterns of neural activity cascading across the brain, running in time from stimulus to response and in space from primary sensory regions to the frontal lobe. Measuring this cascade is key to developing an understanding of brain function. Here we report on a novel methodology that employs multi-modal imaging for inferring this cascade in humans at unprecedented spatiotemporal resolution. Specifically, we develop an encoding model to link simultaneously measured electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) signals to infer high-resolution spatiotemporal brain dynamics during a perceptual decision. After demonstrating replication of results from the literature, we report previously unobserved sequential reactivation of a substantial fraction of the pre-response network whose magnitude correlates with a proxy for decision confidence. Our encoding model, which temporally tags BOLD activations using time localized EEG variability, identifies a coordinated and spatially distributed neural cascade that is associated with a perceptual decision. In general the methodology illuminates complex brain dynamics that would otherwise be unobservable using fMRI or EEG acquired separately.

Electrophysiological Correlates of Emotional Content and Volume Level in Spoken Word Processing

For visual stimuli of emotional content as pictures and written words, stimulus size has been shown to increase emotion effects in the early posterior negativity (EPN), a component of event-related potentials (ERPs) indexing attention allocation during visual sensory encoding. In the present study, we addressed the question whether this enhanced relevance of larger (visual) stimuli might generalize to the auditory domain and whether auditory emotion effects are modulated by volume. Therefore, subjects were listening to spoken words with emotional or neutral content, played at two different volume levels, while ERPs were recorded. Negative emotional content led to an increased frontal positivity and parieto-occipital negativity-a scalp distribution similar to the EPN-between ~370 and 530 ms. Importantly, this emotion-related ERP component was not modulated by differences in volume level, which impacted early auditory processing, as reflected in increased amplitudes of the N1 (80-130 ms) and P2 (130-265 ms) components as hypothesized. However, contrary to effects of stimulus size in the visual domain, volume level did not influence later ERP components. These findings indicate modality-specific and functionally independent processing triggered by emotional content of spoken words and volume level.

EEG-Informed fMRI Reveals a Disturbed Gamma-Band-Specific Network in Subjects at High Risk for Psychosis

OBJECTIVES

Abnormalities of oscillatory gamma activity are supposed to reflect a core pathophysiological mechanism underlying cognitive disturbances in schizophrenia. The auditory evoked gamma-band response (aeGBR) is known to be reduced across all stages of the disease. The present study aimed to elucidate alterations of an aeGBR-specific network mediated by gamma oscillations in the high-risk state of psychosis (HRP) by means of functional magnetic resonance imaging (fMRI) informed by electroencephalography (EEG).

METHODS

EEG and fMRI were simultaneously recorded from 27 HRP individuals and 26 healthy controls (HC) during performance of a cognitively demanding auditory reaction task. We used single trial coupling of the aeGBR with the corresponding blood oxygen level depending response (EEG-informed fMRI).

RESULTS

A gamma-band-specific network was significantly lower active in HRP subjects compared with HC (random effects analysis, P < .01, Bonferroni-corrected for multiple comparisons) accompanied by a worse task performance. This network involved the bilateral auditory cortices, the thalamus and frontal brain regions including the anterior cingulate cortex, as well as the bilateral dorsolateral prefrontal cortex.

CONCLUSIONS

For the first time we report a reduced activation of an aeGBR-specific network in HRP subjects brought forward by EEG-informed fMRI. Because the HRP reflects the clinical risk for conversion to psychotic disorders including schizophrenia and the aeGBR has repeatedly been shown to be altered in patients with schizophrenia the results of our study point towards a potential applicability of aeGBR disturbances as a marker for the prediction of transition of HRP subjects to schizophrenia.

The way you say it, the way I feel it: emotional word processing in accented speech

The present study examined whether processing words with affective connotations in a listener's native language may be modulated by accented speech. To address this question, we used the Event Related Potential (ERP) technique and recorded the cerebral activity of Spanish native listeners, who performed a semantic categorization task, while listening to positive, negative and neutral words produced in standard Spanish or in four foreign accents. The behavioral results yielded longer latencies for emotional than for neutral words in both native and foreign-accented speech, with no difference between positive and negative words. The electrophysiological results replicated previous findings from the emotional language literature, with the amplitude of the Late Positive Complex (LPC), associated with emotional language processing, being larger (more positive) for emotional than for neutral words at posterior scalp sites. Interestingly, foreign-accented speech was found to interfere with the processing of positive valence and go along with a negativity bias, possibly suggesting heightened attention to negative words. The manipulation employed in the present study provides an interesting perspective on the effects of accented speech on processing affective-laden information. It shows that higher order semantic processes that involve emotion-related aspects are sensitive to a speaker's accent.

A novel single-trial event-related potential estimation method based on compressed sensing

Cognitive functions are often studied using event-related potentials (ERPs) that are usually estimated by an averaging algorithm. Clearly, estimation of single-trial ERPs can provide researchers with many more details of cognitive activity than the averaging algorithm. A novel method to estimate single-trial ERPs is proposed in this paper. This method includes two key ideas. First, singular value decomposition was used to construct a matrix, which mapped single-trial electroencephalographic recordings (EEG) into a low-dimensional vector that contained little information from the spontaneous EEG. Second, we used the theory of compressed sensing to build a procedure to restore single-trial ERPs from this low-dimensional vector. ERPs are sparse or approximately sparse in the frequency domain. This fact allowed us to use the theory of compressed sensing. We verified this method in simulated and real data. Our method and dVCA (differentially variable component analysis), another method of single-trial ERPs estimation, were both used to estimate single-trial ERPs from the same simulated data. Results demonstrated that our method significantly outperforms dVCA under various conditions of signal-to-noise ratio. Moreover, the single-trial ERPs estimated from the real data by our method are statistically consistent with the theories of cognitive science.

EEG-fMRI integration for the study of human brain function

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have proved to be extremely valuable tools for the non-invasive study of human brain function. Moreover, due to a notable degree of complementarity between the two modalities, the combination of EEG and fMRI data has been actively sought in the last two decades. Although initially focused on epilepsy, EEG-fMRI applications were rapidly extended to the study of healthy brain function, yielding new insights into its underlying mechanisms and pathways. Nevertheless, EEG and fMRI have markedly different spatial and temporal resolutions, and probe neuronal activity through distinct biophysical processes, many aspects of which are still poorly understood. The remarkable conceptual and methodological challenges associated with EEG-fMRI integration have motivated the development of a wide range of analysis approaches over the years, each relying on more or less restrictive assumptions, and aiming to shed further light on the mechanisms of brain function along with those of the EEG-fMRI coupling itself. Here, we present a review of the most relevant EEG-fMRI integration approaches yet proposed for the study of brain function, supported by a general overview of our current understanding of the biophysical mechanisms coupling the signals obtained from the two modalities.