Temporal precedence of emotion over attention modulations in the lateral amygdala: Intracranial ERP evidence from a patient with temporal lobe epilepsy

Expectancy and attention bias to spiders: Dissecting anticipation and allocation processes using ERPs

The current registered report focused on the temporal dynamics of the relationship between expectancy and attention toward threat, to better understand the mechanisms underlying the prioritization of threat detection over expectancy. In the current event-related potentials experiment, a-priori expectancy was manipulated, and attention bias was measured, using a well-validated paradigm. A visual search array was presented, with one of two targets: spiders (threatening) or birds (neutral). A verbal cue stating the likelihood of encountering a target preceded the array, creating congruent and incongruent trials. Following cue presentation, preparatory processes were examined using the contingent negative variation (CNV) component. Following target presentation, two components were measured: early posterior negativity (EPN) and late positive potential (LPP), reflecting early and late stages of natural selective attention toward emotional stimuli, respectively. Behaviorally, spiders were found faster than birds, and congruency effects emerged for both targets. For the CNV, a non-significant trend of more negative amplitudes following spider cues emerged. As expected, EPN and LPP amplitudes were larger for spider targets compared to bird targets. Data-driven, exploratory, topographical analyses revealed different patterns of activation for bird cues compared to spider cues. Furthermore, 400-500 ms post-target, a congruency effect was revealed only for bird targets. Together, these results demonstrate that while expectancy for spider appearance is evident in differential neural preparation, the actual appearance of spider target overrides this expectancy effect and only in later stages of processing does the cueing effect come again into play.

Amygdala and cortical gamma-band responses to emotional faces are modulated by attention to valence

The amygdala might support an attentional bias for emotional faces. However, whether and how selective attention toward a specific valence modulates this bias is not fully understood. Likewise, it is unclear whether amygdala and cortical signals respond to emotion and attention in a similar way. We recorded gamma-band activity (GBA, > 30 Hz) intracranially in the amygdalae of 11 patients with epilepsy and collected scalp recordings from 19 healthy participants. We presented angry, neutral, and happy faces randomly, and we denoted one valence as the target. Participants detected happy targets most quickly and accurately. In the amygdala, during attention to negative faces, low gamma-band activity (LGBA, < 90 Hz) increased for angry compared with happy faces from 160 ms. From 220 ms onward, amygdala high gamma-band activity (HGBA, > 90 Hz) was higher for angry and neutral faces than for happy ones. Monitoring neutral faces increased amygdala HGBA for emotions compared with neutral faces from 40 ms. Expressions were not differentiated in GBA while monitoring positive faces. On the scalp, only threat monitoring resulted in expression differentiation. Here, posterior LGBA was increased selectively for angry targets from 60 ms. The data show that GBA differentiation of emotional expressions is modulated by attention to valence: Top-down-controlled threat vigilance coordinates widespread GBA in favor of angry faces. Stimulus-driven emotion differentiation in amygdala GBA occurs during a neutral attentional focus. These findings align with a multi-pathway model of emotion processing and specify the role of GBA in this process.

Review of EEG Affective Recognition with a Neuroscience Perspective

Emotions are a series of subconscious, fleeting, and sometimes elusive manifestations of the human innate system. They play crucial roles in everyday life-influencing the way we evaluate ourselves, our surroundings, and how we interact with our world. To date, there has been an abundance of research on the domains of neuroscience and affective computing, with experimental evidence and neural network models, respectively, to elucidate the neural circuitry involved in and neural correlates for emotion recognition. Recent advances in affective computing neural network models often relate closely to evidence and perspectives gathered from neuroscience to explain the models. Specifically, there has been growing interest in the area of EEG-based emotion recognition to adopt models based on the neural underpinnings of the processing, generation, and subsequent collection of EEG data. In this respect, our review focuses on providing neuroscientific evidence and perspectives to discuss how emotions potentially come forth as the product of neural activities occurring at the level of subcortical structures within the brain's emotional circuitry and the association with current affective computing models in recognizing emotions. Furthermore, we discuss whether such biologically inspired modeling is the solution to advance the field in EEG-based emotion recognition and beyond.

Functional connectivity between the amygdala and prefrontal cortex underlies processing of emotion ambiguity

Processing facial expressions of emotion draws on a distributed brain network. In particular, judging ambiguous facial emotions involves coordination between multiple brain areas. Here, we applied multimodal functional connectivity analysis to achieve network-level understanding of the neural mechanisms underlying perceptual ambiguity in facial expressions. We found directional effective connectivity between the amygdala, dorsomedial prefrontal cortex (dmPFC), and ventromedial PFC, supporting both bottom-up affective processes for ambiguity representation/perception and top-down cognitive processes for ambiguity resolution/decision. Direct recordings from the human neurosurgical patients showed that the responses of amygdala and dmPFC neurons were modulated by the level of emotion ambiguity, and amygdala neurons responded earlier than dmPFC neurons, reflecting the bottom-up process for ambiguity processing. We further found parietal-frontal coherence and delta-alpha cross-frequency coupling involved in encoding emotion ambiguity. We replicated the EEG coherence result using independent experiments and further showed modulation of the coherence. EEG source connectivity revealed that the dmPFC top-down regulated the activities in other brain regions. Lastly, we showed altered behavioral responses in neuropsychiatric patients who may have dysfunctions in amygdala-PFC functional connectivity. Together, using multimodal experimental and analytical approaches, we have delineated a neural network that underlies processing of emotion ambiguity.

Nonlocal contrast calculated by the second order visual mechanisms and its significance in identifying facial emotions

Background: Previously obtained results indicate that faces are / preattentively/ detected in the visual scene very fast, and information on facial expression is rapidly extracted at the lower levels of the visual system. At the same time different facial attributes make different contributions in facial expression recognition. However, it is known, among the preattentive mechanisms there are none that would be selective for certain facial features, such as eyes or mouth. The aim of our study was to identify a candidate for the role of such a mechanism. Our assumption was that the most informative areas of the image are those characterized by spatial heterogeneity, particularly with nonlocal contrast changes. These areas may be identified / in the human visual system/ by the second-order visual / mechanisms/ filters selective to contrast modulations of brightness gradients. Methods: We developed a software program imitating the operation of these / mechanisms/ filters and finding areas of contrast heterogeneity in the image. Using this program, we extracted areas with maximum, minimum and medium contrast modulation amplitudes from the initial face images, then we used these to make three variants of one and the same face. The faces were demonstrated to the observers along with other objects synthesized the same way. The participants had to identify faces and define facial emotional expressions. Results: It was found that the greater is the contrast modulation amplitude of the areas shaping the face, the more precisely the emotion is identified. Conclusions: The results suggest that areas with a greater increase in nonlocal contrast are more informative in facial images, and the second-order visual / mechanisms/ filters can claim the role of /filters/ elements that detect areas of interest, attract visual attention and are windows through which subsequent levels of visual processing receive valuable information.

On the dynamics of gain and loss: Electrophysiological evidence from associative learning

Associated relevance affects the sensory encoding of low-level visual features of symbolic stimuli. It is unclear, however, which dimension of low-level visual features benefits from prioritized processing, and how these effects develop throughout the course of relevance acquisition. Moreover, previous evidence is inconclusive regarding the preservation of processing advantage once the association is no longer relevant, as well as its generalization to perceptually similar but novel stimuli. The present study addresses these questions by employing an associative learning paradigm. In two experiments (N = 24 each, between-subject design), different dimensions of low-level visual features of symbolic stimuli were associated with monetary gain, loss, or zero outcome. In a consecutive old/new decision task, associated stimuli were presented together with perceptually similar but novel stimuli. Event-related brain potentials (P1, EPN, LPC) were measured throughout both sessions. Early sensory encoding (P1) was boosted by loss association and appeared to be sensitive to the dimension of the associated low-level visual features. Gain association influenced post-perceptual processing stages (LPC), arising over the course of the learning phase, and are preserved even when the associated outcome was no longer relevant. Gain association also resulted in EPN modulations similar to the effects observed in the case of emotional words. None of the observed effects generalized to perceptually similar stimuli. These results show that acquired relevance can influence the sensory processing of specific dimensions of low-level visual features. Moreover, this study extends previous evidence of a dissociation of early and late neural effects of associated motivational relevance.

How processing emotion affects language control in bilinguals

Research has shown that several variables affect language control among bilingual speakers but the effect of affective processing remains unexplored. Chinese-English bilinguals participated in a novel prime-target language switching experiment in which they first judged the affective valence (i.e., positive or negative) of auditorily presented words and then named pictures with neutral emotional valence in either the same (non-switch trial) or different language (switch trial). Brain activity was monitored using functional magnetic resonance imaging (fMRI). The behavioral performance showed that the typical switch cost (i.e., the calculated difference between switch and non-switch trials) emerged after processing positive words but not after negative words. Brain imaging demonstrated that processing negative words immediately before non-switch picturing naming trials (but not for switch trials) increased activation in brain areas associated with domain-general cognitive control. The opposite patterns were found after processing positive words. These findings suggest that an (emotional) negative priming effect is induced by spontaneous exposure to negative words and that these priming effects may be triggered by reactive emotional processing and that they may interact with higher level cognitive functions.

Rapid Processing of Invisible Fearful Faces in the Human Amygdala

Rapid detection of a threat or its symbol (e.g., fearful face), whether visible or invisible, is critical for human survival. This function is suggested to be enabled by a subcortical pathway to the amygdala independent of the cortex. However, conclusive electrophysiological evidence in humans is scarce. Here, we explored whether the amygdala can rapidly encode invisible fearful faces. We recorded intracranial electroencephalogram (iEEG) responses in the human (both sexes) amygdala to faces with fearful, happy, and neutral emotions rendered invisible by backward masking. We found that a short-latency intracranial event-related potential (iERP) in the amygdala, beginning 88 ms poststimulus onset, was preferentially evoked by invisible fearful faces relative to invisible happy or neutral faces. The rapid iERP exhibited selectivity to the low spatial frequency (LSF) component of the fearful faces. Time-frequency iEEG analyses further identified a rapid amygdala response preferentially for LSF fearful faces at the low gamma frequency band, beginning 45 ms poststimulus onset. In contrast, these rapid responses to invisible fearful faces were absent in cortical regions, including early visual areas, the fusiform gyrus, and the parahippocampal gyrus. These findings provide direct evidence for the existence of a subcortical pathway specific for rapid fear detection in the amygdala and demonstrate that the subcortical pathway can function without conscious awareness and under minimal influence from cortical areas.SIGNIFICANCE STATEMENT Automatic detection of biologically relevant stimuli, such as threats or dangers, has remarkable survival value. Here, we provide direct intracranial electrophysiological evidence that the human amygdala preferentially responds to fearful faces at a rapid speed, despite the faces being invisible. This rapid, fear-selective response is restricted to faces containing low spatial frequency information transmitted by magnocellular neurons and does not appear in cortical regions. These results support the existence of a rapid subcortical pathway independent of cortical pathways to the human amygdala.

Functional connectivity between the amygdala and prefrontal cortex underlies processing of emotion ambiguity

Processing facial expressions of emotion draws on a distributed brain network. In particular, judging ambiguous facial emotions involves coordination between multiple brain areas. Here, we applied multimodal functional connectivity analysis to achieve network-level understanding of the neural mechanisms underlying perceptual ambiguity in facial expressions. We found directional effective connectivity between the amygdala, dorsomedial prefrontal cortex (dmPFC), and ventromedial PFC, supporting both bottom-up affective processes for ambiguity representation/perception and top-down cognitive processes for ambiguity resolution/decision. Direct recordings from the human neurosurgical patients showed that the responses of amygdala and dmPFC neurons were modulated by the level of emotion ambiguity, and amygdala neurons responded earlier than dmPFC neurons, reflecting the bottom-up process for ambiguity processing. We further found parietal-frontal coherence and delta-alpha cross-frequency coupling involved in encoding emotion ambiguity. We replicated the EEG coherence result using independent experiments and further showed modulation of the coherence. EEG source connectivity revealed that the dmPFC top-down regulated the activities in other brain regions. Lastly, we showed altered behavioral responses in neuropsychiatric patients who may have dysfunctions in amygdala-PFC functional connectivity. Together, using multimodal experimental and analytical approaches, we have delineated a neural network that underlies processing of emotion ambiguity.

Significance Statement

A large number of different brain regions participate in emotion processing. However, it remains elusive how these brain regions interact and coordinate with each other and collectively encode emotions, especially when the task requires orchestration between different brain areas. In this study, we employed multimodal approaches that well complemented each other to comprehensively study the neural mechanisms of emotion ambiguity. Our results provided a systematic understanding of the amygdala-PFC network underlying emotion ambiguity with fMRI-based connectivity, EEG coordination of cortical regions, synchronization of brain rhythms, directed information flow of the source signals, and latency of single-neuron responses. Our results further shed light on neuropsychiatric patients who have abnormal amygdala-PFC connectivity.

A generative adversarial model of intrusive imagery in the human brain

The mechanisms underlying the subjective experiences of mental disorders remain poorly understood. This is partly due to long-standing over-emphasis on behavioral and physiological symptoms and a de-emphasis of the patient's subjective experiences when searching for treatments. Here, we provide a new perspective on the subjective experience of mental disorders based on findings in neuroscience and artificial intelligence (AI). Specifically, we propose the subjective experience that occurs in visual imagination depends on mechanisms similar to generative adversarial networks that have recently been developed in AI. The basic idea is that a generator network fabricates a prediction of the world, and a discriminator network determines whether it is likely real or not. Given that similar adversarial interactions occur in the two major visual pathways of perception in people, we explored whether we could leverage this AI-inspired approach to better understand the intrusive imagery experiences of patients suffering from mental illnesses such as post-traumatic stress disorder (PTSD) and acute stress disorder. In our model, a nonconscious visual pathway generates predictions of the environment that influence the parallel but interacting conscious pathway. We propose that in some patients, an imbalance in these adversarial interactions leads to an overrepresentation of disturbing content relative to current reality, and results in debilitating flashbacks. By situating the subjective experience of intrusive visual imagery in the adversarial interaction of these visual pathways, we propose testable hypotheses on novel mechanisms and clinical applications for controlling and possibly preventing symptoms resulting from intrusive imagery.

Emotion and attention in face processing: Complementary evidence from surface event-related potentials and intracranial amygdala recordings

Face processing is biased by emotional and voluntarily directed attention, both of which modulate processing in distributed cortical areas. The amygdala is assumed to contribute to an attentional bias for emotional faces, although its interaction with directed attention awaits further clarification. Here, we studied the interaction of emotion and attention during face processing via scalp EEG potentials of healthy participants and intracranial EEG (iEEG) recordings of the right amygdala in one patient. Three randomized blocks consisting of angry, neutral, and happy facial expressions were presented, and one expression was denoted as the target category in each block. Happy targets were detected fastest and most accurately both in the group study and by the iEEG patient. Occipital scalp potentials revealed emotion differentiation for happy faces in the early posterior negativity (EPN) around 300 ms after stimulus onset regardless of the target condition. A similar response to happy faces occurred in the amygdala only for happy targets. On the scalp, a late positive potential (LPP, around 600 ms) enhancement for targets occurred for all target conditions alike. A simultaneous late signal in the amygdala was largest for emotional targets. No late signal enhancements were found for neutral targets in the amygdala. Cortical modulations, by contrast, showed both attention-independent effects of emotion and emotion-independent effects of attention. These results demonstrate an attention-dependence of amygdala activity during the processing of facial expressions and partly independent cortical mechanisms.

Face-induced gamma oscillations and event-related potentials in patients with epilepsy: an intracranial EEG study

Background

To examine the pathological effect of a mesial temporal seizure onset zone (SOZ) on local and inter-regional response to faces in the amygdala and other structures of the temporal lobe.

Methods

Intracranial EEG data was obtained from the amygdala, hippocampus, fusiform gyrus and parahippocampal gyrus of nine patients with drug-refractory epilepsy during visual stimulation with faces and mosaics. We analyzed event-related potentials (ERP), gamma frequency power, phase-amplitude coupling and phase-slope-index and compared the results between patients with versus without a mesial temporal SOZ.

Results

In the amygdala and fusiform gyrus, faces triggered higher ERP amplitudes compared to mosaics in both patient groups and higher gamma power in patients without a mesial temporal SOZ. In the hippocampus, famous faces triggered higher gamma power for both groups combined but did not affect ERPs in either group. The differentiated ERP response to famous faces in the parahippocampal gyrus was more pronounced in patients without a mesial temporal SOZ. Phase-amplitude coupling and phase-slope-index results yielded bidirectional modulation between amygdala and fusiform gyrus, and predominately unidirectional modulation between parahippocampal gyrus and hippocampus.

Conclusions

A mesial temporal SOZ was associated with an impaired response to faces in the amygdala, fusiform gyrus and parahippocampal gyrus in our patients. Compared to this, the response to faces in the hippocampus was impaired in patients with, as well as without, a mesial temporal SOZ. Our results support existing evidence for face processing deficits in patients with a mesial temporal SOZ and suggest the pathological effect of a mesial temporal SOZ on the amygdala to play a pivotal role in this matter in particular.

Modulation of Emotion Perception via Amygdala Stimulation in Humans

Background

Multiple lines of evidence show that the human amygdala is part of a neural network important for perception of emotion from environmental stimuli, including for processing of intrinsic attractiveness/“goodness” or averseness/“badness,” i.e., affective valence.

Objective/Hypothesis

With this in mind, we investigated the effect of electrical brain stimulation of the human amygdala on perception of affective valence of images taken from the International Affective Picture Set (IAPS).

Methods

Using intracranial electrodes in patients with epilepsy, we first obtained event-related potentials (ERPs) in eight patients as they viewed IAPS images of varying affective valence. Next, in a further cohort of 10 patients (five female and five male), we measured the effect of 50 Hz electrical stimulation of the left amygdala on perception of affective valence from IAPS images.

Results

We recorded distinct ERPs from the left amygdala and found significant differences in the responses between positively and negatively valenced stimuli (p = 0.002), and between neutral and negatively valenced stimuli (p = 0.017) 300–500 ms after stimulus onset. Next, we found that amygdala stimulation did not significantly affect how patients perceived valence for neutral images (p = 0.58), whereas stimulation induced patients to report both positively (p = 0.05) and negatively (< 0.01) valenced images as more neutral.

Conclusion

These results render further evidence that the left amygdala participates in a neural network for perception of emotion from environmental stimuli. These findings support the idea that electrical stimulation disrupts this network and leads to partial disruption of perception of emotion. Harnessing this effect may have clinical implications in treatment of certain neuropsychiatric disorders using deep brain stimulation (DBS) and neuromodulation.

Amygdala function in emotion, cognition, and behavior

The amygdala is a core structure in the anterior medial temporal lobe, with an important role in several brain functions involving memory, emotion, perception, social cognition, and even awareness. As a key brain structure for saliency detection, it triggers and controls widespread modulatory signals onto multiple areas of the brain, with a great impact on numerous aspects of adaptive behavior. Here we discuss the neural mechanisms underlying these functions, as established by animal and human research, including insights provided in both healthy and pathological conditions.

Emotion, Gender and Gaze Discrimination Tasks do not Differentially Impact the Neural Processing of Angry or Happy Facial Expressions-a Mass Univariate ERP Analysis

Facial expression processing is a critical component of social cognition yet, whether it is influenced by task demands at the neural level remains controversial. Past ERP studies have found mixed results with classic statistical analyses, known to increase both Type I and Type II errors, which Mass Univariate statistics (MUS) control better. However, MUS open-access toolboxes can use different fundamental statistics, which may lead to inconsistent results. Here, we compared the output of two MUS toolboxes, LIMO and FMUT, on the same data recorded during the processing of angry and happy facial expressions investigated under three tasks in a within-subjects design. Both toolboxes revealed main effects of emotion during the N170 timing and main effects of task during later time points typically associated with the LPP component. Neither toolbox yielded an interaction between the two factors at the group level, nor at the individual level in LIMO, confirming that the neural processing of these two face expressions is largely independent from task demands. Behavioural data revealed main effects of task on reaction time and accuracy, but no influence of expression or an interaction between the two. Expression processing and task demands are discussed in the context of the consistencies and discrepancies between the two toolboxes and existing literature.

Insular Cortex Response to Static Visual Sexual Stimuli

Abstract. According to previous research, the insula is important for processing salient and emotional stimuli, but its precise role remains elusive. By combining high spatial and temporal resolution, intracranial electroencephalography (iEEG) might contribute to filling this gap. Four drug-resistant epileptic patients with intracranial electrodes in the insula were instructed to watch and rate pictures with sexual content and neutral pictures. Event-related potentials (ERPs) were computed separately for both types of stimuli. Ninety-three percent of the anterior insula (AI) and 85% of the posterior insula (PI) contacts showed differences between ERPs. AI-positive deflections tended to have an earlier onset than PI-positive deflections. The results suggest that the AI generates a P300-like response and contributes to the early phase of the late positive potential, both components found enhanced while viewing emotional stimuli in the ERP literature. The present findings are interpreted as congruent with the role of the AI in maintaining attention to salient stimuli.

Emotional Nuance Enhances Verbatim Retention of Written Materials

Recent studies have demonstrated that details of verbal material are retained in memory. Further, converging evidence points to a memory-enhancing effect of emotion such that memory for emotional events is stronger than memory for neutral events. Building upon this work, it appears likely that verbatim sentence forms will be remembered better when tinged with emotional nuance. Most previous studies have focused on single words. The current study examines the role of emotional nuance in the verbatim retention of longer sentences in written material. In this study, participants silently read transcriptions of spontaneous narratives, half of which had been delivered within a context of emotional expression and the other half with neutral expression. Transcripts were taken from selected narratives that received the highest, most extreme ratings, neutral or emotional. Participants identified written excerpts in a yes/no recognition test. Results revealed that participants' verbatim memory was significantly greater for excerpts from emotionally nuanced narratives than from neutral narratives. It is concluded that the narratives, pre-rated as emotional or neutral, drove this effect of emotion on verbatim retention. These findings expand a growing body of evidence for a role of emotion in memory, and lend support to episodic theories of language and the constructionist account.

In exogenous attention, time is the clue: Brain and heart interactions to survive threatening stimuli

The capture of exogenous attention by negative stimuli has been interpreted as adaptive for survival in a diverse and changing environment. In the present paper, we investigate the neural responses towards two discrete negative emotions with different biological meanings, disgust and fear, and its potential relationships with heart rate variability (HRV) as an index of emotional regulation. With that aim, 30 participants performed a digit categorization task while fear, disgust and neutral distractor pictures were presented. Resting HRV at baseline, behavioral responses, and event-related potentials were recorded. Whereas P1 amplitudes were highest to fear distractors, the disgust stimulation led to augmented P2 amplitudes compared to the rest of distractors. Interestingly, increased N2 amplitudes were also found to disgust distractors, but only in high HRV participants. Neural source estimation data point to the involvement of the insula in this exogenous attentional response to disgust. Additionally, disgust distractors provoked longer reaction times than fear and neutral distractors in the high HRV group. Present findings are interpreted in evolutionary terms suggesting that exogenous attention is captured by negative stimuli following a different time course for fear and disgust. Possible HRV influences on neural mechanisms underlying exogenous attention are discussed considering the potential important role of this variable in emotional regulation processes.

From Gaze Perception to Social Cognition: The Shared-Attention System

When two people look at the same object in the environment and are aware of each other's attentional state, they find themselves in a shared-attention episode. This can occur through intentional or incidental signaling and, in either case, causes an exchange of information between the two parties about the environment and each other's mental states. In this article, we give an overview of what is known about the building blocks of shared attention (gaze perception and joint attention) and focus on bringing to bear new findings on the initiation of shared attention that complement knowledge about gaze following and incorporate new insights from research into the sense of agency. We also present a neurocognitive model, incorporating first-, second-, and third-order social cognitive processes (the shared-attention system, or SAS), building on previous models and approaches. The SAS model aims to encompass perceptual, cognitive, and affective processes that contribute to and follow on from the establishment of shared attention. These processes include fundamental components of social cognition such as reward, affective evaluation, agency, empathy, and theory of mind.

The early processing of fearful and happy facial expressions is independent of task demands - Support from mass univariate analyses

Most ERP studies on facial expressions of emotion have yielded inconsistent results regarding the time course of emotion effects and their possible modulation by task demands. Most studies have used classical statistical methods with a high likelihood of type I and type II errors, which can be limited with Mass Univariate statistics. FMUT and LIMO are currently the only two available toolboxes for Mass Univariate analysis of ERP data and use different fundamental statistics. Yet, no direct comparison of their output has been performed on the same dataset. Given the current push to transition to robust statistics to increase results replicability, here we compared the output of these toolboxes on data previously analyzed using classic approaches (Itier & Neath-Tavares, 2017). The early (0-352 ms) processing of fearful, happy, and neutral faces was investigated under three tasks in a within-subject design that also controlled gaze fixation location. Both toolboxes revealed main effects of emotion and task but neither yielded an interaction between the two, confirming the early processing of fear and happy expressions is largely independent of task demands. Both toolboxes found virtually no difference between neutral and happy expressions, while fearful (compared to neutral and happy) expressions modulated the N170 and EPN but elicited maximum effects after the N170 peak, around 190 ms. Similarities and differences in the spatial and temporal extent of these effects are discussed in comparison to the published classical analysis and the rest of the ERP literature.

Time to Face Language: Embodied Mechanisms Underpin the Inception of Face-Related Meanings in the Human Brain

In construing meaning, the brain recruits multimodal (conceptual) systems and embodied (modality-specific) mechanisms. Yet, no consensus exists on how crucial the latter are for the inception of semantic distinctions. To address this issue, we combined electroencephalographic (EEG) and intracranial EEG (iEEG) to examine when nouns denoting facial body parts (FBPs) and nonFBPs are discriminated in face-processing and multimodal networks. First, FBP words increased N170 amplitude (a hallmark of early facial processing). Second, they triggered fast (~100 ms) activity boosts within the face-processing network, alongside later (~275 ms) effects in multimodal circuits. Third, iEEG recordings from face-processing hubs allowed decoding ~80% of items before 200 ms, while classification based on multimodal-network activity only surpassed ~70% after 250 ms. Finally, EEG and iEEG connectivity between both networks proved greater in early (0-200 ms) than later (200-400 ms) windows. Collectively, our findings indicate that, at least for some lexico-semantic categories, meaning is construed through fast reenactments of modality-specific experience.

Temporal dynamics of amygdala response to emotion- and action-relevance

It has been proposed that the human amygdala may not only encode the emotional value of sensory events, but more generally mediate the appraisal of their relevance for the individual's goals, including relevance for action or task-based needs. However, emotional and non-emotional/action-relevance might drive amygdala activity through distinct neural signals, and the relative timing of both kinds of responses remains undetermined. Here, we recorded intracranial event-related potentials from nine amygdalae of patients undergoing epilepsy surgery, while they performed variants of a Go/NoGo task with faces and abstract shapes, where emotion- and action-relevance were orthogonally manipulated. Our results revealed early amygdala responses to emotion facial expressions starting ~ 130 ms after stimulus-onset. Importantly, the amygdala responded to action-relevance not only with face stimuli but also with abstract shapes (squares), and these relevance effects consistently occurred in later time-windows (starting ~ 220 ms) for both faces and squares. A similar dissociation was observed in gamma activity. Furthermore, whereas emotional responses habituated over time, the action-relevance effect increased during the course of the experiment, suggesting progressive learning based on the task needs. Our results support the hypothesis that the human amygdala mediates a broader relevance appraisal function, with the processing of emotion-relevance preceding temporally that of action-relevance.

The relation between neuronal firing, local field potentials and hemodynamic activity in the human amygdala in response to aversive dynamic visual stimuli

The amygdala is a central part of networks of brain regions underlying perception and cognition, in particular related to processing of emotionally salient stimuli. Invasive electrophysiological and hemodynamic measurements are commonly used to evaluate functions of the human amygdala, but a comprehensive understanding of their relation is still lacking. Here, we aimed at investigating the link between fast and slow frequency amygdalar oscillations, neuronal firing and hemodynamic responses. To this aim, we recorded intracranial electroencephalography (iEEG), hemodynamic responses and single neuron activity from the amygdala of patients with epilepsy. Patients were presented with dynamic visual sequences of fearful faces (aversive condition), interleaved with sequences of neutral landscapes (neutral condition). Comparing responses to aversive versus neutral stimuli across participants, we observed enhanced high gamma power (HGP, >60 ​Hz) during the first 2 ​s of aversive sequence viewing, and reduced delta power (1-4 ​Hz) lasting up to 18 ​s. In 5 participants with implanted microwires, neuronal firing rates were enhanced following aversive stimuli, and exhibited positive correlation with HGP and hemodynamic responses. Our results show that high gamma power, neuronal firing and BOLD responses from the human amygdala are co-modulated. Our findings provide, for the first time, a comprehensive investigation of amygdalar responses to aversive stimuli, ranging from single-neuron spikes to local field potentials and hemodynamic responses.

Vocal threat enhances visual perception as a function of attention and sex

This pre-registered event-related potential study explored how vocal emotions shape visual perception as a function of attention and listener sex. Visual task displays occurred in silence or with a neutral or an angry voice. Voices were task-irrelevant in a single-task block, but had to be categorized by speaker sex in a dual-task block. In the single task, angry voices increased the occipital N2 component relative to neutral voices in women, but not men. In the dual task, angry voices relative to neutral voices increased occipital N1 and N2 components, as well as accuracy, in women and marginally decreased accuracy in men. Thus, in women, vocal anger produced a strong, multifaceted visual enhancement comprising attention-dependent and attention-independent processes, whereas in men, it produced a small, behavior-focused visual processing impairment that was strictly attention-dependent. In sum, these data indicate that attention and listener sex critically modulate whether and how vocal emotions shape visual perception.

Directional interconnectivity of the human amygdala, fusiform gyrus, and orbitofrontal cortex in emotional scene perception

The perception of emotionally arousing scenes modulates neural activity in ventral visual areas via reentrant signals from the amygdala. The orbitofrontal cortex (OFC) shares dense interconnections with amygdala and has been strongly implicated in emotional stimulus processing in primates, but our understanding of the functional contribution of this region to emotional perception in humans is poorly defined. In this study we acquired targeted rapid functional imaging from lateral OFC, amygdala, and fusiform gyrus (FG) over multiple scanning sessions (resulting in over 1,000 trials per participant) in an effort to define the activation amplitude and directional connectivity among these regions during naturalistic scene perception. All regions of interest showed enhanced activation during emotionally arousing, compared with neutral scenes. In addition, we identified bidirectional connectivity between amygdala, FG, and OFC in the great majority of individual subjects, suggesting that human emotional perception is implemented in part via nonhierarchical causal interactions across these three regions.NEW & NOTEWORTHY Due to the practical limitations of noninvasive recording methodologies, there is a scarcity of data regarding the interactions of human amygdala and orbitofrontal cortex (OFC). Using rapid functional MRI sampling and directional connectivity, we found that the human amygdala influences emotional perception via distinct interactions with late-stage ventral visual cortex and OFC, in addition to distinct interactions between OFC and fusiform gyrus. Future efforts may leverage these patterns of directional connectivity to noninvasively distinguish clinical groups from controls with respect to network causal hierarchy.

Human amygdala response to unisensory and multisensory emotion input: No evidence for superadditivity from intracranial recordings

The amygdala is crucially implicated in processing emotional information from various sensory modalities. However, there is dearth of knowledge concerning the integration and relative time-course of its responses across different channels, i.e., for auditory, visual, and audiovisual input. Functional neuroimaging data in humans point to a possible role of this region in the multimodal integration of emotional signals, but direct evidence for anatomical and temporal overlap of unisensory and multisensory-evoked responses in amygdala is still lacking. We recorded event-related potentials (ERPs) and oscillatory activity from 9 amygdalae using intracranial electroencephalography (iEEG) in patients prior to epilepsy surgery, and compared electrophysiological responses to fearful, happy, or neutral stimuli presented either in voices alone, faces alone, or voices and faces simultaneously delivered. Results showed differential amygdala responses to fearful stimuli, in comparison to neutral, reaching significance 100-200 ms post-onset for auditory, visual and audiovisual stimuli. At later latencies, ∼400 ms post-onset, amygdala response to audiovisual information was also amplified in comparison to auditory or visual stimuli alone. Importantly, however, we found no evidence for either super- or subadditivity effects in any of the bimodal responses. These results suggest, first, that emotion processing in amygdala occurs at globally similar early stages of perceptual processing for auditory, visual, and audiovisual inputs; second, that overall larger responses to multisensory information occur at later stages only; and third, that the underlying mechanisms of this multisensory gain may reflect a purely additive response to concomitant visual and auditory inputs. Our findings provide novel insights on emotion processing across the sensory pathways, and their convergence within the limbic system.

Children's sustained attention to emotional facial expressions and their autonomic nervous system reactivity during parent-child interactions

The way individuals process socio-affective information is thought to impact their responses to social interactions, but research testing the relation between these processes is scarce, particularly among children. This study examined if children's attention to socio-affective stimuli was associated with their autonomic nervous system (ANS) reactivity during parent-child interactions. Children's sustained attention to facial expressions of emotion (afraid, happy, sad) was indexed using the late positive potential (LPP) event-related potential (ERP) component during a computer-based task. To measure ANS reactivity, children's respiratory sinus arrhythmia (RSA) was assessed at baseline and during positive and negative parent-child discussions. Enhanced LPP amplitudes in response to all emotional facial expressions, reflecting greater sustained attention to socio-affective stimuli, were associated with increased RSA reactivity during parent-child discussions. These results show correspondence between two psychophysiological substrates of emotion processing in healthy children and highlight how these systems may be synergistic forces contributing to emotion reactivity.

Increased inhibition following negative cues: A possible role for enhanced processing

Based on findings showing that attention is captured by aversive stimuli, previous studies have hypothesized that inhibition of return (IOR) is reduced at spatial locations previously occupied by threat cues. Yet evidence for this view is limited: Only a few studies have demonstrated a reduced degree of IOR following threat cues, while most have not found differences in IOR between aversive and neutral cues. In contrast to previous studies that used the spatial cuing paradigm and for the most part employed mild negative stimuli as cues, we examined the influence of highly aversive, colored and complex pictures of real life situations. As opposed to the stimuli used in previous studies, these pictures are thought to result in enhanced processing as well as in specific enhancement for threat pictures in comparison to neutral ones. Based on evidence indicating that enhanced processing of spatial cues results in increased IOR, we hypothesized that the negative picture cues employed in the present study would yield increased IOR. This hypothesis was confirmed in two experiments. We suggest that the enhancement of IOR following highly threatening cues may be related to efficient spatial orienting of attention in response to stimuli that are important from an evolutionary point of view. The results are discussed in the context of neurocognitive mechanisms that may underlie the modulation of IOR by emotional information.

Veterans with PTSD demonstrate amygdala hyperactivity while viewing threatening faces: A MEG study

Posttraumatic stress disorder (PTSD) is a major psychiatric disorder that is prevalent in combat veterans. Previous neuroimaging studies have found elevated amygdala activity in PTSD in response to threatening stimuli, but previous work has lacked the temporal specificity to study fast bottom-up fear responses involving the amygdala. Forty-four combat veterans, 28 with PTSD and 16 without, completed psychological testing and then a face-processing task during magnetoencephalography (MEG). The resulting MEG data were pre-processed, transformed into the time-frequency domain, and then imaged using a beamforming approach. We found that veterans with PTSD exhibited significantly stronger oscillatory activity from 50 to 450 ms in the left amygdala compared to veterans without PTSD while processing threatening faces. This group difference was not present while viewing neutral faces. The current study shows that amygdala hyperactivity in response to threatening cues begins quickly in PTSD, which makes theoretical sense as an adaptive bottom-up fear response.

Discrimination of emotional states from scalp- and intracranial EEG using multiscale Rényi entropy

A data-adaptive, multiscale version of Rényi's quadratic entropy (RQE) is introduced for emotional state discrimination from EEG recordings. The algorithm is applied to scalp EEG recordings of 30 participants watching 4 emotionally-charged video clips taken from a validated public database. Krippendorff's inter-rater statistic reveals that multiscale RQE of the mid-frontal scalp electrodes best discriminates between five emotional states. Multiscale RQE is also applied to joint scalp EEG, amygdala- and occipital pole intracranial recordings of an implanted patient watching a neutral and an emotionally charged video clip. Unlike for the neutral video clip, the RQEs of the mid-frontal scalp electrodes and the amygdala-implanted electrodes are observed to coincide in the time range where the crux of the emotionally-charged video clip is revealed. In addition, also during this time range, phase synchrony between the amygdala and mid-frontal recordings is maximal, as well as our 30 participants' inter-rater agreement on the same video clip. A source reconstruction exercise using intracranial recordings supports our assertion that amygdala could contribute to mid-frontal scalp EEG. On the contrary, no such contribution was observed for the occipital pole's intracranial recordings. Our results suggest that emotional states discriminated from mid-frontal scalp EEG are likely to be mirrored by differences in amygdala activations in particular when recorded in response to emotionally-charged scenes.

Auditory attention enhances processing of positive and negative words in inferior and superior prefrontal cortex

Visually presented emotional words are processed preferentially and effects of emotional content are similar to those of explicit attention deployment in that both amplify visual processing. However, auditory processing of emotional words is less well characterized and interactions between emotional content and task-induced attention have not been fully understood. Here, we investigate auditory processing of emotional words, focussing on how auditory attention to positive and negative words impacts their cerebral processing. A Functional magnetic resonance imaging (fMRI) study manipulating word valence and attention allocation was performed. Participants heard negative, positive and neutral words to which they either listened passively or attended by counting negative or positive words, respectively. Regardless of valence, active processing compared to passive listening increased activity in primary auditory cortex, left intraparietal sulcus, and right superior frontal gyrus (SFG). The attended valence elicited stronger activity in left inferior frontal gyrus (IFG) and left SFG, in line with these regions' role in semantic retrieval and evaluative processing. No evidence for valence-specific attentional modulation in auditory regions or distinct valence-specific regional activations (i.e., negative > positive or positive > negative) was obtained. Thus, allocation of auditory attention to positive and negative words can substantially increase their processing in higher-order language and evaluative brain areas without modulating early stages of auditory processing. Inferior and superior frontal brain structures mediate interactions between emotional content, attention, and working memory when prosodically neutral speech is processed.

The influence of affective state on exogenous attention to emotional distractors: behavioral and electrophysiological correlates

The interplay between exogenous attention to emotional distractors and the baseline affective state has not been well established yet. The present study aimed to explore this issue through behavioral measures and event-related potentials (ERPs). Participants (N = 30) completed a digit categorization task depicted over negative, positive or neutral distractor background pictures, while they experienced negative, positive and neutral affective states elicited by movie scenes. Behavioral results showed higher error rates and longer reaction times for negative distractors than for neutral and positive ones, irrespective of the current emotional state. Neural indices showed that the participants' affective state modulated N1 amplitudes, irrespective of distractor type, while the emotional charge of distractors modulated N2, irrespective of the emotional state. Importantly, an interaction of state and distractor type was observed in LPP. These results demonstrate that exogenous attention to emotional distractors is independent from modulating effects of the emotional baseline state at early, automatic stages of processing. However, attention to emotional distractors and affective state interact at later latencies.

Amygdala Response to Emotional Stimuli without Awareness: Facts and Interpretations

Over the past two decades, evidence has accumulated that the human amygdala exerts some of its functions also when the observer is not aware of the content, or even presence, of the triggering emotional stimulus. Nevertheless, there is as of yet no consensus on the limits and conditions that affect the extent of amygdala’s response without focused attention or awareness. Here we review past and recent studies on this subject, examining neuroimaging literature on healthy participants as well as brain-damaged patients, and we comment on their strengths and limits. We propose a theoretical distinction between processes involved in attentional unawareness, wherein the stimulus is potentially accessible to enter visual awareness but fails to do so because attention is diverted, and in sensory unawareness, wherein the stimulus fails to enter awareness because its normal processing in the visual cortex is suppressed. We argue this distinction, along with data sampling amygdala responses with high temporal resolution, helps to appreciate the multiplicity of functional and anatomical mechanisms centered on the amygdala and supporting its role in non-conscious emotion processing. Separate, but interacting, networks relay visual information to the amygdala exploiting different computational properties of subcortical and cortical routes, thereby supporting amygdala functions at different stages of emotion processing. This view reconciles some apparent contradictions in the literature, as well as seemingly contrasting proposals, such as the dual stage and the dual route model. We conclude that evidence in favor of the amygdala response without awareness is solid, albeit this response originates from different functional mechanisms and is driven by more complex neural networks than commonly assumed. Acknowledging the complexity of such mechanisms can foster new insights on the varieties of amygdala functions without awareness and their impact on human behavior.

Competition for attentional resources between low spatial frequency content of emotional images and a foreground task in early visual cortex

Low spatial frequency (LSF) image content has been proposed to play a superior functional role in emotional content extraction via the magnocellular pathway biasing attentional resources toward emotional content in visual cortex. We investigated whether emotionally unpleasant complex images that were presented either unfiltered or with LSF content only in the background while subjects performed a foreground task will withdraw more attentional resources from the task compared to unemotional, neutral images (distraction paradigm). We measured steady-state visual evoked potentials (SSVEPs) driven by flickering stimuli of a foreground task. Unfiltered unpleasant images resulted in a significant reduction of SSVEP amplitude compared to neutral images. No statistically significant differences were found with LSF background images. In a behavioral control experiment, we found no significant differences for complexity ratings between unfiltered and LSF pictures. Content identification was possible for unfiltered and LSF picture (correct responses > 74%). An additional EEG study examined typical emotion-related components for complex images presented either as unfiltered, LSF, or high spatial frequency (HSF, as an additional control) filtered, unpleasant, and neutral images. We found a significant main effect of emotional valence in the early posterior negativity. Late positive potential differences were only found for unfiltered and HSF images. Results suggest that, while LSF content is sufficient to allow for content and emotional cue extraction when images were presented alone, LSF content is not salient enough to serve as emotional distractor that withdraws attentional resources from a foreground task in early visual cortex.

Predicting Affective Information - An Evaluation of Repetition Suppression Effects

Both theoretical proposals and empirical studies suggest that the brain interprets sensory input based on expectations to mitigate computational burden. However, as social beings, much of sensory input is affectively loaded – e.g., the smile of a partner, the critical voice of a boss, or the welcoming gesture of a friend. Given that affective information is highly complex and often ambiguous, building up expectations of upcoming affective sensory input may greatly contribute to its rapid and efficient processing. This review points to the role of affective information in the context of the ‘predictive brain’. It particularly focuses on repetition suppression (RS) effects that have recently been linked to prediction processes. The findings are interpreted as evidence for more pronounced prediction processes with affective material. Importantly, it is argued that bottom-up attention inflates the neural RS effect, and because affective stimuli tend to attract more bottom-up attention, it thereby particularly overshadows the magnitude of RS effects for this information. Finally, anxiety disorders, such as social phobia, are briefly discussed as manifestations of modulations in affective prediction.

Precision about the automatic emotional brain

The question of automaticity in emotion processing has been debated under different perspectives in recent years. Satisfying answers to this issue will require a better definition of automaticity in terms of relevant behavioral phenomena, ecological conditions of occurrence, and a more precise mechanistic account of the underlying neural circuits.

Neural processing of fearful and happy facial expressions during emotion-relevant and emotion-irrelevant tasks: A fixation-to-feature approach

Research suggests an important role of the eyes and mouth for discriminating facial expressions of emotion. A gaze-contingent procedure was used to test the impact of fixation to facial features on the neural response to fearful, happy and neutral facial expressions in an emotion discrimination (Exp.1) and an oddball detection (Exp.2) task. The N170 was the only eye-sensitive ERP component, and this sensitivity did not vary across facial expressions. In both tasks, compared to neutral faces, responses to happy expressions were seen as early as 100-120ms occipitally, while responses to fearful expressions started around 150ms, on or after the N170, at both occipital and lateral-posterior sites. Analyses of scalp topographies revealed different distributions of these two emotion effects across most of the epoch. Emotion processing interacted with fixation location at different times between tasks. Results suggest a role of both the eyes and mouth in the neural processing of fearful expressions and of the mouth in the processing of happy expressions, before 350ms.

Selective visual attention to emotional words: Early parallel frontal and visual activations followed by interactive effects in visual cortex

Human brains spontaneously differentiate between various emotional and neutral stimuli, including written words whose emotional quality is symbolic. In the electroencephalogram (EEG), emotional-neutral processing differences are typically reflected in the early posterior negativity (EPN, 200-300 ms) and the late positive potential (LPP, 400-700 ms). These components are also enlarged by task-driven visual attention, supporting the assumption that emotional content naturally drives attention. Still, the spatio-temporal dynamics of interactions between emotional stimulus content and task-driven attention remain to be specified. Here, we examine this issue in visual word processing. Participants attended to negative, neutral, or positive nouns while high-density EEG was recorded. Emotional content and top-down attention both amplified the EPN component in parallel. On the LPP, by contrast, emotion and attention interacted: Explicit attention to emotional words led to a substantially larger amplitude increase than did explicit attention to neutral words. Source analysis revealed early parallel effects of emotion and attention in bilateral visual cortex and a later interaction of both in right visual cortex. Distinct effects of attention were found in inferior, middle and superior frontal, paracentral, and parietal areas, as well as in the anterior cingulate cortex (ACC). Results specify separate and shared mechanisms of emotion and attention at distinct processing stages. Hum Brain Mapp 37:3575-3587, 2016. © 2016 Wiley Periodicals, Inc.

Integrative Processing of Touch and Affect in Social Perception: An fMRI Study

Social perception commonly employs multiple sources of information. The present study aimed at investigating the integrative processing of affective social signals. Task-related and task-free functional magnetic resonance imaging was performed in 26 healthy adult participants during a social perception task concerning dynamic visual stimuli simultaneously depicting facial expressions of emotion and tactile sensations that could be either congruent or incongruent. Confounding effects due to affective valence, inhibitory top-down influences, cross-modal integration, and conflict processing were minimized. The results showed that the perception of congruent, compared to incongruent stimuli, elicited enhanced neural activity in a set of brain regions including left amygdala, bilateral posterior cingulate cortex (PCC), and left superior parietal cortex. These congruency effects did not differ as a function of emotion or sensation. A complementary task-related functional interaction analysis preliminarily suggested that amygdala activity depended on previous processing stages in fusiform gyrus and PCC. The findings provide support for the integrative processing of social information about others' feelings from manifold bodily sources (sensory-affective information) in amygdala and PCC. Given that the congruent stimuli were also judged as being more self-related and more familiar in terms of personal experience in an independent sample of participants, we speculate that such integrative processing might be mediated by the linking of external stimuli with self-experience. Finally, the prediction of task-related responses in amygdala by intrinsic functional connectivity between amygdala and PCC during a task-free state implies a neuro-functional basis for an individual predisposition for the integrative processing of social stimulus content.

Human amygdala activation during rapid eye movements of rapid eye movement sleep: an intracranial study

The amygdaloid complex plays a crucial role in processing emotional signals and in the formation of emotional memories. Neuroimaging studies have shown human amygdala activation during rapid eye movement sleep (REM). Stereotactically implanted electrodes for presurgical evaluation in epileptic patients provide a unique opportunity to directly record amygdala activity. The present study analysed amygdala activity associated with REM sleep eye movements on the millisecond scale. We propose that phasic activation associated with rapid eye movements may provide the amygdala with endogenous excitation during REM sleep. Standard polysomnography and stereo-electroencephalograph (SEEG) were recorded simultaneously during spontaneous sleep in the left amygdala of four patients. Time-frequency analysis and absolute power of gamma activity were obtained for 250 ms time windows preceding and following eye movement onset in REM sleep, and in spontaneous waking eye movements in the dark. Absolute power of the 44-48 Hz band increased significantly during the 250 ms time window after REM sleep rapid eye movements onset, but not during waking eye movements. Transient activation of the amygdala provides physiological support for the proposed participation of the amygdala in emotional expression, in the emotional content of dreams and for the reactivation and consolidation of emotional memories during REM sleep, as well as for next-day emotional regulation, and its possible role in the bidirectional interaction between REM sleep and such sleep disorders as nightmares, anxiety and post-traumatic sleep disorder. These results provide unique, direct evidence of increased activation of the human amygdala time-locked to REM sleep rapid eye movements.