Affective engagement for facial expressions and emotional scenes: the influence of social anxiety

Assessing motivational biases in brain and behavior: Event-related potential and response time concomitants of the approach-avoidance task

The approach-avoidance task (AAT) is designed to measure implicit motivated action biases instantiated by emotional stimuli and alterations in such biases that drive psychiatric disorder. While some research has measured AAT event-related potential (ERP) correlates to establish bias sensitivity even at a neural level, a lack of work with unpleasant, pleasant, and neutral stimuli together and a common focus on psychiatric disorder-matched (rather than generally emotional) content limits conclusions that can be drawn. Thus, current work extends the AAT literature by testing ERP modulations across normatively unpleasant, pleasant, and neutral conditions; and supporting the task's use as an individual difference assessment, it also provides data on AAT reliability and initially explores anxiety-related effects when stimuli are not disorder-matched. In 38 participants including 19 anxiety treatment-seeking individuals, 32 sensor electroencephalography revealed robust N100, N200, and late positive potential (LPP) ERP components and bias-consistent modulations for unpleasant images (reduced N200s on unpleasant push relative to pull trials; enhanced LPP for unpleasant compared to neutral trials). Meanwhile, modulations were less consistent with emotion-driven bias for other conditions-that is, LPPs were enhanced but N200 was not modulated for pleasant images, and for neutral images, N200 was unexpectedly enhanced on push compared to pull trials. Following these analyses, reliability tests revealed excellent raw ERP reliabilities but lower reliabilities for modulation scores, and comparing treatment- to non-treatment-seeking groups showed no preliminary indication of ERP modulation changes when stimuli are not personally relevant. How these findings together inform understanding of AAT as a measure of bias is discussed.

Emotion-gaze interaction affects time-to-collision estimates, but not preferred interpersonal distance towards looming faces

Estimating the time until impending collision (time-to-collision, TTC) of approaching or looming individuals and maintaining a comfortable distance from others (interpersonal distance, IPD) are commonly required in daily life and contribute to survival and social goals. Despite accumulating evidence that facial expressions and gaze direction interactively influence face processing, it remains unclear how these facial features affect the spatiotemporal processing of looming faces. We examined whether facial expressions (fearful vs. neutral) and gaze direction (direct vs. averted) interact on the judgments of TTC and IPD for looming faces, based on the shared signal hypothesis that fear signals the existence of threats in the environment when coupled with averted gaze. Experiment 1 demonstrated that TTC estimates were reduced for fearful faces compared to neutral ones only when the concomitant gaze was averted. In Experiment 2, the emotion-gaze interaction was not observed in the IPD regulation, which is arguably sensitive to affective responses to faces. The results suggest that fearful-averted faces modulate the cognitive extrapolation process of looming motion by communicating environmental threats rather than by altering subjective fear or perceived emotional intensity of faces. The TTC-specific effect may reflect an enhanced defensive response to unseen threats implied by looming fearful-averted faces. Our findings provide insight into how the visual system processes facial features to ensure bodily safety and comfortable interpersonal communication in dynamic environments.

Social concerns about anxious arousal explain the association between neural responses to anxious arousal pictures and social anxiety

Cognitive theories propose that social anxiety disorder involves heightened attention to anxious arousal symptoms due to worries that they may evoke rejection from others. Supporting this, studies have shown that social anxiety is related to greater attention to representations of anxious arousal and to anxiety sensitivity social concerns, which refers to sensitivity to feelings of anxious arousal during social situations. However, this has not yet been tested using neural indices of attention to images depicting anxious arousal. To examine these associations, the current study examined early and sustained attentional bias to anxious arousal images using the P2 and the late positive potential (LPP), respectively. Electroencephalogram data were collected while a non-clinical sample of undergraduate students (N = 106) viewed images of people exhibiting anxious arousal in addition to blocks of negative and neutral images from the IAPS. The neural response to anxious arousal images was isolated using residual scores (e.g., using linear regression to predict the P2 elicited by anxious arousal images from the P2 elicited by neutral images (P2neutral→AA) or negative images (P2negative→AA), then saving the unstandardized residuals). There was an indirect effect of the P2neutral→AA and P2negative→AA waveforms that was explained by anxiety sensitivity social concerns. Additionally, there was an indirect effect of both LPP waveforms on social anxiety symptoms during the early time window of the LPP (400-700 ms). At the later time window of the LPP (700-1000 ms), there was an indirect effect of the LPPneutral→AA residual waveform, but not the LPPnegative→AA, on social anxiety symptoms.

Alpha-band oscillations and emotion: A review of studies on picture perception

Although alpha-band activity has long been a focus of psychophysiological research, its modulation by emotional value during picture perception has only recently been studied systematically. Here, we review these studies and report that the most consistent alpha oscillatory pattern indexing emotional processing is an enhanced desynchronization (ERD) over posterior sensors when viewing emotional compared with neutral pictures. This enhanced alpha ERD is not specific to unpleasant picture content, as previously proposed for other measures of affective response, but has also been observed for pleasant stimuli. Evidence suggests that this effect is not confined to the alpha band but that it also involves a desynchronization of the lower beta frequencies (8-20 Hz). The emotional modulation of alpha ERD occurs even after massive stimulus repetition and when emotional cues serve as task-irrelevant distractors, consistent with the hypothesis that evaluative processes are mandatory in emotional picture processing. A similar enhanced ERD has been observed for other significant cues (e.g., conditioned aversive stimuli, or in anticipation of a potential threat), suggesting that it reflects cortical excitability associated with the engagement of the motivational systems.

Incidental learning of faces during threat: No evidence for enhanced physiological responses to former threat identities

Remembering an unfamiliar person and the contextual conditions of that encounter is important for adaptive future behavior, especially in a potentially dangerous situation. Initiating defensive behavior in the presence of former dangerous circumstances can be crucial. Recent studies showed selective electrocortical processing of faces that were previously seen in a threat context compared to a safety context, however, this was not reflected in conscious recognition performance. Here, we investigated whether previously seen threat-faces, that could not be remembered, were capable to activate defensive psychophysiological response systems. During an encoding phase, 50 participants with low to moderate levels of anxiety viewed 40 face pictures with neutral expressions (6 s each), without an explicit learning instruction (incidental learning task). Each half of the faces were presented with contextual background colors that signaled either threat-of-shock or safety. In the recognition phase, all old and additional new faces (total of 60) were presented intermixed without context information. Participants had to decide whether a face was new or had been presented previously in a threatening or a safe context. Results show moderate face recognition independent of context conditions. Startle reflex and skin conductance responses (SCR) were more pronounced for threat compared to safety during encoding. For SCR, this differentiation was enhanced with higher levels of depression and anxiety. There were no differential startle reflex or SCR effects during recognition. From a clinical perspective, these findings do not support the notion that perceptual biases and physiological arousal directly relate to threat-associated identity recognition deficits in healthy and clinical participants with anxiety and trauma-related disorders.

Neural evidence of face processing in social anxiety disorder: A systematic review with meta-analysis

Numerous previous studies have used event-related potentials (ERPs) to examine facial processing deficits in individuals with social anxiety disorder (SAD). However, researchers still need to determine whether the deficits are general or specific and what the dominant factors are behind different cognitive stages. Meta-analysis was performed to quantitatively identify face processing deficits in individuals with SAD. Ninety-seven results in 27 publications involving 1032 subjects were calculated using Hedges' g. The results suggest that the face itself elicits enlarged P1 amplitudes, threat-related facial expressions induce larger P2 amplitudes, and negative facial expressions lead to enhanced P3/LPP amplitudes in SAD individuals compared with controls. That is, there is face perception attentional bias in the early phase (P1), threat attentional bias in the mid-term phase (P2), and negative emotion attentional bias in the late phase (P3/LPP), which can be summarized into a three-phase SAD face processing deficit model. These findings provide an essential theoretical basis for cognitive behavioral therapy and have significant application value for the initial screening, intervention, and treatment of social anxiety.

Neuroelectric Correlates of Human Sexuality: A Review and Meta-Analysis

Many reviews on sexual arousal in humans focus on different brain imaging methods and behavioral observations. Although neurotransmission in the brain is mainly performed through electrochemical signals, there are no systematic reviews of the electrophysiological correlates of sexual arousal. We performed a systematic search on this subject and reviewed 255 studies including various electrophysiological methods. Our results show how neuroelectric signals have been used to investigate genital somatotopy as well as basic genital physiology during sexual arousal and how cortical electric signals have been recorded during orgasm. Moreover, experiments on the interactions of cognition and sexual arousal in healthy subjects and in individuals with abnormal sexual preferences were analyzed as well as case studies on sexual disturbances associated with diseases of the nervous system. In addition, 25 studies focusing on brain potentials during the interaction of cognition and sexual arousal were eligible for meta-analysis. The results showed significant effect sizes for specific brain potentials during sexual stimulation (P3: Cohen's d = 1.82, N = 300, LPP: Cohen's d = 2.30, N = 510) with high heterogeneity between the combined studies. Taken together, our review shows how neuroelectric methods can consistently differentiate sexual arousal from other emotional states.

Angry and fearful compared to happy or neutral faces as conditional stimuli in human fear conditioning: A systematic review and meta-analysis

Some previous research has shown stronger acquisition and impaired extinction of fear conditioned to angry or fearful compared to happy or neutral face conditional stimuli (CS) - a difference attributed to biological 'preparedness'. A systematic review and meta-analysis of fear conditioning studies comparing face CSs of differing expressions identified thirty studies, eighteen of which were eligible for meta-analysis. Skin conductance responses were larger to angry or fearful faces compared to happy or neutral faces during habituation, acquisition and extinction. Significant differences in differential conditioning between angry, fearful, neutral, and happy face CSs were also found, but differences were more prominent between angry and neutral faces compared to angry/fearful and happy faces. This is likely due to lower arousal elicited by neutral compared to happy faces, which may be more salient as CSs. The findings suggest there are small to moderate differences in differential conditioning when angry or fearful compared to happy or neutral faces are used as CSs. These findings have implications for fear conditioning study design and the preparedness theory.

fMRI replicability during emotional scene viewing: Functional regions and sample size

Recent findings have questioned the replicability of functional magnetic resonance imaging (fMRI) in the study of affective processing, reporting low replicability of emotional enhancement during a face-matching task. However, poor replicability may instead reflect a lack of emotional engagement for face matching. In the current study, replicability of emotional enhancement was tested in a large (N = 160) sample when emotional engagement was assessed during pleasant, neutral, and unpleasant picture viewing, which reliably engages affective reactions in both the brain and the body. Replicability was computed using a subsampling technique, in which random sets of subjects of different sample sizes (N = 20, 40, 60, 80) were selected from the entire dataset, and replicability of emotional enhancement for peaks, clusters, and voxels were averaged across 500 permutations for each sample size. Consistent with previous findings, fMRI replicability increased with increasing sample size. On the other hand, even with relatively small samples, fMRI replicability for peaks, clusters, and voxels during emotional, compared to neutral, scene viewing was good to excellent. Importantly, replicability varied in different brain regions, with excellent replicability at both the cluster and peak level with an N of 40, at the most conservative threshold (p < .001), in the amygdala and the visual cortex. The data argue against general recommendations regarding sample size in fMRI studies of emotion, suggesting instead that degree of replicability depends on successful emotional engagement in task-related brain regions.

Your face scares me: Effects of Perceptual load and Social Anxiety on processing of threatening and neutral faces

Social Anxiety Disorder is among the most widely studied psychiatric conditions. However, the role of attentional and emotional processes in the maintenance of the condition is still not well-established. This study addressed the impact of individual differences in Social Anxiety, by examining the effects of perceptual load and stimulus valence when processing faces vs objects, here used as distractors, within a letter-search task. In addition to RT and accuracy on the letter search task, heart rate, and skin conductance during the task and participants’ self-report emotional evaluation were assessed to help interpret performance effects. Results suggest that distractor stimuli that are either threatening or faces impair performance of high SA participants. Results demonstrate a hypervigilance for threatening faces in SA but indicate that this happens primarily when cognitive resources are available, that is, under low perceptual load.

Psychophysiological Effects of Downregulating Negative Emotions: Insights From a Meta-Analysis of Healthy Adults

Assessing psychophysiological responses of emotion regulation is a cost-efficient way to quantify emotion regulation and to complement subjective report that may be biased. Previous studies have revealed inconsistent results complicating a sound interpretation of these findings. In the present study, we summarized the existing literature through a systematic search of articles. Meta-analyses were used to evaluate effect sizes of instructed downregulation strategies on common autonomic (electrodermal, respiratory, cardiovascular, and pupillometric) and electromyographic (corrugator activity, emotion-modulated startle) measures. Moderator analyses were conducted, with moderators including study design, emotion induction, control instruction and trial duration. We identified k = 78 studies each contributing multiple sub-samples and performed 23 meta-analyses for combinations of emotion regulation strategy and psychophysiological measure. Overall, results showed that effects of reappraisal and suppression on autonomic measures were highly inconsistent across studies with rather small mean effect sizes. Electromyography (startle and corrugator activity) showed medium effect sizes that were consistent across studies. Our findings highlight the diversity as well as the low level of standardization and comparability of research in this area. Significant moderation of effects by study design, trial duration, and control condition emphasizes the need for better standardization of methods. In addition, the small mean effect sizes resulting from our analyses on autonomic measures should be interpreted with caution. Findings corroborate the importance of multi-channel approaches.

Behavioral and neural evidence of enhanced long-term memory for untrustworthy faces

In daily life, we automatically form impressions of other individuals on basis of subtle facial features that convey trustworthiness. Because these face-based judgements influence current and future social interactions, we investigated how perceived trustworthiness of faces affects long-term memory using event-related potentials (ERPs). In the current study, participants incidentally viewed 60 neutral faces differing in trustworthiness, and one week later, performed a surprise recognition memory task, in which the same old faces were presented intermixed with novel ones. We found that after one week untrustworthy faces were better recognized than trustworthy faces and that untrustworthy faces prompted early (350–550 ms) enhanced frontal ERP old/new differences (larger positivity for correctly remembered old faces, compared to novel ones) during recognition. Our findings point toward an enhanced long-lasting, likely familiarity-based, memory for untrustworthy faces. Even when trust judgments about a person do not necessarily need to be accurate, a fast access to memories predicting potential harm may be important to guide social behaviour in daily life.

The effect of being watched on facial EMG and autonomic activity in response to another individual's facial expressions

We tested if facial reactions to another person's facial expressions depend on the self-relevance of the observed expressions. In the present study (n = 44), we measured facial electromyographic (zygomatic and corrugator) activity and autonomic arousal (skin conductance) responses to a live model's smiling and neutral faces. In one condition, the participant and the model were able to see each other normally, whereas in the other condition, the participant was led to believe that the model could not see the participant. The results showed that the increment of zygomatic activity in response to smiling faces versus neutral faces was greater when the participants believed they were being watched than it was when the participants believed they were not being watched. However, zygomatic responses to smiles did not differ between the conditions, while the results suggested that the participants' zygomatic responses to neutral faces seemed to attenuate in the condition of believing they were being watched. Autonomic responses to smiling faces were greater in the belief of being watched than in the belief of not being watched condition. The results suggest that the self-relevance of another individual's facial expression modulates autonomic arousal responses and to a lesser extent facial EMG responses.

Facial Reactivity and Attentional Processing of Facial Expressions and Gaze Direction

Abstract. Patterns of facial reactivity and attentional allocation to emotional facial expressions, and how these are moderated by gaze direction, are not clearly established. Among a sample of undergraduate university students, aged between 17 and 22 years (76% female), corrugator and zygomatic reactivity, as measured by facial electromyography, and attention allocation, as measured by the startle reflex and startle-elicited N100, was examined while viewing happy, neutral, angry and fearful facial expressions, which were presented at either 0- or 30-degree gaze. Results indicated typically observed facial mimicry to happy faces but, unexpectedly, “smiling” facial responses to fearful, and to a lesser extent, angry faces. This facial reactivity was not influenced by gaze direction. Furthermore, emotional facial expressions did not elicit increased attentional allocation. Likewise, matched facial expressions did not elicit increased attentional allocation. Rather, happy and fearful faces with direct (0°) gaze elicited increased controlled attentional allocation, and averted (30°) gaze faces, regardless of emotional expression, elicited preferential, early cortical processing. These findings suggest typical facial mimicry to happy faces, but unexpected facial reactivity to angry and fearful faces, perhaps due to an attempt to regulate social bonds during threat perception. Findings also suggest a divergence in controlled versus preferential, early cortical attentional processing for direct compared to averted gaze faces. These findings relate to young, mostly female, adults attending university. The experiment should be repeated with a larger sample drawn from the general community, with a broader age range and gender balance, and with a stimulus set with validated subjective valence and arousal ratings. This can reduce Type II error and establish normative patterns of facial reactivity and attentional processing of emotional facial expressions with different gaze directions.

Electrophysiological Correlates of Racial In-group Bias in Observing Nonverbal Social Encounters

Despite evidence identifying the role of group membership in social cognition, the neural mechanisms associated with the perception and evaluation of nonverbal behaviors displayed by in-group versus out-group members remain unclear. Here, 42 white participants underwent electroencephalographic recording while observing social encounters involving dynamic displays of nonverbal behaviors by racial in-group and out-group avatar characters. Dynamic behaviors included approach and avoidance poses and expressions, followed by the participants' ratings of the avatars displaying them. Behaviorally, participants showed longer RTs when evaluating in-group approach behavior compared with other behaviors, possibly suggesting increased interest and attention devoted to processing positive social encounters with their in-group members. Analyses of ERPs revealed differential sensitivity of the N450 and late positivity components to social cues, with the former showing initial sensitivity to the presence of a humanoid avatar character at the beginning of social encounters and the latter showing sensitivity to dynamic nonverbal behaviors displayed by the avatars. Moreover, time-frequency analysis of electroencephalography data also identified suppression of beta-range power linked to the observation of dynamic nonverbal behaviors. Notably, the magnitude of these responses was modulated by the degree of behavioral racial in-group bias. This suggests that differential neural sensitivity to nonverbal cues while observing social encounters is associated with subsequent in-group bias manifested in the evaluation of such encounters. Collectively, these findings shed light on the mechanisms of racial in-group bias in social cognition and have implications for understanding factors related to successful interactions with individuals from diverse racial backgrounds.

Emotional bias varies with stimulus type, arousal and task setting: Meta-analytic evidences

Emotional bias, which describes human's asymmetric processing of emotional stimuli, consists of negativity bias (Increased response to negative over positive stimuli) and positivity offset (the reversed phenomenon). Previous studies suggest that stimulus arousal (high/low), stimulus type (scenic/verbal), cultural background (Eastern/Western), and task setting (explicit/implicit) may modulate emotional bias, but with inconclusive findings. To address how the profile of emotional bias varies with these factors, a meta-analysis of emotional P3 event-related potential amplitudes was performed. Forty-nine effect sizes from 38 studies involving 1263 subjects were calculated using Hedges'g. The results highlight significant moderators of arousal, stimulus type, and task setting. Specifically, high-arousal stimuli enhance negativity bias relative to low-arousal stimuli; scenic stimulus leads to a negativity bias while verbal stimulus is linked with a positivity offset; explicit emotion tasks lead to negativity bias, whereas implicit emotion tasks do not exhibit emotional bias. These results indicate that emotional bias is labile depending on stimulus arousal, stimulus type and task setting. The implication of these findings for emotion regulation is discussed.

Reversing Threat to Safety: Incongruence of Facial Emotions and Instructed Threat Modulates Conscious Perception but Not Physiological Responding

Facial expressions inform about other peoples' emotion and motivation and thus are central for social communication. However, the meaning of facial expressions may change depending on what we have learned about the related consequences. For instance, a smile might easily become threatening when displayed by a person who is known to be dangerous. The present study examined the malleability of emotional facial valence by means of social learning. To this end, facial expressions served as cues for verbally instructed threat-of-shock or safety (e.g., "happy faces cue shocks"). Moreover, reversal instructions tested the flexibility of threat/safety associations (e.g., "now happy faces cue safety"). Throughout the experiment, happy, neutral, and angry facial expressions were presented and auditory startle probes elicited defensive reflex activity. Results show that self-reported ratings and physiological reactions to threat/safety cues dissociate. Regarding threat and valence ratings, happy facial expressions tended to be more resistant becoming a threat cue, and angry faces remain threatening even when instructed as safety cue. For physiological response systems, however, we observed threat-potentiated startle reflex and enhanced skin conductance responses for threat compared to safety cues regardless of whether threat was cued by happy or angry faces. Thus, the incongruity of visual and verbal threat/safety information modulates conscious perception, but not the activation of physiological response systems. These results show that verbal instructions can readily overwrite the intrinsic meaning of facial emotions, with clear benefits for social communication as learning and anticipation of threat and safety readjusted to accurately track environmental changes.

Affective startle modulation and psychopathology: Implications for appetitive and defensive brain systems

Startle reflex potentiation versus startle attenuation to unpleasant versus pleasant stimuli likely reflect priming of the defensive versus appetitive motivational systems, respectively. This review summarizes and systemizes the literature on affective startle modulation related to psychopathologies with the aim to reveal underlying mechanisms across psychopathologies. We found evidence for psychopathologies characterized by increased startle potentiation to unpleasant stimuli (anxiety disorders), decreased startle potentiation to unpleasant stimuli (psychopathy), decreased startle attenuation to pleasant stimuli (ADHD), as well as a general hyporeactivity to affective stimuli (depression). Increased versus decreased startle responses to disorder-specific stimuli characterize specific phobia and drug dependence. No psychopathology is characterized by increased startle attenuation to standard pleasant stimuli or a general hyperreactivity to affective stimuli. This review indicates that the defensive and the appetitive systems operate independently mostly in accordance with the motivational priming hypothesis and that affective startle modulation is a highly valuable paradigm to unraveling dysfunctions of the defensive and appetitive systems in psychopathologies as requested by the Research Domain Criteria initiative.

The Startle-Evoked Potential: Negative Affect and Severity of Pathology in Anxiety/Mood Disorders

BACKGROUND

The National Institute of Mental Health Research Domain Criteria initiative encourages a search for dimensional biological measures of psychopathology unconstrained by current diagnostic categories. Consistent with this aim, the presented research studies a large sample of anxiety and mood disorder patients, assessing differences in principal diagnoses and comorbidity patterns, clinicians' ratings, and questionnaire measures of negative affect and life dysfunction as they relate to a potential brain marker of pathology: the amplitude of the event-related potential (ERP) elicited by a startle-evoking stimulus.

METHODS

Patients seeking evaluation or treatment for anxiety and mood disorders (N = 208) participated in two tasks at the University of Florida (Gainesville, FL): 1) imagining emotional and neutral events and 2) viewing emotional and neutral pictures while acoustic startle probes were presented and the ERP was recorded. For a comparison patient group (N = 120), startle probes were administered and ERPs recorded at the University of Greifswald (Greifswald, Germany) while performing the same imagery task.

RESULTS

Reduced positive amplitude of a centroparietal startle-evoked ERP (156-352 ms after onset) significantly predicted higher questionnaire scores of anxiety/depression, reports of increased life dysfunction, greater comorbidity, and clinician ratings of heightened severity and poorer prognosis. The effect was general across principal diagnoses, found for both the Florida and German samples, and consistent in pattern despite differences in the tasks administered.

CONCLUSIONS

The startle-evoked ERP reliably predicts severity and breadth of psychopathology, independent of task context. It is a potential significant contributor to a needed array of biological measures that might improve classification of anxiety and mood disorders.

Face Perception in Social Anxiety: Visuocortical Dynamics Reveal Propensities for Hypervigilance or Avoidance

BACKGROUND

Theories of aberrant attentional processing in social anxiety, and anxiety disorders more broadly, have postulated an initial hypervigilance or facilitation to clinically relevant threats and consequent defensive avoidance. However, existing objective measurements utilized to explore this phenomenon lack the resolution to elucidate attentional dynamics, particularly covert influences.

METHODS

We utilized a continuous measure of visuocortical engagement, the steady-state visual evoked potential in response to naturalistic angry, fearful, happy, and neutral facial expressions. Participants were treatment-seeking patients with principal diagnoses of social anxiety circumscribed to performance situations (n = 21) or generalized across interaction contexts (n = 42), treatment-seeking patients with panic disorder with agoraphobia (n = 25), and 17 healthy participants.

RESULTS

At the principal disorder level, only circumscribed social anxiety patients showed sustained visuocortical facilitation to aversive facial expressions. Control participants as well as patients with panic disorder with agoraphobia and generalized social anxiety showed no bias. More finely stratifying the sample according to clinical judgment of social anxiety severity and interference revealed a linear increase in visuocortical bias to aversive expressions for all but the most severely impaired patients. This group showed an opposing sustained attentional disengagement.

CONCLUSIONS

Rather than shifts between covert vigilance and avoidance of aversive facial expressions, social anxiety appears to confer a sustained bias for one or the other. While vigilant attention reliably increases with social anxiety severity for the majority of patients, the most impaired patients show an opposing avoidance. These distinct patterns of attentional allocation could provide a powerful means of personalizing neuroscience-based interventions to modify attention bias and related impairment.

Hot Speech and Exploding Bombs: Autonomic Arousal During Emotion Classification of Prosodic Utterances and Affective Sounds

Emotional expressions provide strong signals in social interactions and can function as emotion inducers in a perceiver. Although speech provides one of the most important channels for human communication, its physiological correlates, such as activations of the autonomous nervous system (ANS) while listening to spoken utterances, have received far less attention than in other domains of emotion processing. Our study aimed at filling this gap by investigating autonomic activation in response to spoken utterances that were embedded into larger semantic contexts. Emotional salience was manipulated by providing information on alleged speaker similarity. We compared these autonomic responses to activations triggered by affective sounds, such as exploding bombs, and applause. These sounds had been rated and validated as being either positive, negative, or neutral. As physiological markers of ANS activity, we recorded skin conductance responses (SCRs) and changes of pupil size while participants classified both prosodic and sound stimuli according to their hedonic valence. As expected, affective sounds elicited increased arousal in the receiver, as reflected in increased SCR and pupil size. In contrast, SCRs to angry and joyful prosodic expressions did not differ from responses to neutral ones. Pupil size, however, was modulated by affective prosodic utterances, with increased dilations for angry and joyful compared to neutral prosody, although the similarity manipulation had no effect. These results indicate that cues provided by emotional prosody in spoken semantically neutral utterances might be too subtle to trigger SCR, although variation in pupil size indicated the salience of stimulus variation. Our findings further demonstrate a functional dissociation between pupil dilation and skin conductance that presumably origins from their differential innervation.

The special status of sad infant faces: age and valence differences in adults' cortical face processing

Understanding the relative and joint prioritization of age- and valence-related face characteristics in adults’ cortical face processing remains elusive because these two characteristics have not been manipulated in a single study of neural face processing. We used electroencephalography to investigate adults’ P1, N170, P2 and LPP responses to infant and adult faces with happy and sad facial expressions. Viewing infant vs adult faces was associated with significantly larger P1, N170, P2 and LPP responses, with hemisphere and/or participant gender moderating this effect in select cases. Sad faces were associated with significantly larger N170 responses than happy faces. Sad infant faces were associated with significantly larger N170 responses in the right hemisphere than all other combinations of face age and face valence characteristics. We discuss the relative and joint neural prioritization of infant face characteristics and negative facial affect, and their biological value as distinct caregiving and social cues.

Neurophysiological Markers of Emotion Processing in Burnout Syndrome

The substantial body of research employing subjective measures indicates that burnout syndrome is associated with cognitive and emotional dysfunctions. The growing amount of neurophysiological and neuroimaging research helps in broadening existing knowledge of the neural mechanisms underlying core burnout components (emotional exhaustion and depersonalization/cynicism) that are inextricably associated with emotional processing. In the presented EEG study, a group of 93 participants (55 women; mean age = 35.8) were selected for the burnout group or the demographically matched control group on the basis of the results of the Maslach Burnout Inventory – General Survey (MBI-GS) and the Areas of Worklife Survey (AWS). Subjects then participated in an EEG experiment using two experimental procedures: a facial recognition task and viewing of passive pictures. The study focuses on analyzing event-related potentials (ERPs): N170, VPP, EPN, and LPP, as indicators of emotional information processing. Our results show that burnout subjects, as compared to the control group, demonstrate significantly weaker response to affect-evoking stimuli, indexed by a decline in VPP amplitude to emotional faces and decreased EPN amplitude in processing emotional scenes. The analysis of N170 and LPP showed no significant between-group difference. The correlation analyses revealed that VPP and EPN, which are ERP components related to emotional processing, are associated with two core burnout symptoms: emotional exhaustion and cynicism. To our knowledge, we are one of the first research groups to use ERPs to demonstrate such a relationship between neurophysiological activity and burnout syndrome in the context of emotional processing. Thus, in conclusion we emphasized that the decreased amplitude of VPP and EPN components in the burnout group may be a neurophysiological manifestation of emotional blunting and may be considered as neurophysiological markers of emotional exhaustion and cynicism. Additionally, we did not observe a decrease in LPP, which may be considered as a marker that significantly differentiates burnout from depression.

Startling similarity: Effects of facial self-resemblance and familiarity on the processing of emotional faces

Facial self-resemblance has been associated with positive emotional evaluations, but this effect may be biased by self-face familiarity. Here we report two experiments utilizing startle modulation to investigate how the processing of facial expressions of emotion is affected by subtle resemblance to the self as well as to familiar faces. Participants of the first experiment (I) (N = 39) were presented with morphed faces showing happy, neutral, and fearful expressions which were manipulated to resemble either their own or unknown faces. At SOAs of either 300 ms or 3500–4500 ms after picture onset, startle responses were elicited by binaural bursts of white noise (50 ms, 105 dB), and recorded at the orbicularis oculi via EMG. Manual reaction time was measured in a simple emotion discrimination paradigm. Pictures preceding noise bursts by short SOA inhibited startle (prepulse inhibition, PPI). Both affective modulation and PPI of startle in response to emotional faces was altered by physical similarity to the self. As indexed both by relative facilitation of startle and faster manual responses, self-resemblance apparently induced deeper processing of facial affect, particularly in happy faces. Experiment II (N = 54) produced similar findings using morphs of famous faces, yet showed no impact of mere familiarity on PPI effects (or response time, either). The results are discussed with respect to differential (presumably pre-attentive) effects of self-specific vs. familiar information in face processing.

Flashing a smile: Startle eyeblink modulation by masked affective faces

Affective faces are important stimuli with relevance to healthy and abnormal social and affective information processing. The aim of this study was to investigate the effect of brief presentations of affective faces on attention and emotional state across the time course of stimulus processing, as indexed by startle eyeblink response modulation. Healthy adults were presented with happy, neutral, and disgusted male and female faces that were backward masked by neutral faces. Startle responses were elicited at 300, 800, and 3,500 ms following stimulus presentation to probe early and late startle eyeblink modulation, indicative of attention allocation and emotional state, respectively. Results revealed that, at 300 ms, both face expression and face gender modulated startle eyeblink response, suggesting that more attention was allocated to masked happy compared to disgusted female faces, and masked disgusted compared to neutral male faces. There were no effects of either face expression or face gender on startle modulation at 800 ms. At 3,500 ms, target face expression did not modulate startle, but male faces elicited larger startle responses than female faces, indicative of a more negative emotional state. These findings provide a systematic investigation of attention and emotion modulation by brief affective faces across the time course of stimulus processing.

How the Experience of Emotion is Modulated by Facial Feedback

The facial feedback hypothesis states that facial actions modulate subjective experiences of emotion. Using the voluntary facial action technique, in which the participants react with instruction induced smiles and frowns when exposed to positive and negative emotional pictures and then rate the pleasantness of these stimuli, four questions were addressed in the present study. The results in Experiment 1 demonstrated a feedback effect because participants experienced the stimuli as more pleasant during smiling as compared to when frowning. However, this effect was present only during the critical actions of smiling and frowning, with no remaining effects after 5 min or after 1 day. In Experiment 2, feedback effects were found only when the facial action (smile/frown) was incongruent with the presented emotion (positive/negative), demonstrating attenuating but not enhancing modulation. Finally, no difference in the intensity of produced feedback effect was found between smiling and frowning, and no difference in feedback effect was found between positive and negative emotions. In conclusion, facial feedback appears to occur mainly during actual facial actions, and primarily attenuate ongoing emotional states.

The Impact of Top-Down Prediction on Emotional Face Processing in Social Anxiety

There is evidence that people with social anxiety show abnormal processing of emotional faces. To investigate the impact of top-down prediction on emotional face processing in social anxiety, brain responses of participants with high and low social anxiety (LSA) were recorded, while they performed a variation of the emotional task, using high temporal resolution event-related potential techniques. Behaviorally, we reported an effect of prediction with higher accuracy for predictable than unpredictable faces. Furthermore, we found that participants with high social anxiety (HSA), but not with LSA, recognized angry faces more accurately than happy faces. For the P100 and P200 components, HSA participants showed enhanced brain activity for angry faces compared to happy faces, suggesting a hypervigilance to angry faces. Importantly, HSA participants exhibited larger N170 amplitudes in the right hemisphere electrodes than LSA participants when they observed unpredictable angry faces, but not when the angry faces were predictable. This probably reflects the top-down prediction improving the deficiency at building a holistic face representation in HSA participants.

Facing two faces: Defense activation varies as a function of personal relevance

It can be unsettling to be watched by a group of people, and when they express anger or hostility, this can prime defensive behavior. In contrast, when others smile at us, this may be comforting. This study tested to which degree the impact of facial expressions (happy, neutral, and angry) varies with the personal relevance of a social situation. Modelling a triadic situation, two faces looked either directly at the participant, faced each other, or they were back to back. Results confirmed that this variation constitutes a gradient of personal relevance (directed frontally > towards > away), as reflected by corresponding defensive startle modulation and autonomic nervous system activity. This gradient was particularly pronounced for angry faces and it was steeper in participants with higher levels of social anxiety. Thus, sender-recipient constellations modulate the processing of facial emotions in favor of adequate behavioral responding (e.g., avoidance) in group settings.

Empathy-Related Responses to Depicted People in Art Works

Existing theories of empathic response to visual art works postulate the primacy of automatic embodied reaction to images based on mirror neuron mechanisms. Arguing for a more inclusive concept of empathy-related response and integrating four distinct bodies of literature, we discuss contextual, and personal factors which modulate empathic response to depicted people. We then present an integrative model of empathy-related responses to depicted people in art works. The model assumes that a response to empathy-eliciting figural artworks engages the dynamic interaction of two mutually interlinked sets of processes: socio-affective/cognitive processing, related to the person perception, and esthetic processing, primarily concerned with esthetic appreciation and judgment and attention to non-social aspects of the image. The model predicts that the specific pattern of interaction between empathy-related and esthetic processing is co-determined by several sets of factors: (i) the viewer's individual characteristics, (ii) the context variables (which include various modes of priming by narratives and other images), (iii) multidimensional features of the image, and (iv) aspects of a viewer's response. Finally we propose that the model is implemented by the interaction of functionally connected brain networks involved in socio-cognitive and esthetic processing.

Incongruence between Verbal and Non-Verbal Information Enhances the Late Positive Potential

Smooth social communication consists of both verbal and non-verbal information. However, when presented with incongruence between verbal information and nonverbal information, the relationship between an individual judging trustworthiness in those who present the verbal-nonverbal incongruence and the brain activities observed during judgment for trustworthiness are not clear. In the present study, we attempted to identify the impact of incongruencies between verbal information and facial expression on the value of trustworthiness and brain activity using event-related potentials (ERP). Combinations of verbal information [positive/negative] and facial expressions [smile/angry] expressions were presented randomly on a computer screen to 17 healthy volunteers. The value of trustworthiness of the presented facial expression was evaluated by the amount of donation offered by the observer to the person depicted on the computer screen. In addition, the time required to judge the value of trustworthiness was recorded for each trial. Using electroencephalography, ERP were obtained by averaging the wave patterns recorded while the participants judged the value of trustworthiness. The amount of donation offered was significantly lower when the verbal information and facial expression were incongruent, particularly for [negative × smile]. The amplitude of the early posterior negativity (EPN) at the temporal lobe showed no significant difference between all conditions. However, the amplitude of the late positive potential (LPP) at the parietal electrodes for the incongruent condition [negative × smile] was higher than that for the congruent condition [positive × smile]. These results suggest that the LPP amplitude observed from the parietal cortex is involved in the processing of incongruence between verbal information and facial expression.

Repetition and ERPs during emotional scene processing: A selective review

A set of studies are reviewed that investigate the effects of repetition during scene perception on event-related potentials, elucidating perceptual, memory and emotional processes. Repetition suppression was consistently found for the amplitude of early frontal N2 and posterior P2 components, which was greatly enhanced for massed, compared to distributed, repetition. Both repetition suppression and enhancement of the amplitude of a centro-parietal positive potential (LPP) were found in specific contexts. Suppression was reliably found following a massive number of repetitions of few items, whereas enhancement is found when repetitions are spaced; enhancement was apparent both during simple free viewing as well as on an explicit recognition test. Regardless of repetition, an enhanced LPP was always found for emotional, compared to neutral, scenes. Taken together, the data suggest that different effects of massed and distributed repetitions on specific ERP components index perceptual priming, habituation, and spontaneous episodic retrieval.

Evaluative ratings and attention across the life span: emotional arousal and gender

This study was designed to investigate the evolution of emotional processing over the whole adult life span as a function of stimulus arousal and participants' gender. To this end, self-reported affective evaluation and attentional capture prompted by pleasant and unpleasant pictures varying in arousal were measured in a large sample of participants (n = 211) balanced by gender and equally spread across seven decades from 20 to 90 years. Results showed age differences only for affective evaluation of pleasant stimuli, with opposite patterns depending on stimulus arousal. As age increased, low-arousing pleasant cues (e.g. images of babies) were experienced as more pleasant and arousing by both males and females, whereas high-arousing stimuli (e.g. erotic images) were experienced as less pleasant only by females. In contrast, emotional pictures (both pleasant and unpleasant) were effective at capturing attention in a similar way across participants, regardless of age and gender. Taken together, these findings suggest that specific emotional cues prompt different subjective responses across different age groups, while basic mechanisms involved in attentional engagement towards both pleasant and unpleasant stimuli are preserved in healthy ageing.

Emotional facial expressions evoke faster orienting responses, but weaker emotional responses at neural and behavioural levels compared to scenes: A simultaneous EEG and facial EMG study

In the current study, electroencephalography (EEG) was recorded simultaneously with facial electromyography (fEMG) to determine whether emotional faces and emotional scenes are processed differently at the neural level. In addition, it was investigated whether these differences can be observed at the behavioural level via spontaneous facial muscle activity. Emotional content of the stimuli did not affect early P1 activity. Emotional faces elicited enhanced amplitudes of the face-sensitive N170 component, while its counterpart, the scene-related N100, was not sensitive to emotional content of scenes. At 220-280ms, the early posterior negativity (EPN) was enhanced only slightly for fearful as compared to neutral or happy faces. However, its amplitudes were significantly enhanced during processing of scenes with positive content, particularly over the right hemisphere. Scenes of positive content also elicited enhanced spontaneous zygomatic activity from 500-750ms onwards, while happy faces elicited no such changes. Contrastingly, both fearful faces and negative scenes elicited enhanced spontaneous corrugator activity at 500-750ms after stimulus onset. However, relative to baseline EMG changes occurred earlier for faces (250ms) than for scenes (500ms) whereas for scenes activity changes were more pronounced over the whole viewing period. Taking into account all effects, the data suggests that emotional facial expressions evoke faster attentional orienting, but weaker affective neural activity and emotional behavioural responses compared to emotional scenes.

Imaging distributed and massed repetitions of natural scenes: spontaneous retrieval and maintenance

Repetitions that are distributed (spaced) across time prompt enhancement of a memory-related event-related potential, compared to when repetitions are massed (contiguous). Here, we used fMRI to investigate neural enhancement and suppression effects during free viewing of natural scenes that were either novel or repeated four times with massed or distributed repetitions. Distributed repetition was uniquely associated with a repetition enhancement effect in a bilateral posterior parietal cluster that included the precuneus and posterior cingulate and which has previously been implicated in episodic memory retrieval. Unique to massed repetition, conversely, was enhancement in a right dorsolateral prefrontal cluster that has been implicated in short-term maintenance. Repetition suppression effects for both types of spacing were widespread in regions activated during novel picture processing. Taken together, the data are consistent with a hypothesis that distributed repetition prompts spontaneous retrieval of prior occurrences, whereas massed repetition prompts short-term maintenance of the episodic representation, due to contiguous presentation. These processing differences may mediate the classic spacing effect in learning and memory.

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice’s emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.

Prefrontal engagement by cognitive reappraisal of negative faces

Cognitive reappraisal has been associated with increased activation in prefrontal cortex (PFC) and cingulate regions implicated in cognitive control and affect regulation. To date, neuroimaging studies of reappraisal have primarily used emotionally evocative scenes, and it remains unclear whether the same cognitive strategy applied to emotional facial expressions would involve similar or different neural underpinnings. The present study used fMRI to examine brain activation during cognitive reappraisal of negatively valenced facial expressions relative to passive viewing of negative and neutral facial expressions. Twenty-two healthy adults completed a cognitive reappraisal task comprised of three different conditions (Look-Neutral, Maintain-Negative, Reappraise-Negative). Results indicated that reappraisal was associated with a decrease in negative affect and engagement of PFC brain regions implicated in cognitive control and affect regulation (DLPFC, mPFC, and VLPFC). Furthermore, individual differences in habitual reappraisal use were associated with greater DLPFC and mPFC activation, while suppression use was associated with greater amygdala activation. The present study provides preliminary evidence that facial expressions are effective alternative 'targets' of prefrontal engagement during cognitive reappraisal. These findings are particularly relevant for future research probing the neural bases of emotion regulation in populations for whom aversive scenes may be less appropriate (e.g., children) and illnesses in which aberrant responses to social signals of threat and negative feedback are cardinal phenotypes.

The duration of disgusted and fearful faces is judged longer and shorter than that of neutral faces: the attention-related time distortions as revealed by behavioral and electrophysiological measurements

Time perception has been shown to be altered by emotions. This study employed event-related potentials (ERPs) to examine the effects of two threat-related emotions on the judgment of time intervals in the range of 490–910 ms. We demonstrated that disgust and fear have distinct influences on time perception. At the behavioral level, disgusted faces were estimated longer and fearful faces were estimated shorter (i.e., the generalization gradient for the disgusted faces was shifted left while the generalization gradient for the fearful faces was shifted right) when compared with neutral faces. Accordingly, the contingent negative variation, an online ERP index of timing, displayed larger area in disgust and smaller area in fear conditions when compared with neutral condition (disgust = 1.94 ± 2.35 μV_•s, neutral = 1.40 ± 2.5 μV_•s, and fear = 1.00 ± 2.26 μV_•s). These findings indicated that specific neural mechanisms may underlie the attention effects of different subtypes of threat-related emotions on timing; compared with neutral faces, fearful faces are likely to attract more attentional resources while disgusted faces may attract less attentional resources for emotional processing. The major contribution of the current study is to provide neural correlates of fear vs. disgust divergence in the aspect of time perception and to demonstrate beyond the behavioral level that the categorization of threat-related emotions should be refined so to highlight the adaptability of the human defense system.

Selective control of attention supports the positivity effect in aging

There is emerging evidence for a positivity effect in healthy aging, which describes an age-specific increased focus on positive compared to negative information. Life-span researchers have attributed this effect to the selective allocation of cognitive resources in the service of prioritized emotional goals. We explored the basic principles of this assumption by assessing selective attention and memory for visual stimuli, differing in emotional content and self-relevance, in young and old participants. To specifically address the impact of cognitive control, voluntary attentional selection during the presentation of multiple-item displays was analyzed and linked to participants' general ability of cognitive control. Results revealed a positivity effect in older adults' selective attention and memory, which was particularly pronounced for self-relevant stimuli. Focusing on positive and ignoring negative information was most evident in older participants with a generally higher ability to exert top-down control during visual search. Our findings highlight the role of controlled selectivity in the occurrence of a positivity effect in aging. Since the effect has been related to well-being in later life, we suggest that the ability to selectively allocate top-down control might represent a resilience factor for emotional health in aging.

Social and emotional relevance in face processing: happy faces of future interaction partners enhance the late positive potential

Human face perception is modulated by both emotional valence and social relevance, but their interaction has rarely been examined. Event-related brain potentials (ERP) to happy, neutral, and angry facial expressions with different degrees of social relevance were recorded. To implement a social anticipation task, relevance was manipulated by presenting faces of two specific actors as future interaction partners (socially relevant), whereas two other face actors remained non-relevant. In a further control task all stimuli were presented without specific relevance instructions (passive viewing). Face stimuli of four actors (2 women, from the KDEF) were randomly presented for 1s to 26 participants (16 female). Results showed an augmented N170, early posterior negativity (EPN), and late positive potential (LPP) for emotional in contrast to neutral facial expressions. Of particular interest, face processing varied as a function of experimental tasks. Whereas task effects were observed for P1 and EPN regardless of instructed relevance, LPP amplitudes were modulated by emotional facial expression and relevance manipulation. The LPP was specifically enhanced for happy facial expressions of the anticipated future interaction partners. This underscores that social relevance can impact face processing already at an early stage of visual processing. These findings are discussed within the framework of motivated attention and face processing theories.

Threatening scenes but not threatening faces shorten time-to-contact estimates

We previously reported that time-to-contact (TTC) judgments of threatening scene pictures (e.g., frontal attacks) resulted in shortened estimations and were mediated by cognitive processes, and that judgments of threatening (e.g., angry) face pictures resulted in a smaller effect and did not seem cognitively mediated. In the present study, the effects of threatening scenes and faces were compared in two different tasks. An effect of threatening scene pictures occurred in a prediction-motion task, which putatively requires cognitive motion extrapolation, but not in a relative TTC judgment task, which was designed to be less reliant on cognitive processes. An effect of threatening face pictures did not occur in either task. We propose that an object's explicit potential of threat per se, and not only emotional valence, underlies the effect of threatening scenes on TTC judgments and that such an effect occurs only when the task allows sufficient cognitive processing. Results are consistent with distinctions between predator and social fear systems and different underlying physiological mechanisms. Not all threatening information elicits the same responses, and whether an effect occurs at all may depend on the task and the degree to which the task involves cognitive processes.

Crossmodal interactions during affective picture processing

"Natural" crossmodal correspondences, such as the spontaneous tendency to associate high pitches with high spatial locations, are often hypothesized to occur preattentively and independently of task instructions (top-down attention). Here, we investigate bottom-up attentional engagement by using emotional scenes that are known to naturally and reflexively engage attentional resources. We presented emotional (pleasant and unpleasant) or neutral pictures either below or above a fixation cross, while participants were required to discriminate between a high or a low pitch tone (experiment 1). Results showed that despite a robust crossmodal attentional capture of task-irrelevant emotional pictures, the general advantage in classifying the tones for congruent over incongruent visual-auditory stimuli was similar for emotional and neutral pictures. On the other hand, when picture position was task-relevant (experiment 2), task-irrelevant tones did not interact with pictures with regard to their combination of pitch and visual vertical spatial position, but instead they were effective in minimizing the interference effect of emotional picture processing on the ongoing task. These results provide constraints on our current understanding of natural crossmodal correspondences.