Emotional scenes and facial expressions elicit different psychophysiological responses

Effect of Sound Genre on Emotional Responses for Adults With and Without Hearing Loss

OBJECTIVES

Adults with permanent hearing loss exhibit a reduced range of valence ratings in response to nonspeech sounds; however, the degree to which sound genre might affect such ratings is unclear. The purpose of this study was to determine if ratings of valence covary with sound genre (e.g., social communication, technology, music), or only expected valence (pleasant, neutral, unpleasant).

DESIGN

As part of larger study protocols, participants rated valence and arousal in response to nonspeech sounds. For this study, data were reanalyzed by assigning sounds to unidimensional genres and evaluating relationships between hearing loss, age, and gender and ratings of valence. In total, results from 120 adults with normal hearing (M = 46.3 years, SD = 17.7, 33 males and 87 females) and 74 adults with hearing loss (M = 66.1 years, SD = 6.1, 46 males and 28 females) were included.

RESULTS

Principal component analysis confirmed valence ratings loaded onto eight unidimensional factors: positive and negative social communication, positive and negative technology, music, animal, activities, and human body noises. Regression analysis revealed listeners with hearing loss rated some genres as less extreme (less pleasant/less unpleasant) than peers with better hearing, with the relationship between hearing loss and valence ratings being similar across genres within an expected valence category. In terms of demographic factors, female gender was associated with less pleasant ratings of negative social communication, positive and negative technology, activities, and human body noises, while increasing age was related to a subtle rise in valence ratings across all genres.

CONCLUSIONS

Taken together, these results confirm and extend previous findings that hearing loss is related to a reduced range of valence ratings and suggest that this effect is mediated by expected sound valence, rather than sound genre.

The Emotion Authenticity Recognition (EAR) test: normative data of an innovative test using dynamic emotional stimuli to evaluate the ability to recognize the authenticity of emotions expressed by faces

Despite research has massively focused on how emotions conveyed by faces are perceived, the perception of emotions' authenticity is a topic that has been surprisingly overlooked. Here, we present the Emotion Authenticity Recognition (EAR) test, a test specifically developed using dynamic stimuli depicting authentic and posed emotions to evaluate the ability of individuals to correctly identify an emotion (emotion recognition index, ER Index) and classify its authenticity (authenticity recognition index (EA Index). The EAR test has been validated on 522 healthy participants and normative values are provided. Correlations with demographic characteristics, empathy and general cognitive status have been obtained revealing that both indices are negatively correlated with age, and positively with education, cognitive status and different facets of empathy. The EAR test offers a new ecological test to assess the ability to detect emotion authenticity that allow to explore the eventual social cognitive deficit even in patients otherwise cognitively intact.

D, B. VS, A. L, M. L. Down-modulation of functional ventral striatum activation for emotional face stimuli in patients with insula damage

Insula damage results in substantial impairments in facial emotion recognition. In particular, left hemispheric damage appears to be associated with poorer recognition of aversively rated facial expressions. Functional imaging can provide information on differences in the processing of these stimuli in patients with insula lesions when compared to healthy matched controls (HCs). We therefore investigated 17 patients with insula lesions in the chronic stage following stroke and 13 HCs using a passive-viewing task with pictures of facial expressions testing the blood oxygenation dependent (BOLD) effect in predefined regions of interest (ROIs). We expected a decrease in functional activation in an area modulating emotional response (left ventral striatum) but not in the facial recognition areas in the left inferior fusiform gyrus. Quantification of BOLD-response in ROIs but also voxel-based statistics confirmed this hypothesis. The voxel-based analysis demonstrated that the decrease in BOLD in the left ventral striatum was driven by left hemispheric damaged patients (n = 10). In our patient group, insula activation was strongly associated with the intensity rating of facial expressions. In conclusion, the combination of performance testing and functional imaging in patients following circumscribed brain damage is a challenging method for understanding emotion processing in the human brain.

Alpha-band oscillations and emotion: A review of studies on picture perception

Although alpha-band activity has long been a focus of psychophysiological research, its modulation by emotional value during picture perception has only recently been studied systematically. Here, we review these studies and report that the most consistent alpha oscillatory pattern indexing emotional processing is an enhanced desynchronization (ERD) over posterior sensors when viewing emotional compared with neutral pictures. This enhanced alpha ERD is not specific to unpleasant picture content, as previously proposed for other measures of affective response, but has also been observed for pleasant stimuli. Evidence suggests that this effect is not confined to the alpha band but that it also involves a desynchronization of the lower beta frequencies (8-20 Hz). The emotional modulation of alpha ERD occurs even after massive stimulus repetition and when emotional cues serve as task-irrelevant distractors, consistent with the hypothesis that evaluative processes are mandatory in emotional picture processing. A similar enhanced ERD has been observed for other significant cues (e.g., conditioned aversive stimuli, or in anticipation of a potential threat), suggesting that it reflects cortical excitability associated with the engagement of the motivational systems.

Facial expression-based indicators of schizophrenia: Evidence from recent research

Impaired ability to recognize emotion in other's face (decoding) or to express emotion through the face (encoding) are considered critical in schizophrenia. The topic of research draws considerable attention since clinicians rely heavily on the patient's facial expressions for diagnosis and on the patient's ability to understand the clinician's communicative intent. While most researchers argue in favor of a generalized emotion deficit, others indicate an emotion-specific deficit in schizophrenia. An early review (Mandal et al., 1998) indicated a possible breakdown in perception-expression-experience link of emotion; later reviews (Kohler et al., 2010; Chan et al., 2010) pointed to a generalized emotion processing deficit due to perceptual deficits in schizophrenia. The present review (2010-2022) revisits this controversy with 47 published studies (37 decoding, 10 encoding) conducted on 2364 patients in 20 countries. Schizophrenia is characterized by reduced emotion processing ability, especially with negative symptoms and at an acute state of illness. It is however still unclear whether this dysfunction is independent of a generalized face perception deficit or of subjective experience of emotion in schizophrenia.

Time is a great healer: Peak-end memory bias in anxiety - Induced by threat of shock

Recently, we demonstrated that the peak-end memory bias, which is well established in the context of pain, can also be observed in anxiety: Retrospective evaluations of a frightening experience are worse when peak anxiety is experienced at the end of an episode. Here, we set out to conceptually replicate and extend this finding with rigorous experimental control in a threat of shock paradigm. We induced two intensity levels of anxiety by presenting visual cues that indicated different strengths of electric stimuli. Each of the 59 participants went through one of two conditions that only differed in the order of moderate and high threat phases. As a manipulation check, orbicularis-EMG to auditory startle probes, electrodermal activity, and state anxiety confirmed the effects of the specific threat exposure. Critically, after some time had passed, participants for whom exposure had ended with high threat reported more anxiety for the entire episode than those for whom it ended with moderate threat. Moreover, they ranked their experience as more aversive when compared to other unpleasant everyday experiences. This study overcomes several previous limitations and speaks to the generalizability of the peak-end bias. Most notably, the findings bear implications for exposure therapy in clinical anxiety.

New insights on the correspondence between subjective affective experience and physiological responses from representational similarity analysis

Classical views suggest that experienced affect is related to a specific bodily response, whereas recent perspectives challenge this view postulating that similar affective experiences rather evoke different physiological responses. To further advance this debate in the field, we used representational similarity analysis to investigate the correspondence between subjective affect (arousal and valence ratings) and physiological reactions (skin conductance response [SCR], startle blink response, heart rate, and corrugator activity) across various emotion induction contexts (picture viewing task, sound listening task, and imagery task). Significant similarities were exclusively observed between SCR and arousal in the picture viewing task. However, none of the other physiological measures showed a significant relation with valence and arousal ratings in any of the tasks. These findings are discussed within the framework of the Populations hypothesis, suggesting that physiological responses do not depend on the experienced affect but are directly associated with the context in which they are evoked.

Reading Your Emotions in My Physiology? Reliable Emotion Interpretations in Absence of a Robust Physiological Resonance

Affective states are expressed in an individual’s physical appearance, ranging from facial expressions and body postures, to indicators of physiological arousal (e.g., a blush). Confirming the claimed communicative function of these markers, humans are capable of distinguishing between a variety of discrete emotion displays. In an attempt to explain the underlying mechanism, characteristic bodily changes within the observer, including physiological arousal and mimicry, have been suggested to facilitate the interpretation of an expression. The current study aims to create a holistic picture of emotion perception by (1) using three different sources of emotional information (prototypical facial expressions, bodily expressions, and subtle facial cues) and (2) measuring changes in multiple physiological signals (facial electromyography, skin conductance level, skin temperature, and pupil size). While participants clearly discriminated between perceived emotional expressions, there was no overall 1–1 correspondence with their physiological responses. Some specific but robust effects were observed. Angry facial expressions were consistently responded to with a peak in skin conductance level. Furthermore, sad body expressions were associated with a drop in skin temperature. In addition to being the best recognized expression, viewing happy faces elicited congruent facial muscle responses, which supports the potential role of embodied simulation in emotion recognition. Lastly, tears were not only rated as highly emotional intense but also evoked a peak in skin conductance level in the observer. The absence of distinct physiological responses to other expressions could be explained by the lacking functionality of affect sharing in a non-interactive experimental context. Consequentially, emotional alignment in body and mind might especially take place in real social situations, which should be considered in future research.

What's in a face: Automatic facial coding of untrained study participants compared to standardized inventories

Automatic facial coding (AFC) is a novel research tool to automatically analyze emotional facial expressions. AFC can classify emotional expressions with high accuracy in standardized picture inventories of intensively posed and prototypical expressions. However, classification of facial expressions of untrained study participants is more error prone. This discrepancy requires a direct comparison between these two sources of facial expressions. To this end, 70 untrained participants were asked to express joy, anger, surprise, sadness, disgust, and fear in a typical laboratory setting. Recorded videos were scored with a well-established AFC software (FaceReader, Noldus Information Technology). These were compared with AFC measures of standardized pictures from 70 trained actors (i.e., standardized inventories). We report the probability estimates of specific emotion categories and, in addition, Action Unit (AU) profiles for each emotion. Based on this, we used a novel machine learning approach to determine the relevant AUs for each emotion, separately for both datasets. First, misclassification was more frequent for some emotions of untrained participants. Second, AU intensities were generally lower in pictures of untrained participants compared to standardized pictures for all emotions. Third, although profiles of relevant AU overlapped substantially across the two data sets, there were also substantial differences in their AU profiles. This research provides evidence that the application of AFC is not limited to standardized facial expression inventories but can also be used to code facial expressions of untrained participants in a typical laboratory setting.

Autonomic Nervous System Activity During Positive Emotions: A Meta-Analytic Review

Autonomic nervous system (ANS) activity is a fundamental component of emotional responding. It is not clear, however, whether positive emotional states are associated with differential ANS reactivity. To address this issue, we conducted a meta-analytic review of 120 articles (686 effect sizes, total N = 6,546), measuring ANS activity during 11 elicited positive emotions, namely amusement, attachment love, awe, contentment, craving, excitement, gratitude, joy, nurturant love, pride, and sexual desire. We identified a widely dispersed collection of studies. Univariate results indicated that positive emotions produce no or weak and highly variable increases in ANS reactivity. However, the limitations of work to date – which we discuss – mean that our conclusions should be treated as empirically grounded hypotheses that future research should validate.

The relationship between early and recent life stress and emotional expression processing: A functional connectivity study

The aim of this study was to characterize neural activation during the processing of negative facial expressions in a non-clinical group of individuals characterized by two factors: the levels of stress experienced in early life and in adulthood. Two models of stress consequences were investigated: the match/mismatch and cumulative stress models. The match/mismatch model assumes that early adversities may promote optimal coping with similar events in the future through fostering the development of coping strategies. The cumulative stress model assumes that effects of stress are additive, regardless of the timing of the stressors. Previous studies suggested that stress can have both cumulative and match/mismatch effects on brain structure and functioning and, consequently, we hypothesized that effects on brain circuitry would be found for both models. We anticipated effects on the neural circuitry of structures engaged in face perception and emotional processing. Hence, the amygdala, fusiform face area, occipital face area, and posterior superior temporal sulcus were selected as seeds for seed-based functional connectivity analyses. The interaction between early and recent stress was related to alterations during the processing of emotional expressions mainly in to the cerebellum, middle temporal gyrus, and supramarginal gyrus. For cumulative stress levels, such alterations were observed in functional connectivity to the middle temporal gyrus, lateral occipital cortex, precuneus, precentral and postcentral gyri, anterior and posterior cingulate gyri, and Heschl's gyrus. This study adds to the growing body of literature suggesting that both the cumulative and the match/mismatch hypotheses are useful in explaining the effects of stress.

Emotion and Psychophysiological Responses During Emotion-Eliciting Film Clips in an Eating Disorders Sample

Background: Greater vulnerability to negative emotions appears associated with the development and maintenance of eating disorders (EDs). A systematic review of psychophysiological studies using emotion-eliciting film clips reveals that there are no studies examining the effect of standardized validated film clips on psychophysiological response across a range of EDs.

Methods: Using standardized validated film clips without ED-specific content, the present study examined self-reported emotions and psychophysiological responses of women with Binge-Eating Disorder (BED; n = 57), Anorexia Nervosa (AN; n = 16), Bulimia Nervosa (BN; n = 34), and Healthy Controls (HCs; n = 26) at Baseline, during Neutral, Sad, Happy, and Fear-inducing film clips, and at Recovery.

Results: Throughout the protocol, the ED groups reported significantly greater sadness and anxiety than HCs. Additionally, the AN group reported more fear, the BED group more frustration, and the BED and BN groups more tension than HCs. Compared to HCs, the BED group reported stronger urges to binge throughout the protocol, whereas BN group reported stronger urges to binge relative to the HC group only at Baseline and Recovery. The BN and BED groups experienced decreased urges to binge during all film clips compared to Baseline. Respiratory Sinus Arrhythmia levels were significantly lower in the BED group compared to HCs and the BN group throughout the protocol.

Discussion: Standardized validated film clips can be used to elicit expected self-reported emotion and skin conductance responses in ED groups, although individuals with EDs compared HCs report greater negative emotions. Interestingly, film clips appeared to reduce urges to binge in binge-eating groups.

Physiological and affective responses to emoji faces: Effects on facial muscle activity, skin conductance, heart rate, and self-reported affect

The objective of the present study was to evaluate emotional responses to emoji faces through physiological and self-report measures, and evaluate possible differences between men and women. One hundred participants (50 women) observed pictures of happy, neutral, and angry emoji faces, while activity of the zygomatic and corrugator muscles, skin conductance, and heart rate were measured. Self-report measures of emotional experience were also recorded. The results showed an increase in zygomatic muscle activity toward happy emoji faces. An increasing trend in corrugator muscle activity toward angry emoji faces was observed; however, this trend was only marginally significant. Happy emoji faces generated an increase in the skin conductance response. The emotional experience of the participants was also consistent with the emotions that were expressed by the emoji faces. No differences were found between sexes. Overall, the results suggest that emoji faces can especially induce pleasant affective states.

Effects of affective content and motivational context on neural gain functions during naturalistic scene perception

Visual scene processing is modulated by semantic, motivational, and emotional factors, in addition to physical scene statistics. An open question is to what extent those factors affect low-level visual processing. One index of low-level visual processing is the contrast response function (CRF), representing the change in neural or psychophysical gain with increasing stimulus contrast. Here we aimed to (a) establish the use of an electrophysiological technique for assessing CRFs with complex emotional scenes and (b) examine the effects of motivational context and emotional content on CRFs elicited by naturalistic stimuli, including faces and complex scenes (humans, animals). Motivational context varied by expectancy of threat (a noxious noise) versus safety. CRFs were measured in 18 participants by means of sweep steady-state visual evoked potentials. Results showed a facilitation in visuocortical sensitivity (contrast gain) under threat, compared with safe conditions, across all stimulus categories. Facial stimuli prompted heightened neural response gain, compared with scenes. Within the scenes, response gain was smaller for scenes high in emotional arousal, compared with low-arousing scenes, consistent with interference effects of emotional content. These findings support the notion that motivational context alters the contrast sensitivity of cortical tissue, differing from changes in response gain (activation) when visual cues themselves carry motivational/affective relevance.

Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces

Decoding someone's facial expressions provides insights into his or her emotional experience. Recently, Automatic Facial Coding (AFC) software has been developed to provide measurements of emotional facial expressions. Previous studies provided first evidence for the sensitivity of such systems to detect facial responses in study participants. In the present experiment, we set out to generalise these results to affective responses as they can occur in variable social interactions. Thus, we presented facial expressions (happy, neutral, angry) and instructed participants (N = 64) to either actively mimic, to look at them passively (n = 21), or to inhibit their own facial reaction (n = 22). A video stream for AFC and an electromyogram (EMG) of the zygomaticus and corrugator muscles were registered continuously. In the mimicking condition, both AFC and EMG differentiated well between facial expressions in response to the different emotional pictures. In the passive viewing and in the inhibition condition AFC did not detect changes in facial expressions whereas EMG was still highly sensitive. Although only EMG is sensitive when participants intend to conceal their facial reactions, these data extend previous findings that Automatic Facial Coding is a promising tool for the detection of intense facial reaction.

Stress Makes the Difference: Social Stress and Social Anxiety in Decision-Making Under Uncertainty

Stress and anxiety can both influence risk-taking in decision-making. While stress typically increases risk-taking, anxiety often leads to risk-averse choices. Few studies have examined both stress and anxiety in a single paradigm to assess risk-averse choices. We therefore set out to examine emotional decision-making under stress in socially anxious participants. In our study, individuals (N = 87) high or low in social anxiety completed an expanded variation of the Balloon Analogue Risk Task (BART). While inflating a balloon to a larger degree is rewarded, a possible explosion leads to (a) a loss of money and (b) it is followed by an emotional picture (i.e., a calm vs. an angry face). To induce stress before this task, participants were told that they would have to deliver a speech. We operationalized risk-taking by the number of pumps during inflation and its functionality by the amount of monetary gain. In addition, response times were recorded as an index of decisional conflict. Without the stressor, high socially anxious compared to low socially anxious participants did not differ in any of the dependent variables. However, under stress, the low socially anxious group took more risk and earned more money, while high socially anxious individuals remained more cautious and did not change their risk-taking under social stress. Overall, high socially anxious individuals made their decisions more hesitantly compared to low socially anxious individuals. Unexpectedly, there were no main effects or interactions with the valence of the emotional faces. This data shows that stress affects socially anxious individuals differently: in low socially anxious individuals stress fosters risk-taking, whereas high socially anxious individuals did not alter their behavior and remained risk-averse. The novel eBART is a promising research tool to examine the specific factors that influence decision-making.

Model Gender Interacts With Expressed Emotion to Enhance Startle: Angry Male and Happy Female Faces Produce the Greatest Potentiation

Several studies have implied gender differences in startle reaction to emotional facial expressions. However, few studies have been designed to investigate the difference between responding to emotional female vs. male faces, nor gender differences in responses. The present experiment investigated startle EMG responses to a startle probe while viewing pictures of neutral, happy, angry, fearful, and sad facial expressions presented by female and male models. Participants were divided into female and male groups. Results showed that emotional facial expressions interact with model gender to produce startle potentiation to a probe: greater responses were found while viewing angry expressions by male models, and while viewing happy faces by female models. There were no differences in responses between male and female participants. We argue that these findings underline theimportance of controlling for model gender in research using facial expressions as stimuli.

Pain Modulates Responses to Emotional Stimuli

Pain and emotion are common subjective experiences that play vital roles in daily life. Pain has been clinically confirmed to increase depressive mood. However, little is known about how pain modulates cognitive emotional judgment processing. A better understanding of this may help explain the effect of pain on the development of depressive moods. We recruited 30 adult participants to test their responses to pictures of scenes (Experiment 1) and faces (Experiment 2) that represented happy, neutral, and sad emotions, while experiencing painful (induced via topical capsaicin cream) and control (hand cream) treatments. Results showed that participants in the painful condition showed lower accuracy to emotional scene stimuli and longer reaction times to both emotional scene and face stimuli, relative to the control condition. In addition, the difference values of the reaction times between the painful and control conditions were larger for sad scenes than for happy or neutral scenes. These results suggest that pain alters attentional processing of emotional stimuli, especially with regards to sad scene stimuli, which may explain how painful stimuli affect the development of depressive moods.

The spatial effect of fearful faces in the autonomic response

Peripersonal space (PPS) corresponds to the space around the body and it is defined by the location in space where multimodal inputs from bodily and external stimuli are integrated. Its extent varies according to the characteristics of external stimuli, e.g., the salience of an emotional facial expression. In the present study, we investigated the psycho-physiological correlates of the extension phenomenon. Specifically, we investigated whether an approaching human face showing either an emotionally negative (fearful) or positive (joyful) facial expression would differentially modulate PPS representation, compared to the same face with a neutral expression. To this aim, we continuously recorded the skin conductance response (SCR) of 27 healthy participants while they watched approaching 3D avatar faces showing fearful, joyful or neutral expressions, and then pressed a button to respond to tactile stimuli delivered on their cheeks at three possible delays (visuo-tactile trials). The results revealed that the SCR to fearful faces, but not joyful or neutral faces, was modulated by the apparent distance from the participant's body. SCR increased from very far space to far and then to near space. We propose that the proximity of the fearful face provided a cue to the presence of a threat in the environment and elicited a robust and urgent organization of defensive responses. In contrast, there would be no need to organize defensive responses to joyful or neutral faces and, as a consequence, no SCR differences were found across spatial positions. These results confirm the defensive function of PPS.

Read My Face: Automatic Facial Coding Versus Psychophysiological Indicators of Emotional Valence and Arousal

Facial expressions provide insight into a person's emotional experience. To automatically decode these expressions has been made possible by tremendous progress in the field of computer vision. Researchers are now able to decode emotional facial expressions with impressive accuracy in standardized images of prototypical basic emotions. We tested the sensitivity of a well-established automatic facial coding software program to detect spontaneous emotional reactions in individuals responding to emotional pictures. We compared automatically generated scores for valence and arousal of the Facereader (FR; Noldus Information Technology) with the current psychophysiological gold standard of measuring emotional valence (Facial Electromyography, EMG) and arousal (Skin Conductance, SC). We recorded physiological and behavioral measurements of 43 healthy participants while they looked at pleasant, unpleasant, or neutral scenes. When viewing pleasant pictures, FR Valence and EMG were both comparably sensitive. However, for unpleasant pictures, FR Valence showed an expected negative shift, but the signal differentiated not well between responses to neutral and unpleasant stimuli, that were distinguishable with EMG. Furthermore, FR Arousal values had a stronger correlation with self-reported valence than with arousal while SC was sensitive and specifically associated with self-reported arousal. This is the first study to systematically compare FR measurement of spontaneous emotional reactions to standardized emotional images with established psychophysiological measurement tools. This novel technology has yet to make strides to surpass the sensitivity of established psychophysiological measures. However, it provides a promising new measurement technique for non-contact assessment of emotional responses.

The Body-Threat Assessment Battery (BTAB): A new instrument for the quantification of threat-related autonomic affective responses induced via dynamic movie clips

We present a new instrument for the assessment of responses to threat-related imagery directed towards a human body - the Body-Threat Assessment Battery (BTAB). The BTAB consists of a series of high-definition dynamic clips depicting body-threats and matched non-threat baseline behaviours. For body-threat stimuli a perspective manipulation was included to assess the effects of viewing threats from the point-of-view of the observer (POV) or from an external/exocentric perspective (EXO). Green-screen technology was used so that extraneous background information could be removed and standardised in post-production. Categorical normative data for psychological ratings (valence, arousal and pain), psychophysiological, phasic skin conductance responses (SCRs) and tonic skin conductance levels (SCLs) were obtained for all stimuli. Body-threat stimuli evoked significantly higher psychological ratings of arousal and pain, with more negative ratings of valence, relative to baseline stimuli. In addition, threat stimuli also had an increased efficacy at evoking SCRs, and these were significantly stronger relative to baseline stimuli. There were no effects of perspective on psychophysiological or psychological responses to threat imagery. The findings are discussed in the context of the utility and scope of the BTAB for supporting neurocognitive investigations of aversive imagery and body-threats specifically in the study of embodiment, body-processing and self-consciousness.

Psychophysiological Effects of Downregulating Negative Emotions: Insights From a Meta-Analysis of Healthy Adults

Assessing psychophysiological responses of emotion regulation is a cost-efficient way to quantify emotion regulation and to complement subjective report that may be biased. Previous studies have revealed inconsistent results complicating a sound interpretation of these findings. In the present study, we summarized the existing literature through a systematic search of articles. Meta-analyses were used to evaluate effect sizes of instructed downregulation strategies on common autonomic (electrodermal, respiratory, cardiovascular, and pupillometric) and electromyographic (corrugator activity, emotion-modulated startle) measures. Moderator analyses were conducted, with moderators including study design, emotion induction, control instruction and trial duration. We identified k = 78 studies each contributing multiple sub-samples and performed 23 meta-analyses for combinations of emotion regulation strategy and psychophysiological measure. Overall, results showed that effects of reappraisal and suppression on autonomic measures were highly inconsistent across studies with rather small mean effect sizes. Electromyography (startle and corrugator activity) showed medium effect sizes that were consistent across studies. Our findings highlight the diversity as well as the low level of standardization and comparability of research in this area. Significant moderation of effects by study design, trial duration, and control condition emphasizes the need for better standardization of methods. In addition, the small mean effect sizes resulting from our analyses on autonomic measures should be interpreted with caution. Findings corroborate the importance of multi-channel approaches.

The effect of being watched on facial EMG and autonomic activity in response to another individual's facial expressions

We tested if facial reactions to another person's facial expressions depend on the self-relevance of the observed expressions. In the present study (n = 44), we measured facial electromyographic (zygomatic and corrugator) activity and autonomic arousal (skin conductance) responses to a live model's smiling and neutral faces. In one condition, the participant and the model were able to see each other normally, whereas in the other condition, the participant was led to believe that the model could not see the participant. The results showed that the increment of zygomatic activity in response to smiling faces versus neutral faces was greater when the participants believed they were being watched than it was when the participants believed they were not being watched. However, zygomatic responses to smiles did not differ between the conditions, while the results suggested that the participants' zygomatic responses to neutral faces seemed to attenuate in the condition of believing they were being watched. Autonomic responses to smiling faces were greater in the belief of being watched than in the belief of not being watched condition. The results suggest that the self-relevance of another individual's facial expression modulates autonomic arousal responses and to a lesser extent facial EMG responses.

Facial Reactivity and Attentional Processing of Facial Expressions and Gaze Direction

Abstract. Patterns of facial reactivity and attentional allocation to emotional facial expressions, and how these are moderated by gaze direction, are not clearly established. Among a sample of undergraduate university students, aged between 17 and 22 years (76% female), corrugator and zygomatic reactivity, as measured by facial electromyography, and attention allocation, as measured by the startle reflex and startle-elicited N100, was examined while viewing happy, neutral, angry and fearful facial expressions, which were presented at either 0- or 30-degree gaze. Results indicated typically observed facial mimicry to happy faces but, unexpectedly, “smiling” facial responses to fearful, and to a lesser extent, angry faces. This facial reactivity was not influenced by gaze direction. Furthermore, emotional facial expressions did not elicit increased attentional allocation. Likewise, matched facial expressions did not elicit increased attentional allocation. Rather, happy and fearful faces with direct (0°) gaze elicited increased controlled attentional allocation, and averted (30°) gaze faces, regardless of emotional expression, elicited preferential, early cortical processing. These findings suggest typical facial mimicry to happy faces, but unexpected facial reactivity to angry and fearful faces, perhaps due to an attempt to regulate social bonds during threat perception. Findings also suggest a divergence in controlled versus preferential, early cortical attentional processing for direct compared to averted gaze faces. These findings relate to young, mostly female, adults attending university. The experiment should be repeated with a larger sample drawn from the general community, with a broader age range and gender balance, and with a stimulus set with validated subjective valence and arousal ratings. This can reduce Type II error and establish normative patterns of facial reactivity and attentional processing of emotional facial expressions with different gaze directions.

Recall of facial expressions and simple orientations reveals competition for resources at multiple levels of the visual hierarchy

Many studies of visual working memory have tested humans' ability to reproduce primary visual features of simple objects, such as the orientation of a grating or the hue of a color patch, following a delay. A consistent finding of such studies is that precision of responses declines as the number of items in memory increases. Here we compared visual working memory for primary features and high-level objects. We presented participants with memory arrays consisting of oriented gratings, facial expressions, or a mixture of both. Precision of reproduction for all facial expressions declined steadily as the memory load was increased from one to five faces. For primary features, this decline and the specific distributions of error observed, have been parsimoniously explained in terms of neural population codes. We adapted the population coding model for circular variables to the non-circular and bounded parameter space used for expression estimation. Total population activity was held constant according to the principle of normalization and the intensity of expression was decoded by drawing samples from the Bayesian posterior distribution. The model fit the data well, showing that principles of population coding can be applied to model memory representations at multiple levels of the visual hierarchy. When both gratings and faces had to be remembered, an asymmetry was observed. Increasing the number of faces decreased precision of orientation recall, but increasing the number of gratings did not affect recall of expression, suggesting that memorizing faces involves the automatic encoding of low-level features, in addition to higher-level expression information.

Affective startle modulation and psychopathology: Implications for appetitive and defensive brain systems

Startle reflex potentiation versus startle attenuation to unpleasant versus pleasant stimuli likely reflect priming of the defensive versus appetitive motivational systems, respectively. This review summarizes and systemizes the literature on affective startle modulation related to psychopathologies with the aim to reveal underlying mechanisms across psychopathologies. We found evidence for psychopathologies characterized by increased startle potentiation to unpleasant stimuli (anxiety disorders), decreased startle potentiation to unpleasant stimuli (psychopathy), decreased startle attenuation to pleasant stimuli (ADHD), as well as a general hyporeactivity to affective stimuli (depression). Increased versus decreased startle responses to disorder-specific stimuli characterize specific phobia and drug dependence. No psychopathology is characterized by increased startle attenuation to standard pleasant stimuli or a general hyperreactivity to affective stimuli. This review indicates that the defensive and the appetitive systems operate independently mostly in accordance with the motivational priming hypothesis and that affective startle modulation is a highly valuable paradigm to unraveling dysfunctions of the defensive and appetitive systems in psychopathologies as requested by the Research Domain Criteria initiative.

Subjective emotional arousal: an explorative study on the role of gender, age, intensity, emotion regulation difficulties, depression and anxiety symptoms, and meta-emotion

Subjective emotional arousal in typically developing adults was investigated in an explorative study. 177 participants (20-70 years) rated facial expressions and words for self-experienced arousal and perceived intensity, and completed the Difficulties in Emotion Regulation scale and the Hospital Anxiety and Depression scale (HADS-D). Exclusion criteria were psychiatric or neurological diseases, or clinically relevant scores in the HADS-D. Arousal regarding faces and words was significantly predicted by emotional clarity. Separate analyses showed following significant results: arousal regarding faces and arousal regarding words constantly predicted each other; negative faces were predicted by age and intensity; neutral faces by gender and impulse control; positive faces by gender and intensity; negative words by emotional clarity; and neutral words by gender. Males showed higher arousal scores than females regarding neutral faces and neutral words; for the other arousal scores, no explicit group differences were shown. Cluster analysis yielded three distinguished emotional characteristics groups: "emotional difficulties disposition group" (mainly females; highest emotion regulation difficulties, depression and anxiety scores; by trend highest arousal), "low emotional awareness group" (exclusively males; lowest awareness regarding currently experienced emotions; by trend intermediate arousal), and a "low emotional difficulties group" (exclusively females; lowest values throughout). No age effect was shown. Results suggest that arousal elicited by facial expressions and words are specialized parts of a greater emotional processing system and that typically developing adults show some kind of stable, modality-unspecific dispositional baseline of emotional arousal. Emotional awareness and clarity, and impulse control probably are trait aspects of emotion regulation that influence emotional arousal in typically developing adults and can be regarded as aspects of meta-emotion. Different emotional personality styles were shown between as well as within gender groups.

How the visual brain detects emotional changes in facial expressions: Evidence from driven and intrinsic brain oscillations

The processing of facial expressions is often studied using static pictorial cues. Recent work, however, suggests that viewing changing expressions more robustly evokes physiological responses. Here, we examined the sensitivity of steady-state visual evoked potentials and intrinsic oscillatory brain activity to transient emotional changes in facial expressions. Twenty-two participants viewed sequences of grayscale faces periodically turned on and off at a rate of 17.5 Hz, to evoke flicker steady-state visual evoked potentials (ssVEPs) in visual cortex. Each sequence began with a neutral face (flickering for 2290 msec), immediately followed by a face from the same actor (also flickering for 2290 msec) with one of four expressions (happy, angry, fearful, or another neutral expression), followed by the initially presented neutral face (flickering for 1140 msec). The amplitude of the ssVEP and the power of intrinsic brain oscillations were analyzed, comparing the four expression-change conditions. We found a transient perturbation (reduction) of the ssVEP that was more pronounced after the neutral-to-angry change compared to the other conditions, at right posterior sensors. Induced alpha-band (8-13 Hz) power was reduced compared to baseline after each change. This reduction showed a central-occipital topography and was strongest in the subtlest and rarest neutral-to-neutral condition. Thus, the ssVEP indexed involvement of face-sensitive cortical areas in decoding affective expressions, whereas mid-occipital alpha power reduction reflected condition frequency rather than expression-specific processing, consistent with the role of alpha power changes in selective attention.

Verbal instructions override the meaning of facial expressions

Psychological research has long acknowledged that facial expressions can implicitly trigger affective psychophysiological responses. However, whether verbal information can alter the meaning of facial emotions and corresponding response patterns has not been tested. This study examined emotional facial expressions as cues for instructed threat-of-shock or safety, with a focus on defensive responding. In addition, reversal instructions were introduced to test the impact of explicit safety instructions on fear extinction. Forty participants were instructed that they would receive unpleasant electric shocks, for instance, when viewing happy but not angry faces. In a second block, instructions were reversed (e.g., now angry faces cued shock). Happy, neutral, and angry faces were repeatedly presented, and auditory startle probes were delivered in half of the trials. The defensive startle reflex was potentiated for threat compared to safety cues. Importantly, this effect occurred regardless of whether threat was cued by happy or angry expressions. Although the typical pattern of response habituation was observed, defense activation to newly instructed threat cues remained significantly enhanced in the second part of the experiment, and it was more pronounced in more socially anxious participants. Thus, anxious individuals did not exhibit more pronounced defense activation compared to less anxious participants, but their defense activation was more persistent.

Effect of congruence between sound and video on heart rate and self-reported measures of emotion

Most studies of emotional responses have used unimodal stimuli (e.g., pictures or sounds) or congruent bimodal stimuli (e.g., video clips with sound), but little is known about the emotional response to incongruent bimodal stimuli. The aim of the present study was to evaluate the effect of congruence between auditory and visual bimodal stimuli on heart rate and self-reported measures of emotional dimension, valence and arousal. Subjects listened to pleasant, neutral, and unpleasant sounds, accompanied by videos with and without content congruence, and heart rate was recorded. Dimensions of valence and arousal of each bimodal stimulus were then self-reported. The results showed that heart rate depends of the valence of the sounds but not of the congruence of the bimodal stimuli. The valence and arousal scores changed depending on the congruence of the bimodal stimuli. These results suggest that the congruence of bimodal stimuli affects the subjective perception of emotion.

Neural Responses to Consciously and Unconsciously Perceived Emotional Faces: A Spinal fMRI Study

Emotional stimuli modulate activity in brain areas related to attention, perception, and movement. Similar increases in neural activity have been detected in the spinal cord, suggesting that this understudied component of the central nervous system is an important part of our emotional responses. To date, previous studies of emotion-dependent spinal cord activity have utilized long presentations of complex emotional scenes. The current study differs from this research by (1) examining whether emotional faces will lead to enhanced spinal cord activity and (2) testing whether these stimuli require conscious perception to influence neural responses. Fifteen healthy undergraduate participants completed six spinal functional magnetic resonance imaging (fMRI) runs in which three one-minute blocks of fearful, angry, or neutral faces were interleaved with 40-s rest periods. In half of the runs, the faces were clearly visible while in the other half, the faces were displayed for only 17 ms. Spinal fMRI consisted of half-Fourier acquisition single-shot turbo spin-echo (HASTE) sequences targeting the cervical spinal cord. The results indicated that consciously perceived faces expressing anger elicited significantly more activity than fearful or neutral faces in ventral (motoric) regions of the cervical spinal cord. When stimuli were presented below the threshold of conscious awareness, neutral faces elicited significantly more activity than angry or fearful faces. Together, these data suggest that the emotional modulation of spinal cord activity is most impactful when the stimuli are consciously perceived and imply a potential threat toward the observer.

Experimental and methodological factors affecting test-retest reliability of amygdala BOLD responses

Previous studies reported poor to fair test-retest reliability of amygdala BOLD responses to emotional stimuli. However, these findings are very heterogeneous across and within studies. The present study sought to systematically examine experimental and methodological factors that contribute to this heterogeneity. Forty-six young subjects were scanned twice with a mean test-retest interval of 7 weeks. We compared amygdala reliability across three tasks: A face-matching task, passive viewing of emotional faces, and passive viewing of emotional scenes. We also explored whether extraction of physiological noise can affect the stability of amygdala responses. We assessed test-retest reliability of amygdala mean amplitudes at the individual level and spatial repeatability (i.e., stability of the spatial distribution of activation) of the amygdala BOLD signal at the group and individual level. All three tasks evoked robust amygdala activation at the group level. At the individual level, amygdala spatial repeatability was poor during passive viewing of scenes and faces and fair or close to fair in the face-matching task. On the other hand, reliability of amygdala mean responses was very poor in the face-matching task while it was significantly higher during passive viewing of faces and scenes. Physiological noise correction changed reliability rates but not uniformly across the three tasks. The current work suggests that the presence of a concurrent task during emotion processing affects amygdala reliability. The dissociation between spatial repeatability and reliability of mean amplitudes highlights the importance of taking into account both measures for a multidimensional assessment of the reliability of BOLD responses.

Threat vs. Threat: Attention to Fear-Related Animals and Threatening Faces

It is generally thought to be adaptive that fear relevant stimuli in the environment can capture and hold our attention; and in psychopathology attentional allocation is thought to be cue-specific. Such hypervigilance toward threatening cues or difficulty to disengage attention from threat has been demonstrated for a variety of stimuli, for example, toward evolutionary prepared animals or toward socially relevant facial expressions. Usually, specific stimuli have been examined in individuals with particular fears (e.g., animals in animal fearful and faces in socially fearful participants). However, different kinds of stimuli are rarely examined in one study. Thus, it is unknown how different categories of threatening stimuli compete for attention and how specific kinds of fears modulate these attentional processes. In this study, we used a free viewing paradigm: pairs of pictures with threat-related content (spiders or angry faces) or neutral content (butterflies or neutral faces) were presented side by side (i.e., spiders and angry faces, angry and neutral faces, spiders and butterflies, butterflies and neutral faces). Eye-movements were recorded while spider fearful, socially anxious, or non-anxious participants viewed the picture pairs. Results generally replicate the finding that unpleasant pictures more effectively capture attention in the beginning of a trial compared to neutral pictures. This effect was more pronounced in spider fearful participants: the higher the fear the quicker they were in looking at spiders. This was not the case for high socially anxious participants and pictures of angry faces. Interestingly, when presented next to each other, there was no preference in initial orientation for either spiders or angry faces. However, neutral faces were looked at more quickly than butterflies. Regarding sustained attention, we found no general preference for unpleasant pictures compared to neutral pictures.

Cardiac responses to viewing facial emotion differentiate frontotemporal dementias

Objective

To establish proof‐of‐principle for the use of heart rate responses as objective measures of degraded emotional reactivity across the frontotemporal dementia spectrum, and to demonstrate specific relationships between cardiac autonomic responses and anatomical patterns of neurodegeneration.

Methods

Thirty‐two patients representing all major frontotemporal dementia syndromes and 19 healthy older controls performed an emotion recognition task, viewing dynamic, naturalistic videos of facial emotions while ECG was recorded. Cardiac reactivity was indexed as the increase in interbeat interval at the onset of facial emotions. Gray matter associations of emotional reactivity were assessed using voxel‐based morphometry of patients’ brain MR images.

Results

Relative to healthy controls, all patient groups had impaired emotion identification, whereas cardiac reactivity was attenuated in those groups with predominant fronto‐insular atrophy (behavioral variant frontotemporal dementia and nonfluent primary progressive aphasia), but preserved in syndromes focused on the anterior temporal lobes (right temporal variant frontotemporal dementia and semantic variant primary progressive aphasia). Impaired cardiac reactivity correlated with gray matter atrophy in a fronto‐cingulo‐insular network that overlapped correlates of cognitive emotion processing.

Interpretation

Autonomic indices of emotional reactivity dissociate from emotion categorization ability, stratifying frontotemporal dementia syndromes and showing promise as novel biomarkers. Attenuated cardiac responses to the emotions of others suggest a core pathophysiological mechanism for emotional blunting and degraded interpersonal reactivity in these diseases.

Hot Speech and Exploding Bombs: Autonomic Arousal During Emotion Classification of Prosodic Utterances and Affective Sounds

Emotional expressions provide strong signals in social interactions and can function as emotion inducers in a perceiver. Although speech provides one of the most important channels for human communication, its physiological correlates, such as activations of the autonomous nervous system (ANS) while listening to spoken utterances, have received far less attention than in other domains of emotion processing. Our study aimed at filling this gap by investigating autonomic activation in response to spoken utterances that were embedded into larger semantic contexts. Emotional salience was manipulated by providing information on alleged speaker similarity. We compared these autonomic responses to activations triggered by affective sounds, such as exploding bombs, and applause. These sounds had been rated and validated as being either positive, negative, or neutral. As physiological markers of ANS activity, we recorded skin conductance responses (SCRs) and changes of pupil size while participants classified both prosodic and sound stimuli according to their hedonic valence. As expected, affective sounds elicited increased arousal in the receiver, as reflected in increased SCR and pupil size. In contrast, SCRs to angry and joyful prosodic expressions did not differ from responses to neutral ones. Pupil size, however, was modulated by affective prosodic utterances, with increased dilations for angry and joyful compared to neutral prosody, although the similarity manipulation had no effect. These results indicate that cues provided by emotional prosody in spoken semantically neutral utterances might be too subtle to trigger SCR, although variation in pupil size indicated the salience of stimulus variation. Our findings further demonstrate a functional dissociation between pupil dilation and skin conductance that presumably origins from their differential innervation.

Face-to-face: Perceived personal relevance amplifies face processing

The human face conveys emotional and social information, but it is not well understood how these two aspects influence face perception. In order to model a group situation, two faces displaying happy, neutral or angry expressions were presented. Importantly, faces were either facing the observer, or they were presented in profile view directed towards, or looking away from each other. In Experiment 1 (n = 64), face pairs were rated regarding perceived relevance, wish-to-interact, and displayed interactivity, as well as valence and arousal. All variables revealed main effects of facial expression (emotional > neutral), face orientation (facing observer > towards > away) and interactions showed that evaluation of emotional faces strongly varies with their orientation. Experiment 2 (n = 33) examined the temporal dynamics of perceptual-attentional processing of these face constellations with event-related potentials. Processing of emotional and neutral faces differed significantly in N170 amplitudes, early posterior negativity (EPN), and sustained positive potentials. Importantly, selective emotional face processing varied as a function of face orientation, indicating early emotion-specific (N170, EPN) and late threat-specific effects (LPP, sustained positivity). Taken together, perceived personal relevance to the observer-conveyed by facial expression and face direction-amplifies emotional face processing within triadic group situations.

The influence of frontal alpha-asymmetry on the processing of approach- and withdrawal-related stimuli-A multichannel psychophysiology study

The approach-withdrawal model of hemispheric activation suggests that left frontal cortical areas mediate approach, while right frontal cortical areas mediate withdrawal motivation. Within this framework, the present study investigates the association of frontal cortical asymmetry with attentional and emotional responses toward approach- and withdrawal-related emotional stimuli. Resting frontal asymmetry was measured from 43 students before they passively viewed negative, neutral, and positive emotional pictures. The startle reflex, skin conductance response, and subjective ratings of valence and arousal were assessed to quantify emotional responding, while attention was assessed with ERPs. We also assessed frontal asymmetry in response to the pictures. Results indicated that relatively stronger right frontal cortical activation was associated with increased N1 amplitudes and more negative subjective emotional evaluation of all stimuli. Furthermore, enhanced right frontal asymmetry (state and trait) was associated with diminished emotional modulation of the late positive potential. In contrast, no association of frontal asymmetry with defensive reflex physiology or activation of sympathetic nervous system activity was found. The current data suggest dissociable influence of resting frontal brain asymmetry on attentional and physiological processing of withdrawal- and approach-related stimuli. That is, asymmetrical frontal cortical brain activation might not modulate approach-/withdrawal-related motor responses and sympathetic arousal directly, but instead enhances allocation of attentional resources to subjectively significant stimuli. The results are discussed in terms of their potential importance for emotion perception in anxiety disorders and their contribution to the understanding of frontal asymmetry.

Facing two faces: Defense activation varies as a function of personal relevance

It can be unsettling to be watched by a group of people, and when they express anger or hostility, this can prime defensive behavior. In contrast, when others smile at us, this may be comforting. This study tested to which degree the impact of facial expressions (happy, neutral, and angry) varies with the personal relevance of a social situation. Modelling a triadic situation, two faces looked either directly at the participant, faced each other, or they were back to back. Results confirmed that this variation constitutes a gradient of personal relevance (directed frontally > towards > away), as reflected by corresponding defensive startle modulation and autonomic nervous system activity. This gradient was particularly pronounced for angry faces and it was steeper in participants with higher levels of social anxiety. Thus, sender-recipient constellations modulate the processing of facial emotions in favor of adequate behavioral responding (e.g., avoidance) in group settings.

In Your Face: Startle to Emotional Facial Expressions Depends on Face Direction

Although faces are often included in the broad category of emotional visual stimuli, the affective impact of different facial expressions is not well documented. The present experiment investigated startle electromyographic responses to pictures of neutral, happy, angry, and fearful facial expressions, with a frontal face direction (directed) and at a 45° angle to the left (averted). Results showed that emotional facial expressions interact with face direction to produce startle potentiation: Greater responses were found for angry expressions, compared with fear and neutrality, with directed faces. When faces were averted, fear and neutrality produced larger responses compared with anger and happiness. These results are in line with the notion that startle is potentiated to stimuli signaling threat. That is, a forward directed angry face may signal a threat toward the observer, and a fearful face directed to the side may signal a possible threat in the environment.

Initial Heart Rate Reactivity to Socioemotional Pictures in Early-Onset Alzheimer's Disease

Patients with Alzheimer's disease (AD) often have generalized anxiety, particularly in early-onset AD (EOAD) or the first stages of their disease. This increased anxiety could be associated with decreased sensorimotor gating with increased attention to significant stimuli from AD pathology in the entorhinal cortex. We investigated whether widening initial attention to socioemotional stimuli was association with anxiety among 16 patients with first stage EOAD compared to 19 normal controls (NCs). The participants underwent assessment of their initial heart rate deceleration ("orienting response"; OR), a measure of attentional refocusing, to pictures (International Affective Picture Stimuli) varying in pleasant-unpleasant valence and social-nonsocial content. The results showed group differences; the EOAD patients had significantly larger ORs than the NCs across conditions, with larger ORs in each valence and social condition. In addition, the EOAD patients, but not the NCs, showed ORs to normally less threatening stimuli, particularly pleasant, but also less significantly, social stimuli. On the Neuropsychiatric Inventory, the ORs among the EOAD patients significantly correlated with anxiety scores. Together, these findings suggest that anxiety in mild EOAD may be associated with widening attentional refocusing to socioemotional stimuli, possibly reflecting decreased sensorimotor gating in the entorhinal cortex. This finding could be a potential biomarker for the first stages of AD.

Multimodal analysis of startle type responses

BACKGROUND AND OBJECTIVE

This article presents a multimodal analysis of startle type responses using a variety of physiological, facial, and speech features. These multimodal components of the startle type response reflect complex brain-body reactions to a sudden and intense stimulus. Additionally, the proposed multimodal evaluation of reflexive and emotional reactions associated with the startle eliciting stimuli and underlying neural networks and pathways could be applied in diagnostics of different psychiatric and neurological diseases. Different startle type stimuli can be compared in the strength of their elicitation of startle responses, i.e. their potential to activate stress-related neural pathways, underlying biomarkers and corresponding behavioral reactions.

METHODS

An innovative method for measuring startle type responses using multimodal stimuli and multimodal feature analysis has been introduced. Individual's multimodal reflexive and emotional expressions during startle type elicitation have been assessed by corresponding physiological, speech and facial features on ten female students of psychology. Different startle eliciting stimuli like noise and airblast probes, as well as a variety of visual and auditory stimuli of different valence and arousal levels, based on International Affective Picture System (IAPS) images and/or sounds from International Affective Digitized Sounds (IADS) database, have been designed and tested. Combined together into more complex startle type stimuli, such composite stimuli can potentiate the evoked response of underlying neural networks, and corresponding neurotransmitters and neuromodulators as well; this is referred to as increased power of response elicitation. The intensity and magnitude of multimodal responses to selected startle type stimuli have been analyzed using effect sizes and medians of dominant multimodal features, i.e. skin conductance, eye blink, head movement, speech fundamental frequency and energy. The significance of the observed effects and comparisons between paradigms were evaluated using one-tailed t-tests and ANOVA methods, respectively. Skin conductance response habituation was analyzed using ANOVA and post hoc multiple comparison tests with the Dunn-Šidák correction.

RESULTS

The results revealed specific physiological, facial and vocal reflexive and emotional responses on selected five stimuli paradigms which included: (1) acoustic startle probes, (2) airblasts, (3) IAPS images, (4) IADS sounds, and (5) image-sound-airblast composite stimuli. Overall, composite and airblast paradigms resulted in the largest responses across all analyzed features, followed by sound and acoustic startle paradigms, while paradigm using images consistently elicited the smallest responses. In this context, power of response elicitation of the selected stimuli paradigms can be described according to the aggregated magnitude of the participants' multimodal responses. We also observed a habituation effect only in skin conductance response to acoustic startle, airblast and sound paradigms.

CONCLUSIONS

This study developed a system for paradigm design and stimuli generation, as well as real-time multimodal signal processing and feature calculation. Experimental paradigms for monitoring individual responses to stressful startle type stimuli were designed in order to compare the response elicitation power across various stimuli. The developed system, applied paradigms and obtained results might be useful in further research for evaluation of individuals' multimodal responses when they are faced with a variety of aversive emotional distractors and stressful situations.

Emotional Distraction and Bodily Reaction: Modulation of Autonomous Responses by Anodal tDCS to the Prefrontal Cortex

Prefrontal electric stimulation has been demonstrated to effectively modulate cognitive processing. Specifically, the amelioration of cognitive control (CC) over emotional distraction by transcranial direct current stimulation (tDCS) points toward targeted therapeutic applications in various psychiatric disorders. In addition to behavioral measures, autonomous nervous system (ANS) responses are fundamental bodily signatures of emotional information processing. However, interactions between the modulation of CC by tDCS and ANS responses have received limited attention. We here report on ANS data gathered in healthy subjects that performed an emotional CC task parallel to the modulation of left prefrontal cortical activity by 1 mA anodal or sham tDCS. Skin conductance responses (SCRs) to negative and neutral pictures of human scenes were reduced by anodal as compared to sham tDCS. Individual SCR amplitude variations were associated with the amount of distraction. Moreover, the stimulation-driven performance- and SCR-modulations were related in form of a quadratic, inverse-U function. Thus, our results indicate that non-invasive brain stimulation (i.e., anodal tDCS) can modulate autonomous responses synchronous to behavioral improvements, but the range of possible concurrent improvements from prefrontal stimulation is limited. Interactions between cognitive, affective, neurophysiological, and vegetative responses to emotional content can shape brain stimulation effectiveness and require theory-driven integration in potential treatment protocols.

Emotional facial expressions evoke faster orienting responses, but weaker emotional responses at neural and behavioural levels compared to scenes: A simultaneous EEG and facial EMG study

In the current study, electroencephalography (EEG) was recorded simultaneously with facial electromyography (fEMG) to determine whether emotional faces and emotional scenes are processed differently at the neural level. In addition, it was investigated whether these differences can be observed at the behavioural level via spontaneous facial muscle activity. Emotional content of the stimuli did not affect early P1 activity. Emotional faces elicited enhanced amplitudes of the face-sensitive N170 component, while its counterpart, the scene-related N100, was not sensitive to emotional content of scenes. At 220-280ms, the early posterior negativity (EPN) was enhanced only slightly for fearful as compared to neutral or happy faces. However, its amplitudes were significantly enhanced during processing of scenes with positive content, particularly over the right hemisphere. Scenes of positive content also elicited enhanced spontaneous zygomatic activity from 500-750ms onwards, while happy faces elicited no such changes. Contrastingly, both fearful faces and negative scenes elicited enhanced spontaneous corrugator activity at 500-750ms after stimulus onset. However, relative to baseline EMG changes occurred earlier for faces (250ms) than for scenes (500ms) whereas for scenes activity changes were more pronounced over the whole viewing period. Taking into account all effects, the data suggests that emotional facial expressions evoke faster attentional orienting, but weaker affective neural activity and emotional behavioural responses compared to emotional scenes.

The color red attracts attention in an emotional context. An ERP study

The color red is known to influence psychological functioning, having both negative (e.g., blood, fire, danger), and positive (e.g., sex, food) connotations. The aim of our study was to assess the attentional capture by red-colored images, and to explore the modulatory role of the emotional valence in this process, as postulated by Elliot and Maier (2012) color-in-context theory. Participants completed a dot-probe task with each cue comprising two images of equal valence and arousal, one containing a prominent red object and the other an object of different coloration. Reaction times were measured, as well as the event-related lateralizations of the EEG. Modulation of the lateralized components revealed that the color red captured and later held the attention in both positive and negative conditions, but not in a neutral condition. An overt motor response to the target stimulus was affected mainly by attention lingering over the visual field where the red cue had been flashed. However, a weak influence of the valence could still be detected in reaction times. Therefore, red seems to guide attention, specifically in emotionally-valenced circumstances, indicating that an emotional context can alter color’s impact both on attention and motor behavior.

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice’s emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.