Face-to-face: Perceived personal relevance amplifies face processing

Electrophysiological, emotional and behavioural responses of female targets of sexual objectification

Sexual objectification and the interiorized objectifying gaze (self-objectification) are dangerous phenomena for women's psychological wellness. However, their specific effects on women's socio-affective reactions are still poorly understood, and their neural activity has never been explored before. In the present study, we investigated women's emotional and electrophysiological responses during simulated computer-based objectifying social interactions, and we examined consequent punishing behaviours towards the perpetrator using the ultimatum game. Behavioural results (N = 36) showed that during objectifying encounters women generally felt angrier/disgusted and tended to punish the perpetrator in later interactions. However, the more the women self-objectified, the more they felt ashamed (p = 0.011) and tended to punish the perpetrators less (p = 0.008). At a neural level (N = 32), objectifying interactions modulated female participants' neural signal elicited during the processing of the perpetrator, increasing early (N170) and later (EPN, LPP) ERP components. In addition, only the amplitude of the LPP positively correlated with shame (p = 0.006) and the level of self-objectification (p = 0.018). This finding provides first evidence for the specific time-course of sexual objectification, self-objectification and its associated shame response, and proves that emotional and social consequences of sexual objectification in women may depend on their tendency to self-objectify.

The impact of emotional facial expressions on reflexive attention depends on the aim of dynamic gaze changes: An ERP study

The emotional expression and gaze direction of a face are important cues for human social interactions. However, the interplay of these factors and their neural correlates are only partially understood. In the current study, we investigated ERP correlates of gaze and emotion processing following the initial presentation of faces with different emotional expressions (happy, neutral, angry) and an averted or direct gaze direction as well as following a subsequent change in gaze direction that occurred in half of the trials. We focused on the time course and scalp topography of the N170 and EPN components. The N170 amplitude was larger to averted than direct gaze for the initial face presentation and larger to gaze changes from direct to averted than from averted to direct in response to the gaze change. For the EPN component in response to the initial face presentation, we replicate classic effects of emotion, which did not interact with gaze direction. As a major new finding, changes from direct to averted gaze elicited an EPN-like effect when the face showed a happy expression. No such effect was seen for angry expressions. We conclude that happy faces reflexively attract attention when they look at the observer rather than away from the observer. These results for happy expressions are in line with the shared signal hypothesis that posits a better processing of expressions if their approach or avoidance tendency is consistent with gaze direction. However, the shared signal hypothesis is not supported by the present results for angry faces.

How facial masks alter the interaction of gaze direction, head orientation, and emotion recognition

The COVID-19 pandemic has altered the way we interact with each other: mandatory mask-wearing obscures facial information that is crucial for emotion recognition. Whereas the influence of wearing a mask on emotion recognition has been repeatedly investigated, little is known about the impact on interaction effects among emotional signals and other social signals. Therefore, the current study sought to explore how gaze direction, head orientation, and emotional expression interact with respect to emotion perception, and how these interactions are altered by wearing a face mask. In two online experiments, we presented face stimuli from the Radboud Faces Database displaying different facial expressions (anger, fear, happiness, neutral, and sadness), gaze directions (−13°, 0°, and 13°), and head orientations (−45°, 0°, and 45°) – either without (Experiment 1) or with mask (Experiment 2). Participants categorized the displayed emotional expressions. Not surprisingly, masks impaired emotion recognition. Surprisingly, without the mask, emotion recognition was unaffected by averted head orientations and only slightly affected by gaze direction. The mask strongly interfered with this ability. The mask increased the influence of head orientation and gaze direction, in particular for the emotions that were poorly recognized with mask. The results suggest that in case of uncertainty due to ambiguity or absence of signals, we seem to unconsciously factor in extraneous information.

What's in a face: Automatic facial coding of untrained study participants compared to standardized inventories

Automatic facial coding (AFC) is a novel research tool to automatically analyze emotional facial expressions. AFC can classify emotional expressions with high accuracy in standardized picture inventories of intensively posed and prototypical expressions. However, classification of facial expressions of untrained study participants is more error prone. This discrepancy requires a direct comparison between these two sources of facial expressions. To this end, 70 untrained participants were asked to express joy, anger, surprise, sadness, disgust, and fear in a typical laboratory setting. Recorded videos were scored with a well-established AFC software (FaceReader, Noldus Information Technology). These were compared with AFC measures of standardized pictures from 70 trained actors (i.e., standardized inventories). We report the probability estimates of specific emotion categories and, in addition, Action Unit (AU) profiles for each emotion. Based on this, we used a novel machine learning approach to determine the relevant AUs for each emotion, separately for both datasets. First, misclassification was more frequent for some emotions of untrained participants. Second, AU intensities were generally lower in pictures of untrained participants compared to standardized pictures for all emotions. Third, although profiles of relevant AU overlapped substantially across the two data sets, there were also substantial differences in their AU profiles. This research provides evidence that the application of AFC is not limited to standardized facial expression inventories but can also be used to code facial expressions of untrained participants in a typical laboratory setting.

The mere sight of loved ones does not inhibit psychophysiological defense mechanisms when threatened

Looking at pictures of loved ones, such as one's romantic partner or good friends, has been shown to alleviate the experience of pain and reduce defensive reactions. However, little is known about such modulatory effects on threat and safety learning and the psychophysiological processes involved. Here, we explored the hypothesis that beloved faces serve as implicit safety cues and attenuate the expression of fear responses and/or accelerate extinction learning in a threatening context. Thirty-two participants viewed pictures of their loved ones (romantic partner, parents, and best friend) as well as of unknown individuals within contextual background colors indicating threat-of-shock or safety. Focusing on the extinction of non-reinforced threat associations (no shocks were given), the experiment was repeated on two more test days while the defensive startle-EMG, SCR, and threat ratings were obtained. Results confirmed pronounced defensive responding to instructed threat-of-shock relative to safety context (e.g., threat-enhanced startle reflex and SCR). Moreover, threat-potentiated startle response slowly declined across test days indicating passive extinction learning in the absence of shocks. Importantly, neither a main effect of face category (loved vs. unknown) nor a significant interaction with threat/safety instructions was observed. Thus, a long-term learning history of beneficial relations (e.g., with supportive parents) did not interfere with verbal threat learning and aversive apprehensions. These findings reflect the effects of worries and apprehensions that persist despite the repeated experience of safety and the pictorial presence of loved ones. How to counter such aversive expectations is key to changing mal-adaptive behaviors (e.g., avoidance or stockpiling), biased risk perceptions, and stereotypes.

Multimodal Evidence of Atypical Processing of Eye Gaze and Facial Emotion in Children With Autistic Traits

According to the shared signal hypothesis (SSH) the impact of facial expressions on emotion processing partially depends on whether the gaze is directed toward or away from the observer. In autism spectrum disorder (ASD) several aspects of face processing have been found to be atypical, including attention to eye gaze and the identification of emotional expressions. However, there is little research on how gaze direction affects emotional expression processing in typically developing (TD) individuals and in those with ASD. This question is investigated here in two multimodal experiments. Experiment 1 required processing eye gaze direction while faces differed in emotional expression. Forty-seven children (aged 9-12 years) participated. Their Autism Diagnostic Observation Schedule (ADOS) scores ranged from 0 to 6 in the experiment. Event-related potentials (ERPs) were sensitive to gaze direction and emotion, but emotion processing did not depend on gaze direction. However, for angry faces the gaze direction effect on the N170 amplitude, as typically observed in TD individuals, diminished with increasing ADOS score. For neutral expressions this correlation was not significant. Experiment 2 required explicit emotion classifications in a facial emotion composite task while eye gaze was manipulated incidentally. A group of 22 children with ASD was compared to a propensity score-matched group of TD children (mean age = 13 years). The same comparison was carried out for a subgroup of nine children with ASD who were less trained in social cognition, according to clinician's report. The ASD group performed overall worse in emotion recognition than the TD group, independently of emotion or gaze direction. However, for disgust expressions, eye tracking data revealed that TD children fixated relatively longer on the eyes of the stimulus face with a direct gaze as compared with averted gaze. In children with ASD we observed no such modulation of fixation behavior as a function of gaze direction. Overall, the present findings from ERPs and eye tracking confirm the hypothesis of an impaired sensitivity to gaze direction in children with ASD or elevated autistic traits, at least for specific emotions. Therefore, we conclude that multimodal investigations of the interaction between emotional processing and stimulus gaze direction are promising to understand the characteristics of individuals differing along the autism trait dimension.

Your happy expressions encourage me to take risks: ERP evidence from an interpersonal gambling game

Although the influence of endogenous emotion on decision-making has been widely studied, the effect of interpersonal emotions on risk decision-making is less understood. To address this issue, participants were asked to perform an interpersonal gambling game after perceiving their cooperator's facial emotions. The results found that the cooperator's happy expressions increased individuals' risk-approaching choice compared with angry expressions. Moreover, happy expressions induced larger P300 potentials in the option assessment stage, and diminished the differences between losses and wins in feedback-related FRN/RewP in the outcome valuation stage. Additionally, single-trial analysis found that the neural response induced by interpersonal expressions and feedback could predict participants' subsequent decision-making. These findings suggest that interpersonal emotions shape individuals' risk preference through enhancing in-depth valuation in the option assessment stage and early motivational salience valuation in the outcome valuation stage.

Electrophysiological responses to negative evaluative person-knowledge: Effects of individual differences

Faces transmit rich information about a unique personal identity. Recent studies examined how negative evaluative information affects event-related potentials (ERPs), the relevance of individual differences, such as trait anxiety, neuroticism, or agreeableness, for these effects is unclear. In this preregistered study, participants (N = 80) were presented with neutral faces, either associated with highly negative or neutral biographical information. Faces were shown under three different task conditions that varied the attentional focus on face-unrelated features, perceptual face information, or emotional information. Results showed a task-independent increase of the N170 component for faces associated with negative information, while interactions occurred for the Early Posterior Negativity (EPN) and the Late Positive Potential (LPP), showing ERP differences only when paying attention to the evaluative information. Trait anxiety and neuroticism did not influence ERP differences. Low agreeableness increased EPN differences during perceptual distraction. Thus, we observed that low agreeableness leads to early increased processing of potentially hostile faces, although participants were required to attend to a face-unrelated feature.

Stress Makes the Difference: Social Stress and Social Anxiety in Decision-Making Under Uncertainty

Stress and anxiety can both influence risk-taking in decision-making. While stress typically increases risk-taking, anxiety often leads to risk-averse choices. Few studies have examined both stress and anxiety in a single paradigm to assess risk-averse choices. We therefore set out to examine emotional decision-making under stress in socially anxious participants. In our study, individuals (N = 87) high or low in social anxiety completed an expanded variation of the Balloon Analogue Risk Task (BART). While inflating a balloon to a larger degree is rewarded, a possible explosion leads to (a) a loss of money and (b) it is followed by an emotional picture (i.e., a calm vs. an angry face). To induce stress before this task, participants were told that they would have to deliver a speech. We operationalized risk-taking by the number of pumps during inflation and its functionality by the amount of monetary gain. In addition, response times were recorded as an index of decisional conflict. Without the stressor, high socially anxious compared to low socially anxious participants did not differ in any of the dependent variables. However, under stress, the low socially anxious group took more risk and earned more money, while high socially anxious individuals remained more cautious and did not change their risk-taking under social stress. Overall, high socially anxious individuals made their decisions more hesitantly compared to low socially anxious individuals. Unexpectedly, there were no main effects or interactions with the valence of the emotional faces. This data shows that stress affects socially anxious individuals differently: in low socially anxious individuals stress fosters risk-taking, whereas high socially anxious individuals did not alter their behavior and remained risk-averse. The novel eBART is a promising research tool to examine the specific factors that influence decision-making.

Here's looking at you, kid? Maternal depression and adolescent attention to self- or other-directed emotional faces

BACKGROUND

Maternal depression history represents a significant risk factor for developing psychopathology in children, altered emotional responding may represent a central risk pathway. However, additional research is needed on factors that affect the strength or direction of response alterations in relation to depression-risk in youth. In particular, facial orientation and gaze direction may alter personal relevance, with emotions directed towards an individual heightening motivational salience, compared to emotions directed away.

METHODS

Mother-daughter dyads (N = 56) were recruited based on presence or absence of maternal depression history and absence of youth depression. In line with theoretical perspectives suggesting diminished sensitivity to emotional context in relation to depression risk, we examined three Event-Related Potential (ERP) components in relation to forward versus averted emotional faces in a sample of girls with and without a maternal history of depression: the N200, N400, and Late Positive Potential (LPP).

RESULTS

Results showed a significant maternal depression history by face-orientation effect. Low-risk girls exhibited more negative N200 and N400 amplitudes for straight (M = -3.72, SE = 0.83; M = -3.57, SE = 0.86) versus averted (M = -2.15, SE = 0.76; M = -1.68, SE = 0.81) faces, while girls of mothers with histories of depression showed undifferentiated N200 or N400 responses in relation to face orientation. For LPP amplitudes, low-risk girls exhibited significantly more positive LPP amplitudes than high-risk girls, but only for averted faces (M = 0.69, SE = 0.59 and M = -2.63, SE = 0.74, respectively).

LIMITATIONS

Cross-sectional design and limited sample.

CONCLUSIONS

Results indicate that familial depression risk is associated with altered responsivity to face-orientation, these were interpreted as representing differential sensitivity to the personal-relevance of emotional stimuli.

Theory of mind and decision science: Towards a typology of tasks and computational models

The ability to form a Theory of Mind (ToM), i.e., to theorize about others' mental states to explain and predict behavior in relation to attributed intentional states, constitutes a hallmark of human cognition. These abilities are multi-faceted and include a variety of different cognitive sub-functions. Here, we focus on decision processes in social contexts and review a number of experimental and computational modeling approaches in this field. We provide an overview of experimental accounts and formal computational models with respect to two dimensions: interactivity and uncertainty. Thereby, we aim at capturing the nuances of ToM functions in the context of social decision processes. We suggest there to be an increase in ToM engagement and multiplexing as social cognitive decision-making tasks become more interactive and uncertain. We propose that representing others as intentional and goal directed agents who perform consequential actions is elicited only at the edges of these two dimensions. Further, we argue that computational models of valuation and beliefs follow these dimensions to best allow researchers to effectively model sophisticated ToM-processes. Finally, we relate this typology to neuroimaging findings in neurotypical (NT) humans, studies of persons with autism spectrum (AS), and studies of nonhuman primates.

Eye contact in active and passive viewing: Event-related brain potential evidence from a combined eye tracking and EEG study

Eye contact is a salient social cue, which is assumed to influence already early neural correlates of face perception. Specifically, the N170 component of the event-related potential (ERP) has often been found to be larger for faces with an averted gaze as compared to faces that directly look at the observer. In most existing ERP studies, effects of eye contact were investigated under comparatively artificial conditions where participants were instructed to maintain a steady fixation while they passively observed gaze changes in the stimulus face. It is therefore unclear to what extent neural correlates of eye contact generalize to more naturalistic situations that involve a continuous interplay between directed and averted gaze between the communication partners. To start bridging this gap, the present study compared the passive viewing of gaze changes to an active condition in which the participant's own gaze (measured online with an eye tracker) interacted with the gaze position of a continuously presented stimulus face. We also investigated whether eye contact effects were modulated by the face's emotional expression. In both the passive and the active viewing condition, N170 amplitudes were larger when the gaze of the stimulus faces was averted rather than directed towards the participant. Furthermore, eye contact decreased P300 amplitudes in both conditions. The emotional expression of the face also modulated the N170, but this effect did not interact with that of gaze direction. We conclude that the neural correlates of gaze perception during active gaze interactions are comparable to those found during passive viewing, encouraging the further study of eye contact effects in more naturalistic settings.

Contextual information resolves uncertainty about ambiguous facial emotions: Behavioral and magnetoencephalographic correlates

Environmental conditions bias our perception of other peoples' facial emotions. This becomes quite relevant in potentially threatening situations, when a fellow's facial expression might indicate potential danger. The present study tested the prediction that a threatening environment biases the recognition of facial emotions. To this end, low- and medium-expressive happy and fearful faces (morphed to 10%, 20%, 30%, or 40% emotional) were presented within a context of instructed threat-of-shock or safety. Self-reported data revealed that instructed threat led to a biased recognition of fearful, but not happy facial expressions. Magnetoencephalographic correlates revealed spatio-temporal clusters of neural network activity associated with emotion recognition and contextual threat/safety in early to mid-latency time intervals in the left parietal cortex, bilateral prefrontal cortex, and the left temporal pole regions. Early parietal activity revealed a double dissociation of face-context information as a function of the expressive level of facial emotions: When facial expressions were difficult to recognize (low-expressive), contextual threat enhanced fear processing and contextual safety enhanced processing of subtle happy faces. However, for rather easily recognizable faces (medium-expressive) the left hemisphere (parietal cortex, PFC, and temporal pole) showed enhanced activity to happy faces during contextual threat and fearful faces during safety. Thus, contextual settings reduce the salience threshold and boost early face processing of low-expressive congruent facial emotions, whereas face-context incongruity or mismatch effects drive neural activity of easier recognizable facial emotions. These results elucidate how environmental settings help recognize facial emotions, and the brain mechanisms underlying the recognition of subtle nuances of fear.

Turning Heads: The Effects of Face View and Eye Gaze Direction on the Perceived Attractiveness of Expressive Faces

We investigated whether the perceived attractiveness of expressive faces was influenced by head turn and eye gaze towards or away from the observer. In all experiments, happy faces were consistently rated as more attractive than angry faces. A head turn towards the observer, whereby a full-face view was shown, was associated with relatively higher attractiveness ratings when gaze direction was aligned with face view (Experiment 1). However, preference for full-face views of happy faces was not affected by gaze shifts towards or away from the observer (Experiment 2a). In Experiment 3, the relative duration of each face view (front-facing or averted at 15°) during a head turn away or towards the observer was manipulated. There was benefit on attractiveness ratings for happy faces shown for a longer duration from the front view, regardless of the direction of head turn. Our findings support previous studies indicating a preference for positive expressions on attractiveness judgements, which is further enhanced by the front views of faces, whether presented during a head turn or shown statically. In sum, our findings imply a complex interaction between cues of social attention, indicated by the view of the face shown, and reward on attractiveness judgements of unfamiliar faces.

Modulation of face- and emotion-selective ERPs by the three most common types of face image manipulations

In neuroscientific studies, the naturalness of face presentation differs; a third of published studies makes use of close-up full coloured faces, a third uses close-up grey-scaled faces and another third employs cutout grey-scaled faces. Whether and how these methodological choices affect emotion-sensitive components of the event-related brain potentials (ERPs) is yet unclear. Therefore, this pre-registered study examined ERP modulations to close-up full-coloured and grey-scaled faces as well as cutout fearful and neutral facial expressions, while attention was directed to no-face oddballs. Results revealed no interaction of face naturalness and emotion for any ERP component, but showed, however, large main effects for both factors. Specifically, fearful faces and decreasing face naturalness elicited substantially enlarged N170 and early posterior negativity amplitudes and lower face naturalness also resulted in a larger P1.This pattern reversed for the LPP, showing linear increases in LPP amplitudes with increasing naturalness. We observed no interaction of emotion with face naturalness, which suggests that face naturalness and emotion are decoded in parallel at these early stages. Researchers interested in strong modulations of early components should make use of cutout grey-scaled faces, while those interested in a pronounced late positivity should use close-up coloured faces.

Reversing Threat to Safety: Incongruence of Facial Emotions and Instructed Threat Modulates Conscious Perception but Not Physiological Responding

Facial expressions inform about other peoples' emotion and motivation and thus are central for social communication. However, the meaning of facial expressions may change depending on what we have learned about the related consequences. For instance, a smile might easily become threatening when displayed by a person who is known to be dangerous. The present study examined the malleability of emotional facial valence by means of social learning. To this end, facial expressions served as cues for verbally instructed threat-of-shock or safety (e.g., "happy faces cue shocks"). Moreover, reversal instructions tested the flexibility of threat/safety associations (e.g., "now happy faces cue safety"). Throughout the experiment, happy, neutral, and angry facial expressions were presented and auditory startle probes elicited defensive reflex activity. Results show that self-reported ratings and physiological reactions to threat/safety cues dissociate. Regarding threat and valence ratings, happy facial expressions tended to be more resistant becoming a threat cue, and angry faces remain threatening even when instructed as safety cue. For physiological response systems, however, we observed threat-potentiated startle reflex and enhanced skin conductance responses for threat compared to safety cues regardless of whether threat was cued by happy or angry faces. Thus, the incongruity of visual and verbal threat/safety information modulates conscious perception, but not the activation of physiological response systems. These results show that verbal instructions can readily overwrite the intrinsic meaning of facial emotions, with clear benefits for social communication as learning and anticipation of threat and safety readjusted to accurately track environmental changes.

Conceptual, Regulatory and Strategic Imperatives in the Early Days of EEG-Based Biomarker Validation for Neurodevelopmental Disabilities

Biological treatment development for syndromal neuropsychiatric conditions such as autism has seen slow progress for decades. Speeding drug discovery may result from the judicious development and application of biomarker measures of brain function to select patients for clinical trials, to confirm target engagement and to optimize drug dose. For neurodevelopmental disorders, electrophysiology (EEG) offers considerable promise because of its ability to monitor brain activity with high temporal resolution and its more ready application for pediatric populations relative to MRI. Here, we discuss conceptual/definitional issues related to biomarker development, discuss practical implementation issues, and suggest preliminary guidelines for validating EEG approaches as biomarkers with a context of use in neurodevelopmental disorder drug development.

The effect of emotional and self-referential contexts on ERP responses towards surprised faces

The perception of surprised faces is demonstrably modulated by emotional context. However, the influence of self-relevance and its interaction with emotional context have not been explored. The present study investigated the effects of contextual valence and self-reference on the perception of surprised faces. Our results revealed that faces in a negative context elicited a larger N170 than those in a neutral context. The EPN was affected by the interaction between contextual valence and self-reference, with larger amplitudes for faces in self-related positive contexts and sender-related negative contexts. Additionally, LPP amplitudes were enhanced for faces in negative contexts relative to neutral and positive contexts, as well as for self-related contexts in comparison to sender-related contexts. Together, these findings help to elucidate the psychophysiological mechanisms underlying the effects of emotional and self-referential contexts on the perception of surprised faces, which are characterized by distinctive ERPs.

Verbal instructions override the meaning of facial expressions

Psychological research has long acknowledged that facial expressions can implicitly trigger affective psychophysiological responses. However, whether verbal information can alter the meaning of facial emotions and corresponding response patterns has not been tested. This study examined emotional facial expressions as cues for instructed threat-of-shock or safety, with a focus on defensive responding. In addition, reversal instructions were introduced to test the impact of explicit safety instructions on fear extinction. Forty participants were instructed that they would receive unpleasant electric shocks, for instance, when viewing happy but not angry faces. In a second block, instructions were reversed (e.g., now angry faces cued shock). Happy, neutral, and angry faces were repeatedly presented, and auditory startle probes were delivered in half of the trials. The defensive startle reflex was potentiated for threat compared to safety cues. Importantly, this effect occurred regardless of whether threat was cued by happy or angry expressions. Although the typical pattern of response habituation was observed, defense activation to newly instructed threat cues remained significantly enhanced in the second part of the experiment, and it was more pronounced in more socially anxious participants. Thus, anxious individuals did not exhibit more pronounced defense activation compared to less anxious participants, but their defense activation was more persistent.