Faces in context: a review and systematization of contextual influences on affective face processing

The influence of contextual uncertainty on facial expression processing: Evidence from behavior and ERPs

The brain helps individuals build expectations based on emotional prediction, facilitating the processing of faces in social interactions. Due to the intricacy of the environment, accurately predicting emotions remains a formidable task. Contextual uncertainty represents a state characterized by the inability to predict when, how, and why events occur. It leads to intensified sentiments and triggers adverse emotions like anxiety. Therefore, comprehending the influences of contextual uncertainty carries importance. The present study utilized event-related potentials (ERPs) technology to investigate contextual uncertainty's influence on facial expression processing. We employed a novel S1-S2 paradigm, using scene images as S1 and faces as S2. From the learning phase into the testing phase, the certain to uncertain group (CER to UNC) experienced more unpredictability (increased uncertainty), whereas the uncertain to certain group (UNC to CER) experienced more predictability (decreased uncertainty). This allowed for manipulating dynamic alterations in predictive relationships. The behavioral results showed that the valence ratings of neutral facial expressions were more negative in the CER to UNC group with increased contextual uncertainty. The ERP results showed that the more negative SPN (stimulus preceding negativity) amplitudes and positive LPP (late positive potential) amplitudes were observed in the UNC to CER group with decreased contextual uncertainty, compared to the CER to UNC group with increased contextual uncertainty. These findings have indicated that contextual uncertainty affects facial expression processing. In summary, these results contributed to comprehending the contextual uncertainty.

Emotion-gaze interaction affects time-to-collision estimates, but not preferred interpersonal distance towards looming faces

Estimating the time until impending collision (time-to-collision, TTC) of approaching or looming individuals and maintaining a comfortable distance from others (interpersonal distance, IPD) are commonly required in daily life and contribute to survival and social goals. Despite accumulating evidence that facial expressions and gaze direction interactively influence face processing, it remains unclear how these facial features affect the spatiotemporal processing of looming faces. We examined whether facial expressions (fearful vs. neutral) and gaze direction (direct vs. averted) interact on the judgments of TTC and IPD for looming faces, based on the shared signal hypothesis that fear signals the existence of threats in the environment when coupled with averted gaze. Experiment 1 demonstrated that TTC estimates were reduced for fearful faces compared to neutral ones only when the concomitant gaze was averted. In Experiment 2, the emotion-gaze interaction was not observed in the IPD regulation, which is arguably sensitive to affective responses to faces. The results suggest that fearful-averted faces modulate the cognitive extrapolation process of looming motion by communicating environmental threats rather than by altering subjective fear or perceived emotional intensity of faces. The TTC-specific effect may reflect an enhanced defensive response to unseen threats implied by looming fearful-averted faces. Our findings provide insight into how the visual system processes facial features to ensure bodily safety and comfortable interpersonal communication in dynamic environments.

Emotional text messages affect the early processing of emoticons depending on their emotional congruence: evidence from the N170 and EPN event related potentials

Emoticons have been considered pragmatic cues that enhance emotional expressivity during computer-mediated communication. Yet, it is unclear how emoticons are processed in ambiguous text-based communication due to incongruences between the emoticon's emotional valence and its context. In this study, we investigated the electrophysiological correlates of contextual influence on the early emotional processing of emoticons, during an emotional congruence judgment task. Participants were instructed to judge the congruence between a text message expressing an emotional situation (positive or negative), and a subsequent emoticon expressing positive or negative emotions. We analyzed early event-related potentials elicited by emoticons related to face processing (N170) and emotional salience in visual perception processing (Early Posterior Negativity, EPN). Our results show that accuracy and Reaction Times depend on the interaction between the emotional valence of the context and the emoticon. Negative emoticons elicited a larger N170, suggesting that the emotional information of the emoticon is integrated at the early stages of the perceptual process. During emoticon processing, a valence effect was observed with enhanced EPN amplitudes in occipital areas for emoticons representing negative valences. Moreover, we observed a congruence effect in parieto-temporal sites within the same time-window, with larger amplitudes for the congruent condition. We conclude that, similar to face processing, emoticons are processed differently according to their emotional content and the context in which they are embedded. A congruent context might enhance the emotional salience of the emoticon (and therefore, its emotional expression) during the early stages of their processing.

Reexamining the Kuleshov effect: Behavioral and neural evidence from authentic film experiments

Film cognition explores the influence of cinematic elements, such as editing and film color, on viewers' perception. The Kuleshov effect, a famous example of how editing influences viewers' emotional perception, was initially proposed to support montage theory through the Kuleshov experiment. This effect, which has since been recognized as a manifestation of point-of-view (POV) editing practices, posits that the emotional interpretation of neutral facial expressions is influenced by the accompanying emotional scene in a face-scene-face sequence. However, concerns persist regarding the validity of previous studies, often employing inauthentic film materials like static images, leaving the question of its existence in authentic films unanswered. This study addresses these concerns by utilizing authentic films in two experiments. In Experiment 1, multiple film clips were captured under the guidance of a professional film director and seamlessly integrated into authentic film sequences. 59 participants viewed these face-scene-face film sequences and were tasked with rating the valence and emotional intensity of neutral faces. The findings revealed that the accompanying fearful or happy scenes significantly influence the interpretation of emotion on neutral faces, eliciting perceptions of negative or positive emotions from the neutral face. These results affirm the existence of the Kuleshov effect within authentic films. In Experiment 2, 31 participants rated the valence and arousal of neutral faces while undergoing functional magnetic resonance imaging (fMRI). The behavioral results confirm the Kuleshov effect in the MRI scanner, while the neural data identify neural correlates that support its existence at the neural level. These correlates include the cuneus, precuneus, hippocampus, parahippocampal gyrus, post cingulate gyrus, orbitofrontal cortex, fusiform gyrus, and insula. These findings also underscore the contextual framing inherent in the Kuleshov effect. Overall, the study integrates film theory and cognitive neuroscience experiments, providing robust evidence supporting the existence of the Kuleshov effect through both subjective ratings and objective neuroimaging measurements. This research also contributes to a deeper understanding of the impact of film editing on viewers' emotional perception from the contemporary POV editing practices and neurocinematic perspective, advancing the knowledge of film cognition.

Religious labels and food preferences, but not country of origin, support opposing face aftereffects

Face templates can be experimentally manipulated, and category-contingent aftereffects suggest discrete templates across social groups. We tested whether 1) explicit religious labels, 2) food preferences, and 3) country of origin would support religion-contingent aftereffects across Christians and Muslims face sets. While viewing face images, ninety-three participants heard audio that stated either 1) a character's religious identity, 2) preferred food, or 3) country of origin. Participants viewed contracted Christian faces and expanded Muslim faces during the training phase. To measure adaptation, before and after the training phases, participants selected the face out of a pair of expanded and contracted Christian or Muslim faces that they found more attractive. Contingent aftereffects were found in the religious explicit (t(30) = 2.49, p = 0.02, Cohen's d = 0.58) and food conditions (t(30) = -3.77, p < 0.01, Cohen's d = -0.82), but not the country condition (t(30) = 1.64, p = 0.11, Cohen's d = 0.31). This suggests that religious labels and food preferences create socially meaningful groups, but country of origin does not. This is evidence of an impact of social categorization on visual processing.

Ensemble Coding of Crowd with Cross-Category Facial Expressions

Ensemble coding allows observers to form an average to represent a set of elements. However, it is unclear whether observers can extract an average from a cross-category set. Previous investigations on this issue using low-level stimuli yielded contradictory results. The current study addressed this issue by presenting high-level stimuli (i.e., a crowd of facial expressions) simultaneously (Experiment 1) or sequentially (Experiment 2), and asked participants to complete a member judgment task. The results showed that participants could extract average information from a group of cross-category facial expressions with a short perceptual distance. These findings demonstrate cross-category ensemble coding of high-level stimuli, contributing to the understanding of ensemble coding and providing inspiration for future research.

Morality or competence is more important? The effect of evaluation dimensions on ERP responses to neutral faces depends on contextual valence and self-relevance

Recent studies have shown that the processing of neutral facial expressions could be modulated by the valence and self-relevance of preceding verbal evaluations. However, these studies have not distinguished the dimension (i.e., morality and competence) from verbal evaluations. In fact, there is a hot controversy about whether morality or competence receives more weight. Therefore, using the ERP technique, the current study aimed to address this issue by comparing the influence of morality and competence evaluations on behavioral and neural responses to neutral facial expressions when these evaluations varied with contextual valence and self-relevance. Our ERP results revealed that the early EPN amplitudes were larger for neutral faces after receiving evaluations about self relative to evaluations about senders. Moreover, the EPN was more negative after a competence evaluation relative to a morality evaluation when these evaluations were positive, while this effect was absent when these evaluations were negative. The late LPP was larger after a morality evaluation compared to a competence evaluation when these evaluations were negative and directed to self. However, no significant LPP effect between morality and competence evaluations was observed when these evaluations were positive. The present study extended previous studies by showing that early and late processing stages of faces are affected by the evaluation dimension in a top-down manner and further modulated by contextual valence and self-relevance.

Emotional anticipation for dynamic emotional faces is not modulated by schizotypal traits: A Representational Momentum study

Schizotypy, a personality structure that resembles schizophrenia symptoms, is often associated with abnormal facial emotion perception. Based on the prevailing sense of threat in psychotic experiences, and the immediate perceptual history of seeing others' facial expressions, individuals with high schizotypal traits may exhibit a heightened tendency to anticipate anger. To test this, we used insights from Representational Momentum (RM), a perceptual phenomenon in which the endpoint of a dynamic event is systematically displaced forward, into the immediate future. Angry-to-ambiguous and happy-to-ambiguous avatar faces were presented, each followed by a probe with the same (ambiguous) expression as the endpoint, or one slightly changed to express greater happiness/anger. Participants judged if the probe was "equal" to the endpoint and rated how confident they were. The sample was divided into high (N = 46) and low (N = 49) schizotypal traits using the Schizotypal Personality Questionnaire (SPQ). First, a forward bias was found in happy-to-ambiguous faces, suggesting emotional anticipation solely for dynamic faces changing towards a potential threat (anger). This may reflect an adaptative mechanism, as it is safer to anticipate any hostility from a conspecific than the opposite. Second, contrary to our hypothesis, high schizotypal traits did not heighten RM for happy-to-ambiguous faces, nor did they lead to overconfidence in biased judgements. This may suggest a typical pattern of emotional anticipation in non-clinical schizotypy, but caution is needed due to the use of self-report questionnaires, university students, and a modest sample size. Future studies should also investigate if the same holds for clinical manifestations of schizophrenia.

Perception of emotionally incongruent cues: evidence for overreliance on body vs. face expressions in Parkinson's disease

Individuals with Parkinson's disease (PD) may exhibit impaired emotion perception. However, research demonstrating this decline has been based almost entirely on the recognition of isolated emotional cues. In real life, emotional cues such as expressive faces are typically encountered alongside expressive bodies. The current study investigated emotion perception in individuals with PD (n = 37) using emotionally incongruent composite displays of facial and body expressions, as well as isolated face and body expressions, and congruent composite displays as a baseline. In addition to a group of healthy controls (HC) (n = 50), we also included control individuals with schizophrenia (SZ) (n = 30), who display, as in PD, similar motor symptomology and decreased emotion perception abilities. The results show that individuals with PD showed an increased tendency to categorize incongruent face-body combinations in line with the body emotion, whereas those with HC showed a tendency to classify them in line with the facial emotion. No consistent pattern for prioritizing the face or body was found in individuals with SZ. These results were not explained by the emotional recognition of the isolated cues, cognitive status, depression, or motor symptoms of individuals with PD and SZ. As real-life expressions may include inconsistent cues in the body and face, these findings may have implications for the way individuals with PD and SZ interpret the emotions of others.

Visual Field Restriction in the Recognition of Basic Facial Expressions: A Combined Eye Tracking and Gaze Contingency Study

Uncertainties and discrepant results in identifying crucial areas for emotional facial expression recognition may stem from the eye tracking data analysis methods used. Many studies employ parameters of analysis that predominantly prioritize the examination of the foveal vision angle, ignoring the potential influences of simultaneous parafoveal and peripheral information. To explore the possible underlying causes of these discrepancies, we investigated the role of the visual field aperture in emotional facial expression recognition with 163 volunteers randomly assigned to three groups: no visual restriction (NVR), parafoveal and foveal vision (PFFV), and foveal vision (FV). Employing eye tracking and gaze contingency, we collected visual inspection and judgment data over 30 frontal face images, equally distributed among five emotions. Raw eye tracking data underwent Eye Movements Metrics and Visualizations (EyeMMV) processing. Accordingly, the visual inspection time, number of fixations, and fixation duration increased with the visual field restriction. Nevertheless, the accuracy showed significant differences among the NVR/FV and PFFV/FV groups, despite there being no difference in NVR/PFFV. The findings underscore the impact of specific visual field areas on facial expression recognition, highlighting the importance of parafoveal vision. The results suggest that eye tracking data analysis methods should incorporate projection angles extending to at least the parafoveal level.

Additive effects of emotional expression and stimulus size on the perception of genuine and artificial facial expressions: an ERP study

Seeing an angry individual in close physical proximity can not only result in a larger retinal representation of that individual and an enhanced resolution of emotional cues, but may also increase motivation for rapid visual processing and action preparation. The present study investigated the effects of stimulus size and emotional expression on the perception of happy, angry, non-expressive, and scrambled faces. We analyzed event-related potentials (ERPs) and behavioral responses of N = 40 participants who performed a naturalness classification task on real and artificially created facial expressions. While the emotion-related effects on accuracy for recognizing authentic expressions were modulated by stimulus size, ERPs showed only additive effects of stimulus size and emotional expression, with no significant interaction with size. This contrasts with previous research on emotional scenes and words. Effects of size were present in all included ERPs, whereas emotional expressions affected the N170, EPN, and LPC, irrespective of size. These results imply that the decoding of emotional valence in faces can occur even for small stimuli. Supra-additive effects in faces may necessitate larger size ranges or dynamic stimuli that increase arousal.

Event-related alpha power in early stage of facial expression processing in social anxiety: Influence of language context

Accurate interpretation of the emotional information conveyed by others' facial expressions is crucial for social interactions. Event-related alpha power, measured by time-frequency analysis, is a frequently used EEG index of emotional information processing. However, it is still unclear how event-related alpha power varies in emotional information processing in social anxiety groups. In the present study, we recorded event-related potentials (ERPs) while participants from the social anxiety and healthy control groups viewed facial expressions (angry, happy, neutral) preceded by contextual sentences conveying either a positive or negative evaluation of the subject. The impact of context on facial expression processing in both groups of participants was explored by assessing behavioral ratings and event-related alpha power (0-200 ms after expression presentation). In comparison to the healthy control group, the social anxiety group exhibited significantly lower occipital alpha power in response to angry facial expressions in negative contexts and neutral facial expressions in positive contexts. The influence of language context on facial expression processing in individuals with social anxiety may occur at an early stage of processing.

Distrust before first sight? Examining knowledge- and appearance-based effects of trustworthiness on the visual consciousness of faces

The present EEG study with 32 healthy participants investigated whether affective knowledge about a person influences the visual awareness of their face, additionally considering the impact of facial appearance. Faces differing in perceived trustworthiness based on appearance were associated with negative or neutral social information and shown as target stimuli in an attentional blink task. As expected, participants showed enhanced awareness of faces associated with negative compared to neutral social information. On the neurophysiological level, this effect was connected to differences in the time range of the early posterior negativity (EPN)-a component associated with enhanced attention and facilitated processing of emotional stimuli. The findings indicate that the social-affective relevance of a face based on emotional knowledge is accessed during a phase of attentional enhancement for conscious perception and can affect prioritization for awareness. In contrast, no clear evidence for influences of facial trustworthiness during the attentional blink was found.

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.

Emotional interference and attentional control in schizophrenia-spectrum disorders: The special case of neutral faces

BACKGROUND AND OBJECTIVES

Schizophrenia-spectrum disorders (SSD) are characterized by impaired emotion processing and attention. SSD patients are more sensitive to the presence of emotional distractors. But despite growing interest on the emotion-attention interplay, emotional interference in SSD is far from fully understood. Moreover, research to date has not established the link between emotional interference and attentional control in SSD. This study thus aimed to investigate the effects of facial expression and attentional control in SSD, by manipulating perceptual load.

METHODS

Twenty-two SSD patients and 22 healthy controls performed a target-letter discrimination task with task-irrelevant angry, happy, and neutral faces. Target-letter was presented among homogenous (low load) or heterogenous (high load) distractor-letters. Accuracy and RT were analysed using (generalized) linear mixed-effect models.

RESULTS

Accuracy was significantly lower in SSD patients than controls, regardless of perceptual load and facial expression. Concerning RT, SSD patients were significantly slower than controls in the presence of neutral faces, but only at high load. No group differences were observed for angry and happy faces.

LIMITATIONS

Heterogeneity of SSD, small sample size, lack of clinical control group, medication.

CONCLUSIONS

One possible explanation is that neutral faces captured exogenous attention to a greater extent in SSD, thus challenging attentional control in perceptually demanding conditions. This may reflect abnormal processing of neutral faces in SSD. If replicated, these findings will help to understand the interplay between exogenous attention, attentional control, and emotion processing in SSD, which may unravel the mechanism underlying socioemotional dysfunction in SSD.

The reciprocal relationship between smiles and situational contexts

ABSTRACTSmiles provide information about a social partner's affect and intentions during social interaction. Although always encountered within a specific situation, the influence of contextual information on smile evaluation has not been widely investigated. Moreover, little is known about the reciprocal effect of smiles on evaluations of their accompanying situations. In this research, we assessed how different smile types and situational contexts affected participants' social evaluations. In Study 1, 85 participants rated reward, affiliation, and dominance smiles embedded within either enjoyable, polite, or negative (unpleasant) situations. Context had a strong effect on smile ratings, such that smiles in enjoyable situations were rated as more genuine and joyful, as well as indicating less superiority than those in negative situations. In Study 2, 200 participants evaluated the situations that these smiles were perceived within (rather than the smiles themselves). Although situations paired with reward (vs. affiliation) smiles tended to be rated more positively, this effect was absent for negative situations. Ultimately, the findings point toward a reciprocal relationship between smiles and contexts, whereby the face influences evaluations of the situation and vice versa.

Gender congruence and emotion effects in cross-modal associative learning: Insights from ERPs and pupillary responses

Social and emotional cues from faces and voices are highly relevant and have been reliably demonstrated to attract attention involuntarily. However, there are mixed findings as to which degree associating emotional valence to faces occurs automatically. In the present study, we tested whether inherently neutral faces gain additional relevance by being conditioned with either positive, negative, or neutral vocal affect bursts. During learning, participants performed a gender-matching task on face-voice pairs without explicit emotion judgments of the voices. In the test session on a subsequent day, only the previously associated faces were presented and had to be categorized regarding gender. We analyzed event-related potentials (ERPs), pupil diameter, and response times (RTs) of N = 32 subjects. Emotion effects were found in auditory ERPs and RTs during the learning session, suggesting that task-irrelevant emotion was automatically processed. However, ERPs time-locked to the conditioned faces were mainly modulated by the task-relevant information, that is, the gender congruence of the face and voice, but not by emotion. Importantly, these ERP and RT effects of learned congruence were not limited to learning but extended to the test session, that is, after removing the auditory stimuli. These findings indicate successful associative learning in our paradigm, but it did not extend to the task-irrelevant dimension of emotional relevance. Therefore, cross-modal associations of emotional relevance may not be completely automatic, even though the emotion was processed in the voice.

Incidental learning of faces during threat: No evidence for enhanced physiological responses to former threat identities

Remembering an unfamiliar person and the contextual conditions of that encounter is important for adaptive future behavior, especially in a potentially dangerous situation. Initiating defensive behavior in the presence of former dangerous circumstances can be crucial. Recent studies showed selective electrocortical processing of faces that were previously seen in a threat context compared to a safety context, however, this was not reflected in conscious recognition performance. Here, we investigated whether previously seen threat-faces, that could not be remembered, were capable to activate defensive psychophysiological response systems. During an encoding phase, 50 participants with low to moderate levels of anxiety viewed 40 face pictures with neutral expressions (6 s each), without an explicit learning instruction (incidental learning task). Each half of the faces were presented with contextual background colors that signaled either threat-of-shock or safety. In the recognition phase, all old and additional new faces (total of 60) were presented intermixed without context information. Participants had to decide whether a face was new or had been presented previously in a threatening or a safe context. Results show moderate face recognition independent of context conditions. Startle reflex and skin conductance responses (SCR) were more pronounced for threat compared to safety during encoding. For SCR, this differentiation was enhanced with higher levels of depression and anxiety. There were no differential startle reflex or SCR effects during recognition. From a clinical perspective, these findings do not support the notion that perceptual biases and physiological arousal directly relate to threat-associated identity recognition deficits in healthy and clinical participants with anxiety and trauma-related disorders.

Are vipers prototypic fear-evoking snakes? A cross-cultural comparison of Somalis and Czechs

Snakes are known as highly fear-evoking animals, eliciting preferential attention and fast detection in humans. We examined the human fear response to snakes in the context of both current and evolutionary experiences, conducting our research in the cradle of humankind, the Horn of Africa. This region is characterized by the frequent occurrence of various snake species, including deadly venomous viperids (adders) and elapids (cobras and mambas). We conducted experiments in Somaliland and compared the results with data from Czech respondents to address the still unresolved questions: To which extent is human fear of snakes affected by evolutionary or current experience and local culture? Can people of both nationalities recognize venomous snakes as a category, or are they only afraid of certain species that are most dangerous in a given area? Are respondents of both nationalities equally afraid of deadly snakes from both families (Viperidae, Elapidae)? We employed a well-established picture-sorting approach, consisting of 48 snake species belonging to four distinct groups. Our results revealed significant agreement among Somali as well as Czech respondents. We found a highly significant effect of the stimulus on perceived fear in both populations. Vipers appeared to be the most salient stimuli in both populations, as they occupied the highest positions according to the reported level of subjectively perceived fear. The position of vipers strongly contrasts with the fear ranking of deadly venomous elapids, which were in lower positions. Fear scores of vipers were significantly higher in both populations, and their best predictor was the body width of the snake. The evolutionary, cultural, and cognitive aspects of this phenomenon are discussed.

Beyond facial expressions: A systematic review on effects of emotional relevance of faces on the N170

The N170 is the most prominent electrophysiological signature of face processing. While facial expressions reliably modulate the N170, there is considerable variance in N170 modulations by other sources of emotional relevance. Therefore, we systematically review and discuss this research area using different methods to manipulate the emotional relevance of inherently neutral faces. These methods were categorized into (1) existing pre-experimental affective person knowledge (e.g., negative attitudes towards outgroup faces), (2) experimentally instructed affective person knowledge (e.g., negative person information), (3) contingency-based affective learning (e.g., fear-conditioning), or (4) the immediate affective context (e.g., emotional information directly preceding the face presentation). For all categories except the immediate affective context category, the majority of studies reported significantly increased N170 amplitudes depending on the emotional relevance of faces. Furthermore, the potentiated N170 was observed across different attention conditions, supporting the role of the emotional relevance of faces on the early prioritized processing of configural facial information, regardless of low-level differences. However, we identified several open research questions and suggest venues for further research.

Motivated attention and task relevance in the processing of cross-modally associated faces: Behavioral and electrophysiological evidence

It has repeatedly been shown that visually presented stimuli can gain additional relevance by their association with affective stimuli. Studies have shown effects of associated affect in event-related potentials (ERP) like the early posterior negativity (EPN), late positive complex (LPC), and even earlier components as the P1 or N170. However, findings are mixed as to the extent associated affect requires directed attention to the emotional quality of a stimulus and which ERP components are sensitive to task instructions during retrieval. In this preregistered study ( https://osf.io/ts4pb ), we tested cross-modal associations of vocal affect-bursts (positive, negative, neutral) to faces displaying neutral expressions in a flash-card-like learning task, in which participants studied face-voice pairs and learned to correctly assign them to each other. In the subsequent EEG test session, we applied both an implicit ("old-new") and explicit ("valence-classification") task to investigate whether the behavior at retrieval and neurophysiological activation of the affect-based associations were dependent on the type of motivated attention. We collected behavioral and neurophysiological data from 40 participants who reached the preregistered learning criterium. Results showed EPN effects of associated negative valence after learning and independent of the task. In contrast, modulations of later stages (LPC) by positive and negative associated valence were restricted to the explicit, i.e., valence-classification, task. These findings highlight the importance of the task at different processing stages and show that cross-modal affect can successfully be associated to faces.

Deepfake smiles matter less-the psychological and neural impact of presumed AI-generated faces

High-quality AI-generated portraits ("deepfakes") are becoming increasingly prevalent. Understanding the responses they evoke in perceivers is crucial in assessing their societal implications. Here we investigate the impact of the belief that depicted persons are real or deepfakes on psychological and neural measures of human face perception. Using EEG, we tracked participants' (N = 30) brain responses to real faces showing positive, neutral, and negative expressions, after being informed that they are either real or fake. Smiling faces marked as fake appeared less positive, as reflected in expression ratings, and induced slower evaluations. Whereas presumed real smiles elicited canonical emotion effects with differences relative to neutral faces in the P1 and N170 components (markers of early visual perception) and in the EPN component (indicative of reflexive emotional processing), presumed deepfake smiles showed none of these effects. Additionally, only smiles presumed as fake showed enhanced LPP activity compared to neutral faces, suggesting more effortful evaluation. Negative expressions induced typical emotion effects, whether considered real or fake. Our findings demonstrate a dampening effect on perceptual, emotional, and evaluative processing of presumed deepfake smiles, but not angry expressions, adding new specificity to the debate on the societal impact of AI-generated content.

These Aren't The Droids You Are Looking for: Promises and Challenges for the Intersection of Affective Science and Robotics/AI

AI research focused on interactions with humans, particularly in the form of robots or virtual agents, has expanded in the last two decades to include concepts related to affective processes. Affective computing is an emerging field that deals with issues such as how the diagnosis of affective states of users can be used to improve such interactions, also with a view to demonstrate affective behavior towards the user. This type of research often is based on two beliefs: (1) artificial emotional intelligence will improve human computer interaction (or more specifically human robot interaction), and (2) we understand the role of affective behavior in human interaction sufficiently to tell artificial systems what to do. However, within affective science the focus of research is often to test a particular assumption, such as "smiles affect liking." Such focus does not provide the information necessary to synthesize affective behavior in long dynamic and real-time interactions. In consequence, theories do not play a large role in the development of artificial affective systems by engineers, but self-learning systems develop their behavior out of large corpora of recorded interactions. The status quo is characterized by measurement issues, theoretical lacunae regarding prevalence and functions of affective behavior in interaction, and underpowered studies that cannot provide the solid empirical foundation for further theoretical developments. This contribution will highlight some of these challenges and point towards next steps to create a rapprochement between engineers and affective scientists with a view to improving theory and solid applications.

Facial affect recognition in context in adults with and without TBI

Introduction

Several studies have reported impaired emotion recognition in adults with traumatic brain injury (TBI), but studies have two major design features that limit application of results to real-world contexts: (1) participants choose from among lists of basic emotions, rather than generating emotion labels, and (2) images are typically presented in isolation rather than in context. To address these limitations, we created an open-labeling task with faces shown alone or in real-life scenes, to more closely approximate how adults with TBI label facial emotions beyond the lab.

Methods

Participants were 55 adults (29 female) with moderate to severe TBI and 55 uninjured comparison peers, individually matched for race, sex, and age. Participants viewed 60 photographs of faces, either alone or in the pictured person's real-life context, and were asked what that person was feeling. We calculated the percent of responses that were standard forced-choice-task options, and also used sentiment intensity analysis to compare verbal responses between the two groups. We tracked eye movements for a subset of participants, to explore whether gaze duration or number of fixations helped explain any group differences in labels.

Results

Over 50% of responses in both groups were words other than basic emotions on standard affect tasks, highlighting the importance of eliciting open-ended responses. Valence of labels by participants with TBI was attenuated relative to valence of Comparison group labels, i.e., TBI group responses were less positive to positive images and the same was true for negative images, although the TBI group responses had higher lexical diversity. There were no significant differences in gaze duration or number of fixations between groups.

Discussion

Results revealed qualitative differences in affect labels between adults with and without TBI that would not have emerged on standard forced-choice tasks. Verbal differences did not appear to be attributable to differences in gaze patterns, leaving open the question of mechanisms of atypical affect processing in adults with TBI.

The roles of inflammation, affect, and interoception in predicting social perception

"Sickness behavior" is an orchestrated suite of symptoms that commonly occur in the context of inflammation, and is characterized by changes in affect, social experience, and behavior. However, recent evidence suggests that inflammation may not always produce the same set of sickness behavior (e.g., fatigue, anhedonia, and social withdrawal). Rather, inflammation may be linked with different behavior across contexts and/or across individuals, though research in this area is under-developed to-date. In the present study (n = 30), we evaluated the influence of affective context and individual differences in difficulty detecting bodily sensations (i.e., interoceptive difficulty) on social perception following an inflammatory challenge. Inflammation was induced using the influenza vaccine and inflammatory reactivity was operationalized as changes in circulating levels of interleukin-6 (IL-6) before the vaccine and approximately 24 h later. Twenty-four hours after administration of the influenza vaccine, we manipulated affective context using a well-validated affect misattribution task in which participants made trustworthiness judgments of individuals with neutral facial expressions following the rapid presentation of "prime" images that were positive or negative in affective content. Interoceptive difficulty was measured at baseline using a validated self-report measure. Results revealed significant interactions between inflammatory reactivity to the influenza vaccine and affective context on social perception. Specifically, individuals with greater inflammatory reactivity were more biased by affective context when judging the trustworthiness of neutral faces. In addition, interoceptive difficulty and affective context interacted to predict social perception such that individuals with greater interoceptive difficulty were more biased by affective context in these judgments. In sum, we provide some of the first evidence that inflammation may amplify the saliency of affective cues during social decision making. Our findings also replicate prior work linking interoceptive ability to the use of affect-as-information during social perception, but in the novel context of inflammation.

Analysis of attentional biases in anxiety using 24 facial priming sequences

The processing of emotional facial expressions helps people to adjust to the physical and social environment. Furthermore, mental disorders such as anxiety have been linked to attentional biases in the processing of this type of information. Nevertheless, there are still contradictory results that might be due to the methodology used and to individual differences in the manifestation of anxiety. Our research goal was to use 24 facial priming sequences to analyse attentional biases in the detection of facial expressions of fear, considering the levels and the ways in which individuals express anxiety. With higher levels of cognitive anxiety and general trait anxiety, those sequences that began in the upper half (vs. lower half) elicited a speedier response in the detection of fear. The results are discussed within the context of other techniques and disorders that prompt a deficit in the processing of facial information.

Semantic satiation of emotional words impedes facial expression processing in two stages

To explore the mechanism of emotional words semantic satiation effect on facial expression processing, participants were asked to judge the facial expression (happiness or sadness) after an emotional word ((cry) or (smile)) or a neutral word ((Ah), baseline condition) was presented for 20 s. The results revealed that participants were slower in judging valence-congruent facial expressions and reported a more enlarged (Experiment 1) and prolonged (Experiment 2) N170 component than the baseline condition. No significant difference in behavior and N170 appeared between the valence-incongruent and the baseline condition. However, the amplitude of LPC (Late Positive Complex) under both valence-congruent/incongruent conditions was smaller than the baseline condition. It indicates that, in the early stage, the impeding effect of satiated emotional words is specifically constrained to facial expressions with the same emotional valence; in the late stage, such an impeding effect might spread to facial expressions with the opposite valence of the satiated emotional word.

Maintaining her image: A social comparative evaluation of the particularity of mothers in the Chinese cultural context

In Chinese culture, the mother holds a special meaning in one's self-concept, and is perceived as being stablyincorporated into and consistent with the self. However, it is unclear whether the evaluation of mothers by individuals is affected following the initiation of upward and downward social comparisons (USC and DSC). This experiment manipulated USC and DSC by evaluating positive and negative public figures and used functional near-infrared spectroscopy to record changes in brain activity during the evaluation. It was found that participants' evaluations of their mothers and their brain activity did not differ from the self during USC, verifying the equivalence of the mother and the self. In DSC, participants made significantly more positive social judgments about their mothers, accompanied by greater activation of the left temporal lobe. These results suggest that the mother was not only stably incorporated into the self but was in a position of even greater importance than the self. In DSC in particular, individuals are more likely to maintain a positive image of their mother.

Age similarities in the anchoring effect in emotion intensity judgment

BACKGROUND

The anchoring effect refers to the tendency that an individual's numerical judgment would assimilate to an anchor (a numerical value) that appears before that judgment. This study investigated whether the anchoring effect exists in the emotion judgment of younger and older adults and observed the age-related characteristics. This could not only broaden the explanation of the anchoring effect but also link this classic judgment bias with daily emotion judgment to refresh our understanding of older adults' ability in emotional perspective taking.

METHOD

Participants (older adults: n = 64, age range: 60-74, 27 males; younger adults: n = 68, age range: 18-34, 34 males) read a brief emotional story and compared the protagonist's emotion intensity to a given numerical anchor (lower or higher than the anchor) and then estimated the protagonist's possible emotion intensity in that story. The task was divided into two cases according to anchor relevance (anchors are relevant or irrelevant relative to the judgment target).

RESULTS

The results showed that the estimates were higher under high-anchor than low-anchor conditions, suggesting the robust anchoring effect. Further, the anchoring effect was greater for anchor-relevant than anchor-irrelevant tasks and for negative rather than positive emotions. No age differences were found.

DISCUSSION AND CONCLUSIONS

The results indicated that the anchoring effect is robust and stable for younger and older adults, even though the anchor information seemed irrelevant. Finally, perceiving others' negative emotions is a crucial but rather difficult aspect of empathy, which could be a challenge and requires more caution for accurate interpretation.

Electrophysiological, emotional and behavioural responses of female targets of sexual objectification

Sexual objectification and the interiorized objectifying gaze (self-objectification) are dangerous phenomena for women's psychological wellness. However, their specific effects on women's socio-affective reactions are still poorly understood, and their neural activity has never been explored before. In the present study, we investigated women's emotional and electrophysiological responses during simulated computer-based objectifying social interactions, and we examined consequent punishing behaviours towards the perpetrator using the ultimatum game. Behavioural results (N = 36) showed that during objectifying encounters women generally felt angrier/disgusted and tended to punish the perpetrator in later interactions. However, the more the women self-objectified, the more they felt ashamed (p = 0.011) and tended to punish the perpetrators less (p = 0.008). At a neural level (N = 32), objectifying interactions modulated female participants' neural signal elicited during the processing of the perpetrator, increasing early (N170) and later (EPN, LPP) ERP components. In addition, only the amplitude of the LPP positively correlated with shame (p = 0.006) and the level of self-objectification (p = 0.018). This finding provides first evidence for the specific time-course of sexual objectification, self-objectification and its associated shame response, and proves that emotional and social consequences of sexual objectification in women may depend on their tendency to self-objectify.

Face processing in young adults with autism and ADHD: An event related potentials study

Background

Atypicalities in perception and interpretation of faces and emotional facial expressions have been reported in both autism and attention-deficit/hyperactivity disorder (ADHD) during childhood and adulthood. Investigation of face processing during young adulthood (18 to 25 years), a transition period to full-fledged adulthood, could provide important information on the adult outcomes of autism and ADHD.

Methods

In this study, we investigated event-related potentials (ERPs) related to visual face processing in autism, ADHD, and co-occurring autism and ADHD in a large sample of young adults (N = 566). The groups were based on the Diagnostic Interview for ADHD in Adults 2.0 (DIVA-2) and the Autism Diagnostic Observation Schedule-2 (ADOS-2). We analyzed ERPs from two passive viewing tasks previously used in childhood investigations: (1) upright and inverted faces with direct or averted gaze; (2) faces expressing different emotions.

Results

Across both tasks, we consistently found lower amplitude and longer latency of N170 in participants with autism compared to those without. Longer P1 latencies and smaller P3 amplitudes in response to emotional expressions and longer P3 latencies for upright faces were also characteristic to the autistic group. Those with ADHD had longer N170 latencies, specific to the face-gaze task. Individuals with both autism and ADHD showed additional alterations in gaze modulation and a lack of the face inversion effect indexed by a delayed N170.

Conclusion

Alterations in N170 for autistic young adults is largely consistent with studies on autistic adults, and some studies in autistic children. These findings suggest that there are identifiable and measurable socio-functional atypicalities in young adults with autism.

Neural response and representation: Facial expressions in scenes

Previous studies have shown that the brain generates expectations based on scenes, which affects facial expression recognition. However, although facial expressions are known to interact with perception, the mechanism underlying this interaction remains poorly understood. Here, we used frequency labeling and decoding techniques to reveal the effects of scene-based expectation on the amplitude and representational strength of neural activity. We also reduced the relative reliability between expectation and sensory input by blurring facial expressions to further investigate the effects of this relative reliability on the pattern of neural activation and representation. Participants viewed emotional changes in unblurred or blurred facial expressions, which flickered at a rate of 6 Hz within a scene. We found that facial expressions that were congruent with the emotional significance of the scene elicited a larger steady-state visual evoked potential amplitude than did facial expressions that were incongruent with the emotional significance of a scene, in both unblurred and blurred conditions. We also found that expected facial expression representations were stronger than unexpected representations during the unblurred condition. In the blurred condition, unexpected representations were stronger than expected representations. Taken together, these results suggested that facial expression processing in the visual cortex is modulated by top-down signals. The relative reliability of expectation and sensory input moderated the influence of a scene on facial expression representation. Furthermore, our study showed that neural activation amplitudes did not correspond to representational strength.

Testing, explaining, and exploring models of facial expressions of emotions

Models are the hallmark of mature scientific inquiry. In psychology, this maturity has been reached in a pervasive question-what models best represent facial expressions of emotion? Several hypotheses propose different combinations of facial movements [action units (AUs)] as best representing the six basic emotions and four conversational signals across cultures. We developed a new framework to formalize such hypotheses as predictive models, compare their ability to predict human emotion categorizations in Western and East Asian cultures, explain the causal role of individual AUs, and explore updated, culture-accented models that improve performance by reducing a prevalent Western bias. Our predictive models also provide a noise ceiling to inform the explanatory power and limitations of different factors (e.g., AUs and individual differences). Thus, our framework provides a new approach to test models of social signals, explain their predictive power, and explore their optimization, with direct implications for theory development.

Degree of abstraction rather than ambiguity is crucial for driving mentalizing involvement commentary on "A-EM: a neurocognitive model for understanding mixed and ambiguous emotions and morality"

Willems (this issue) proposes a neurocognitive model with a central role allotted to ambiguity in perceived morality and emotion in driving involvement of reflective/mentalizing processes. We argue that abstractness of representation has more explanatory power in this respect. We illustrate this with examples from the verbal and non-verbal domain showing a) concrete-ambiguous emotions processed through reflexive systems and b) abstract-unambiguous emotions processed through the mentalizing system, counter to MA-EM model predictions. However, due to the natural correlation between ambiguity and abstractness, both accounts will typically make convergent predictions.

The neural correlates of context driven changes in the emotional response: An fMRI study

Emotional flexibility reflects the ability to adjust the emotional response to the changing environmental context. To understand how context can trigger a change in emotional response, i.e., how it can upregulate the initial emotional response or trigger a shift in the valence of emotional response, we used a task consisting of picture pairs during functional magnetic resonance imaging sessions. In each pair, the first picture was a smaller detail (a decontextualized photograph depicting emotions using primarily facial and postural expressions) from the second (contextualized) picture, and the neural response to a decontextualized picture was compared with the same picture in a context. Thirty-one healthy participants (18 females; mean age: 24.44 ± 3.4) were involved in the study. In general, context (vs. pictures without context) increased activation in areas involved in facial emotional processing (e.g., middle temporal gyrus, fusiform gyrus, and temporal pole) and affective mentalizing (e.g., precuneus, temporoparietal junction). After excluding the general effect of context by using an exclusive mask with activation to context vs. no-context, the automatic shift from positive to negative valence induced by the context was associated with increased activation in the thalamus, caudate, medial frontal gyrus and lateral orbitofrontal cortex. When the meaning changed from negative to positive, it resulted in a less widespread activation pattern, mainly in the precuneus, middle temporal gyrus, and occipital lobe. Providing context cues to facial information recruited brain areas that induced changes in the emotional responses and interpretation of the emotional situations automatically to support emotional flexibility.

The role of face masks within in-patient psychotherapy: Results of a survey among inpatients and healthcare professionals

Introduction

Face-to-face medical and psychotherapeutic treatments during the Corona pandemic often involve patients and health care providers wearing face masks. We performed a pilot survey assessing the subjective experience of wearing face masks during psychotherapy sessions regarding (i) feasibility, (ii) psychotherapeutic treatment and (iii) communication, emotion and working alliance in patients and healthcare professionals.

Methods

A total of n = 62 inpatients (RR = 95.4%) and n = 33 healthcare professionals (RR = 86.8%) at an academic department of Psychosomatic Medicine and Psychotherapy participated in this survey anonymously. The items of the questionnaire were created by the interprofessional expert team and were based on existing instruments: (i) the Therapeutic Relationship Questionnaire and (ii) the German translation of Yalom's Questionnaire on Experiencing in Group Psychotherapy.

Results

The majority of patients rate their psychotherapy as highly profitable despite the mask. In individual therapy, face masks seem to have a rather low impact on subjective experience of psychotherapy and the relationship to the psychotherapist. Most patients reported using alternative facial expressions and expressions. In the interactional group therapy, masks were rather hindering. On the healthcare professional side, there were more frequent negative associations of face masks in relation to (i) experiencing connectedness with colleagues, (ii) forming relationships, and (iii) therapeutic treatment.

Discussion

Information should be given to patients about the possible effects of face masks on the recognition of emotions, possible misinterpretations and compensation possibilities through alternative stimuli (e.g., eye area) and they should be encouraged to ask for further information. Especially in group therapy, with patients from other cultural backgrounds and in cases of need for help (e.g., hearing impairment) or complex disorders, appropriate non-verbal gestures and body language should be used to match the intended emotional expression.

The Eyes Have It: Psychotherapy in the Era of Masks

Nonverbal communication is integral to the success of psychotherapy and facial expression is an important component of nonverbal communication. The SARS CoV-2 pandemic has caused alterations in how psychotherapy services are provided. In this paper, potential issues that may arise from conducting psychotherapy when both the patient and therapist are wearing masks are explored. These include higher likelihood of misidentifying facial expression, especially when expression is incongruent with body language, and when the lower face is more important for correct identification of emotion. These issues may be particularly problematic for patient populations for whom emotion recognition may be a problem at baseline, or for those more prone to biases in emotional recognition. Suggestions are made for therapists to consider when seeing patients in-person when masks are necessary.

Therapist reactions to patient personality: A pilot study of clinicians’ emotional and neural responses using three clinical vignettes from in treatment series

Introduction

Therapists’ responses to patients play a crucial role in psychotherapy and are considered a key component of the patient–clinician relationship, which promotes successful treatment outcomes. To date, no empirical research has ever investigated therapist response patterns to patients with different personality disorders from a neuroscience perspective.

Methods

In the present study, psychodynamic therapists (N = 14) were asked to complete a battery of instruments (including the Therapist Response Questionnaire) after watching three videos showing clinical interactions between a therapist and three patients with narcissistic, histrionic/borderline, and depressive personality disorders, respectively. Subsequently, participants’ high-density electroencephalography (hdEEG) was recorded as they passively viewed pictures of the patients’ faces, which were selected from the still images of the previously shown videos. Supervised machine learning (ML) was used to evaluate whether: (1) therapists’ responses predicted which patient they observed during the EEG task and whether specific clinician reactions were involved in distinguishing between patients with different personality disorders (using pairwise comparisons); and (2) therapists’ event-related potentials (ERPs) predicted which patient they observed during the laboratory experiment and whether distinct ERP components allowed this forecast.

Results

The results indicated that therapists showed distinct patterns of criticized/devalued and sexualized reactions to visual depictions of patients with different personality disorders, at statistically systematic and clinically meaningful levels. Moreover, therapists’ late positive potentials (LPPs) in the hippocampus were able to determine which patient they observed during the EEG task, with high accuracy.

Discussion

These results, albeit preliminary, shed light on the role played by therapists’ memory processes in psychotherapy. Clinical and neuroscience implications of the empirical investigation of therapist responses are discussed.

Threat induction biases processing of emotional expressions

Threats can derive from our physical or social surroundings and bias the way we perceive and interpret a given situation. They can be signaled by peers through facial expressions, as expressed anger or fear can represent the source of perceived threat. The current study seeks to investigate enhanced attentional state and defensive reflexes associated with contextual threat induced through aversive sounds presented in an emotion recognition paradigm. In a sample of 120 healthy participants, response and gaze behavior revealed differences in perceiving emotional facial expressions between threat and safety conditions: Responses were slower under threat and less accurate. Happy and neutral facial expressions were classified correctly more often in a safety context and misclassified more often as fearful under threat. This unidirectional misclassification suggests that threat applies a negative filter to the perception of neutral and positive information. Eye movements were initiated later under threat, but fixation changes were more frequent and dwell times shorter compared to a safety context. These findings demonstrate that such experimental paradigms are capable of providing insight into how context alters emotion processing at cognitive, physiological, and behavioral levels. Such alterations may derive from evolutionary adaptations necessary for biasing cognitive processing to survive disadvantageous situations. This perspective sets up new testable hypotheses regarding how such levels of explanation may be dysfunctional in patient populations.

Diminished ability to integrate target stimuli with context during emotional recognition in individuals with broad autism phenotype

Individuals with broad autism phenotype (BAP) have a tendency not to integrate emotional stimuli with the surrounding context. They have also shown different patterns and abilities in processing positive and negative emotions. This study aimed to examine whether the effect of context on target stimuli could vary depending on the type of target emotion in individuals with BAP. Based on the Broad Autism Phenotype Questionnaire (BAPQ) and Autism-Spectrum Quotient (AQ), 36 individuals with BAP and 33 healthy controls were selected. All the participants performed an overlap-emotional task consisting of six conditions: 2 (congruence: congruent and incongruent) × 3 (emotion: fearful, sad, and happy). Reaction time and accuracy were measured as dependent variables. The results revealed that the individuals with BAP showed no difference in reaction time between the condition of congruence and incongruence, but that the control group was faster to categorize facial expression on the condition of congruence than that of incongruence regardless of the type of target emotion. There were no differences between the two groups in any of the conditions with regard to accuracy. These findings indicate that individuals with BAP tend not to integrate target emotions with contextual information, a feature that could worsen the speed of emotional recognition in individuals with BAP. This study confirmed that the individuals with BAP have different cognition patterns in emotional recognition than the control group.

Emotional context can reduce the negative impact of face masks on inferring emotions

While face masks prevent the spread of disease, they occlude lower face parts and thus impair facial emotion recognition. Since emotions are often also contextually situated, it remains unknown whether providing a descriptive emotional context alongside the facial emotion may reduce some of the negative impact of facial occlusion on emotional communication. To address this question, here we examined how emotional inferences were affected by facial occlusion and the availability of emotional context. Participants were presented with happy or sad emotional faces who were either fully visible or partially obstructed by an opaque surgical mask. The faces were shown either within an emotionally congruent (e.g., "Her cat was found/lost yesterday afternoon") or neutral ("Get ready to see the next person") context. Participants were asked to infer the emotional states of the protagonists by rating their emotional intensity and valence. Facial occlusion by masks impacted the ratings, such that protagonists were judged to feel less intense and more neutral emotions when they wore masks relative to when their face was fully visible. Importantly, this negative impact of visual occlusion by mask was reduced but not fully eliminated when the faces were presented within a congruent emotional context. Thus, visual occlusion of facial emotions impairs understanding of emotions, with this negative effect of face masks partially mitigated by the availability of a larger emotional context.

Moral content influences facial emotion processing development during early-to-middle childhood

Facial emotions are often processed in light of moral information, which can assist in predicting and interpreting the intentions of another. Neurophysiological measures of facial emotion processing (FEP) may be sensitive to moral content. Relatively little is known, however, about the relationship between moral content and FEP during early-to-middle childhood, and how this relationship may change across development. Eighty-four children aged 4-12 years completed a task assessing whether child faces primed within the moral harm/care domain influenced face sensitive event-related potentials (ERPs; N170 and LPP). Results demonstrated that N170 amplitude decreased with age for faces primed with positive moral content, whilst LPP amplitude decreased with age for faces primed with negative moral content. Collectively, this suggests that morally relevant content within the harm/care domain is integrated during the early stages of FEP in early-to-middle childhood. Moreover, stronger language ability was positively correlated with the LPP for fearful faces primed with negative moral content. Overall, findings provide novel evidence to suggest that FEP development may be modulated by moral content, and emotion-specific results may be influenced by language. Findings from this research highlight the complex relationship between broader social cognitive skills during child development.

Neural computations underlying contextual processing in humans

Context shapes our perception of facial expressions during everyday social interactions. We interpret a person’s face in a hostile situation negatively and judge the same face under pleasant circumstances positively. Critical to our adaptive fitness, context provides situation-specific framing to resolve ambiguity and guide our interpersonal behavior. This context-specific modulation of facial expression is thought to engage the amygdala, hippocampus, and orbitofrontal cortex; however, the underlying neural computations remain unknown. Here we use human intracranial electroencephalograms (EEGs) directly recorded from these regions and report bidirectional theta-gamma interactions within the amygdala-hippocampal network, facilitating contextual processing. Contextual information is subsequently represented in the orbitofrontal cortex, where a theta phase shift binds context and face associations within theta cycles, endowing faces with contextual meanings at behavioral timescales. Our results identify theta phase shifts as mediating associations between context and face processing, supporting flexible social behavior.

Context influences our perception of facial expressions. Zheng et al. show that contextual modulation of faces relies on medial temporal lobe-orbitofrontal cortex communications in humans. High gamma bursts occur in rhythm with theta oscillations, with cross-regional theta-gamma phase shifts binding context-face associations.

Influence of affective verbal context on emotional facial expression perception of social anxiety

Previous studies have shown that the perception of ambiguous facial expressions for individuals with social anxiety was influenced by the affective verbal context. However, it is still unknown how emotional facial expressions are perceived by individuals with social anxiety in the context of the verbal context. In this study, we used event-related potentials (ERPs) technology to examine how individuals with social anxiety perceive emotional facial expressions in positive and negative contexts. The results showed that: (1) Within the negative verbal contexts, the amplitude of P1 induced by facial expressions in the social anxiety group was significantly higher than that induced by the healthy control group; The N170 amplitude induced by facial expressions in social anxiety group was less negative than that in the healthy control group, and was not affected by the context. (2) The social anxiety group had significantly higher LPP in negative contexts elicited by angry expressions than by happy expressions. This study proved that the perception of emotional facial expressions was influenced by top-down information in the early and late stages of visual perception for individuals with social anxiety.

Contextual cues about reciprocity impact ratings of smile sincerity

Research has shown that context influences how sincere a smile appears to observers. That said, most studies on this topic have focused exclusively on situational cues (e.g. smiling while at a party versus smiling during a job interview) and few have examined other elements of context. One important element concerns any knowledge an observer might have about the smiler as an individual (e.g. their habitual behaviours, traits or attitudes). In this manuscript, we present three experiments that explored the influence of such knowledge on ratings of smile sincerity. In Experiments 1 and 2, participants rated the sincerity of Duchenne and non-Duchenne smiles after having been exposed to cues about the smiler's tendency to reciprocate (this person always, never or occasionally returns favours). In Experiment 3 they performed the same task but with cues about the smiler's love of learning (this person always, never or occasionally enjoys learning new tasks). The results show that cues about the smiler's reciprocity tendency influenced participants' ratings of smile sincerity and did so in a stronger manner than cues about the smiler's love of learning. Overall, these results both strengthen and broaden the literature on the role of context on judgements of smile sincerity.

Displays of negative facial affect during parent-adolescent conflict and the bidirectional transmission of social anxiety

BACKGROUND

Symptoms of social anxiety rise rapidly during adolescence, particularly for girls. Pervasive displays of parental negative affect may increase adolescents' fear of negative evaluation (FNE), thereby increasing risk for social anxiety symptoms. Adolescent displays of negative affect may also exacerbate parents' social anxiety symptoms (via FNE of their child or their parenting skills), yet little research has tested transactional pathways of transmission in families. By early adolescence, rates of parent-child conflict rise, and offspring become increasingly independent in their own displays of negative affect, increasing opportunities for hypothesized transactional pathways between parent-adolescent displays of negative affect and social anxiety symptoms.

METHODS

This study included 129 parents and daughters (11-13; no baseline social anxiety disorder), two-thirds of whom were at high risk for social anxiety due to a shy/fearful temperament. We used actor-partner interdependence models (APIM) to test whether displays of negative facial affect, assessed individually for each parent and daughter during a conflict discussion, would predict their partner's social anxiety symptoms two years later. Automated facial affect coding assessed the frequency of negative affect during the discussion. Clinician ratings of social anxiety symptoms were completed at baseline and two-year follow-up.

RESULTS

Both parents and daughters who displayed more frequent negative facial affect at baseline had partners with higher follow-up social anxiety symptoms, an effect that was maintained after accounting for actors' and partners' baseline symptoms.

CONCLUSIONS

Findings are consistent with intergenerational models positing that parental negative affective behaviors increase risk for adolescent social anxiety symptoms but also suggest that adolescent negative facial affect may exacerbate parental social anxiety symptoms. These bidirectional effects improve understanding of how social anxiety is maintained within a transactional family structure and highlight that displays of negative affect during parent-adolescent interaction may warrant future examination as a potential treatment target for adolescent social anxiety.

Emotional context effect on recognition of varying facial emotion expression intensities in depression

BACKGROUND

Previous research has indicated that Major Depressive Disorder (MDD) patients have deficits in the process of facial emotion recognition. In most of these studies, isolated emotional faces were used, and the effect of the surrounding context of the face was neglected. We aimed to investigate how context emotion (sad or happy) affects facial emotion recognition and whether this effect is different in depressive individuals compared to the control group.

METHODS

Happy, sad, neutral facial expressions with congruent and incongruent visual contexts were presented to 51 MDD patients and 42 matched healthy controls. Emotional facial expressions are presented as morphs gradually expressing happiness or sadness with 40% and 80% intensity levels. Mean reaction time, mean accuracy rate, and mean emotion intensity rating score was calculated for each condition.

RESULTS

The performances on facial emotion intensity rating and accuracy rate were similar between MDD patients and controls. MDD patients were slower to recognize all facial emotions and to recognize facial emotions with emotionally incongruent backgrounds compared to congruent ones.

LIMITATIONS

Antidepressant therapy of patients might have affected our results.

CONCLUSIONS

Emotional contextual features have an important role in facial emotion recognition but this effect is independent of depression. Longer reaction time in depression may be related to some cognitive impairments.