Fashioning the Face: Sensorimotor Simulation Contributes to Facial Expression Recognition

Anatomo-functional basis of emotional and motor resonance elicited by facial expressions

Simulation theories predict that the observation of other's expressions modulates neural activity in the same centres controlling their production. This hypothesis has been developed by two models, postulating that the visual input is directly projected either to the motor system for action recognition (motor resonance) or to emotional/interoceptive regions for emotional contagion and social synchronization (emotional resonance). Here we investigated the role of frontal/insular regions in the processing of observed emotional expressions by combining intracranial recording, electrical stimulation and effective connectivity. First, we intracranially recorded from prefrontal, premotor or anterior insular regions of 44 patients during the passive observation of emotional expressions, finding widespread modulations in prefrontal/insular regions (anterior cingulate cortex, anterior insula, orbitofrontal cortex and inferior frontal gyrus) and motor territories (Rolandic operculum and inferior frontal junction). Subsequently, we electrically stimulated the activated sites, finding that (i) in the anterior cingulate cortex and anterior insula, the stimulation elicited emotional/interoceptive responses, as predicted by the 'emotional resonance model'; (ii) in the Rolandic operculum it evoked face/mouth sensorimotor responses, in line with the 'motor resonance' model; and (iii) all other regions were unresponsive or revealed functions unrelated to the processing of facial expressions. Finally, we traced the effective connectivity to sketch a network-level description of these regions, finding that the anterior cingulate cortex and the anterior insula are reciprocally interconnected while the Rolandic operculum is part of the parieto-frontal circuits and poorly connected with the former. These results support the hypothesis that the pathways hypothesized by the 'emotional resonance' and the 'motor resonance' models work in parallel, differing in terms of spatio-temporal fingerprints, reactivity to electrical stimulation and connectivity patterns.

Confrontation with others' emotions changes bodily resonance differently in those with low and high levels of depersonalization

We present novel research on the cortical dynamics of atypical perceptual and emotional processing in people with symptoms of depersonalization-derealization disorder (DP-DR). We used electroencephalography (EEG)/event-related potentials (ERPs) to delineate the early perceptual mechanisms underlying emotional face recognition and mirror touch in adults with low and high levels of DP-DR symptoms (low-DP and high-DP groups). Face-sensitive visual N170 showed markedly less differentiation for emotional versus neutral face-voice stimuli in the high- than in the low-DP group. This effect was related to self-reported bodily symptoms like disembodiment. Emotional face-voice primes altered mirror touch at somatosensory cortical components P45 and P100 differently in the two groups. In the high-DP group, mirror touch occurred only when seeing touch after being confronted with angry face-voice primes. Mirror touch in the low-DP group, however, was unaffected by preceding emotions. Modulation of mirror touch following angry others was related to symptoms of self-other confusion. Results suggest that others' negative emotions affect somatosensory processes in those with an altered sense of bodily self. Our findings are in line with the idea that disconnecting from one's body and self (core symptom of DP-DR) may be a defence mechanism to protect from the threat of negative feelings, which may be exacerbated through self-other confusion. This article is part of the theme issue 'Sensing and feeling: an integrative approach to sensory processing and emotional experience'.

The other person's smiling amount affects one's smiling response during face-to-face conversations

Introduction

Smiling during conversation occurs interactively between people and is known to build good interpersonal relationships. However, whether and how much the amount that an individual smiles is influenced by the other person's smile has remained unclear. This study aimed to quantify the amount of two individuals' smiles during conversations and investigate the dependency of one's smile amount (i.e., intensity and frequency) on that of the other.

Method

Forty participants (20 females) engaged in three-minute face-to-face conversations as speakers with a listener (male or female), under three conditions, where the amount of smiling response by listeners was controlled as "less," "moderate," and "greater." The amount of the smiles was quantified based on their facial movements through automated facial expression analysis.

Results

The results showed that the amount of smiling by the speaker changed significantly depending on the listener's smile amount; when the listeners smiled to a greater extent, the speakers tended to smile more, especially when they were of the same gender (i.e., male-male and female-female pairs). Further analysis revealed that the smiling intensities of the two individuals changed in a temporally synchronized manner.

Discussion

These results provide quantitative evidence for the dependence of one's smile on the other's smile, and the differential effect between gender pairs.

Expression perceptive fields explain individual differences in the recognition of facial emotions

Humans can use the facial expressions of another to infer their emotional state, although it remains unknown how this process occurs. Here we suppose the presence of perceptive fields within expression space, analogous to feature-tuned receptive-fields of early visual cortex. We developed genetic algorithms to explore a multidimensional space of possible expressions and identify those that individuals associated with different emotions. We next defined perceptive fields as probabilistic maps within expression space, and found that they could predict the emotions that individuals infer from expressions presented in a separate task. We found profound individual variability in their size, location, and specificity, and that individuals with more similar perceptive fields had similar interpretations of the emotion communicated by an expression, providing possible channels for social communication. Modelling perceptive fields therefore provides a predictive framework in which to understand how individuals infer emotions from facial expressions.

The roles of symmetry and elongation in developing reference frames

Previous studies showed that elongation and symmetry (two ubiquitous aspects of natural stimuli) are important attributes in object perception and recognition, which in turn suggests that these geometrical factors may contribute to the selection of perceptual reference-frames. However, whether and how these attributes guide the selection of reference-frames is still poorly understood. The goal of this study was to examine systematically the roles of elongation and symmetry, as well as their combination, in the selection of reference axis and how these axes are developed for unfamiliar objects. We designed our experiments to eliminate two potential confounding factors: (i) extraneous environmental cues, such as edges of the screen, etc. (by using VR) and (ii) pre-learned cues for familiar objects and shapes (by using reinforcement learning of novel shapes). We used algorithmically generated textures with different orientations having specified levels of symmetry and elongation as the stimuli. In each trial, we presented only one stimulus and asked observers to report if the stimulus was in its original form or a flipped (mirror-image) one. Feedback was provided at the end of each trial. Based on previous studies on mental rotation, we hypothesized that the selection of a reference-frame defined by symmetry and/or elongation would be revealed by a linear relationship between reaction-times and the angular-deviation from either the most symmetrical or the most elongated orientation. Our results are consistent with this hypothesis. We found that subjects performed mental rotation to transform images to their reference axes and used the most symmetrical or elongated orientation as the reference axis when only one factor was presented, and they used a "winner-take-all" strategy when both factors were presented, with elongation being more dominant than symmetry. We discuss theoretical implications of these findings, in particular in the context of "canonical sensorimotor theory."

Facial mimicry interference reduces working memory accuracy for facial emotion expressions

Facial mimicry, the tendency to imitate facial expressions of other individuals, has been shown to play a critical role in the processing of emotion expressions. At the same time, there is evidence suggesting that its role might change when the cognitive demands of the situation increase. In such situations, understanding another person is dependent on working memory. However, whether facial mimicry influences working memory representations for facial emotion expressions is not fully understood. In the present study, we experimentally interfered with facial mimicry by using established behavioral procedures, and investigated how this interference influenced working memory recall for facial emotion expressions. Healthy, young adults (N = 36) performed an emotion expression n-back paradigm with two levels of working memory load, low (1-back) and high (2-back), and three levels of mimicry interference: high, low, and no interference. Results showed that, after controlling for block order and individual differences in the perceived valence and arousal of the stimuli, the high level of mimicry interference impaired accuracy when working memory load was low (1-back) but, unexpectedly, not when load was high (2-back). Working memory load had a detrimental effect on performance in all three mimicry conditions. We conclude that facial mimicry might support working memory for emotion expressions when task load is low, but that the supporting effect possibly is reduced when the task becomes more cognitively challenging.

Early parent-child intervention with Dialogic Book-Sharing: effects on child communicative and socio-emotional development and on parenting. Study protocol for a multicentre randomised controlled trial in Italy

BACKGROUND

Research in the neurosciences has highlighted the importance of intersubjective relationships in promoting neuromental development of the child. Children's learning in early childhood occurs mainly in a dyadic context of an interaction with their parents: from this perspective, good dialogic parent-child communication is required to be promoted also through good educational practices. Dialogic Book-Sharing (DBS), a dialogic form of parent-child communication through the use of wordless picture books, provides a privileged 'intersubjective' space and is highly effective in promoting communication, language, attention, behavioural development and the parent-child relationship. DBS programme, successfully previously trialled in South Africa and the UK, will be applied for the first time in Italy for research purposes in Italian health, educational and maternal-child centres.

METHODS

A multicentre randomised controlled trial is being conducted to evaluate DBS parenting intervention for children aged between 14 and 20 months. Parent-child dyads are randomly allocated to a book-sharing intervention group or to a wait-list control group. In the intervention, parents are trained in supportive book-sharing with their children by local staff of the centres. DBS intervention is carried out in small groups over a period of 4 weeks. Data are collected at baseline, post-intervention and at 6 months post-intervention with a questionnaire and video recording of parent-child interaction.

DISCUSSION

DBS programme in early childhood could enhance the educational resources offered by Italian health, educational and maternal-child centres, in support of child's development and parenting. DBS represents a strategic opportunity for bringing about positive effects, also in terms of prevention of socio-emotional and cognitive difficulties. As such it represents a promising response to the new social, health and educational needs of the post-COVID-19 pandemic era caused by the social isolation measures. Furthermore, the application of the DBS methodology is a way to promote the use of books, and thereby counteract the excessive use of technological devices already present in early childhood.

TRIAL REGISTRATION

The trial is registered on the International Standard Randomised Controlled Trial Number database, registration number ISRCTN11755019 Registered on 2 November 2023. This is version 1 of the protocol for the trial.

Sensorimotor Simulation's Influence on Stress: EEG and Autonomic Responses in Digital Interviews

This study explored the role of sensorimotor simulation in modulating the stress response in individuals exposed to stressful digital simulated interviews. Participants were assigned to two different versions of a Digital Social Stress Test: a simulated version with a dynamic-realistic examining committee (Dyn-DSST) and a version with a static examining committee (Stat-DSST). During interview preparation, behavioral indices reflecting stress regulation and resistance, response times, and electroencephalographic (EEG) and autonomic indices were collected. Higher regulation scores were found for the Stat-DSST group compared to the Dyn-DSST group, probably induced by the presence of limited external sensory input in time and space, perceived as less stressful. The EEG results revealed a distinct contribution of the low- and high-frequency bands for both groups. Dyn-DSST required greater cognitive regulation effort due to the presence of a continuous flow of information, which can enhance sensory and motor activation in the brain. The SCR increased in the Dyn-DSST group compared to the Stat-DSST group, reflecting greater emotional involvement in the Dyn-DSST group and reduced sensory stimulation in the static version. In conclusion, the results suggest that sensorimotor simulation impacts the stress response differently in dynamic interviews compared to static ones, with distinct profiles based on behavioral, EEG, and autonomic measures.

Facial representations of complex affective states combining pain and a negative emotion

Pain is rarely communicated alone, as it is often accompanied by emotions such as anger or sadness. Communicating these affective states involves shared representations. However, how an individual conceptually represents these combined states must first be tested. The objective of this study was to measure the interaction between pain and negative emotions on two types of facial representations of these states, namely visual (i.e., interactive virtual agents; VAs) and sensorimotor (i.e., one's production of facial configurations). Twenty-eight participants (15 women) read short written scenarios involving only pain or a combined experience of pain and a negative emotion (anger, disgust, fear, or sadness). They produced facial configurations representing these experiences on the faces of the VAs and on their face (own production or imitation of VAs). The results suggest that affective states related to a direct threat to the body (i.e., anger, disgust, and pain) share a similar facial representation, while those that present no immediate danger (i.e., fear and sadness) differ. Although visual and sensorimotor representations of these states provide congruent affective information, they are differently influenced by factors associated with the communication cycle. These findings contribute to our understanding of pain communication in different affective contexts.

How and Why People Synchronize: An Integrated Perspective

Academic AbstractInterpersonal synchrony, the alignment of behavior and/or physiology during interactions, is a pervasive phenomenon observed in diverse social contexts. Here we synthesize across contexts and behaviors to classify the different forms and functions of synchrony. We provide a concise framework for classifying the manifold forms of synchrony along six dimensions: periodicity, discreteness, spatial similarity, directionality, leader-follower dynamics, and observability. We also distill the various proposed functions of interpersonal synchrony into four interconnected functions: reducing complexity and improving understanding, accomplishing joint tasks, strengthening social connection, and influencing partners' behavior. These functions derive from first principles, emerge from each other, and are accomplished by some forms of synchrony more than others. Effective synchrony flexibly adapts to social goals and more synchrony is not always better. Our synthesis offers a shared framework and language for the field, allowing for better cross-context and cross-behavior comparisons, generating new hypotheses, and highlighting future research directions.

Facial icons as indexes of emotions and intentions

Various objects and artifacts incorporate representations of faces, encompassing artworks like portraits, as well as ethnographic or industrial artifacts such as masks or humanoid robots. These representations exhibit diverse degrees of human-likeness, serving different functions and objectives. Despite these variations, they share common features, particularly facial attributes that serve as building blocks for facial expressions-an effective means of communicating emotions. To provide a unified conceptualization for this broad spectrum of face representations, we propose the term "facial icons" drawing upon Peirce's semiotic concepts. Additionally, based on these semiotic principles, we posit that facial icons function as indexes of emotions and intentions, and introduce a significant anthropological theory aligning with our proposition. Subsequently, we support our assertions by examining processes related to face and facial expression perception, as well as sensorimotor simulation processes involved in discerning others' mental states, including emotions. Our argumentation integrates cognitive and experimental evidence, reinforcing the pivotal role of facial icons in conveying mental states.

Perception of emotionally incongruent cues: evidence for overreliance on body vs. face expressions in Parkinson's disease

Individuals with Parkinson's disease (PD) may exhibit impaired emotion perception. However, research demonstrating this decline has been based almost entirely on the recognition of isolated emotional cues. In real life, emotional cues such as expressive faces are typically encountered alongside expressive bodies. The current study investigated emotion perception in individuals with PD (n = 37) using emotionally incongruent composite displays of facial and body expressions, as well as isolated face and body expressions, and congruent composite displays as a baseline. In addition to a group of healthy controls (HC) (n = 50), we also included control individuals with schizophrenia (SZ) (n = 30), who display, as in PD, similar motor symptomology and decreased emotion perception abilities. The results show that individuals with PD showed an increased tendency to categorize incongruent face-body combinations in line with the body emotion, whereas those with HC showed a tendency to classify them in line with the facial emotion. No consistent pattern for prioritizing the face or body was found in individuals with SZ. These results were not explained by the emotional recognition of the isolated cues, cognitive status, depression, or motor symptoms of individuals with PD and SZ. As real-life expressions may include inconsistent cues in the body and face, these findings may have implications for the way individuals with PD and SZ interpret the emotions of others.

Effects of facial biofeedback on hypomimia, emotion recognition, and affect in Parkinson's disease

OBJECTIVES

Facial expressions are a core component of emotions and nonverbal social communication. Therefore, hypomimia as secondary symptom of Parkinson's disease (PD) has adverse effects like social impairment, stigmatization, under-diagnosis and under-treatment of depression, and a generally lower quality of life. Beside unspecific dopaminergic treatment, specific treatment options for hypomimia in PD are rarely investigated. This quasi-randomized controlled trial evaluated the short-term effects of facial electromyogram (EMG) based biofeedback to enhance facial expression and emotion recognition as nonverbal social communication skills in PD patients. Furthermore effects on affect are examined.

METHOD

A sample of 34 in-patients with PD were allocated either to facial EMG-biofeedback as experimental group or non-facial exercises as control group. Facial expression during posing of emotions (measured via EMG), facial emotion recognition, and positive and negative affect were assessed before and after treatment. Stronger improvements were expected in the EMG-biofeedback in comparison to the control group.

RESULTS

The facial EMG-biofeedback group showed significantly greater improvements in overall facial expression, and especially for happiness and disgust. Also, overall facial emotion recognition abilities improved significantly stronger in the experimental group. Positive affect was significantly increased in both groups with no significant differences between them, while negative affect did not change within both groups.

CONCLUSIONS

The study provides promising evidence for facial EMG-biofeedback as a tool to improve facial expression and emotion recognition in PD. Embodiment theories are discussed as working mechanism.

Reading of ingroup politicians' smiles triggers smiling in the corner of one's eyes

Spontaneous smiles in response to politicians can serve as an implicit barometer for gauging electorate preferences. However, it is unclear whether a subtle Duchenne smile-an authentic expression involving the coactivation of the zygomaticus major (ZM) and orbicularis oculi (OO) muscles-would be elicited while reading about a favored politician smiling, indicating a more positive disposition and political endorsement. From an embodied simulation perspective, we investigated whether written descriptions of a politician's smile would trigger morphologically different smiles in readers depending on shared or opposing political orientation. In a controlled reading task in the laboratory, participants were presented with subject-verb phrases describing left and right-wing politicians smiling or frowning. Concurrently, their facial muscular reactions were measured via electromyography (EMG) recording at three facial muscles: the ZM and OO, coactive during Duchenne smiles, and the corrugator supercilii (CS) involved in frowning. We found that participants responded with a Duchenne smile detected at the ZM and OO facial muscles when exposed to portrayals of smiling politicians of same political orientation and reported more positive emotions towards these latter. In contrast, when reading about outgroup politicians smiling, there was a weaker activation of the ZM muscle and no activation of the OO muscle, suggesting a weak non-Duchenne smile, while emotions reported towards outgroup politicians were significantly more negative. Also, a more enhanced frown response in the CS was found for ingroup compared to outgroup politicians' frown expressions. Present findings suggest that a politician's smile may go a long way to influence electorates through both non-verbal and verbal pathways. They add another layer to our understanding of how language and social information shape embodied effects in a highly nuanced manner. Implications for verbal communication in the political context are discussed.

Application of facial neuromuscular electrical stimulation (fNMES) in psychophysiological research: Practical recommendations based on a systematic review of the literature

Facial neuromuscular electrical stimulation (fNMES), which allows for the non-invasive and physiologically sound activation of facial muscles, has great potential for investigating fundamental questions in psychology and neuroscience, such as the role of proprioceptive facial feedback in emotion induction and emotion recognition, and may serve for clinical applications, such as alleviating symptoms of depression. However, despite illustrious origins in the 19th-century work of Duchenne de Boulogne, the practical application of fNMES remains largely unknown to today's researchers in psychology. In addition, published studies vary dramatically in the stimulation parameters used, such as stimulation frequency, amplitude, duration, and electrode size, and in the way they reported them. Because fNMES parameters impact the comfort and safety of volunteers, as well as its physiological (and psychological) effects, it is of paramount importance to establish recommendations of good practice and to ensure studies can be better compared and integrated. Here, we provide an introduction to fNMES, systematically review the existing literature focusing on the stimulation parameters used, and offer recommendations on how to safely and reliably deliver fNMES and on how to report the fNMES parameters to allow better cross-study comparison. In addition, we provide a free webpage, to easily visualise fNMES parameters and verify their safety based on current density. As an example of a potential application, we focus on the use of fNMES for the investigation of the facial feedback hypothesis.

Task-related modulation of motor response to emotional bodies: A TMS motor-evoked potential study

Exposure to emotional body postures during perceptual decision-making tasks has been linked to transient suppression of motor reactivity, supporting the monitoring of emotionally relevant information. However, it remains unclear whether this effect occurs implicitly, i.e., when emotional information is irrelevant to the task. To investigate this issue, we used single-pulse transcranial magnetic stimulation (TMS) to assess motor excitability while healthy participants were asked to categorize pictures of body expressions as emotional or neutral (emotion recognition task) or as belonging to a male or a female actor (gender recognition task) while receiving TMS over the motor cortex at 100 and 125 ms after picture onset. Results demonstrated that motor-evoked potentials (MEPs) were reduced for emotional body postures relative to neutral postures during the emotion recognition task. Conversely, MEPs increased for emotional body postures relative to neutral postures during the gender recognition task. These findings indicate that motor inhibition, contingent upon observing emotional body postures, is selectively associated with actively monitoring emotional features. In contrast, observing emotional body postures prompts motor facilitation when task-relevant features are non-emotional. These findings contribute to embodied cognition models that link emotion perception and action tendencies.

Mimicking Facial Expressions Facilitates Working Memory for Stimuli in Emotion-Congruent Colours

It is one thing for everyday phrases like "seeing red" to link some emotions with certain colours (e.g., anger with red), but can such links measurably bias information processing? We investigated whether emotional face information (angry/happy/neutral) held in visual working memory (VWM) enhances memory for shapes presented in a conceptually consistent colour (red or green) (Experiment 1). Although emotional information held in VWM appeared not to bias memory for coloured shapes in Experiment 1, exploratory analyses suggested that participants who physically mimicked the face stimuli were better at remembering congruently coloured shapes. Experiment 2 confirmed this finding by asking participants to hold the faces in mind while either mimicking or labelling the emotional expressions of face stimuli. Once again, those who mimicked the expressions were better at remembering shapes with emotion-congruent colours, whereas those who simply labelled them were not. Thus, emotion-colour associations appear powerful enough to guide attention, but-consistent with proposed impacts of "embodied emotion" on cognition-such effects emerged when emotion processing was facilitated through facial mimicry.

An investigation of the modulatory effects of empathic and autistic traits on emotional and facial motor responses during live social interactions

A close relationship between emotional contagion and spontaneous facial mimicry has been theoretically proposed and is supported by empirical data. Facial expressions are essential in terms of both emotional and motor synchrony. Previous studies have demonstrated that trait emotional empathy enhanced spontaneous facial mimicry, but the relationship between autistic traits and spontaneous mimicry remained controversial. Moreover, previous studies presented faces that were static or videotaped, which may lack the "liveliness" of real-life social interactions. We addressed this limitation by using an image relay system to present live performances and pre-recorded videos of smiling or frowning dynamic facial expressions to 94 healthy female participants. We assessed their subjective experiential valence and arousal ratings to infer the amplitude of emotional contagion. We measured the electromyographic activities of the zygomaticus major and corrugator supercilii muscles to estimate spontaneous facial mimicry. Individual differences measures included trait emotional empathy (empathic concern) and the autism-spectrum quotient. We did not find that live performances enhanced the modulatory effect of trait differences on emotional contagion or spontaneous facial mimicry. However, we found that a high trait empathic concern was associated with stronger emotional contagion and corrugator mimicry. We found no two-way interaction between the autism spectrum quotient and emotional condition, suggesting that autistic traits did not modulate emotional contagion or spontaneous facial mimicry. Our findings imply that previous findings regarding the relationship between emotional empathy and emotional contagion/spontaneous facial mimicry using videos and photos could be generalized to real-life interactions.

Social cognition in hyperkinetic movement disorders: a systematic review

Numerous lines of research indicate that our social brain involves a network of cortical and subcortical brain regions that are responsible for sensing and controlling body movements. However, it remains unclear whether movement disorders have a systematic impact on social cognition. To address this question, we conducted a systematic review examining the influence of hyperkinetic movement disorders (including Huntington disease, Tourette syndrome, dystonia, and essential tremor) on social cognition. Following the PRISMA guidelines and registering the protocol in the PROSPERO database (CRD42022327459), we analyzed 50 published studies focusing on theory of mind (ToM), social perception, and empathy. The results from these studies provide evidence of impairments in ToM and social perception in all hyperkinetic movement disorders, particularly during the recognition of negative emotions. Additionally, individuals with Huntington's Disease and Tourette syndrome exhibit empathy disorders. These findings support the functional role of subcortical structures (such as the basal ganglia and cerebellum), which are primarily responsible for movement disorders, in deficits related to social cognition.

To see or not to see: the parallel processing of self-relevance and facial expressions

The self, like the concept of central "gravity", facilitates the processing of information that is directly relevant to the self. This phenomenon is known as the self-prioritization effect. However, it remains unclear whether the self-prioritization effect extends to the processing of emotional facial expressions. To fill this gap, we used a self-association paradigm to investigate the impact of self-relevance on the recognition of emotional facial expressions while controlling for confounding factors such as familiarity and overlearning. Using a large and diverse sample, we replicated the effect of self-relevance on face processing but found no evidence for a modulation of self-relevance on facial emotion recognition. We propose two potential theoretical explanations to account for these findings and emphasize that further research with different experimental designs and a multitasks measurement approach is needed to understand this mechanism fully. Overall, our study contributes to the literature on the parallel cognitive processing of self-relevance and facial emotion recognition, with implications for both social and cognitive psychology.

Connectome-based modeling reveals a resting-state functional network that mediates the relationship between social rejection and rumination

Background

Rumination impedes problem solving and is one of the most important factors in the onset and maintenance of multiple psychiatric disorders. The current study aims to investigate the impact of social rejection on rumination and explore the underlying neural mechanisms involved in this process.

Methods

We utilized psychological questionnaire and resting-state brain imaging data from a sample of 560 individuals. The predictive model for rumination scores was constructed using resting-state functional connectivity data through connectome-based predictive modeling. Additionally, a mediation analysis was conducted to investigate the mediating role of the prediction network in the relationship between social rejection and rumination.

Results

A positive correlation between social rejection and rumination was found. We obtained the prediction model of rumination and found that the strongest contributions came from the intra- and internetwork connectivity within the default mode network (DMN), dorsal attention network (DAN), frontoparietal control network (FPCN), and sensorimotor networks (SMN). Analysis of node strength revealed the significance of the supramarginal gyrus (SMG) and angular gyrus (AG) as key nodes in the prediction model. In addition, mediation analysis showed that the strength of the prediction network mediated the relationship between social rejection and rumination.

Conclusion

The findings highlight the crucial role of functional connections among the DMN, DAN, FPCN, and SMN in linking social rejection and rumination, particular in brain regions implicated in social cognition and emotion, namely the SMG and AG regions. These results enhance our understanding of the consequences of social rejection and provide insights for novel intervention strategies targeting rumination.

Emotion regulation from an action-control perspective

Despite increasing interest in emotional processes in cognitive science, theories on emotion regulation have remained rather isolated, predominantly focused on cognitive regulation strategies such as reappraisal. However, recent neurocognitive evidence suggests that early emotion regulation may involve sensorimotor control in addition to other emotion-regulation processes. We propose an action-oriented view of emotion regulation, in which feedforward predictions develop from action-selection mechanisms. Those can account for acute emotional-action control as well as more abstract instances of emotion regulation such as cognitive reappraisal. We argue the latter occurs in absence of overt motor output, yet in the presence of full-blown autonomic, visceral, and subjective changes. This provides an integrated framework with testable neuro-computational predictions and concrete starting points for intervention to improve emotion control in affective disorders.

Facial mimicry is not modulated by dopamine D2/3 and opioid receptor antagonism

RATIONALE

According to theories of embodied cognition, facial mimicry - the spontaneous, low-intensity imitation of a perceived emotional facial expression - is first an automatic motor response, whose accompanying proprioceptive feedback contributes to emotion recognition. Alternative theoretical accounts, however, view facial mimicry as an emotional response to a rewarding stimulus, and/or an affiliative signal, and thus reject the view of an automatic motor copy.

OBJECTIVES

To contribute to this debate and further investigate the neural basis of facial mimicry, as well as its relation to reward processing, we measured facial reactions to dynamic happy and angry faces after pharmacologically manipulating the opioid and dopamine systems - respectively, thought to subserve 'liking' and 'wanting' of rewards.

METHODS

In a placebo-controlled, double-blind experiment, 130 volunteers received in a between-subjects design 50 mg of the opioidergic antagonist naltrexone, 400 mg of the dopaminergic antagonist amisulpride, or placebo.

RESULTS

Clear occurrence of facial mimicry, measured 4 h after drug intake with electromyography (EMG) of the zygomaticus major and corrugator supercilii muscles, was found. However, facial mimicry was not affected by either compound, as shown with both frequentist statistics, and a Bayesian asymptotic regression model.

CONCLUSIONS

This null finding does not support the hypothesis that facial mimicry (of happiness) reflects an emotional response to a rewarding stimulus, leaving open the possibility of facial mimicry being an automatic motor copy. The results are relevant to the discussion about the psychological nature and the neural basis of facial mimicry, although they should be considered preliminary, given the challenges of interpreting null findings when targeting a novel effect of unknown size.

Advancing Naturalistic Affective Science with Deep Learning

People express their own emotions and perceive others' emotions via a variety of channels, including facial movements, body gestures, vocal prosody, and language. Studying these channels of affective behavior offers insight into both the experience and perception of emotion. Prior research has predominantly focused on studying individual channels of affective behavior in isolation using tightly controlled, non-naturalistic experiments. This approach limits our understanding of emotion in more naturalistic contexts where different channels of information tend to interact. Traditional methods struggle to address this limitation: manually annotating behavior is time-consuming, making it infeasible to do at large scale; manually selecting and manipulating stimuli based on hypotheses may neglect unanticipated features, potentially generating biased conclusions; and common linear modeling approaches cannot fully capture the complex, nonlinear, and interactive nature of real-life affective processes. In this methodology review, we describe how deep learning can be applied to address these challenges to advance a more naturalistic affective science. First, we describe current practices in affective research and explain why existing methods face challenges in revealing a more naturalistic understanding of emotion. Second, we introduce deep learning approaches and explain how they can be applied to tackle three main challenges: quantifying naturalistic behaviors, selecting and manipulating naturalistic stimuli, and modeling naturalistic affective processes. Finally, we describe the limitations of these deep learning methods, and how these limitations might be avoided or mitigated. By detailing the promise and the peril of deep learning, this review aims to pave the way for a more naturalistic affective science.

Construction and validation of the Dalian emotional movement open-source set (DEMOS)

Human body movements are important for emotion recognition and social communication and have received extensive attention from researchers. In this field, emotional biological motion stimuli, as depicted by point-light displays, are widely used. However, the number of stimuli in the existing material library is small, and there is a lack of standardized indicators, which subsequently limits experimental design and conduction. Therefore, based on our prior kinematic dataset, we constructed the Dalian Emotional Movement Open-source Set (DEMOS) using computational modeling. The DEMOS has three views (i.e., frontal 0°, left 45°, and left 90°) and in total comprises 2664 high-quality videos of emotional biological motion, each displaying happiness, sadness, anger, fear, disgust, and neutral. All stimuli were validated in terms of recognition accuracy, emotional intensity, and subjective movement. The objective movement for each expression was also calculated. The DEMOS can be downloaded for free from https://osf.io/83fst/ . To our knowledge, this is the largest multi-view emotional biological motion set based on the whole body. The DEMOS can be applied in many fields, including affective computing, social cognition, and psychiatry.

What Is Written on a Dog's Face? Evaluating the Impact of Facial Phenotypes on Communication between Humans and Canines

Facial phenotypes are significant in communication with conspecifics among social primates. Less is understood about the impact of such markers in heterospecific encounters. Through behavioral and physical phenotype analyses of domesticated dogs living in human households, this study aims to evaluate the potential impact of superficial facial markings on dogs' production of human-directed facial expressions. That is, this study explores how facial markings, such as eyebrows, patches, and widow's peaks, are related to expressivity toward humans. We used the Dog Facial Action Coding System (DogFACS) as an objective measure of expressivity, and we developed an original schematic for a standardized coding of facial patterns and coloration on a sample of more than 100 male and female dogs (N = 103), aged from 6 months to 12 years, representing eight breed groups. The present study found a statistically significant, though weak, correlation between expression rate and facial complexity, with dogs with plainer faces tending to be more expressive (r = -0.326, p ≤ 0.001). Interestingly, for adult dogs, human companions characterized dogs' rates of facial expressivity with more accuracy for dogs with plainer faces. Especially relevant to interspecies communication and cooperation, within-subject analyses revealed that dogs' muscle movements were distributed more evenly across their facial regions in a highly social test condition compared to conditions in which they received ambiguous cues from their owners. On the whole, this study provides an original evaluation of how facial features may impact communication in human-dog interactions.

Embodied language of emotions: Predicting human intuitions with linguistic distributions in blind and sighted individuals

Recent constructionist theories have suggested that language and sensory experience play a crucial role not only in how individuals categorise emotions but also in how they experience and shape them, helping to acquire abstract concepts that are used to make sense of bodily perceptions associated with specific emotions. Here, we aimed to investigate the role of sensory experience in conceptualising bodily felt emotions by asking 126 Italian blind participants to freely recall in which part of the body they commonly feel specific emotions (N = 15). Participants varied concerning visual experience in terms of blindness onset (i.e., congenital vs late) and degree of visual experience (i.e., total vs partial sensory loss). Using an Italian semantic model to estimate to what extent discrete emotions are associated with body parts in language experience, we found that all participants' reports correlated with the model predictions. Interestingly, blind - and especially congenitally blind - participants' responses were more strongly correlated with the model, suggesting that language might be one of the possible compensative mechanisms for the lack of visual feedback in constructing bodily felt emotions. Our findings present theoretical implications for the study of emotions, as well as potential real-world applications for blind individuals, by revealing, on the one hand, that vision plays an essential role in the construction of felt emotions and the way we talk about our related bodily (emotional) experiences. On the other hand, evidence that blind individuals rely more strongly on linguistic cues suggests that vision is a strong cue to acquire emotional information from the surrounding world, influencing how we experience emotions. While our findings do not suggest that blind individuals experience emotions in an atypical and dysfunctional way, they nonetheless support the view that promoting the use of non-visual emotional signs and body language since early on might help the blind child to develop a good emotional awareness as well as good emotion regulation abilities.

Differential responses in the mirror neuron system during imitation of individual emotional facial expressions and association with autistic traits

The mirror neuron system (MNS), including the inferior frontal gyrus (IFG), inferior parietal lobule (IPL) and superior temporal sulcus (STS) plays an important role in action representation and imitation and may be dysfunctional in autism spectrum disorder (ASD). However, it's not clear how these three regions respond and interact during the imitation of different basic facial expressions and whether the pattern of responses is influenced by autistic traits. Thus, we conducted a natural facial expression (happiness, angry, sadness and fear) imitation task in 100 healthy male subjects where expression intensity was measured using facial emotion recognition software (FaceReader) and MNS responses were recorded using functional near-infrared spectroscopy (fNIRS). Autistic traits were measured using the Autism Spectrum Quotient questionnaire. Results showed that imitation of happy expressions produced the highest expression intensity but a small deactivation in MNS responses, suggesting a lower processing requirement compared to other expressions. A cosine similarity analysis indicated a distinct pattern of MNS responses during imitation of each facial expression with functional intra-hemispheric connectivity between the left IPL and left STS being significantly higher during happy compared to other expressions, while inter-hemispheric connectivity between the left and right IPL differed between imitation of fearful and sad expressions. Furthermore, functional connectivity changes during imitation of each different expression could reliably predict autistic trait scores. Overall, the results provide evidence for distinct patterns of functional connectivity changes between MNS regions during imitation of different emotions which are also associated with autistic traits.

Hidden faces, altered perceptions: the impact of face masks on interpersonal perception

The pandemic has made wearing masks commonplace, prompting researchers to investigate their effects on interpersonal perception. Findings indicate masks obstruct face identification and expression recognition, with lower face cues being most affected. When judging attractiveness, masks can enhance the appeal of less attractive faces, but reduce the appeal of more attractive faces. Trust and speech perception outcomes are inconclusive. Future studies could focus on individual differences in how masks influence our perception of others.

The role of affective touch in modulating emotion processing among preschool children

Recognizing emotional expressions is a prerequisite for understanding others' feelings and intentions, a key component of social interactions that develops throughout childhood. In multisensory social environments, touch may be crucial for emotion processing, linking external sensory information with internal affective states. The current study investigated whether affective touch facilitates recognition of emotional expressions throughout childhood. Preschool children (N = 121 3- to 6-year-olds) were presented with different tactile stimulations followed by an emotion-matching task. Results revealed that affective touch fosters the recognition of negative emotions and increases the speed of association of positive emotions, highlighting the centrality of tactile experiences for socioemotional understanding. The current research opens new perspectives on how to support emotional recognition with potential consequences for the development of social functioning.

Facial mimicry and metacognitive judgments in emotion recognition are distinctly modulated by social anxiety and autistic traits

Facial mimicry as well as the accurate assessment of one's performance when judging others' emotional expressions have been suggested to inform successful emotion recognition. Differences in the integration of these two information sources might explain alterations in the perception of others' emotions in individuals with Social Anxiety Disorder and individuals on the autism spectrum. Using a non-clinical sample (N = 57), we examined the role of social anxiety and autistic traits in the link between facial mimicry, or confidence in one's performance, and emotion recognition. While participants were presented with videos of spontaneous emotional facial expressions, we measured their facial muscle activity, asked them to label the expressions and indicate their confidence in accurately labelling the expressions. Our results showed that confidence in emotion recognition was lower with higher social anxiety traits even though actual recognition was not related to social anxiety traits. Higher autistic traits, in contrast, were associated with worse recognition, and a weakened link between facial mimicry and performance. Consequently, high social anxiety traits might not affect emotion recognition itself, but the top-down evaluation of own abilities in emotion recognition contexts. High autistic traits, in contrast, may be related to lower integration of sensorimotor simulations, which promote emotion recognition.

Testing EEG functional connectivity between sensorimotor and face processing visual regions in individuals with congenital facial palsy

Moebius syndrome (MBS) is characterized by the congenital absence or underdevelopment of cranial nerves VII and VI, leading to facial palsy and impaired lateral eye movements. As a result, MBS individuals cannot produce facial expressions and did not develop motor programs for facial expressions. In the latest model of sensorimotor simulation, an iterative communication between somatosensory, motor/premotor cortices, and visual regions has been proposed, which should allow more efficient discriminations among subtle facial expressions. Accordingly, individuals with congenital facial motor disability, specifically with MBS, should exhibit atypical communication within this network. Here, we aimed to test this facet of the sensorimotor simulation models. We estimated the functional connectivity between the visual cortices for face processing and the sensorimotor cortices in healthy and MBS individuals. To this aim, we studied the strength of beta band functional connectivity between these two systems using high-density EEG, combined with a change detection task with facial expressions (and a control condition involving non-face stimuli). The results supported our hypothesis such that when discriminating subtle facial expressions, participants affected by congenital facial palsy (compared to healthy controls) showed reduced connectivity strength between sensorimotor regions and visual regions for face processing. This effect was absent for the condition with non-face stimuli. These findings support sensorimotor simulation models and the communication between sensorimotor and visual areas during subtle facial expression processing.

Facial expression and emotion

Human facial expressions are unique in their ability to express our emotions and communicate them to others. The mimic expression of basic emotions is very similar across different cultures and has also many features in common with other mammals. This suggests a common genetic origin of the association between facial expressions and emotion. However, recent studies also show cultural influences and differences. The recognition of emotions from facial expressions, as well as the process of expressing one's emotions facially, occurs within an extremely complex cerebral network. Due to the complexity of the cerebral processing system, there are a variety of neurological and psychiatric disorders that can significantly disrupt the coupling of facial expressions and emotions. Wearing masks also limits our ability to convey and recognize emotions through facial expressions. Through facial expressions, however, not only "real" emotions can be expressed, but also acted ones. Thus, facial expressions open up the possibility of faking socially desired expressions and also of consciously faking emotions. However, these pretenses are mostly imperfect and can be accompanied by short-term facial movements that indicate the emotions that are actually present (microexpressions). These microexpressions are of very short duration and often barely perceptible by humans, but they are the ideal application area for computer-aided analysis. This automatic identification of microexpressions has not only received scientific attention in recent years, but its use is also being tested in security-related areas. This article summarizes the current state of knowledge of facial expressions and emotions.

Neural mechanisms for emotional contagion and spontaneous mimicry of live facial expressions

Viewing a live facial expression typically elicits a similar expression by the observer (facial mimicry) that is associated with a concordant emotional experience (emotional contagion). The model of embodied emotion proposes that emotional contagion and facial mimicry are functionally linked although the neural underpinnings are not known. To address this knowledge gap, we employed a live two-person paradigm (n = 20 dyads) using functional near-infrared spectroscopy during live emotive face-processing while also measuring eye-tracking, facial classifications and ratings of emotion. One dyadic partner, 'Movie Watcher', was instructed to emote natural facial expressions while viewing evocative short movie clips. The other dyadic partner, 'Face Watcher', viewed the Movie Watcher's face. Task and rest blocks were implemented by timed epochs of clear and opaque glass that separated partners. Dyadic roles were alternated during the experiment. Mean cross-partner correlations of facial expressions (r = 0.36 ± 0.11 s.e.m.) and mean cross-partner affect ratings (r = 0.67 ± 0.04) were consistent with facial mimicry and emotional contagion, respectively. Neural correlates of emotional contagion based on covariates of partner affect ratings included angular and supramarginal gyri, whereas neural correlates of the live facial action units included motor cortex and ventral face-processing areas. Findings suggest distinct neural components for facial mimicry and emotional contagion. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

"When You're Smiling": How Posed Facial Expressions Affect Visual Recognition of Emotions

Facial imitation occurs automatically during the perception of an emotional facial expression, and preventing it may interfere with the accuracy of emotion recognition. In the present fMRI study, we evaluated the effect of posing a facial expression on the recognition of ambiguous facial expressions. Since facial activity is affected by various factors, such as empathic aptitudes, the Interpersonal Reactivity Index (IRI) questionnaire was administered and scores were correlated with brain activity. Twenty-six healthy female subjects took part in the experiment. The volunteers were asked to pose a facial expression (happy, disgusted, neutral), then to watch an ambiguous emotional face, finally to indicate whether the emotion perceived was happiness or disgust. As stimuli, blends of happy and disgusted faces were used. Behavioral results showed that posing an emotional face increased the percentage of congruence with the perceived emotion. When participants posed a facial expression and perceived a non-congruent emotion, a neural network comprising bilateral anterior insula was activated. Brain activity was also correlated with empathic traits, particularly with empathic concern, fantasy and personal distress. Our findings support the idea that facial mimicry plays a crucial role in identifying emotions, and that empathic emotional abilities can modulate the brain circuits involved in this process.

Using support vector machine to explore the difference of function connection between deficit and non-deficit schizophrenia based on gray matter volume

Objective

Schizophrenia can be divided into deficient schizophrenia (DS) and non-deficient schizophrenia (NDS) according to the presence of primary and persistent negative symptoms. So far, there are few studies that have explored the differences in functional connectivity (FC) between the different subtypes based on the region of interest (ROI) from GMV (Gray matter volume), especially since the characteristics of brain networks are still unknown. This study aimed to investigate the alterations of functional connectivity between DS and NDS based on the ROI obtained by machine learning algorithms and differential GMV. Then, the relationships between the alterations and the clinical symptoms were analyzed. In addition, the thalamic functional connection imbalance in the two groups was further explored.

Methods

A total of 16 DS, 31 NDS, and 38 health controls (HC) underwent resting-state fMRI scans, patient group will further be evaluated by clinical scales including the Brief Psychiatric Rating Scale (BPRS), the Scale for the Assessment of Negative Symptoms (SANS), and the Scale for the Assessment of Positive Symptoms (SAPS). Based on GMV image data, a support vector machine (SVM) is used to classify DS and NDS. Brain regions with high weight in the classification were used as seed points in whole-brain FC analysis and thalamic FC imbalance analysis. Finally, partial correlation analysis explored the relationships between altered FC and clinical scale in the two subtypes.

Results

The relatively high classification accuracy is obtained based on the SVM. Compared to HC, the FC increased between the right inferior parietal lobule (IPL.R) bilateral thalamus, and lingual gyrus, and between the right inferior temporal gyrus (ITG.R) and the Salience Network (SN) in NDS. The FC between the right thalamus (THA.R) and Visual network (VN), between ITG.R and right superior occipital gyrus in the DS group was higher than that in HC. Furthermore, compared with NDS, the FC between the ITG.R and the left superior and middle frontal gyrus decreased in the DS group. The thalamic FC imbalance, which is characterized by frontotemporal-THA.R hypoconnectivity and sensory motor network (SMN)-THA.R hyperconnectivity was found in both subtypes. The FC value of THA.R and SMN was negatively correlated with the SANS score in the DS group but positively correlated with the SAPS score in the NDS group.

Conclusion

Using an SVM classification method and based on an ROI from GMV, we highlighted the difference in functional connectivity between DS and NDS from the local to the brain network, which provides new information for exploring the neural physiopathology of the two subtypes of schizophrenic.

Whose emotion is it? Perspective matters to understand brain-body interactions in emotions

Feeling happy, or judging whether someone else is feeling happy are two distinct facets of emotions that nevertheless rely on similar physiological and neural activity. Differentiating between these two states, also called Self/Other distinction, is an essential aspect of empathy, but how exactly is it implemented? In non-emotional cognition, the transient neural response evoked at each heartbeat, or heartbeat evoked response (HER), indexes the self and signals Self/Other distinction. Here, using electroencephalography (n = 32), we probe whether HERs' role in Self/Other distinction extends also to emotion - a domain where brain-body interactions are particularly relevant. We asked participants to rate independently validated affective scenes, reporting either their own emotion (Self) or the emotion expressed by people in the scene (Other). During the visual cue indicating to adopt the Self or Other perspective, before the affective scene, HERs distinguished between the two conditions, in visual cortices as well as in the right frontal operculum. Physiological reactivity (facial electromyogram, skin conductance, heart rate) during affective scene co-varied as expected with valence and arousal ratings, but also with the Self- or Other- perspective adopted. Finally, HERs contributed to the subjective experience of valence in the Self condition, in addition to and independently from physiological reactivity. We thus show that HERs represent a trans-domain marker of Self/Other distinction, here specifically contributing to experienced valence. We propose that HERs represent a form of evidence related to the 'I' part of the judgement 'To which extent do I feel happy'. The 'I' related evidence would be combined with the affective evidence collected during affective scene presentation, accounting at least partly for the difference between feeling an emotion and identifying it in someone else.

A method for diagnosing depression: Facial expression mimicry is evaluated by facial expression recognition

BACKGROUND

Considerable evidence has shown that facial expression mimicry is impaired in patients with depression. We aimed to evaluate voluntary expression mimicry by facial expression recognition for diagnosing depression.

METHODS

A total of 168 participants performed voluntary expression mimicry task, posing anger, disgust, fear, happiness, neutrality, sadness, and surprise. 9 healthy raters performed facial expression recognition task through the observer scoring method, and evaluated seven expressions imitated by participants. Emotional scores were calculated to measure any differences between two groups of participants and provided a basis for clinical diagnosis of depression.

RESULTS

Compared with the control group, the depression group had lower accuracy in imitating happiness. Compared with the control group, the depression group imitated a higher neutrality bias for sadness, surprise, happiness and disgust, while sadness and surprise had a lower happiness bias; for imitating happiness, the depression group showed higher anger, disgust, fear, neutrality, and surprise bias; for imitating neutrality, the depression group showed higher sadness bias, and lower happiness bias. Compared with the control group, the raters had a higher reaction time to recognize the happiness imitated by depression group, and it was positively correlated with severity of depression. The severity of depression was also negatively correlated with accuracy in imitating happiness, and positively correlated with neutrality bias of imitating surprise.

LIMITATIONS

The ecological effectiveness of static stimulus materials is lower than that of dynamic stimuli. Without synchronized functional imaging, there is no way to link brain activation patterns.

CONCLUSION

The ability of patients with depression to voluntarily imitate facial expressions declines, which is mainly reflected in accuracy, bias and recognizability. Our experiment has discovered deficits in these aspects of patients with depression, which will be used as a method for diagnosising depression.

Non-conscious processing of fear faces: a function of the implicit self-concept of anxiety

BACKGROUND

Trait anxiety refers to a stable tendency to experience fears and worries across many situations. High trait anxiety is a vulnerability factor for the development of psychopathologies. Self-reported trait anxiety appears to be associated with an automatic processing advantage for threat-related information. Self-report measures assess aspects of the explicit self-concept of anxiety. Indirect measures can tap into the implicit self-concept of anxiety.

METHODS

We examined automatic brain responsiveness to non-conscious threat as a function of trait anxiety using functional magnetic resonance imaging. Besides a self-report instrument, we administered the Implicit Association Test (IAT) to assess anxiety. We used a gender-decision paradigm presenting brief (17 ms) and backward-masked facial expressions depicting disgust and fear.

RESULTS

Explicit trait anxiety was not associated with brain responsiveness to non-conscious threat. However, a relation of the implicit self-concept of anxiety with masked fear processing in the thalamus, precentral gyrus, and lateral prefrontal cortex was observed.

CONCLUSIONS

We provide evidence that a measure of the implicit self-concept of anxiety is a valuable predictor of automatic neural responses to threat in cortical and subcortical areas. Hence, implicit anxiety measures could be a useful addition to explicit instruments. Our data support the notion that the thalamus may constitute an important neural substrate in biased non-conscious processing in anxiety.

An Audience Facilitates Facial Feedback: A Social-Context Hypothesis Reconciling Original Study and Nonreplication

Nonreplications of previously undisputed phenomena tend to leave a theoretical vacuum. This theoretical perspective seeks to fill the gap left by the failure to replicate unobtrusive facial feedback. In the emblematic original study, participants who held a pen between the teeth (i.e., requiring activity of the zygomaticus major muscle) rated cartoons more positively than participants who held the pen between the lips. We argue that the same social mechanisms (e.g., the presence of an audience) modulate facial feedback to emotion as are involved in the feed-forward shaping of facial actions by emotions. Differing social contexts could thus help explain the contrast between original findings and failures to obtain unobtrusive facial feedback. An exploratory analysis that included results only from (unobtrusive) facial-feedback studies without explicit reference to emotion in the facial manipulation provided preliminary support for this hypothesis. Studies with a social context (e.g., due to experimenter presence) showed a medium-sized aggregate facial-feedback effect, whereas studies without a social context (e.g., when facial actions were only filmed), revealed a small effect. Video awareness strengthened facial feedback considerably within an engaging social context, but seemed to reduce it without a social context. We provisionally conclude that a (pro-)social interpretation of facial actions facilitates feedback to (primarily positive) emotion, and suggest further research explicitly manipulating this context.

The neural basis of smile authenticity judgments and the potential modulatory role of the oxytocin receptor gene (OXTR)

Accurate perception of genuine vs. posed smiles is crucial for successful social navigation in humans. While people vary in their ability to assess the authenticity of smiles, little is known about the specific biological mechanisms underlying this variation. We investigated the neural substrates of smile authenticity judgments using functional magnetic resonance imaging (fMRI). We also tested a preliminary hypothesis that a common polymorphism in the oxytocin receptor gene (OXTR) rs53576 would modulate the behavioral and neural indices of accurate smile authenticity judgments. A total of 185 healthy adult participants (Neuroimaging arm: N = 44, Behavioral arm: N = 141) determined the authenticity of dynamic facial expressions of genuine and posed smiles either with or without fMRI scanning. Correctly identified genuine vs. posed smiles activated brain areas involved with reward processing, facial mimicry, and mentalizing. Activation within the inferior frontal gyrus and dorsomedial prefrontal cortex correlated with individual differences in sensitivity (d') and response criterion (C), respectively. Our exploratory genetic analysis revealed that rs53576 G homozygotes in the neuroimaging arm had a stronger tendency to judge posed smiles as genuine than did A allele carriers and showed decreased activation in the medial prefrontal cortex when viewing genuine vs. posed smiles. Yet, OXTR rs53576 did not modulate task performance in the behavioral arm, which calls for further studies to evaluate the legitimacy of this result. Our findings extend previous literature on the biological foundations of smile authenticity judgments, particularly emphasizing the involvement of brain regions implicated in reward, facial mimicry, and mentalizing.

The Role of Alpha Oscillations among the Main Neuropsychiatric Disorders in the Adult and Developing Human Brain: Evidence from the Last 10 Years of Research

Alpha oscillations (7-13 Hz) are the dominant rhythm in both the resting and active brain. Accordingly, translational research has provided evidence for the involvement of aberrant alpha activity in the onset of symptomatological features underlying syndromes such as autism, schizophrenia, major depression, and Attention Deficit and Hyperactivity Disorder (ADHD). However, findings on the matter are difficult to reconcile due to the variety of paradigms, analyses, and clinical phenotypes at play, not to mention recent technical and methodological advances in this domain. Herein, we seek to address this issue by reviewing the literature gathered on this topic over the last ten years. For each neuropsychiatric disorder, a dedicated section will be provided, containing a concise account of the current models proposing characteristic alterations of alpha rhythms as a core mechanism to trigger the associated symptomatology, as well as a summary of the most relevant studies and scientific contributions issued throughout the last decade. We conclude with some advice and recommendations that might improve future inquiries within this field.

Mimicry of partially occluded emotional faces: do we mimic what we see or what we know?

Facial electromyography (EMG) was used to investigate patterns of facial mimicry in response to partial facial expressions in two contexts that differ in how naturalistic and socially significant the faces are. Experiment 1 presented participants with either the upper- or lower-half of facial expressions and used a forced-choice emotion categorisation task. This task emphasises cognition at the expense of ecological and social validity. Experiment 2 presented whole heads and expressions were occluded by clothing. Additionally, the emotion recognition task is more open-ended. This context has greater social validity. We found mimicry in both experiments, however mimicry differed in terms of which emotions were mimicked and the extent to which the mimicry involved muscle sites that were not observed. In the more cognitive context, there was relatively more motor matching (i.e. mimicking only what was seen). In the more socially valid context, participants were less likely to mimic only what they saw - and instead mimicked what they knew. Additionally, participants mimicked anger in the cognitive context but not the social context. These findings suggest that mimicry involves multiple mechanisms and that the more social the context, the more likely it is to reflect a mechanism of social regulation.

The different shades of laughter: when do we laugh and when do we mimic other's laughter?

Laughter is an ambiguous phenomenon in response to both positive and negative events and a social signal that coordinates social interactions. We assessed (i) who laughs and why, and (ii) if the type of laughter and whether the observer approves of it impact on facial mimicry as a proxy for shared laughter. For this, 329 participants watched funny, schadenfreude and disgusting scenes and then saw individuals who purportedly reacted to each scene while participants' facial expressions were recorded and analysed. Participants laughed more in response to funny than in response to schadenfreude scenes and least in response to disgust scenes, and laughter within each scene could be explained both by situational perceptions of the scenes as well as by individual differences. Furthermore, others' laughter in response to funny scenes was perceived as more appropriate, elicited more closeness and more laughter mimicry than others' laughter in response to schadenfreude and especially in response to disgust scenes. Appropriateness and closeness as well as individual differences could explain laughter mimicry within each scene. This is in line with the notion that laughter is not per se an affiliative signal and that different types of laughter have distinct social implications. This article is part of the theme issue 'Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience'.

Degenerate pathway for processing smile and other emotional expressions in congenital facial palsy: an hdEEG investigation

Influential theoretical models argue that an internal simulation mechanism (motor or sensorimotor simulation) supports the recognition of facial expressions. However, despite numerous converging sources of evidence, recent studies testing patients with congenital facial palsy (i.e. Moebius syndrome) seem to refute these theoretical models. However, these results do not consider the principles of neuroplasticity and degeneracy that could support the involvement of an alternative neural processing pathway in these patients. In the present study, we tested healthy participants and participants with Moebius syndrome in a highly sensitive facial expression discrimination task and concomitant high-density electroencephalographic recording. The results, both at the scalp and source levels, indicate the activation of two different pathways of facial expression processing in healthy participants and participants with Moebius syndrome, compatible, respectively, with a dorsal pathway that includes premotor areas and a ventral pathway. Therefore, these results support the reactivation of sensorimotor representations of facial expressions (i.e. simulation) in healthy subjects, in the place of an alternative processing pathway in subjects with congenital facial palsy.

This article is part of the theme issue ‘Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience’.

Are emotions contagious? A conceptual review of studies in language education

Considering the great role those affective variables play in language learning, it seems wise to hypothesize whether teacher and learner emotions are contagious in the language classroom or not. The existing literature on emotion contagion in other fields of study accepts this hypothesis and reveals significant points about the verbal and non-verbal cues for emotion contagion in class. The present paper introduces and defines the topic and goes on with summarizing the procedure of transmitting the emotion. Then, it reviews the existing research on emotion contagion in different areas and creates a link between them and the L2 studies. It draws attention to the dearth of research on emotion contagion in the second language acquisition (SLA) domain and emphasizes the need for innovative dynamic approaches to research to explore emotion contagion in the ecology of the English as a Foreign Language (EFL) classroom. It also reviews the limited body of research in this regard in the SLA domain and shows how the significant findings can be useful for L2 educators. The findings of these studies show that more relevant studies in the future can be hoped to provide insightful remarks about how different emotions are transmitted between the teacher and students or among students in L2 classes. These studies are supposed to adopt a dynamic approach as well as a longitudinal data collection and analysis procedure. These can have implications for teacher education programs in the English language teaching (ELT) domain.

Wearing a KN95/FFP2 facemask induces subtle yet significant brain functional connectivity modifications restricted to the salience network

Background

The use of facemasks is one of the consequences of the coronavirus disease 2019 (COVID-19) pandemic. We used resting-state functional magnetic resonance imaging (fMRI) to search for subtle changes in brain functional connectivity, expected notably related to the high-level salience network (SN) and default mode network (DMN).

Methods

Prospective crossover design resting 3-T fMRI study with/without wearing a tight FFP2/KN95 facemask, including 23 community-dwelling male healthy controls aged 29.9 ± 6.9 years (mean ± standard deviation). Physiological parameters, respiration frequency, and heart rate were monitored. The data analysis was performed using the CONN toolbox.

Results

Wearing an FFP2/KN95 facemask did not impact respiration or heart rate but resulted in a significant reduction in functional connectivity between the SN as the seed region and the left middle frontal and precentral gyrus. No difference was found when the DMN, sensorimotor, visual, dorsal attention, or language networks were used as seed regions. In the absence of significant changes of physiological parameter respiration and heart rate, and in the absence of changes in lower-level functional networks, we assume that those subtle modifications are cognitive consequence of wearing facemasks.

Conclusions

The effect of wearing a tight FFP2/KN95 facemask in men is limited to high-level functional networks. Using the SN as seed network, we observed subtle yet significant decreases between the SN and the left middle frontal and precentral gyrus. Our observations suggest that wearing a facemask may change the patterns of functional connectivity with the SN known to be involved in communication, social behavior, and self-awareness.

Enhanced Mirror Neuron Network Activity and Effective Connectivity during Live Interaction Among Female Subjects

Facial expressions are indispensable in daily human communication. Previous neuroimaging studies investigating facial expression processing have presented pre-recorded stimuli and lacked live face-to-face interaction. Our paradigm alternated between presentations of real-time model performance and pre-recorded videos of dynamic facial expressions to participants. Simultaneous functional magnetic resonance imaging (fMRI) and facial electromyography activity recordings, as well as post-scan valence and arousal ratings were acquired from 44 female participants. Live facial expressions enhanced the subjective valence and arousal ratings as well as facial muscular responses. Live performances showed greater engagement of the right posterior superior temporal sulcus (pSTS), right inferior frontal gyrus (IFG), right amygdala and right fusiform gyrus, and modulated the effective connectivity within the right mirror neuron system (IFG, pSTS, and right inferior parietal lobule). A support vector machine algorithm could classify multivoxel activation patterns in brain regions involved in dynamic facial expression processing in the mentalizing networks (anterior and posterior cingulate cortex). These results indicate that live social interaction modulates the activity and connectivity of the right mirror neuron system and enhances spontaneous mimicry, further facilitating emotional contagion.

Restricting movements of lower face leaves recognition of emotional vocalizations intact but introduces a valence positivity bias

Blocking facial mimicry can disrupt recognition of emotion stimuli. Many previous studies have focused on facial expressions, and it remains unclear whether this generalises to other types of emotional expressions. Furthermore, by emphasizing categorical recognition judgments, previous studies neglected the role of mimicry in other processing stages, including dimensional (valence and arousal) evaluations. In the study presented herein, we addressed both issues by asking participants to listen to brief non-verbal vocalizations of four emotion categories (anger, disgust, fear, happiness) and neutral sounds under two conditions. One of the conditions included blocking facial mimicry by creating constant tension on the lower face muscles, in the other condition facial muscles remained relaxed. After each stimulus presentation, participants evaluated sounds' category, valence, and arousal. Although the blocking manipulation did not influence emotion recognition, it led to higher valence ratings in a non-category-specific manner, including neutral sounds. Our findings suggest that somatosensory and motor feedback play a role in the evaluation of affect vocalizations, perhaps introducing a directional bias. This distinction between stimulus recognition, stimulus categorization, and stimulus evaluation is important for understanding what cognitive and emotional processing stages involve somatosensory and motor processes.

A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm

Multiple types of brain-control systems have been applied in the field of rehabilitation. As an alternative scheme for balancing user fatigue and the classification accuracy of brain–computer interface (BCI) systems, facial-expression-based brain control technologies have been proposed in the form of novel BCI systems. Unfortunately, existing machine learning algorithms fail to identify the most relevant features of electroencephalogram signals, which further limits the performance of the classifiers. To address this problem, an improved classification method is proposed for facial-expression-based BCI (FE-BCI) systems, using a convolutional neural network (CNN) combined with a genetic algorithm (GA). The CNN was applied to extract features and classify them. The GA was used for hyperparameter selection to extract the most relevant parameters for classification. To validate the superiority of the proposed algorithm used in this study, various experimental performance results were systematically evaluated, and a trained CNN-GA model was constructed to control an intelligent car in real time. The average accuracy across all subjects was 89.21 ± 3.79%, and the highest accuracy was 97.71 ± 2.07%. The superior performance of the proposed algorithm was demonstrated through offline and online experiments. The experimental results demonstrate that our improved FE-BCI system outperforms the traditional methods.