Blocking facial mimicry affects recognition of facial and body expressions

Facial mimicry interference reduces working memory accuracy for facial emotion expressions

Facial mimicry, the tendency to imitate facial expressions of other individuals, has been shown to play a critical role in the processing of emotion expressions. At the same time, there is evidence suggesting that its role might change when the cognitive demands of the situation increase. In such situations, understanding another person is dependent on working memory. However, whether facial mimicry influences working memory representations for facial emotion expressions is not fully understood. In the present study, we experimentally interfered with facial mimicry by using established behavioral procedures, and investigated how this interference influenced working memory recall for facial emotion expressions. Healthy, young adults (N = 36) performed an emotion expression n-back paradigm with two levels of working memory load, low (1-back) and high (2-back), and three levels of mimicry interference: high, low, and no interference. Results showed that, after controlling for block order and individual differences in the perceived valence and arousal of the stimuli, the high level of mimicry interference impaired accuracy when working memory load was low (1-back) but, unexpectedly, not when load was high (2-back). Working memory load had a detrimental effect on performance in all three mimicry conditions. We conclude that facial mimicry might support working memory for emotion expressions when task load is low, but that the supporting effect possibly is reduced when the task becomes more cognitively challenging.

Individual differences in intracortical inhibition predict action control when facing emotional stimuli

Efficient inhibitory control in the context of prepotent actions is vital. However, such action inhibition may be profoundly influenced by affective states. Interestingly, research indicates that action control can be either impaired or improved by emotional stimuli. Thus, a great deal of confusion surrounds our knowledge of the complex dynamics subtending emotions and action control. Here, we aimed to investigate whether negative stimuli, even when non-consciously presented and task-irrelevant, can affect action control relative to neutral stimuli. Additionally, we tested whether individual differences in intracortical excitability may predict action control capabilities. To address these issues, we asked participants to complete a modified version of the Stop Signal Task (SST) in which fearful or neutral stimuli were subliminally presented before the go signals as primes. Moreover, we assessed participants' resting-state corticospinal excitability, short intracortical inhibition (SICI), and intracortical facilitation (ICF). Results demonstrated better action control capabilities when fearful stimuli were subliminally presented and interindividual SICI predicted stronger action inhibition capabilities. Taken together, these results shed new light on the intricate dynamics between action, consciousness, and motor control, suggesting that intracortical measures can be used as potential biomarkers of reduced motor inhibition in research and clinical settings.

Development and testing of a questionnaire on the expectations and experiences with wearing face masks in inpatient and day hospitals

BACKGROUND

The Covid-19 pandemic made wearing of face masks mandatory in the psychotherapeutic context. Against this background, the present study aimed to compare the expectations of patients undergoing day-hospital or inpatient treatment regarding wearing a mask in psychotherapy before the start of therapy with the final experience after the end of therapy. The study also investigated the extent to which expectations and experiences were influenced by other factors such as socio-demographic characteristics, patients' general attitudes towards wearing a mask, duration of treatment, or mental health diagnoses.

METHODS

Patients' expectations and experiences were recorded using two versions of a self-developed questionnaire: the pre-version, which was administered before the start of therapy and recorded expectations, and the post-version, which was administered after the end of therapy and recorded the final experiences. An exploratory factor analysis was conducted for the questionnaire's pre- and post-version. T-tests for paired samples were calculated to compare the patients' expectations regarding the extracted factors with the final experiences. Bivariate correlations were calculated to explore the association of other potential factors with expectations and experiences.

RESULTS

The exploratory factor analysis revealed a three-factor structure: communication barriers, self-confidence, and infection protection. The communication barriers expected by the patients before the start of the therapy turned out to be significantly higher than ultimately experienced after the therapy. Higher age correlated significantly negatively with expectations and experiences, with less self-confidence expected and experienced in therapy with a mask by older patients. There was a significant positive correlation between the expectations and the duration of treatment. Patients' general attitudes correlated significantly with their expectations and experiences.

CONCLUSION

Based on the results, wearing a mask does not appear to negatively impact the success of psychotherapy from the patient's perspective. However, patient-specific characteristics also appear to play a role in this context.

Facial icons as indexes of emotions and intentions

Various objects and artifacts incorporate representations of faces, encompassing artworks like portraits, as well as ethnographic or industrial artifacts such as masks or humanoid robots. These representations exhibit diverse degrees of human-likeness, serving different functions and objectives. Despite these variations, they share common features, particularly facial attributes that serve as building blocks for facial expressions-an effective means of communicating emotions. To provide a unified conceptualization for this broad spectrum of face representations, we propose the term "facial icons" drawing upon Peirce's semiotic concepts. Additionally, based on these semiotic principles, we posit that facial icons function as indexes of emotions and intentions, and introduce a significant anthropological theory aligning with our proposition. Subsequently, we support our assertions by examining processes related to face and facial expression perception, as well as sensorimotor simulation processes involved in discerning others' mental states, including emotions. Our argumentation integrates cognitive and experimental evidence, reinforcing the pivotal role of facial icons in conveying mental states.

Reading of ingroup politicians' smiles triggers smiling in the corner of one's eyes

Spontaneous smiles in response to politicians can serve as an implicit barometer for gauging electorate preferences. However, it is unclear whether a subtle Duchenne smile-an authentic expression involving the coactivation of the zygomaticus major (ZM) and orbicularis oculi (OO) muscles-would be elicited while reading about a favored politician smiling, indicating a more positive disposition and political endorsement. From an embodied simulation perspective, we investigated whether written descriptions of a politician's smile would trigger morphologically different smiles in readers depending on shared or opposing political orientation. In a controlled reading task in the laboratory, participants were presented with subject-verb phrases describing left and right-wing politicians smiling or frowning. Concurrently, their facial muscular reactions were measured via electromyography (EMG) recording at three facial muscles: the ZM and OO, coactive during Duchenne smiles, and the corrugator supercilii (CS) involved in frowning. We found that participants responded with a Duchenne smile detected at the ZM and OO facial muscles when exposed to portrayals of smiling politicians of same political orientation and reported more positive emotions towards these latter. In contrast, when reading about outgroup politicians smiling, there was a weaker activation of the ZM muscle and no activation of the OO muscle, suggesting a weak non-Duchenne smile, while emotions reported towards outgroup politicians were significantly more negative. Also, a more enhanced frown response in the CS was found for ingroup compared to outgroup politicians' frown expressions. Present findings suggest that a politician's smile may go a long way to influence electorates through both non-verbal and verbal pathways. They add another layer to our understanding of how language and social information shape embodied effects in a highly nuanced manner. Implications for verbal communication in the political context are discussed.

Empathy bodyssence: temporal dynamics of sensorimotor and physiological responses and the subjective experience in synchrony with the other's suffering

Background

Empathy is foundational in our intersubjective interactions, connecting with others across bodily, emotional, and cognitive dimensions. Previous evidence suggests that observing individuals in painful situations elicits whole bodily responses, unveiling the interdependence of the body and empathy. Although the role of the body has been extensively described, the temporal structure of bodily responses and its association with the comprehension of subjective experiences remain unclear.

Objective

Building upon the enactive approach, our study introduces and examines "bodyssence," a neologism formed from "body" and "essence." Our primary goal is to analyze the temporal dynamics, physiological, and phenomenological elements in synchrony with the experiences of sportspersons suffering physical accidents.

Methods

Using the empirical 5E approach, a refinement of Varela's neurophenomenological program, we integrated both objective third-person measurements (postural sway, electrodermal response, and heart rate) and first-person descriptions (phenomenological data). Thirty-five participants watched videos of sportspersons experiencing physical accidents during extreme sports practice, as well as neutral videos, while standing on a force platform and wearing electrodermal and heart electrodes. Subsequently, micro-phenomenological interviews were conducted.

Results

Bodyssence is composed of three distinct temporal dynamics. Forefeel marks the commencement phase, encapsulating the body's pre-reflective consciousness as participants anticipate impending physical accidents involving extreme sportspersons, manifested through minimal postural movement and high heart rate. Fullfeel, capturing the zenith of empathetic engagement, is defined by profound negative emotions, and significant bodily and kinesthetic sensations, with this stage notably featuring an increase in postural movement alongside a reduction in heart rate. In the Reliefeel phase, participants report a decrease in emotional intensity, feeling a sense of relief, as their postural control starts to reach a state of equilibrium, and heart rate remaining low. Throughout these phases, the level of electrodermal activity consistently remains high.

Conclusion

This study through an enactive approach elucidates the temporal attunement of bodily experience to the pain experienced by others. The integration of both first and third-person perspectives through an empirical 5E approach reveals the intricate nature of bodyssence, offering an innovative approach to understanding the dynamic nature of empathy.

Task-related modulation of motor response to emotional bodies: A TMS motor-evoked potential study

Exposure to emotional body postures during perceptual decision-making tasks has been linked to transient suppression of motor reactivity, supporting the monitoring of emotionally relevant information. However, it remains unclear whether this effect occurs implicitly, i.e., when emotional information is irrelevant to the task. To investigate this issue, we used single-pulse transcranial magnetic stimulation (TMS) to assess motor excitability while healthy participants were asked to categorize pictures of body expressions as emotional or neutral (emotion recognition task) or as belonging to a male or a female actor (gender recognition task) while receiving TMS over the motor cortex at 100 and 125 ms after picture onset. Results demonstrated that motor-evoked potentials (MEPs) were reduced for emotional body postures relative to neutral postures during the emotion recognition task. Conversely, MEPs increased for emotional body postures relative to neutral postures during the gender recognition task. These findings indicate that motor inhibition, contingent upon observing emotional body postures, is selectively associated with actively monitoring emotional features. In contrast, observing emotional body postures prompts motor facilitation when task-relevant features are non-emotional. These findings contribute to embodied cognition models that link emotion perception and action tendencies.

Social cognition in hyperkinetic movement disorders: a systematic review

Numerous lines of research indicate that our social brain involves a network of cortical and subcortical brain regions that are responsible for sensing and controlling body movements. However, it remains unclear whether movement disorders have a systematic impact on social cognition. To address this question, we conducted a systematic review examining the influence of hyperkinetic movement disorders (including Huntington disease, Tourette syndrome, dystonia, and essential tremor) on social cognition. Following the PRISMA guidelines and registering the protocol in the PROSPERO database (CRD42022327459), we analyzed 50 published studies focusing on theory of mind (ToM), social perception, and empathy. The results from these studies provide evidence of impairments in ToM and social perception in all hyperkinetic movement disorders, particularly during the recognition of negative emotions. Additionally, individuals with Huntington's Disease and Tourette syndrome exhibit empathy disorders. These findings support the functional role of subcortical structures (such as the basal ganglia and cerebellum), which are primarily responsible for movement disorders, in deficits related to social cognition.

Effect of Action Units, Viewpoint and Immersion on Emotion Recognition Using Dynamic Virtual Faces

Facial affect recognition is a critical skill in human interactions that is often impaired in psychiatric disorders. To address this challenge, tests have been developed to measure and train this skill. Recently, virtual human (VH) and virtual reality (VR) technologies have emerged as novel tools for this purpose. This study investigates the unique contributions of different factors in the communication and perception of emotions conveyed by VHs. Specifically, it examines the effects of the use of action units (AUs) in virtual faces, the positioning of the VH (frontal or mid-profile), and the level of immersion in the VR environment (desktop screen versus immersive VR). Thirty-six healthy subjects participated in each condition. Dynamic virtual faces (DVFs), VHs with facial animations, were used to represent the six basic emotions and the neutral expression. The results highlight the important role of the accurate implementation of AUs in virtual faces for emotion recognition. Furthermore, it is observed that frontal views outperform mid-profile views in both test conditions, while immersive VR shows a slight improvement in emotion recognition. This study provides novel insights into the influence of these factors on emotion perception and advances the understanding and application of these technologies for effective facial emotion recognition training.

Increasing associative plasticity in temporo-occipital back-projections improves visual perception of emotions

The posterior superior temporal sulcus (pSTS) is a critical node in a network specialized for perceiving emotional facial expressions that is reciprocally connected with early visual cortices (V1/V2). Current models of perceptual decision-making increasingly assign relevance to recursive processing for visual recognition. However, it is unknown whether inducing plasticity into reentrant connections from pSTS to V1/V2 impacts emotion perception. Using a combination of electrophysiological and neurostimulation methods, we demonstrate that strengthening the connectivity from pSTS to V1/V2 selectively increases the ability to perceive facial expressions associated with emotions. This behavior is associated with increased electrophysiological activity in both these brain regions, particularly in V1/V2, and depends on specific temporal parameters of stimulation that follow Hebbian principles. Therefore, we provide evidence that pSTS-to-V1/V2 back-projections are instrumental to perception of emotion from facial stimuli and functionally malleable via manipulation of associative plasticity.

Mask wearing provides psychological ease but does not affect facial expression intensity estimation

During the COVID-19 pandemic, wearing a face mask became a global daily practice. Japanese people were already accustomed to wearing masks due to their collectivistic culture, which prioritizes conformity and group harmony. In such a culture, where individuals are concerned about how others perceive them and their actions, wearing masks can be a self-protective action to prevent, escape, or reduce the severity of perceived negative feelings. Previous studies indicate that people experiencing anxiety tend to have negative biases when evaluating emotional expressions on faces. Therefore, we hypothesized that wearing a mask can reduce the negative feelings caused by social pressure, emotion processing, especially intensity perception. While our findings confirmed that wearing a mask reduced negative feelings caused by social pressure, there was no significant change in emotion intensity recognition performance. This null result might be attributed to the small effect size of the association between negative bias in emotion processing and an individual's state. In future studies, it would be valuable to include participants from non-collectivistic cultures to gain a broader understanding of the impact of wearing masks on emotion processing.

Moving beyond the lab: investigating empathy through the Empirical 5E approach

Empathy is a complex and multifaceted phenomenon that plays a crucial role in human social interactions. Recent developments in social neuroscience have provided valuable insights into the neural underpinnings and bodily mechanisms underlying empathy. This methodology often prioritizes precision, replicability, internal validity, and confound control. However, fully understanding the complexity of empathy seems unattainable by solely relying on artificial and controlled laboratory settings, while overlooking a comprehensive view of empathy through an ecological experimental approach. In this article, we propose articulating an integrative theoretical and methodological framework based on the 5E approach (the "E"s stand for embodied, embedded, enacted, emotional, and extended perspectives of empathy), highlighting the relevance of studying empathy as an active interaction between embodied agents, embedded in a shared real-world environment. In addition, we illustrate how a novel multimodal approach including mobile brain and body imaging (MoBi) combined with phenomenological methods, and the implementation of interactive paradigms in a natural context, are adequate procedures to study empathy from the 5E approach. In doing so, we present the Empirical 5E approach (E5E) as an integrative scientific framework to bridge brain/body and phenomenological attributes in an interbody interactive setting. Progressing toward an E5E approach can be crucial to understanding empathy in accordance with the complexity of how it is experienced in the real world.

"When You're Smiling": How Posed Facial Expressions Affect Visual Recognition of Emotions

Facial imitation occurs automatically during the perception of an emotional facial expression, and preventing it may interfere with the accuracy of emotion recognition. In the present fMRI study, we evaluated the effect of posing a facial expression on the recognition of ambiguous facial expressions. Since facial activity is affected by various factors, such as empathic aptitudes, the Interpersonal Reactivity Index (IRI) questionnaire was administered and scores were correlated with brain activity. Twenty-six healthy female subjects took part in the experiment. The volunteers were asked to pose a facial expression (happy, disgusted, neutral), then to watch an ambiguous emotional face, finally to indicate whether the emotion perceived was happiness or disgust. As stimuli, blends of happy and disgusted faces were used. Behavioral results showed that posing an emotional face increased the percentage of congruence with the perceived emotion. When participants posed a facial expression and perceived a non-congruent emotion, a neural network comprising bilateral anterior insula was activated. Brain activity was also correlated with empathic traits, particularly with empathic concern, fantasy and personal distress. Our findings support the idea that facial mimicry plays a crucial role in identifying emotions, and that empathic emotional abilities can modulate the brain circuits involved in this process.

A Utilização da Imitação Facial em Tarefas de Reconhecimento de Expressões Emocionais

Resumo A imitação facial é um comportamento involuntário capaz de facilitar a transmissão de informações não verbais relevantes em diferentes contextos sociais. Este estudo teve por objetivo analisar a capacidade de reconhecimento de expressões emocionais enquanto o observador tensiona a própria face ou imita a face-alvo. A hipótese utilizada foi a de que indivíduos que tensionam a própria face terão menor probabilidade de acertos na execução das tarefas de reconhecimento de expressões emocionais e aqueles que imitam a expressão terão uma maior probabilidade de acertos na execução das mesmas tarefas. A amostra foi composta por 30 participantes, divididos em dois grupos experimentais: o Grupo Imitação (GI) e o Grupo Ruído (GR), ambos com 18 participantes do sexo feminino e 12 do sexo masculino. O experimento consistiu em apresentar fotos de atores expressando facialmente uma emoção básica por 10 segundos. Neste período, os participantes deveriam, então, observar ou intervir facialmente, imitando ou tensionando a própria face (de acordo com o grupo alocado, Imitação ou Ruído). Após os 10 segundos executando a instrução (observar, imitar ou interferir), o participante deveria responder - entre as opções alegria, tristeza, nojo, raiva, surpresa e medo - a emoção correspondente à imagem. Os resultados apresentaram diferenças significativas quando comparadas as tarefas de tensionar ou imitar a face-alvo, sugerindo que a alteração da própria face do observador pode influenciar durante o desempenho de uma tarefa de reconhecimento de emoções em faces.

Mask-wearing selectivity alters observers’ face perception

Face masks became prevalent across the globe as an efficient tool to stop the spread of COVID-19. A host of studies already demonstrated that masks lead to changes in facial identification and emotional expression processing. These changes were documented across ages and were consistent even with the increased exposure to masked faces. Notably, mask-wearing also changes the state of the observers in regard to their own bodies and other agents. Previous research has already demonstrated a plausible association between observers’ states and their perceptual behaviors. Thus, an outstanding question is whether mask-wearing would alter face recognition abilities. To address this question, we conducted a set of experiments in which participants were asked to recognize non-masked faces (Experiment 1), masked faces (Experiment 2) and novel objects (Experiment 3) while they were either masked or unmasked. Mask wearing hindered face perception abilities but did not modulate object recognition ability. Finally, we demonstrated that the decrement in face perception ability relied on wearing the mask on distinctive facial features (Experiment 4). Together, these findings reveal a novel effect of mask-wearing on face recognition. We discuss these results considering the plausible effect of somatosensory stimulation on visual processing as well as the effect of involuntary perspective taking.

Degenerate pathway for processing smile and other emotional expressions in congenital facial palsy: an hdEEG investigation

Influential theoretical models argue that an internal simulation mechanism (motor or sensorimotor simulation) supports the recognition of facial expressions. However, despite numerous converging sources of evidence, recent studies testing patients with congenital facial palsy (i.e. Moebius syndrome) seem to refute these theoretical models. However, these results do not consider the principles of neuroplasticity and degeneracy that could support the involvement of an alternative neural processing pathway in these patients. In the present study, we tested healthy participants and participants with Moebius syndrome in a highly sensitive facial expression discrimination task and concomitant high-density electroencephalographic recording. The results, both at the scalp and source levels, indicate the activation of two different pathways of facial expression processing in healthy participants and participants with Moebius syndrome, compatible, respectively, with a dorsal pathway that includes premotor areas and a ventral pathway. Therefore, these results support the reactivation of sensorimotor representations of facial expressions (i.e. simulation) in healthy subjects, in the place of an alternative processing pathway in subjects with congenital facial palsy.

This article is part of the theme issue ‘Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience’.

Restricting movements of lower face leaves recognition of emotional vocalizations intact but introduces a valence positivity bias

Blocking facial mimicry can disrupt recognition of emotion stimuli. Many previous studies have focused on facial expressions, and it remains unclear whether this generalises to other types of emotional expressions. Furthermore, by emphasizing categorical recognition judgments, previous studies neglected the role of mimicry in other processing stages, including dimensional (valence and arousal) evaluations. In the study presented herein, we addressed both issues by asking participants to listen to brief non-verbal vocalizations of four emotion categories (anger, disgust, fear, happiness) and neutral sounds under two conditions. One of the conditions included blocking facial mimicry by creating constant tension on the lower face muscles, in the other condition facial muscles remained relaxed. After each stimulus presentation, participants evaluated sounds' category, valence, and arousal. Although the blocking manipulation did not influence emotion recognition, it led to higher valence ratings in a non-category-specific manner, including neutral sounds. Our findings suggest that somatosensory and motor feedback play a role in the evaluation of affect vocalizations, perhaps introducing a directional bias. This distinction between stimulus recognition, stimulus categorization, and stimulus evaluation is important for understanding what cognitive and emotional processing stages involve somatosensory and motor processes.

Investigating the Relationship between Facial Mimicry and Empathy

Facial expressions play a key role in interpersonal communication when it comes to negotiating our emotions and intentions, as well as interpreting those of others. Research has shown that we can connect to other people better when we exhibit signs of empathy and facial mimicry. However, the relationship between empathy and facial mimicry is still debated. Among the factors contributing to the difference in results across existing studies is the use of different instruments for measuring both empathy and facial mimicry, as well as often ignoring the differences across various demographic groups. This study first looks at the differences in the empathetic abilities of people across different demographic groups based on gender, ethnicity and age. The empathetic ability is measured based on the Empathy Quotient, capturing a balanced representation of both emotional and cognitive empathy. Using statistical and machine learning methods, this study then investigates the correlation between the empathetic ability and facial mimicry of subjects in response to images portraying different emotions displayed on a computer screen. Unlike the existing studies measuring facial mimicry using electromyography, this study employs a technology detecting facial expressions based on video capture and deep learning. This choice was made in the context of increased online communication during and after the COVID-19 pandemic. The results of this study confirm the previously reported difference in the empathetic ability between females and males. However, no significant difference in empathetic ability was found across different age and ethnic groups. Furthermore, no strong correlation was found between empathy and facial reactions to faces portraying different emotions shown on a computer screen. Overall, the results of this study can be used to inform the design of online communication technologies and tools for training empathy team leaders, educators, social and healthcare providers.

Mirror neurons 30 years later: implications and applications

Mirror neurons (MNs) were first described in a seminal paper in 1992 as a class of monkey premotor cells discharging during both action execution and observation. Despite their debated origin and function, recent studies in several species, from birds to humans, revealed that beyond MNs properly so called, a variety of cell types distributed among multiple motor, sensory, and emotional brain areas form a 'mirror mechanism' more complex and flexible than originally thought, which has an evolutionarily conserved role in social interaction. Here, we trace the current limits and envisage the future trends of this discovery, showing that it inspired translational research and the development of new neurorehabilitation approaches, and constitutes a point of no return in social and affective neuroscience.

An examination of autonomic and facial responses to prototypical facial emotion expressions in psychopathy

Meta-analyses have found that people high in psychopathy categorize (or “recognize”) others’ prototypical facial emotion expressions with reduced accuracy. However, these have been contested with remaining questions regarding the strength, specificity, and mechanisms of this ability in psychopathy. In addition, few studies have tested holistically whether psychopathy is related to reduced facial mimicry or autonomic arousal in response to others’ dynamic facial expressions. Therefore, the current study presented 6 s videos of a target person making prototypical emotion expressions (anger, fear, disgust, sadness, joy, and neutral) to N = 88 incarcerated adult males while recording facial electromyography, skin conductance response (SCR), and heart rate. Participants identified the emotion category and rated the valence and intensity of the target person’s emotion. Psychopathy was assessed via the Psychopathy Checklist-Revised (PCL-R). We predicted that overall PCL-R scores and scores for the interpersonal/affective traits, in particular, would be related to reduced emotion categorization accuracy, valence ratings, intensity ratings, facial mimicry, SCR amplitude, and cardiac deceleration in response to the prototypical facial emotion expressions. In contrast to our hypotheses, PCL-R scores were unrelated to emotion categorization accuracy, valence ratings, and intensity ratings. Stimuli failed to elicit facial mimicry from the full sample, which does not allow drawing conclusions about the relationship between psychopathy and facial mimicry. However, participants displayed general autonomic arousal responses, but not to prototypical emotion expressions per se. PCL-R scores were also unrelated to SCR and cardiac deceleration. These findings failed to identify aberrant behavioral and physiological responses to prototypical facial emotion expressions in relation to psychopathy.

Facial emotion perception abilities are related to grey matter volume in the culmen of cerebellum anterior lobe in drug-naïve patients with first-episode schizophrenia

Impaired capability for understanding and interpreting the expressions on other people's faces manifests itself as a core feature of schizophrenia, contributing to social dysfunction. With the purpose of better understanding of the neurobiological basis of facial emotion perception deficits in schizophrenia, we investigated facial emotion perception abilities and regional structural brain abnormalities in drug-naïve patients with first-episode schizophrenia, and then examined the correlation between them. Fifty-two drug-naive patients with first-episode schizophrenia and 29 group-matched healthy controls were examined for facial emotion perception abilities assessed with the Facial Emotion Categorization and performed magnetic resonance imaging. The Facial Emotion Categorization data were inserted into a logistic function model so as to calculate shift point and slope as outcome measurements. Voxel-based morphometry was applied to investigate regional grey matter volume (GMV) alterations. The relationship between facial emotion perception and GMV was explored in patients using voxel-wise correlation analysis within brain regions that showed a significant GMV alterations in patients compared with controls. The schizophrenic patients performed differently on Facial Emotion Categorization tasks from the controls and presented a higher shift point and a steeper slope. Relative to the controls, patients showed GMV reductions in the superior temporal gyrus, middle occipital gyrus, parahippocampa gyrus, posterior cingulate, the culmen of cerebellum anterior lobe, cerebellar tonsil, and the declive of cerebellum posterior lobe. Importantly, abnormal performance on Facial Emotion Categorization was found correlated with GMV alterations in the culmen of cerebellum anterior lobe in schizophrenia. This study suggests that reduced GMV in the culmen of cerebellum anterior lobe occurs in first-episode schizophrenia, constituting a potential neuropathological basis for the impaired facial emotion perception in schizophrenia.

Sensorimotor inhibition during emotional processing

Visual processing of emotional stimuli has been shown to engage complex cortical and subcortical networks, but it is still unclear how it affects sensorimotor integration processes. To fill this gap, here, we used a TMS protocol named short-latency afferent inhibition (SAI), capturing sensorimotor interactions, while healthy participants were observing emotional body language (EBL) and International Affective Picture System (IAPS) stimuli. Participants were presented with emotional (fear- and happiness-related) or non-emotional (neutral) EBL and IAPS stimuli while SAI was tested at 120 ms and 300 ms after pictures presentation. At the earlier time point (120 ms), we found that fear-related EBL and IAPS stimuli selectively enhanced SAI as indexed by the greater inhibitory effect of somatosensory afferents on motor excitability. Larger early SAI enhancement was associated with lower scores at the Behavioural Inhibition Scale (BIS). At the later time point (300 ms), we found a generalized SAI decrease for all kind of stimuli (fear, happiness or neutral). Because the SAI index reflects integrative activity of cholinergic sensorimotor circuits, our findings suggest greater sensitivity of such circuits during early (120 ms) processing of threat-related information. Moreover, the correlation with BIS score may suggest increased attention and sensory vigilance in participants with greater anxiety-related dispositions. In conclusion, the results of this study show that sensorimotor inhibition is rapidly enhanced while processing threatening stimuli and that SAI protocol might be a valuable option in evaluating emotional-motor interactions in physiological and pathological conditions.

Audio and visual speech emotion activate the left pre-supplementary motor area

Sensorimotor brain areas have been implicated in the recognition of emotion expressed on the face and through nonverbal vocalizations. However, no previous study has assessed whether sensorimotor cortices are recruited during the perception of emotion in speech-a signal that includes both audio (speech sounds) and visual (facial speech movements) components. To address this gap in the literature, we recruited 24 participants to listen to speech clips produced in a way that was either happy, sad, or neutral in expression. These stimuli also were presented in one of three modalities: audio-only (hearing the voice but not seeing the face), video-only (seeing the face but not hearing the voice), or audiovisual. Brain activity was recorded using electroencephalography, subjected to independent component analysis, and source-localized. We found that the left presupplementary motor area was more active in response to happy and sad stimuli than neutral stimuli, as indexed by greater mu event-related desynchronization. This effect did not differ by the sensory modality of the stimuli. Activity levels in other sensorimotor brain areas did not differ by emotion, although they were greatest in response to visual-only and audiovisual stimuli. One possible explanation for the pre-SMA result is that this brain area may actively support speech emotion recognition by using our extensive experience expressing emotion to generate sensory predictions that in turn guide our perception.

Simulated proximity enhances perceptual and physiological responses to emotional facial expressions

Physical proximity is important in social interactions. Here, we assessed whether simulated physical proximity modulates the perceived intensity of facial emotional expressions and their associated physiological signatures during observation or imitation of these expressions. Forty-four healthy volunteers rated intensities of dynamic angry or happy facial expressions, presented at two simulated locations, proximal (0.5 m) and distant (3 m) from the participants. We tested whether simulated physical proximity affected the spontaneous (in the observation task) and voluntary (in the imitation task) physiological responses (activity of the corrugator supercilii face muscle and pupil diameter) as well as subsequent ratings of emotional intensity. Angry expressions provoked relative activation of the corrugator supercilii muscle and pupil dilation, whereas happy expressions induced a decrease in corrugator supercilii muscle activity. In proximal condition, these responses were enhanced during both observation and imitation of the facial expressions, and were accompanied by an increase in subsequent affective ratings. In addition, individual variations in condition related EMG activation during imitation of angry expressions predicted increase in subsequent emotional ratings. In sum, our results reveal novel insights about the impact of physical proximity in the perception of emotional expressions, with early proximity-induced enhancements of physiological responses followed by an increased intensity rating of facial emotional expressions.

Influence of bodily resonances on emotional prosody perception

Introduction

Emotional prosody is defined as suprasegmental and segmental changes in the human voice and related acoustic parameters that can inform the listener about the emotional state of the speaker. While the processing of emotional prosody is well represented in the literature, the mechanism of embodied cognition in emotional voice perception is very little studied. This study aimed to investigate the influence of induced bodily vibrations-through a vibrator placed close to the vocal cords-in the perception of emotional vocalizations. The main hypothesis was that induced body vibrations would constitute a potential interoceptive feedback that can influence the auditory perception of emotions. It was also expected that these effects would be greater for stimuli that are more ambiguous.

Methods

Participants were presented with emotional vocalizations expressing joy or anger which varied from low-intensity vocalizations, considered as ambiguous, to high-intensity ones, considered as non-ambiguous. Vibrations were induced simultaneously in half of the trials and expressed joy or anger congruently with the voice stimuli. Participants had to evaluate each voice stimulus using four visual analog scales (joy, anger, and surprise, sadness as control scales).

Results

A significant effect of the vibrations was observed on the three behavioral indexes-discrimination, confusion and accuracy-with vibrations confusing rather than facilitating vocal emotion processing.

Conclusion

Over all, this study brings new light on a poorly documented topic, namely the potential use of vocal cords vibrations as an interoceptive feedback allowing humans to modulate voice production and perception during social interactions.

Feelings first? Sex differences in affective and cognitive processes in emotion recognition

The recognition of emotional expressions is important for social understanding and interaction, but findings on the relationship between emotion recognition, empathy, and theory of mind, as well as sex differences in these relationships, have been inconsistent. This may reflect the relative involvement of affective and cognitive processes at different stages of emotion recognition and in different experimental paradigms. In this study, images of faces were morphed from neutral to full expression of five basic emotions (anger, disgust, fear, happiness, and sadness), which participants were asked to identify as quickly and accurately as possible. Accuracy and response times from healthy males (n = 46) and females (n = 43) were analysed in relation to self-reported empathy (Empathy Quotient; EQ) and mentalising/theory of mind (Reading the Mind in the Eyes Test). Females were faster and more accurate than males in recognising dynamic emotions. Linear mixed-effects modelling showed that response times were inversely related to the emotional empathy subscale of the EQ, but this was accounted for by a female advantage on both measures. Accuracy was unrelated to EQ scores but was predicted independently by sex and Eyes Test scores. These findings suggest that rapid processing of dynamic emotional expressions is strongly influenced by sex, which may reflect the greater involvement of affective processes at earlier stages of emotion recognition.

Love and Imitation: Facial Mimicry and Relationship Satisfaction in Younger and Older Couples

OBJECTIVE

Social mimicry, the imitation of one's conversation partner, is associated with empathy, liking, and affiliation. Because previous research has mainly focused on young adulthood and zero acquaintances, little is known about age differences in mimicry and its role for romantic relationships.

METHODS

In this study, 37 younger and 41 older couples talked about an ongoing problem faced by one of the partners while being video-recorded. Three independent observers assessed partners' facial mimicry. After the conversation, couples evaluated the quality of their relationship.

RESULTS

Younger couples imitated each other more than older couples. The link between mimicry and relationship quality was mixed. While facial mimicry was not linked to subjective closeness in either younger or older couples, there was a significant positive association between mimicry and relationship satisfaction in young, but not older, couples.

DISCUSSION

Overall, the results suggest that facial mimicry is a social process that becomes less prevalent and might differ in their function as individuals age.

Mask exposure during COVID-19 changes emotional face processing

Faces are one of the key ways that we obtain social information about others. They allow people to identify individuals, understand conversational cues, and make judgements about others' mental states. When the COVID-19 pandemic hit the United States, widespread mask-wearing practices were implemented, causing a shift in the way Americans typically interact. This introduction of masks into social exchanges posed a potential challenge-how would people make these important inferences about others when a large source of information was no longer available? We conducted two studies that investigated the impact of mask exposure on emotion perception. In particular, we measured how participants used facial landmarks (visual cues) and the expressed valence and arousal (affective cues), to make similarity judgements about pairs of emotion faces. Study 1 found that in August 2020, participants with higher levels of mask exposure used cues from the eyes to a greater extent when judging emotion similarity than participants with less mask exposure. Study 2 measured participants' emotion perception in both April and September 2020 -before and after widespread mask adoption-in the same group of participants to examine changes in the use of facial cues over time. Results revealed an overall increase in the use of visual cues from April to September. Further, as mask exposure increased, people with the most social interaction showed the largest increase in the use of visual facial cues. These results provide evidence that a shift has occurred in how people process faces such that the more people are interacting with others that are wearing masks, the more they have learned to focus on visual cues from the eye area of the face.

Altering Facial Movements Abolishes Neural Mirroring of Facial Expressions

People tend to automatically imitate others' facial expressions of emotion. That reaction, termed "facial mimicry" has been linked to sensorimotor simulation-a process in which the observer's brain recreates and mirrors the emotional experience of the other person, potentially enabling empathy and deep, motivated processing of social signals. However, the neural mechanisms that underlie sensorimotor simulation remain unclear. This study tests how interfering with facial mimicry by asking participants to hold a pen in their mouth influences the activity of the human mirror neuron system, indexed by the desynchronization of the EEG mu rhythm. This response arises from sensorimotor brain areas during observed and executed movements and has been linked with empathy. We recorded EEG during passive viewing of dynamic facial expressions of anger, fear, and happiness, as well as nonbiological moving objects. We examine mu desynchronization under conditions of free versus altered facial mimicry and show that desynchronization is present when adult participants can freely move but not when their facial movements are inhibited. Our findings highlight the importance of motor activity and facial expression in emotion communication. They also have important implications for behaviors that involve occupying or hiding the lower part of the face.

Early Right Motor Cortex Response to Happy and Fearful Facial Expressions: A TMS Motor-Evoked Potential Study

The ability to rapidly process others' emotional signals is crucial for adaptive social interactions. However, to date it is still unclear how observing emotional facial expressions affects the reactivity of the human motor cortex. To provide insights on this issue, we employed single-pulse transcranial magnetic stimulation (TMS) to investigate corticospinal motor excitability. Healthy participants observed happy, fearful and neutral pictures of facial expressions while receiving TMS over the left or right motor cortex at 150 and 300 ms after picture onset. In the early phase (150 ms), we observed an enhancement of corticospinal excitability for the observation of happy and fearful emotional faces compared to neutral expressions specifically in the right hemisphere. Interindividual differences in the disposition to experience aversive feelings (personal distress) in interpersonal emotional contexts predicted the early increase in corticospinal excitability for emotional faces. No differences in corticospinal excitability were observed at the later time (300 ms) or in the left M1. These findings support the notion that emotion perception primes the body for action and highlights the role of the right hemisphere in implementing a rapid and transient facilitatory response to emotional arousing stimuli, such as emotional facial expressions.

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies.

Effects of posed smiling on memory for happy and sad facial expressions

The perception and storage of facial emotional expressions constitutes an important human skill that is essential for our daily social interactions. While previous research revealed that facial feedback can influence the perception of facial emotional expressions, it is unclear whether facial feedback also plays a role in memory processes of facial emotional expressions. In the present study we investigated the impact of facial feedback on the performance in emotional visual working memory (WM). For this purpose, 37 participants underwent a classical facial feedback manipulation (FFM) (holding a pen with the teeth—inducing a smiling expression vs. holding a pen with the non-dominant hand—as a control condition) while they performed a WM task on varying intensities of happy or sad facial expressions. Results show that the smiling manipulation improved memory performance selectively for happy faces, especially for highly ambiguous facial expressions. Furthermore, we found that in addition to an overall negative bias specifically for happy faces (i.e. happy faces are remembered as more negative than they initially were), FFM induced a positivity bias when memorizing emotional facial information (i.e. faces were remembered as being more positive than they actually were). Finally, our data demonstrate that men were affected more by FFM: during induced smiling men showed a larger positive bias than women did. These data demonstrate that facial feedback not only influences our perception but also systematically alters our memory of facial emotional expressions.

Authentic and posed emotional vocalizations trigger distinct facial responses

The ability to recognize the emotions of others is a crucial skill. In the visual modality, sensorimotor mechanisms provide an important route for emotion recognition. Perceiving facial expressions often evokes activity in facial muscles and in motor and somatosensory systems, and this activity relates to performance in emotion tasks. It remains unclear whether and how similar mechanisms extend to audition. Here we examined facial electromyographic and electrodermal responses to nonverbal vocalizations that varied in emotional authenticity. Participants (N = 100) passively listened to laughs and cries that could reflect an authentic or a posed emotion. Bayesian mixed models indicated that listening to laughter evoked stronger facial responses than listening to crying. These responses were sensitive to emotional authenticity. Authentic laughs evoked more activity than posed laughs in the zygomaticus and orbicularis, muscles typically associated with positive affect. We also found that activity in the orbicularis and corrugator related to subjective evaluations in a subsequent authenticity perception task. Stronger responses in the orbicularis predicted higher perceived laughter authenticity. Stronger responses in the corrugator, a muscle associated with negative affect, predicted lower perceived laughter authenticity. Moreover, authentic laughs elicited stronger skin conductance responses than posed laughs. This arousal effect did not predict task performance, however. For crying, physiological responses were not associated with authenticity judgments. Altogether, these findings indicate that emotional authenticity affects peripheral nervous system responses to vocalizations. They also point to a role of sensorimotor mechanisms in the evaluation of authenticity in the auditory modality.

Blocking facial mimicry during binocular rivalry modulates visual awareness of faces with a neutral expression

Several previous studies have interfered with the observer’s facial mimicry during a variety of facial expression recognition tasks providing evidence in favor of the role of facial mimicry and sensorimotor activity in emotion processing. In this theoretical context, a particularly intriguing facet has been neglected, namely whether blocking facial mimicry modulates conscious perception of facial expressions of emotions. To address this issue, we used a binocular rivalry paradigm, in which two dissimilar stimuli presented to the two eyes alternatingly dominate conscious perception. On each trial, female participants (N = 32) were exposed to a rivalrous pair of a neutral and a happy expression of the same individual through anaglyph glasses in two conditions: in one, they could freely use their facial mimicry, in the other they had to keep a chopstick between their lips, constraining the mobility of the zygomatic muscle and producing ‘noise’ for sensorimotor simulation. We found that blocking facial mimicry affected the perceptual dominance in terms of cumulative time favoring neutral faces, but it did not change the time before the first dominance was established. Taken together, our results open a door to future investigation of the intersection between sensorimotor simulation models and conscious perception of emotional facial expressions.

A New Neurorehabilitative Postsurgery Intervention for Facial Palsy Based on Smile Observation and Hand-Mouth Motor Synergies

Objective

To perform a preliminary test of a new rehabilitation treatment (FIT-SAT), based on mirror mechanisms, for gracile muscles after smile surgery.

Method

A pre- and postsurgery longitudinal design was adopted to study the efficacy of FIT-SAT. Four patients with bilateral facial nerve paralysis (Moebius syndrome) were included. They underwent two surgeries with free muscle transfers, one year apart from each other. The side of the face first operated on was rehabilitated with the traditional treatment, while the second side was rehabilitated with FIT-SAT. The FIT-SAT treatment includes video clips of an actor performing a unilateral or a bilateral smile to be imitated (FIT condition). In addition to this, while smiling, the participants close their hand in order to exploit the overlapped cortical motor representation of the hand and the mouth, which may facilitate the synergistic activity of the two effectors during the early phases of recruitment of the transplanted muscles (SAT). The treatment was also aimed at avoiding undesired movements such as teeth grinding. Discussion. Results support FIT-SAT as a viable alternative for smile rehabilitation after free muscle transfer. We propose that the treatment potentiates the effect of smile observation by activating the same neural structures responsible for the execution of the smile and therefore by facilitating its production. Closing of the hand induces cortical recruitment of hand motor neurons, recruiting the transplanted muscles, and reducing the risk of associating other unwanted movements such as teeth clenching to the smile movements.

Facial emotion mimicry in older adults with and without cognitive impairments due to Alzheimer's disease

Facial expression of humans is one of the main channels of everyday communication. The reported research work investigated communication regarding the pattern of emotional expression of healthy older adults and with mild cognitive impairments (MCI) or Alzheimer's disease (AD). It focuses on mimicking of displayed emotional facial expression on a sample of 25 older adults (healthy, MCI and AD patients). The adequacy of the patients' individual facial expressions in six basic emotions was measured with the Kinect 3D recording of the participants' facial expressions and compared to their own typical emotional facial expressions. The reactions were triggered by mimicking 49 still pictures of emotional facial expressions. No statistically significant differences in terms of frequency nor adequacy of emotional facial expression were reported in healthy and MCI groups. Unique patterns of emotional expressions have been observed in the AD group. Further investigating the pattern of older adults' facial expression may decrease the misunderstandings and increase the quality of life of the patients.

Investigating automatic emotion processing in boys with autism via eye tracking and facial mimicry recordings

Difficulties in automatic emotion processing in individuals with autism spectrum disorder (ASD) might remain concealed in behavioral studies due to compensatory strategies. To gain more insight in the mechanisms underlying facial emotion recognition, we recorded eye tracking and facial mimicry data of 20 school-aged boys with ASD and 20 matched typically developing controls while performing an explicit emotion recognition task. Proportional looking times to specific face regions (eyes, nose, and mouth) and face exploration dynamics were analyzed. In addition, facial mimicry was assessed. Boys with ASD and controls were equally capable to recognize expressions and did not differ in proportional looking times, and number and duration of fixations. Yet, specific facial expressions elicited particular gaze patterns, especially within the control group. Both groups showed similar face scanning dynamics, although boys with ASD demonstrated smaller saccadic amplitudes. Regarding the facial mimicry, we found no emotion specific facial responses and no group differences in the responses to the displayed facial expressions. Our results indicate that boys with and without ASD employ similar eye gaze strategies to recognize facial expressions. Smaller saccadic amplitudes in boys with ASD might indicate a less exploratory face processing strategy. Yet, this slightly more persistent visual scanning behavior in boys with ASD does not imply less efficient emotion information processing, given the similar behavioral performance. Results on the facial mimicry data indicate similar facial responses to emotional faces in boys with and without ASD. LAY SUMMARY: We investigated (i) whether boys with and without autism apply different face exploration strategies when recognizing facial expressions and (ii) whether they mimic the displayed facial expression to a similar extent. We found that boys with and without ASD recognize facial expressions equally well, and that both groups show similar facial reactions to the displayed facial emotions. Yet, boys with ASD visually explored the faces slightly less than the boys without ASD.

Modulation of Response Times During Processing of Emotional Body Language

The investigation of how humans perceive and respond to emotional signals conveyed by the human body has been for a long time secondary compared with the investigation of facial expressions and emotional scenes recognition. The aims of this behavioral study were to assess the ability to process emotional body postures and to test whether motor response is mainly driven by the emotional content of the picture or if it is influenced by motor resonance. Emotional body postures and scenes (IAPS) divided into three clusters (fear, happiness, and neutral) were shown to 25 healthy subjects (13 males, mean age ± SD: 22.3 ± 1.8 years) in a three-alternative forced choice task. Subjects were asked to recognize the emotional content of the pictures by pressing one of three keys as fast as possible in order to estimate response times (RTs). The rating of valence and arousal was also performed. We found shorter RTs for fearful body postures as compared with happy and neutral postures. In contrast, no differences across emotional categories were found for the IAPS stimuli. Analysis on valence and arousal and the subsequent item analysis showed an excellent reliability of the two sets of images used in the experiment. Our results show that fearful body postures are rapidly recognized and processed, probably thanks to the automatic activation of a series of central nervous system structures orchestrating the defensive threat reactions, strengthening and supporting previous neurophysiological and behavioral findings in body language processing.

Validation of dynamic virtual faces for facial affect recognition

The ability to recognise facial emotions is essential for successful social interaction. The most common stimuli used when evaluating this ability are photographs. Although these stimuli have proved to be valid, they do not offer the level of realism that virtual humans have achieved. The objective of the present paper is the validation of a new set of dynamic virtual faces (DVFs) that mimic the six basic emotions plus the neutral expression. The faces are prepared to be observed with low and high dynamism, and from front and side views. For this purpose, 204 healthy participants, stratified by gender, age and education level, were recruited for assessing their facial affect recognition with the set of DVFs. The accuracy in responses was compared with the already validated Penn Emotion Recognition Test (ER-40). The results showed that DVFs were as valid as standardised natural faces for accurately recreating human-like facial expressions. The overall accuracy in the identification of emotions was higher for the DVFs (88.25%) than for the ER-40 faces (82.60%). The percentage of hits of each DVF emotion was high, especially for neutral expression and happiness emotion. No statistically significant differences were discovered regarding gender. Nor were significant differences found between younger adults and adults over 60 years. Moreover, there is an increase of hits for avatar faces showing a greater dynamism, as well as front views of the DVFs compared to their profile presentations. DVFs are as valid as standardised natural faces for accurately recreating human-like facial expressions of emotions.

Emotion recognition in nonhuman primates: How experimental research can contribute to a better understanding of underlying mechanisms

Recognising conspecifics' emotional expressions is important for nonhuman primates to navigate their physical and social environment. We address two possible mechanisms underlying emotion recognition: emotional contagion, the automatic matching of the observer's emotions to the emotional state of the observed individual, and cognitive empathy, the ability to understand the meaning and cause of emotional expressions while maintaining a distinction between own and others' emotions. We review experimental research in nonhuman primates to gain insight into the evolution of emotion recognition. Importantly, we focus on how emotional contagion and cognitive empathy can be studied experimentally. Evidence for aspects of cognitive empathy in different nonhuman primate lineages suggests that a wider range of primates than commonly assumed can infer emotional meaning from emotional expressions. Possibly, analogous rather than homologous evolution underlies emotion recognition. However, conclusions regarding its exact evolutionary course require more research in different modalities and species.

Two Neural Networks for Laughter: A Tractography Study

Laughter is a complex motor behavior occurring in both emotional and nonemotional contexts. Here, we investigated whether the different functions of laughter are mediated by distinct networks and, if this is the case, which are the white matter tracts sustaining them. We performed a multifiber tractography investigation placing seeds in regions involved in laughter production, as identified by previous intracerebral electrical stimulation studies in humans: the pregenual anterior cingulate (pACC), ventral temporal pole (TPv), frontal operculum (FO), presupplementary motor cortex, and ventral striatum/nucleus accumbens (VS/NAcc). The primary motor cortex (M1) and two subcortical territories were also studied to trace the descending projections. Results provided evidence for the existence of two relatively distinct networks. A first network, including pACC, TPv, and VS/NAcc, is interconnected through the anterior cingulate bundle, the accumbofrontal tract, and the uncinate fasciculus, reaching the brainstem throughout the mamillo-tegmental tract. This network is likely involved in the production of emotional laughter. A second network, anchored to FO and M1, projects to the brainstem motor nuclei through the internal capsule. It is most likely the neural basis of nonemotional and conversational laughter. The two networks interact throughout the pre-SMA that is connected to both pACC and FO.