Facial mimicry in its social setting

Computer Vision Analysis of Reduced Interpersonal Affect Coordination in Youth With Autism Spectrum Disorder

Atypical social-emotional reciprocity is a core feature of autism spectrum disorder (ASD) but can be difficult to operationalize. One measurable manifestation of reciprocity may be interpersonal coordination, the tendency to align the form and timing of one's behaviors (including facial affect) with others. Interpersonal affect coordination facilitates sharing and understanding of emotional cues, and there is evidence that it is reduced in ASD. However, most research has not measured this process in true social contexts, due in part to a lack of tools for measuring dynamic facial expressions over the course of an interaction. Automated facial analysis via computer vision provides an efficient, granular, objective method for measuring naturally occurring facial affect and coordination. Youth with ASD and matched typically developing youth participated in cooperative conversations with their mothers and unfamiliar adults. Time-synchronized videos were analyzed with an open-source computer vision toolkit for automated facial analysis, for the presence and intensity of facial movements associated with positive affect. Both youth and adult conversation partners exhibited less positive affect during conversations when the youth partner had ASD. Youth with ASD also engaged in less affect coordination over the course of conversations. When considered dimensionally across youth with and without ASD, affect coordination significantly predicted scores on rating scales of autism-related social atypicality, adaptive social skills, and empathy. Findings suggest that affect coordination is an important interpersonal process with implications for broader social-emotional functioning. This preliminary study introduces a promising novel method for quantifying moment-to-moment facial expression and emotional reciprocity during natural interactions. LAY SUMMARY: This study introduces a novel, automated method for measuring social-emotional reciprocity during natural conversations, which may improve assessment of this core autism diagnostic behavior. We used computerized methods to measure facial affect and the degree of affect coordination between conversation partners. Youth with autism displayed reduced affect coordination, and reduced affect coordination predicted lower scores on measures of broader social-emotional skills.

The Influence of Reward on Facial Mimicry: No Evidence for a Significant Effect of Oxytocin

Recent findings suggest a role of oxytocin on the tendency to spontaneously mimic the emotional facial expressions of others. Oxytocin-related increases of facial mimicry, however, seem to be dependent on contextual factors. Given previous literature showing that people preferentially mimic emotional expressions of individuals associated with high (vs. low) rewards, we examined whether the reward value of the mimicked agent is one factor influencing the oxytocin effects on facial mimicry. To test this hypothesis, 60 male adults received 24 IU of either intranasal oxytocin or placebo in a double-blind, between-subject experiment. Next, the value of male neutral faces was manipulated using an associative learning task with monetary rewards. After the reward associations were learned, participants watched videos of the same faces displaying happy and angry expressions. Facial reactions to the emotional expressions were measured with electromyography. We found that participants judged as more pleasant the face identities associated with high reward values than with low reward values. However, happy expressions by low rewarding faces were more spontaneously mimicked than high rewarding faces. Contrary to our expectations, we did not find a significant direct effect of intranasal oxytocin on facial mimicry, nor on the reward-driven modulation of mimicry. Our results support the notion that mimicry is a complex process that depends on contextual factors, but failed to provide conclusive evidence of a role of oxytocin on the modulation of facial mimicry.

I'll cry instead: Mu suppression responses to tearful facial expressions

Tears are a facial expression of emotion that readily elicit empathic responses from observers. It is currently unknown whether these empathic responses to tears are influenced by specific neural substrates. The EEG mu rhythm is one method of investigating the human mirror neuron system, purported to underlie the sharing of affective states and a facilitator of social cognition. The purpose of this research was to explore the mu response to tearful expressions of emotion. Sixty-eight participants viewed happy and sad faces, both with and without tears, in addition to a neutral control condition. Participants first completed an emotion discrimination task, and then an imitation condition where they were required to mimic the displayed expression. Mu enhancement was found in response to the discrimination task, whilst suppression was demonstrated in response to the imitation condition. Examination of the suppression scores revealed that greater suppression was observed in response to happy-tear and sad tear-free expressions. Planned contrasts exploring suppression to neutral faces revealed no significant differences between emotional and neutral conditions. The mu response to neutral expressions resembled that of the happy-tear and the sad tear-free conditions, lending support to the idea that ambiguous emotional expressions require greater sensorimotor engagement. This study provides preliminary evidence for the role of the mirror neuron system in discerning tearful expressions of emotion in the absence of context.

Protective effects of elevated anandamide on stress and fear-related behaviors: translational evidence from humans and mice

Post-traumatic stress disorder (PTSD) is a common, debilitating condition with limited treatment options. Extinction of fear memories through prolonged exposure therapy, the primary evidence-based behavioral treatment for PTSD, has only partial efficacy. In mice, pharmacological inhibition of fatty acid amide hydrolase (FAAH) produces elevated levels of anandamide (AEA) and promotes fear extinction, suggesting that FAAH inhibitors may aid fear extinction-based treatments. A human FAAH 385C->A substitution encodes an FAAH enzyme with reduced catabolic efficacy. Individuals homozygous for the FAAH 385A allele may therefore offer a genetic model to evaluate the impact of elevations in AEA signaling in humans, helping to inform whether FAAH inhibitors have the potential to facilitate fear extinction therapy for PTSD. To overcome the challenge posed by low frequency of the AA genotype (appr. 5%), we prospectively genotyped 423 individuals to examine the balanced groups of CC, AC, and AA individuals (n = 25/group). Consistent with its loss-of-function nature, the A allele was dose dependently associated with elevated basal AEA levels, facilitated fear extinction, and enhanced the extinction recall. Moreover, the A-allele homozygotes were protected against stress-induced decreases in AEA and negative emotional consequences of stress. In a humanized mouse model, AA homozygous mice were similarly protected against stress-induced decreases in AEA, both in the periphery, and also in the amygdala and prefrontal cortex, brain structures critically involved in fear extinction and regulation of stress responses. Collectively, these data suggest that AEA signaling can temper aspects of the stress response and that FAAH inhibition may aid the treatment for stress-related psychiatric disorders, such as PTSD.

Achieving affective human-virtual agent communication by enabling virtual agents to imitate positive expressions

Affective communication, communicating with emotion, during face-to-face communication is critical for social interaction. Advances in artificial intelligence have made it essential to develop affective human–virtual agent communication. A person’s belief during human–virtual agent interaction that the agent is a computer program affects social-cognitive processes. Whether this belief interferes with affective communication is an open question. We hypothesized that the imitation of a positive emotional expression by a virtual agent induces a positive emotion, regardless of the belief. To test this hypothesis, we conducted an fMRI study with 39 healthy volunteers, who were made to believe that a virtual agent was either a person or a computer. They were instructed to smile, and immediately afterwards, the virtual agent displayed a positive, negative, or neutral expression. The participants reported a positive emotion only when their smile was imitated by the agent’s positive expression regardless of their belief. This imitation activated the participants’ medial prefrontal cortex and precuneus, which are involved in anthropomorphism and contingency, respectively. These results suggest that a positive congruent response by a virtual agent can overcome the effect of believing that the agent is a computer program and thus contribute to achieving affective human–virtual agent communication.

Perception of Discrete Emotions in Others: Evidence for Distinct Facial Mimicry Patterns

Covert facial mimicry involves subtle facial muscle activation in observers when they perceive the facial emotional expressions of others. It remains uncertain whether prototypical facial features in emotional expressions are being covertly mimicked and also whether covert facial mimicry involves distinct facial muscle activation patterns across muscles per emotion category, or simply distinguishes positive versus negative valence in observed facial emotions. To test whether covert facial mimicry is emotion-specific, we measured facial electromyography (EMG) from five muscle sites (corrugator supercilii, levator labii, frontalis lateralis, depressor anguli oris, zygomaticus major) whilst participants watched videos of people expressing 9 different basic and complex emotions and a neutral expression. This study builds upon previous research by including a greater number of facial muscle measures and emotional expressions. It is the first study to investigate activation patterns across muscles during facial mimicry and to provide evidence for distinct patterns of facial muscle activation when viewing individual emotion categories, suggesting that facial mimicry is emotion-specific, rather than just valence-based.

Towards the automatic detection of social biomarkers in autism spectrum disorder: introducing the simulated interaction task (SIT)

Social interaction deficits are evident in many psychiatric conditions and specifically in autism spectrum disorder (ASD), but hard to assess objectively. We present a digital tool to automatically quantify biomarkers of social interaction deficits: the simulated interaction task (SIT), which entails a standardized 7-min simulated dialog via video and the automated analysis of facial expressions, gaze behavior, and voice characteristics. In a study with 37 adults with ASD without intellectual disability and 43 healthy controls, we show the potential of the tool as a diagnostic instrument and for better description of ASD-associated social phenotypes. Using machine-learning tools, we detected individuals with ASD with an accuracy of 73%, sensitivity of 67%, and specificity of 79%, based on their facial expressions and vocal characteristics alone. Especially reduced social smiling and facial mimicry as well as a higher voice fundamental frequency and harmony-to-noise-ratio were characteristic for individuals with ASD. The time-effective and cost-effective computer-based analysis outperformed a majority vote and performed equal to clinical expert ratings.

Blocking facial mimicry affects recognition of facial and body expressions

Facial mimicry is commonly defined as the tendency to imitate-at a sub-threshold level-facial expressions of other individuals. Numerous studies support a role of facial mimicry in recognizing others' emotions. However, the underlying functional mechanism is unclear. A prominent hypothesis considers facial mimicry as based on an action-perception loop, leading to the prediction that facial mimicry should be observed only when processing others' facial expressions. Nevertheless, previous studies have also detected facial mimicry during observation of emotional bodily expressions. An emergent alternative hypothesis is that facial mimicry overtly reflects the simulation of an "emotion", rather than the reproduction of a specific observed motor pattern. In the present study, we tested whether blocking mimicry ("Bite") on the lower face disrupted recognition of happy expressions conveyed by either facial or body expressions. In Experiment 1, we tested participants' ability to identify happy, fearful and neutral expressions in the Bite condition and in two control conditions. In Experiment 2, to ensure that such a manipulation selectively affects emotion recognition, we tested participants' ability to recognize emotional expressions, as well as the actors' gender, under the Bite condition and a control condition. Finally, we investigated the relationship between dispositional empathy and emotion recognition under the condition of blocked mimicry. Our findings demonstrated that blocking mimicry on the lower face hindered recognition of happy facial and body expressions, while the recognition of neutral and fearful expressions was not affected by the mimicry manipulation. The mimicry manipulation did not affect the gender discrimination task. Furthermore, the impairment of happy expression recognition correlated with empathic traits. These results support the role of facial mimicry in emotion recognition and suggest that facial mimicry reflects a global sensorimotor simulation of others' emotions rather than a muscle-specific reproduction of an observed motor expression.

The Mimicry Among Us: Intra- and Inter-Personal Mechanisms of Spontaneous Mimicry

This review explores spontaneous mimicry in the context of three questions. The first question concerns the role of spontaneous mimicry in processing conceptual information. The second question concerns the debate whether spontaneous mimicry is driven by simple associative processes or reflects higher-order processes such as goals, intentions, and social context. The third question addresses the implications of these debates for understanding atypical individuals and states. We review relevant literature and argue for a dynamic, context-sensitive role of spontaneous mimicry in social cognition and behavior. We highlight how the modulation of mimicry is often adaptive but also point out some cases of maladaptive modulations that impair an individuals’ engagement in social life.

Synchronization in Interpersonal Speech

During both positive and negative dyadic exchanges, individuals will often unconsciously imitate their partner. A substantial amount of research has been made on this phenomenon, and such studies have shown that synchronization between communication partners can improve interpersonal relationships. Automatic computational approaches for recognizing synchrony are still in their infancy. In this study, we extend on previous work in which we applied a novel method utilizing hand-crafted low-level acoustic descriptors and autoencoders (AEs) to analyse synchrony in the speech domain. For this purpose, a database consisting of 394 in-the-wild speakers from six different cultures, is used. For each speaker in the dyadic exchange, two AEs are implemented. Post the training phase, the acoustic features for one of the speakers is tested using the AE trained on their dyadic partner. In this same way, we also explore the benefits that deep representations from audio may have, implementing the state-of-the-art Deep Spectrum toolkit. For all speakers at varied time-points during their interaction, the calculation of reconstruction error from the AE trained on their respective dyadic partner is made. The results obtained from this acoustic analysis are then compared with the linguistic experiments based on word counts and word embeddings generated by our word2vec approach. The results demonstrate that there is a degree of synchrony during all interactions. We also find that, this degree varies across the 6 cultures found in the investigated database. These findings are further substantiated through the use of 4,096 dimensional Deep Spectrum features.

Unfolding political attitudes through the face: facial expressions when reading emotion language of left- and right-wing political leaders

Spontaneous emotionally congruent facial responses (ECFR) to others’ emotional expressions can occur by simply observing others’ faces (i.e., smiling) or by reading emotion related words (i.e., to smile). The goal of the present study was to examine whether language describing political leaders’ emotions affects voters by inducing emotionally congruent facial reactions as a function of readers’ and politicians’ shared political orientation. Participants read sentences describing politicians’ emotional expressions, while their facial muscle activation was measured by means of electromyography (EMG). Results showed that reading sentences describing left and right-wing politicians “smiling” or “frowning” elicits ECFR for ingroup but not outgroup members. Remarkably, ECFR were sensitive to attitudes toward individual leaders beyond the ingroup vs. outgroup political divide. Through integrating behavioral and physiological methods we were able to consistently tap on a ‘favored political leader effect’ thus capturing political attitudes towards an individual politician at a given moment of time, at multiple levels (explicit responses and automatic ECFR) and across political party membership lines. Our findings highlight the role of verbal behavior of politicians in affecting voters’ facial expressions with important implications for social judgment and behavioral outcomes.

A case of V180I genetic Creutzfeldt-Jakob disease presenting with conspicuous facial mimicry

Although there have been no reports of facial mimicry in patients with Creutzfeldt-Jakob disease (CJD), we encountered a patient with genetic CJD with prion protein gene codon 180 mutation (V180I gCJD) who apparently showed this interesting clinical finding. The patient was an 87-year-old Japanese woman, and the first observed CJD symptom was poor spontaneity. She gradually showed cognitive dysfunction and subsequently gait disturbance. A prion protein gene analysis revealed a V180I mutation with methionine homozygosity at codon 129. Facial mimicry was observed 7 months after disease onset and continued for approximately 9 months. Pathological laughing and startle reaction were also observed during approximately the same period, whereas myoclonus was observed at a later stage, 12 months after disease onset, and was very mild in degree. Electroencephalography studies showed a diffuse slow basic pattern without periodic sharp wave complexes. Diffusion-weighted magnetic resonance imaging showed extensive hyperintensity in the cerebral cortex, and there was also hyperintensity with edematous swelling in the same regions on T2-weighted and fluid-attenuated inversion recovery images. On the basis of the magnetic resonance imaging findings and the findings of previous case reports of V180I gCJD, we speculate that the characteristic extensive cerebrocortical involvement observed in V180I gCJD was implicated in the pathogenesis of the facial mimicry observed in this case.

Increased Medial Prefrontal Cortex and Decreased Zygomaticus Activation in Response to Disliked Smiles Suggest Top-Down Inhibition of Facial Mimicry

Spontaneous facial mimicry is modulated by many factors, and often needs to be suppressed to comply with social norms. The neural basis for the inhibition of facial mimicry was investigated in a combined functional magnetic resonance imaging and electromyography study in 39 healthy participants. In an operant conditioning paradigm, face identities were associated with reward or punishment and were later shown expressing dynamic smiles and anger expressions. Face identities previously associated with punishment, compared to reward, were disliked by participants overall, and their smiles generated less mimicry. Consistent with previous research on the inhibition of finger/hand movements, the medial prefrontal cortex (mPFC) was activated when previous conditioning was incongruent with the valence of the expression. On such trials there was also greater functional connectivity of the mPFC with insula and premotor cortex as tested with psychophysiological interaction, suggesting inhibition of areas associated with the production of facial mimicry and the processing of facial feedback. The findings suggest that the mPFC supports the inhibition of facial mimicry, and support the claim of theories of embodied cognition that facial mimicry constitutes a spontaneous low-level motor imitation.

The modulation of facial mimicry by attachment tendencies and their underlying affiliation motives in 3-year-olds: An EMG study

From early in life, facial mimicry represents an important example of implicit non-verbal communication. Facial mimicry is conceived of as the automatic tendency to mimic another person's facial expressions and is thought to serve as a social glue among interaction partners. Although in adults mimicry has been shown to be moderated by the social context and one's needs to affiliate with others, evidence from behavioural mimicry studies suggest that 3-year-olds do not yet show sensitivity to social dynamics. Here, we examined whether attachment tendencies, as a proxy for interindividual differences in affiliation motivation, modulates facial mimicry in 3-year-olds. Resistant and avoidant insecure attachment tendencies are characterized by high and low affiliation motivation, respectively, and these were hypothesized to lead to either enhancement or suppression of mimicry. Additionally, we hypothesized that these effects will be moderated by inhibitory control skills. Facial mimicry of happy and sad expressions was recorded with electromyography (EMG), attachment tendencies were assessed with a parent-report questionnaire and inhibitory control with the gift delay task. The final sample consisted of 42 children, with overall scores suggesting secure attachment. Our findings revealed that 3-year-olds mimicked happy and sad facial expressions. Moreover, resistant tendencies predicted enhanced sad but not happy facial mimicry, whereas avoidant tendencies were not significantly related to mimicry. These effects were not moderated by inhibitory control skills. In conclusion, these findings provide the first evidence for the modulation of mimicry by attachment tendencies and their underlying motivation for affiliation in young children, specifically for negatively-valenced emotional expressions.

Objective assessment of blinking and facial expressions in Parkinson's disease using a vertical electro-oculogram and facial surface electromyography

OBJECTIVE

Hypomimia is a common and early symptom of Parkinson's disease (PD), which reduces the ability of PD patients to manifest emotions. Currently, it is visually evaluated by the neurologist during neurological examinations for PD diagnosis, as described in task 3.2 of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Since such an evaluation is semi-quantitative and affected by inter-variability, this paper aims to measure the physiological parameters related to eye blink and facial expressions extracted from a vertical electro-oculogram (VEOG) and facial surface electromyography (fsEMG) to differentiate PD patients from healthy control subjects (HCs).

APPROACH

The spontaneous eye blink rate-minute (sEBR), its maximum amplitude (BMP), and facial cutaneous muscle activity were measured in 24 PD patients and 24 HCs while the subjects looked at a visual-tester composed of three main parts: static vision, dynamic vision and reading silently. Specificity and sensitivity for each parameter were calculated.

MAIN RESULTS

The VEOG and the fsEMG allowed the identification of some parameters related to eye blink and facial expressions (i.e. sEBR, BMP, frontal and peribuccal muscular activities), being able to distinguish between PD patients and HCs with high sensitivity and specificity.

SIGNIFICANCE

The demonstration that the combination of parameters related to eye blink and facial expressions can discriminate (with high accuracy) between PD patients versus HCs, thus resulting in a useful tool to support the neurologist in objective assessment of hypomimia for improving PD diagnosis.

Empathic Responses for Pain in Facial Muscles Are Modulated by Actor's Attractiveness and Gender, and Perspective Taken by Observer

Although empathy for pain is an often studied phenomenon, only few studies employing electromyography (EMG) have investigated either emotional responses to the pain of others or factors that modulate these responses. The present study investigated whether the sex and attractiveness of persons experiencing pain affected muscle activity associated with empathy for pain, the corrugator supercili (CS) and orbicularis oculi (OO) muscles, in male and female participants in two conditions: adopting a perspective of “the other” or “the self.” Fifty one participants (27 females) watched movies showing situations that included the expression of pain, with female and male and more and less attractive actors under both conditions, while the CS and OO EMG were recorded. Perspective did not affect CS muscle activity, but OO muscle activity tended to be higher in women than men under the imagine-self condition. CS muscle activity, but not OO muscle activity, was modulated by the actors’ gender and attractiveness. CS muscle activity was stronger in response to the pain of less attractive than more attractive actors, and to the pain of female actors compared to male actors. Moreover, a positive correlation was found between empathic concern, as a trait, and CS muscle activity, but only in the imagine-self condition.

Empathy in Facial Mimicry of Fear and Disgust: Simultaneous EMG-fMRI Recordings During Observation of Static and Dynamic Facial Expressions

Real-life faces are dynamic by nature, particularly when expressing emotion. Increasing evidence suggests that the perception of dynamic displays enhances facial mimicry and induces activation in widespread brain structures considered to be part of the mirror neuron system, a neuronal network linked to empathy. The present study is the first to investigate the relations among facial muscle responses, brain activity, and empathy traits while participants observed static and dynamic (videos) facial expressions of fear and disgust. During display presentation, blood-oxygen level-dependent (BOLD) signal as well as muscle reactions of the corrugator supercilii and levator labii were recorded simultaneously from 46 healthy individuals (21 females). It was shown that both fear and disgust faces caused activity in the corrugator supercilii muscle, while perception of disgust produced facial activity additionally in the levator labii muscle, supporting a specific pattern of facial mimicry for these emotions. Moreover, individuals with higher, compared to individuals with lower, empathy traits showed greater activity in the corrugator supercilii and levator labii muscles; however, these responses were not differentiable between static and dynamic mode. Conversely, neuroimaging data revealed motion and emotional-related brain structures in response to dynamic rather than static stimuli among high empathy individuals. In line with this, there was a correlation between electromyography (EMG) responses and brain activity suggesting that the Mirror Neuron System, the anterior insula and the amygdala might constitute the neural correlates of automatic facial mimicry for fear and disgust. These results revealed that the dynamic property of (emotional) stimuli facilitates the emotional-related processing of facial expressions, especially among whose with high trait empathy.

Affiliative zygomatic synchrony in co-present strangers

In social contexts individuals frequently act as social chameleons, synchronizing their responses with those of others. Such synchrony is believed to play an important role, promoting mutual emotional and social states. However, synchrony in facial signals, which serve as the main communicative channel between people, has not been systematically studied. To address this gap, we investigated the social spread of smiling dynamics in a naturalistic social setting and assessed its affiliative function. We also studied whether smiling synchrony between people is linked with convergence in their autonomic and emotional responses. To that aim we measured moment-by-moment changes in zygomatic electromyography and cardiovascular activity in dyads of previously unacquainted participants, who co-viewed and subsequently rated emotional movies. We found a robust, dyad-specific zygomatic synchrony in co-viewing participants. During the positive movie, such zygomatic synchrony co-varied with cardiovascular synchrony and with convergence in positive feelings. No such links were found for the negative movie. Centrally, zygomatic synchrony in both emotional contexts predicted the subsequently reported affiliative feelings of dyad members. These results demonstrate that a naturally unfolding smiling behavior is highly contagious. They further suggest that zygomatic synchrony functions as a social facilitator, eliciting affiliation towards previously unknown others.

Integrating Tinbergen's inquiries: Mimicry and play in humans and other social mammals

Visual signals convey emotions and intentions between individuals. Darwin underlined that human facial expressions represent a shared heritage between our species and many other social mammals. Social play is a fertile field to examine the role and the potential communicative function of facial expressions. The relaxed open-mouth (or play face) is a context-specific playful expression, which is widespread in human and non-human mammals. Here, we focus on playful communication by applying Tinbergen's four areas of inquiry: proximate causation, ontogeny, function, and evolution. First of all we explore mimicry by focusing on its neural substrates and factors of modulation within playful and non-playful context (proximate causation). Play face is one of the earliest facial expressions to appear and be mimicked in neonates. The motor resonance between infants and their caregivers is essential later in life when individuals begin to engage in increasingly complex social interactions, including play (ontogeny). The success of a playful session can be evaluated by its duration in time. Mirroring facial expressions prolongs the session by favoring individuals to fine-tune their own motor sequences accordingly (function). Finally, through a comparative approach we also demonstrate that the elements constituting play communication and mimicry are sensitive to the quality of interindividual relationships of a species, thus reflecting the nature of its social network and style (evolution). In conclusion, our goal is to integrate Tinbergen's four areas of ethological inquiry to provide a broader framework regarding the importance of communication and mimicry in the play domain of humans and other social mammals.

Behind the Robot's Smiles and Frowns: In Social Context, People Do Not Mirror Android's Expressions But React to Their Informational Value

Facial actions are key elements of non-verbal behavior. Perceivers' reactions to others' facial expressions often represent a match or mirroring (e.g., they smile to a smile). However, the information conveyed by an expression depends on context. Thus, when shown by an opponent, a smile conveys bad news and evokes frowning. The availability of anthropomorphic agents capable of facial actions raises the question of how people respond to such agents in social context. We explored this issue in a study where participants played a strategic game with or against a facially expressive android. Electromyography (EMG) recorded participants' reactions over zygomaticus muscle (smiling) and corrugator muscle (frowning). We found that participants' facial responses to android's expressions reflect their informational value, rather than a direct match. Overall, participants smiled more, and frowned less, when winning than losing. Critically, participants' responses to the game outcome were similar regardless of whether it was conveyed via the android's smile or frown. Furthermore, the outcome had greater impact on people's facial reactions when it was conveyed through android's face than a computer screen. These findings demonstrate that facial actions of artificial agents impact human facial responding. They also suggest a sophistication in human-robot communication that highlights the signaling value of facial expressions.

Neural Correlates of Facial Mimicry: Simultaneous Measurements of EMG and BOLD Responses during Perception of Dynamic Compared to Static Facial Expressions

Facial mimicry (FM) is an automatic response to imitate the facial expressions of others. However, neural correlates of the phenomenon are as yet not well established. We investigated this issue using simultaneously recorded EMG and BOLD signals during perception of dynamic and static emotional facial expressions of happiness and anger. During display presentations, BOLD signals and zygomaticus major (ZM), corrugator supercilii (CS) and orbicularis oculi (OO) EMG responses were recorded simultaneously from 46 healthy individuals. Subjects reacted spontaneously to happy facial expressions with increased EMG activity in ZM and OO muscles and decreased CS activity, which was interpreted as FM. Facial muscle responses correlated with BOLD activity in regions associated with motor simulation of facial expressions [i.e., inferior frontal gyrus, a classical Mirror Neuron System (MNS)]. Further, we also found correlations for regions associated with emotional processing (i.e., insula, part of the extended MNS). It is concluded that FM involves both motor and emotional brain structures, especially during perception of natural emotional expressions.

Matching Your Face or Appraising the Situation: Two Paths to Emotional Contagion

Emotions are believed to converge both through emotional mimicry and social appraisal. The present study compared contagion of anger and happiness. In Experiment 1, participants viewed dynamic angry and happy faces, with facial electromyography recorded from the zygomaticus major and corrugator supercilii as emotional mimicry. Self-reported emotional experiences were analyzed as emotional contagion. Experiment 2 manipulated social appraisal as the gaze of expression toward the target. The results showed that there was emotional contagion for angry and happy expressions both in Experiment 1 and Experiment 2. Experiment 1 indicated an overt mimicry pattern for happy faces, but not for angry faces. Experiment 2 found an influence of social appraisal on angry contagion but not on happy diffusion. The two experiments suggest that the underlying processes of emotional mimicry and social appraisal are differentially relevant for different emotional contagion, with happiness processing following a mimicry-based path to emotional contagion, and anger processing requiring social appraisal.

The development of spontaneous facial responses to others' emotions in infancy: An EMG study

Viewing facial expressions often evokes facial responses in the observer. These spontaneous facial reactions (SFRs) are believed to play an important role for social interactions. However, their developmental trajectory and the underlying neurocognitive mechanisms are still little understood. In the current study, 4- and 7-month old infants were presented with facial expressions of happiness, anger, and fear. Electromyography (EMG) was used to measure activation in muscles relevant for forming these expressions: zygomaticus major (smiling), corrugator supercilii (frowning), and frontalis (forehead raising). The results indicated no selective activation of the facial muscles for the expressions in 4-month-old infants. For 7-month-old infants, evidence for selective facial reactions was found especially for happy (leading to increased zygomaticus major activation) and fearful faces (leading to increased frontalis activation), while angry faces did not show a clear differential response. These results suggest that emotional SFRs may be the result of complex neurocognitive mechanisms which lead to partial mimicry but are also likely to be influenced by evaluative processes. Such mechanisms seem to undergo important developments at least until the second half of the first year of life.

Mimicking emotions

Emotional mimicry refers to the tendency to mimic other's emotions in order to share minds. We present new evidence that supports our Contextual Model of Emotional Mimicry, showing that emotional mimicry serves affiliative goals that vary across social contexts. This also implies the opposite, namely that we (unconsciously) refrain from mimicking others' emotions if we want to keep emotional distance. Facial mimicry of emotions is further suggested to be a largely top-down process, based on goals and representations, rather than on mere watching others' facial movements.

Are Vocal Pitch Changes in Response to Facial Expressions of Emotions Potential Cues of Empathy? A Preliminary Report

Previous research has demonstrated that even brief exposures to facial expressions of emotions elicit facial mimicry in receivers in the form of corresponding facial muscle movements. As well, vocal and verbal patterns of speakers converge in conversations, a type of vocal mimicry. There is also evidence of cross-modal mimicry in which emotional vocalizations elicit corresponding facial muscle activity. Further, empathic capacity has been associated with enhanced tendency towards facial mimicry as well as verbal synchrony. We investigated a type of potential cross-modal mimicry in a simulated dyadic situation. Specifically, we examined the influence of facial expressions of happy, sad, and neutral emotions on the vocal pitch of receivers, and its potential association with empathy. Results indicated that whereas both mean pitch and variability of pitch varied somewhat in the predicted directions, empathy was correlated with the difference in the variability of pitch while speaking to the sad and neutral faces. Discussion of results considers the dimensional nature of emotional vocalizations and possible future directions.

Social Context Modulates Facial Imitation of Children's Emotional Expressions

Children use emotional facial expressions of others for guiding their behavior, a process which is important to a child’s social-emotional development. Earlier studies on facial interaction demonstrate that imitation of emotional expressions of others is automatic, yet can be dynamically modulated depending on contextual information. Considering the value of emotional expressions for children especially, we tested whether and to what extent information about children’s temperament and domestic situation alters mimicry of their emotional expressions. Results show that angry expressions of children displaying negative behavior resulted in stronger imitation, which may serve as a corrective signal. Sad facial expressions resulted in stronger imitation towards those behaving positively but only when exposed to a difficult domestic situation, indicating increased empathy towards these children. These findings shed new light on the dynamic implicit communicative processes that shape interaction with children of different social-emotional backgrounds.

Emotional Empathy and Facial Mimicry for Static and Dynamic Facial Expressions of Fear and Disgust

Facial mimicry is the tendency to imitate the emotional facial expressions of others. Increasing evidence suggests that the perception of dynamic displays leads to enhanced facial mimicry, especially for happiness and anger. However, little is known about the impact of dynamic stimuli on facial mimicry for fear and disgust. To investigate this issue, facial EMG responses were recorded in the corrugator supercilii, levator labii, and lateral frontalis muscles, while participants viewed static (photos) and dynamic (videos) facial emotional expressions. Moreover, we tested whether emotional empathy modulated facial mimicry for emotional facial expressions. In accordance with our predictions, the highly empathic group responded with larger activity in the corrugator supercilii and levator labii muscles. Moreover, dynamic compared to static facial expressions of fear revealed enhanced mimicry in the high-empathic group in the frontalis and corrugator supercilii muscles. In the low-empathic group the facial reactions were not differentiated between fear and disgust for both dynamic and static facial expressions. We conclude that highly empathic subjects are more sensitive in their facial reactions to the facial expressions of fear and disgust compared to low empathetic counterparts. Our data confirms that personal characteristics, i.e., empathy traits as well as modality of the presented stimuli, modulate the strength of facial mimicry. In addition, measures of EMG activity of the levator labii and frontalis muscles may be a useful index of empathic responses of fear and disgust.

Do Dynamic Compared to Static Facial Expressions of Happiness and Anger Reveal Enhanced Facial Mimicry?

Facial mimicry is the spontaneous response to others’ facial expressions by mirroring or matching the interaction partner. Recent evidence suggested that mimicry may not be only an automatic reaction but could be dependent on many factors, including social context, type of task in which the participant is engaged, or stimulus properties (dynamic vs static presentation). In the present study, we investigated the impact of dynamic facial expression and sex differences on facial mimicry and judgment of emotional intensity. Electromyography recordings were recorded from the corrugator supercilii, zygomaticus major, and orbicularis oculi muscles during passive observation of static and dynamic images of happiness and anger. The ratings of the emotional intensity of facial expressions were also analysed. As predicted, dynamic expressions were rated as more intense than static ones. Compared to static images, dynamic displays of happiness also evoked stronger activity in the zygomaticus major and orbicularis oculi, suggesting that subjects experienced positive emotion. No muscles showed mimicry activity in response to angry faces. Moreover, we found that women exhibited greater zygomaticus major muscle activity in response to dynamic happiness stimuli than static stimuli. Our data support the hypothesis that people mimic positive emotions and confirm the importance of dynamic stimuli in some emotional processing.

Spontaneous Facial Mimicry Is Enhanced by the Goal of Inferring Emotional States: Evidence for Moderation of "Automatic" Mimicry by Higher Cognitive Processes

A number of studies have shown that individuals often spontaneously mimic the facial expressions of others, a tendency known as facial mimicry. This tendency has generally been considered a reflex-like "automatic" response, but several recent studies have shown that the degree of mimicry may be moderated by contextual information. However, the cognitive and motivational factors underlying the contextual moderation of facial mimicry require further empirical investigation. In this study, we present evidence that the degree to which participants spontaneously mimic a target's facial expressions depends on whether participants are motivated to infer the target's emotional state. In the first study we show that facial mimicry, assessed by facial electromyography, occurs more frequently when participants are specifically instructed to infer a target's emotional state than when given no instruction. In the second study, we replicate this effect using the Facial Action Coding System to show that participants are more likely to mimic facial expressions of emotion when they are asked to infer the target's emotional state, rather than make inferences about a physical trait unrelated to emotion. These results provide convergent evidence that the explicit goal of understanding a target's emotional state affects the degree of facial mimicry shown by the perceiver, suggesting moderation of reflex-like motor activities by higher cognitive processes.

Fashioning the Face: Sensorimotor Simulation Contributes to Facial Expression Recognition

When we observe a facial expression of emotion, we often mimic it. This automatic mimicry reflects underlying sensorimotor simulation that supports accurate emotion recognition. Why this is so is becoming more obvious: emotions are patterns of expressive, behavioral, physiological, and subjective feeling responses. Activation of one component can therefore automatically activate other components. When people simulate a perceived facial expression, they partially activate the corresponding emotional state in themselves, which provides a basis for inferring the underlying emotion of the expresser. We integrate recent evidence in favor of a role for sensorimotor simulation in emotion recognition. We then connect this account to a domain-general understanding of how sensory information from multiple modalities is integrated to generate perceptual predictions in the brain.

Judged and Remembered Trustworthiness of Faces Is Enhanced by Experiencing Multisensory Synchrony and Asynchrony in the Right Order

This work builds on the enfacement effect. This effect occurs when experiencing a rhythmic stimulation on one’s cheek while seeing someone else’s face being touched in a synchronous way. This typically leads to cognitive and social-cognitive effects similar to self-other merging. In two studies, we demonstrate that this multisensory stimulation can change the evaluation of the other’s face. In the first study, participants judged the stranger’s face and similar faces as being more trustworthy after synchrony, but not after asynchrony. Synchrony interacted with the order of the stroking; hence trustworthiness only changed when the synchronous stimulation occurred before the asynchronous one. In the second study, a synchronous stimulation caused participants to remember the stranger’s face as more trustworthy, but again only when the synchronous stimulation came before the asynchronous one. The results of both studies show that order of stroking creates a context in which multisensory synchrony can affect the trustworthiness of faces.