The contribution of facial dynamics to subtle expression recognition in typical viewers and developmental visual agnosia

Positive and negative facial valence perception are modulated differently by eccentricity in the parafovea: Replication from KDEF to NimStim

While perceiving the emotional state of others may be crucial for our behavior even when this information is present outside of central vision, emotion perception studies typically focus on central visual field. We have recently investigated emotional valence (pleasantness) perception across the parafovea (≤ 4°) and found that for briefly presented (200 ms) emotional face images (from the established KDEF image-set), positive (happy) valence was the least affected by eccentricity (distance from the central visual field) and negative (fearful) valence the most. Furthermore, we found that performance at 2° predicted performance at 4°. Here we tested (n = 37) whether these effects replicate with face stimuli of different identities from a different well-established image-set (NimStim). All our prior findings replicated and eccentricity-based modulation magnitude was smaller with NimStim (~ 16.6% accuracy reduction at 4°) than with KDEF stimuli (~ 27.3% reduction). Our current investigations support our earlier findings that for briefly presented parafoveal stimuli, positive and negative valence perception are differently affected by eccentricity and may be dissociated. Furthermore, our results highlight the importance of investigating emotions beyond central vision and demonstrate commonalities and differences across different image sets in the parafovea, emphasizing the contribution of replication studies to substantiate our knowledge about perceptual mechanisms.

Recognizing facial expressions of emotion amid noise: A dynamic advantage

Humans communicate internal states through complex facial movements shaped by biological and evolutionary constraints. Although real-life social interactions are flooded with dynamic signals, current knowledge on facial expression recognition mainly arises from studies using static face images. This experimental bias might stem from previous studies consistently reporting that young adults minimally benefit from the richer dynamic over static information, whereas children, the elderly, and clinical populations very strongly do (Richoz, Jack, Garrod, Schyns, & Caldara, 2015, Richoz, Jack, Garrod, Schyns, & Caldara, 2018b). These observations point to a near-optimal facial expression decoding system in young adults, almost insensitive to the advantage of dynamic over static cues. Surprisingly, no study has yet tested the idea that such evidence might be rooted in a ceiling effect. To this aim, we asked 70 healthy young adults to perform static and dynamic facial expression recognition of the six basic expressions while parametrically and randomly varying the low-level normalized phase and contrast signal (0%-100%) of the faces. As predicted, when 100% face signals were presented, static and dynamic expressions were recognized with equal efficiency with the exception of those with the most informative dynamics (i.e., happiness and surprise). However, when less signal was available, dynamic expressions were all better recognized than their static counterpart (peaking at ∼20%). Our data show that facial movements increase our ability to efficiently identify emotional states of others under the suboptimal visual conditions that can occur in everyday life. Dynamic signals are more effective and sensitive than static ones for decoding all facial expressions of emotion for all human observers.

The Interplay between Chronotype and Emotion Regulation in the Recognition of Facial Expressions of Emotion

Emotion regulation strategies affect the experience and processing of emotions and emotional stimuli. Chronotype has also been shown to influence the processing of emotional stimuli, with late chronotypes showing a bias towards better processing of negative stimuli. Additionally, greater eveningness has been associated with increased difficulties in emotion regulation and preferential use of expressive suppression strategies. Therefore, the present study aimed to understand the interplay between chronotype and emotion regulation on the recognition of dynamic facial expressions of emotion. To that end, 287 participants answered self-report measures and performed an online facial emotion recognition task from short video clips where a neutral face gradually morphed into a full-emotion expression (one of the six basic emotions). Participants should press the spacebar to stop each video as soon as they could recognize the emotional expression, and then identify it from six provided labels/emotions. Greater eveningness was associated with shorter response times (RT) in the identification of sadness, disgust and happiness. Higher scores of expressive suppression were associated with longer RT in identifying sadness, disgust, anger and surprise. Expressive suppression significantly moderated the relationship between chronotype and the recognition of sadness and anger, with chronotype being a significant predictor of emotion recognition times only at higher levels of expressive suppression. No significant effects were observed for cognitive reappraisal. These results are consistent with a negative bias in emotion processing in late chronotypes and increased difficulty in anger and sadness recognition for expressive suppressor morning-types.

Positive and negative facial valence perception are modulated differently by eccentricity in the parafovea

Understanding whether people around us are in a good, bad or neutral mood can be critical to our behavior, both when looking directly at them or when they are in our peripheral visual field. However, facial expressions of emotions are often investigated at central visual field or at locations right or left of fixation. Here we assumed that perception of facial emotional valence (the emotion's pleasantness) changes with distance from central visual field (eccentricity) and that different emotions may be influenced differently by eccentricity. Participants (n = 58) judged the valence of emotional faces across the parafovea (≤ 4°, positive (happy), negative (fearful), or neutral)) while their eyes were being tracked. As expected, performance decreased with eccentricity. Positive valence perception was least affected by eccentricity (accuracy reduction of 10-19% at 4°) and negative the most (accuracy reduction of 35-38% at 4°), and this was not a result of speed-accuracy trade-off or response biases. Within-valence (but not across-valence) performance was associated across eccentricities suggesting perception of different valences is supported by different mechanisms. While our results may not generalize to all positive and negative emotions, they indicate that beyond-foveal investigations can reveal additional characteristics of the mechanisms that underlie facial expression processing and perception.

Automatic facial reactions to facial, body, and vocal expressions: A stimulus-response compatibility study

When perceiving emotional facial expressions there is an automatic tendency to react with a matching facial expression. A classic explanation of this phenomenon, termed the matched motor hypothesis, highlights the importance of topographic matching, that is, the correspondence in body parts, between perceived and produced actions. More recent studies using mimicry paradigms have challenged this classic account, producing ample evidence against the matched motor hypothesis. However, research using stimulus-response compatibility (SRC) paradigms usually assumed the effect relies on topographic matching. While mimicry and SRC share some characteristics, critical differences between the paradigms suggest conclusions cannot be simply transferred from one to another. Thus, our aim in the present study was to directly test the matched motor hypothesis using SRC. Specifically, we investigated whether observing emotional body postures or hearing emotional vocalizations produces a tendency to respond with one's face, despite completely different motor actions being involved. In three SRC experiments, participants were required to either smile or frown in response to a color cue, presented concurrently with stimuli of happy and angry facial (experiment 1), body (experiment 2), or vocal (experiment 3) expressions. Reaction times were measured using facial EMG. Whether presenting facial, body, or vocal expressions, we found faster responses in compatible, compared to incompatible trials. These results demonstrate that the SRC effect of emotional expressions does not require topographic matching. Our findings question interpretations of previous research and suggest further examination of the matched motor hypothesis.

Measuring emotion recognition by people with Parkinson's disease using eye-tracking with dynamic facial expressions

BACKGROUND

Motion is an important cue to emotion recognition, and it has been suggested that we recognize emotions via internal simulation of others' expressions. There is a reduction of facial expression in Parkinson's disease (PD), which may influence the ability to use motion to recognise emotions in others. However, the majority of previous work in PD has used only static expressions. Moreover, few studies have used eye-tracking to explore emotion processing in PD.

NEW METHOD

We measured accuracy and eye movements in people with PD and healthy controls when identifying emotions from both static and dynamic facial expressions.

RESULTS

The groups did not differ overall in emotion recognition accuracy, but motion significantly increased recognition only in the control group. Participants made fewer and longer fixations when viewing dynamic expressions, and interest area analysis revealed increased gaze to the mouth region and decreased gaze to the eyes for dynamic stimuli, although the latter was specific to the control group.

COMPARISON WITH EXISTING METHODS

Ours is the first study to directly compare recognition of static and dynamic emotional expressions in PD using eye-tracking, revealing subtle differences between groups that may otherwise be undetected.

CONCLUSIONS

It is feasible and informative to use eye-tracking with dynamic expressions to investigate emotion recognition in PD. Our findings suggest that people with PD may differ from healthy older adults in how they utilise motion during facial emotion recognition. Nonetheless, gaze patterns indicate some effects of motion on emotional processing, highlighting the need for further investigation in this area.

Use and Usefulness of Dynamic Face Stimuli for Face Perception Studies-a Review of Behavioral Findings and Methodology

Faces that move contain rich information about facial form, such as facial features and their configuration, alongside the motion of those features. During social interactions, humans constantly decode and integrate these cues. To fully understand human face perception, it is important to investigate what information dynamic faces convey and how the human visual system extracts and processes information from this visual input. However, partly due to the difficulty of designing well-controlled dynamic face stimuli, many face perception studies still rely on static faces as stimuli. Here, we focus on evidence demonstrating the usefulness of dynamic faces as stimuli, and evaluate different types of dynamic face stimuli to study face perception. Studies based on dynamic face stimuli revealed a high sensitivity of the human visual system to natural facial motion and consistently reported dynamic advantages when static face information is insufficient for the task. These findings support the hypothesis that the human perceptual system integrates sensory cues for robust perception. In the present paper, we review the different types of dynamic face stimuli used in these studies, and assess their usefulness for several research questions. Natural videos of faces are ecological stimuli but provide limited control of facial form and motion. Point-light faces allow for good control of facial motion but are highly unnatural. Image-based morphing is a way to achieve control over facial motion while preserving the natural facial form. Synthetic facial animations allow separation of facial form and motion to study aspects such as identity-from-motion. While synthetic faces are less natural than videos of faces, recent advances in photo-realistic rendering may close this gap and provide naturalistic stimuli with full control over facial motion. We believe that many open questions, such as what dynamic advantages exist beyond emotion and identity recognition and which dynamic aspects drive these advantages, can be addressed adequately with different types of stimuli and will improve our understanding of face perception in more ecological settings.