Biases in the perception and affective valence of neutral facial expressions induced by the immediate perceptual history

Vicarious facilitation of facial responses to pain: Does the others' expression need to be painful?

INTRODUCTION

Prior exposure to others' facial expressions of pain can lead to a facilitation of pain responses, including its corresponding response channel, namely facial responses to pain. It has been questioned, however, whether this vicarious pain facilitation occurs only when observing others' pain or whether the observation of other negative expressions can trigger similar facilitation of facial responses to pain. The study aimed to test this, by comparing the impact of viewing others' facial expressions of pain versus another negative expression (sadness) and two control expressions (neutral, happiness) on facial responses to pain.

METHOD

Participants (N = 56; 31 females), watched short video clips of computer-generated facial expressions (pain, sadness, neutral & happiness) before they received painful and non-painful heat stimuli. Facial responses were analysed using the Facial Action Coding System. In addition, subjective and autonomic responses were assessed.

RESULTS

The prior exposure to others' expressions of pain and sadness versus neutral did not lead to significantly increased facial responses to pain. Likewise, subjective and autonomic pain responses were not facilitated. However, viewing others' expressions of happiness, consistently reduced facial as well as subjective and autonomic responses to pain compared to others' negative or neutral expressions. This dampening effect was not observed for non-painful heat.

DISCUSSION

Facial and other pain responses were most strongly affected by prior exposure to others' facial expressions of happiness, which led to a pain-dampening effect. In contrast, the evidence for vicarious facilitation of pain was rather weak in the present study, with no evidence of pain-specificity.

SIGNIFICANCE STATEMENT

Facial responses to pain - along with subjective and autonomic responses - are reduced when observing others' expressions of happiness, demonstrating pain modulation by positive affective social signals, which may also transfer to clinical contexts.

Emotional anticipation for dynamic emotional faces is not modulated by schizotypal traits: A Representational Momentum study

Schizotypy, a personality structure that resembles schizophrenia symptoms, is often associated with abnormal facial emotion perception. Based on the prevailing sense of threat in psychotic experiences, and the immediate perceptual history of seeing others' facial expressions, individuals with high schizotypal traits may exhibit a heightened tendency to anticipate anger. To test this, we used insights from Representational Momentum (RM), a perceptual phenomenon in which the endpoint of a dynamic event is systematically displaced forward, into the immediate future. Angry-to-ambiguous and happy-to-ambiguous avatar faces were presented, each followed by a probe with the same (ambiguous) expression as the endpoint, or one slightly changed to express greater happiness/anger. Participants judged if the probe was "equal" to the endpoint and rated how confident they were. The sample was divided into high (N = 46) and low (N = 49) schizotypal traits using the Schizotypal Personality Questionnaire (SPQ). First, a forward bias was found in happy-to-ambiguous faces, suggesting emotional anticipation solely for dynamic faces changing towards a potential threat (anger). This may reflect an adaptative mechanism, as it is safer to anticipate any hostility from a conspecific than the opposite. Second, contrary to our hypothesis, high schizotypal traits did not heighten RM for happy-to-ambiguous faces, nor did they lead to overconfidence in biased judgements. This may suggest a typical pattern of emotional anticipation in non-clinical schizotypy, but caution is needed due to the use of self-report questionnaires, university students, and a modest sample size. Future studies should also investigate if the same holds for clinical manifestations of schizophrenia.

Are adaptation aftereffects for facial emotional expressions affected by prior knowledge about the emotion?

Accurate perception of the emotional signals conveyed by others is crucial for successful social interaction. Such perception is influenced not only by sensory input, but also by knowledge we have about the others' emotions. This study addresses the issue of whether knowing that the other's emotional state is congruent or incongruent with their displayed emotional expression ("genuine" and "fake", respectively) affects the neural mechanisms underpinning the perception of their facial emotional expressions. We used a visual adaptation paradigm to investigate this question in three experiments employing increasing adaptation durations. The adapting stimuli consisted of photographs of emotional facial expressions of joy and anger, purported to reflect (in-)congruency between felt and expressed emotion, displayed by professional actors. A Validity checking procedure ensured participants had the correct knowledge about the (in-)congruency. Significantly smaller adaptation aftereffects were obtained when participants knew that the displayed expression was incongruent with the felt emotion, following all tested adaptation periods. This study shows that knowledge relating to the congruency between felt and expressed emotion modulates face expression aftereffects. We argue that this reflects that the neural substrate responsible for the perception of facial expressions of emotion incorporates the presumed felt emotion underpinning the expression.

Perception of “Live” Facial Expressions

The article provides a review of experimental studies of interpersonal perception on the material of static and dynamic facial expressions as a unique source of information about the person’s inner world. The focus is on the patterns of perception of a moving face, included in the processes of communication and joint activities (an alternative to the most commonly studied perception of static images of a person outside of a behavioral context). The review includes four interrelated topics: face statics and dynamics in the recognition of emotional expressions; specificity of perception of moving face expressions; multimodal integration of emotional cues; generation and perception of facial expressions in communication processes. The analysis identifies the most promising areas of research of face in motion. We show that the static and dynamic modes of facial perception complement each other, and describe the role of qualitative features of the facial expression dynamics in assessing the emotional state of a person. Facial expression is considered as part of a holistic multimodal manifestation of emotions. The importance of facial movements as an instrument of social interaction is emphasized.

The Role of Pattern Extrapolation in the Perception of Dynamic Facial Expressions in Autism Spectrum Disorder

Changes in the intensity and type of facial expressions reflect alterations in the emotional state of the agent. Such "direct" access to the other's affective state might, top-down, influence the perception of the facial expressions that gave rise to the affective state inference. Previously, we described a perceptual bias occurring when the last, neutral, expression of offsets of facial expressions (joy-to-neutral and anger-to-neutral), was evaluated. Individuals with high-functioning autism (HFA) and matched typically developed (TD) individuals rated the neutral expression at the end of the joy-offset videos as slightly angry and the identical neutral expression at the end of the anger-offset videos as slightly happy ("overshoot" bias). That study suggested that the perceptual overshoot response bias in the TD group could be best explained by top-down "emotional anticipation," i.e., the involuntary/automatic anticipation of the agent's next emotional state of mind, generated by the immediately preceding perceptual history (low-level mind reading). The experimental manipulations further indicated that in the HFA group the "overshoot" was better explained by contrast effects between the first and last facial expressions, both presented for a relatively long period of 400 ms. However, in principle, there is another, more parsimonious, explanation, which is pattern extrapolation or representational momentum (RM): the extrapolation of a pattern present in the dynamic sequence. This hypothesis is tested in the current study, in which 18 individuals with HFA and a matched control group took part. In a base-line condition, joy-offset and anger-offset video-clips were presented. In the new experimental condition, the clips were modified so as to create an offset-onset-offset pattern within each sequence (joy-to-anger-to-neutral and anger-to-joy-to-neutral). The final neutral expressions had to be evaluated. The overshoot bias was confirmed in the base-line condition for both TD and HFA groups, while the experimental manipulation removed the bias in both groups. This outcome ruled out pattern extrapolation or RM as explanation for the perceptual "overshoot" bias in the HFA group and suggested a role for facial contrast effects in HFA. This is compatible with the view that ASD individuals tend to lack the spontaneous "tracking" of changes in the others' affective state and hence show no or reduced emotional anticipation.

The neuroelectric dynamics of the emotional anticipation of other people's pain

When we observe a dynamic emotional facial expression, we usually automatically anticipate how that expression will develop. Our objective was to study a neurocognitive biomarker of this anticipatory process for facial pain expressions, operationalized as a mismatch effect. For this purpose, we studied the behavioral and neuroelectric (Event-Related Potential, ERP) correlates, of a match or mismatch, between the intensity of an expression of pain anticipated by the participant, and the intensity of a static test expression of pain displayed with the use of a representational momentum paradigm. Here, the paradigm consisted in displaying a dynamic facial pain expression which suddenly disappeared, and participants had to memorize the final intensity of the dynamic expression. We compared ERPs in response to congruent (intensity the same as the one memorized) and incongruent (intensity different from the one memorized) static expression intensities displayed after the dynamic expression. This paradigm allowed us to determine the amplitude and direction of this intensity anticipation by measuring the observer's memory bias. Results behaviorally showed that the anticipation was backward (negative memory bias) for high intensity expressions of pain (participants expected a return to a neutral state) and more forward (memory bias less negative, or even positive) for less intense expressions (participants expected increased intensity). Detecting mismatch (incongruent intensity) led to faster responses than detecting match (congruent intensity). The neuroelectric correlates of this mismatch effect in response to the testing of expression intensity ranged from P100 to LPP (Late Positive Potential). Path analysis and source localization suggested that the medial frontal gyrus was instrumental in mediating the mismatch effect through top-down influence on both the occipital and temporal regions. Moreover, having the facility to detect incongruent expressions, by anticipating emotional state, could be useful for prosocial behavior and the detection of trustworthiness.

Person perception from changing emotional expressions: primacy, recency, or averaging effect?

Dynamic changes in emotional expressions are a valuable source of information in social interactions. As the expressive behaviour of a person changes, the inferences drawn from the behaviour may also change. Here, we test the possibility that dynamic changes in emotional expressions affect person perception in terms of stable trait attributions. Across three experiments, we examined perceivers' inferences about others' personality traits from changing emotional expressions. Expressions changed from one emotion ("start emotion") to another emotion ("end emotion"), allowing us to disentangle potential primacy, recency, and averaging effects. Drawing on three influential models of person perception, we examined perceptions of dominance and affiliation (Experiment 1a), competence and warmth (Experiment 1b), and dominance and trustworthiness (Experiment 2). A strong recency effect was consistently found across all trait judgments, that is, the end emotion of dynamic expressions had a strong impact on trait ratings. Evidence for a primacy effect was also observed (i.e. the information of start emotions was integrated), but less pronounced, and only for trait ratings relating to affiliation, warmth, and trustworthiness. Taken together, these findings suggest that, when making trait judgements about others, observers weigh the most recently displayed emotion in dynamic expressions more heavily than the preceding emotion.

The role of temporal inversion in the perception of realistic and morphed dynamic transitions between facial expressions

Recent studies suggest that video recordings of human facial expressions are perceived differently than linear morphing between the first and last frames of these records. Also, observers can differentiate dynamic expressions presented in normal versus time-reversed frame orders. To date, the simultaneous influence of dynamics (natural or linear) and timeline (normal or reversed) has not yet been tested on a wide range of dynamic emotional expressions and the transitions between them. We compared the perception of dynamic transitions between basic emotions in realistic (human-posed) and artificial (linearly morphed) stimuli which were presented in reversed or non-reversed order. The nonlinearity of realistic stimuli was demonstrated by automated facial structure analysis. The results of the behavioral study revealed that the recognition of emotions in time-reversed stimuli significantly differed from recognition of the normally presented ones, and this difference was substantially higher for videos of a dynamic human face than for linear morphs. Emotions displayed at the end of the transitions were recognized better than the first-frame emotions in all types of stimuli except in the time-reversed videos, which showed a similar recognition rate for both the starting and ending emotions. Our findings suggest that nonlinearity, which is present in a realistic facial display but absent in linear morphing, is an important cue for emotion perception, and that unnatural perceptual conditions (inversion in time) make the recognition of emotions more difficult. These results confirm the ability of the human visual system to use subtle dynamic cues on an interlocutor's face, and reveal its sensitivity to the timeline organization of the displayed emotions.

Context sensitivity in the detection of changes in facial emotion

In social contexts, reading subtle changes in others’ facial expressions is a crucial communication skill. To measure this ability, we developed an expression-change detection task, wherein a series of pictures of changes in an individual’s facial expressions within contextual scenes were presented. The results demonstrated that the detection of subtle changes was highly sensitive to contextual stimuli. That is, participants identified the direction of facial-expression changes more accurately and more quickly when they were ‘appropriate’—consistent with the valence of the contextual stimulus change—than when they were ‘inappropriate’. Moreover, individual differences in sensitivity to contextual stimuli were correlated with scores on the Toronto Alexithymia Scale, a commonly used measure of alexithymia tendencies. These results suggest that the current behavioural task might facilitate investigations of the role of context in human social cognition.

Atypical emotional anticipation in high-functioning autism

BACKGROUND

Understanding and anticipating others' mental or emotional states relies on the processing of social cues, such as dynamic facial expressions. Individuals with high-functioning autism (HFA) may process these cues differently from individuals with typical development (TD) and purportedly use a 'mechanistic' rather than a 'mentalistic' approach, involving rule- and contingency-based interpretations of the stimuli. The study primarily aimed at examining whether the judgments of facial expressions made by individuals with TD and HFA would be similarly affected by the immediately preceding dynamic perceptual history of that face. A second aim was to explore possible differences in the mechanisms underpinning the perceptual judgments in the two groups.

METHODS

Twenty-two adults with HFA and with TD, matched for age, gender and IQ, were tested in three experiments in which dynamic, 'ecologically valid' offsets of happy and angry facial expressions were presented. Participants evaluated the expression depicted in the last frame of the video clip by using a 5-point scale ranging from slightly angry via neutral to slightly happy. Specific experimental manipulations prior to the final facial expression of the video clip allowed examining contributions of bottom-up mechanisms (sequential contrast/context effects and representational momentum) and a top-down mechanism (emotional anticipation) to distortions in the perception of the final expression.

RESULTS

In experiment 1, the two groups showed a very similar perceptual bias for the final expression of joy-to-neutral and anger-to-neutral videos (overshoot bias). In experiment 2, a change in the actor's identity during the clip removed the bias in the TD group, but not in the HFA group. In experiment 3, neutral-to-joy/anger-to-neutral sequences generated an undershoot bias (opposite to the overshoot) in the TD group, whereas no bias was observed in the HFA group.

CONCLUSIONS

We argue that in TD individuals the perceptual judgments of other's facial expressions were underpinned by an automatic emotional anticipation mechanism. In contrast, HFA individuals were primarily influenced by visual features, most notably the contrast between the start and end expressions, or pattern extrapolation. We critically discuss the proposition that automatic emotional anticipation may be induced by motor simulation of the perceived dynamic facial expressions and discuss its implications for autism.

Perception of temporal asymmetries in dynamic facial expressions

In the current study we examined whether timeline-reversals and emotional direction of dynamic facial expressions affect subjective experience of human observers. We recorded natural movies of faces that increased or decreased their expressions of fear, and played them either in the natural frame order or reversed from last to first frame (reversed timeline). This led to four conditions of increasing or decreasing fear, either following the natural or reversed temporal trajectory of facial dynamics. This 2-by-2 factorial design controlled for visual low-level properties, static visual content, and motion energy across the different factors. It allowed us to examine perceptual consequences that would occur if the timeline trajectory of facial muscle movements during the increase of an emotion are not the exact mirror of the timeline during the decrease. It additionally allowed us to study perceptual differences between increasing and decreasing emotional expressions. Perception of these time-dependent asymmetries have not yet been quantified. We found that three emotional measures, emotional intensity, artificialness of facial movement, and convincingness or plausibility of emotion portrayal, were affected by timeline-reversals as well as by the emotional direction of the facial expressions. Our results imply that natural dynamic facial expressions contain temporal asymmetries, and show that deviations from the natural timeline lead to a reduction of perceived emotional intensity and convincingness, and to an increase of perceived artificialness of the dynamic facial expression. In addition, they show that decreasing facial expressions are judged as less plausible than increasing facial expressions. Our findings are of relevance for both, behavioral as well as neuroimaging studies, as processing and perception are influenced by temporal asymmetries.

Facial motion engages predictive visual mechanisms

We employed a novel cuing paradigm to assess whether dynamically versus statically presented facial expressions differentially engaged predictive visual mechanisms. Participants were presented with a cueing stimulus that was either the static depiction of a low intensity expressed emotion; or a dynamic sequence evolving from a neutral expression to the low intensity expressed emotion. Following this cue and a backwards mask, participants were presented with a probe face that displayed either the same emotion (congruent) or a different emotion (incongruent) with respect to that displayed by the cue although expressed at a high intensity. The probe face had either the same or different identity from the cued face. The participants' task was to indicate whether or not the probe face showed the same emotion as the cue. Dynamic cues and same identity cues both led to a greater tendency towards congruent responding, although these factors did not interact. Facial motion also led to faster responding when the probe face was emotionally congruent to the cue. We interpret these results as indicating that dynamic facial displays preferentially invoke predictive visual mechanisms, and suggest that motoric simulation may provide an important basis for the generation of predictions in the visual system.

Beyond face value: does involuntary emotional anticipation shape the perception of dynamic facial expressions?

Emotional facial expressions are immediate indicators of the affective dispositions of others. Recently it has been shown that early stages of social perception can already be influenced by (implicit) attributions made by the observer about the agent's mental state and intentions. In the current study possible mechanisms underpinning distortions in the perception of dynamic, ecologically-valid, facial expressions were explored. In four experiments we examined to what extent basic perceptual processes such as contrast/context effects, adaptation and representational momentum underpinned the perceptual distortions, and to what extent 'emotional anticipation', i.e. the involuntary anticipation of the other's emotional state of mind on the basis of the immediate perceptual history, might have played a role. Neutral facial expressions displayed at the end of short video-clips, in which an initial facial expression of joy or anger gradually morphed into a neutral expression, were misjudged as being slightly angry or slightly happy, respectively (Experiment 1). This response bias disappeared when the actor's identity changed in the final neutral expression (Experiment 2). Videos depicting neutral-to-joy-to-neutral and neutral-to-anger-to-neutral sequences again produced biases but in opposite direction (Experiment 3). The bias survived insertion of a 400 ms blank (Experiment 4). These results suggested that the perceptual distortions were not caused by any of the low-level perceptual mechanisms (adaptation, representational momentum and contrast effects). We speculate that especially when presented with dynamic, facial expressions, perceptual distortions occur that reflect 'emotional anticipation' (a low-level mindreading mechanism), which overrules low-level visual mechanisms. Underpinning neural mechanisms are discussed in relation to the current debate on action and emotion understanding.

Repeated short presentations of morphed facial expressions change recognition and evaluation of facial expressions

This study investigated whether sensitivity to and evaluation of facial expressions varied with repeated exposure to non-prototypical facial expressions for a short presentation time. A morphed facial expression was presented for 500 ms repeatedly, and participants were required to indicate whether each facial expression was happy or angry. We manipulated the distribution of presentations of the morphed facial expressions for each facial stimulus. Some of the individuals depicted in the facial stimuli expressed anger frequently (i.e., anger-prone individuals), while the others expressed happiness frequently (i.e., happiness-prone individuals). After being exposed to the faces of anger-prone individuals, the participants became less sensitive to those individuals' angry faces. Further, after being exposed to the faces of happiness-prone individuals, the participants became less sensitive to those individuals' happy faces. We also found a relative increase in the social desirability of happiness-prone individuals after exposure to the facial stimuli.

Upper-lower visual field asymmetries in oculomotor inhibition of emotional distractors

The present study investigated oculomotor inhibition of emotional faces as indicated by saccade curvatures. In Experiment 1, participants saccaded towards a target that appeared above or below fixation while single facial distractors depicting neutral, happy, and angry expressions appeared in one of the four quadrants of the screen. In Experiment 2, participants selected between two objects on the screen by saccading towards a predefined target, while again single facial emotional distractors were presented in one of the four screen quadrants. In both experiments, saccade trajectories curved most strongly away from angry distractors indicating enhanced attentional capture by angry faces. This effect occurred with upright faces but not with inverted faces. The emotion effect was restricted to targets at the lower vertical meridian. The lower visual field has been argued to be specialized for action in peripersonal space and near vision. The modulation by target location might be attributed to activation of near space representation by saccades toward a lower target, inducing increased vigilance for stimuli of action relevance to protect the peripersonal space from interference.