Rapid reactions to direct and averted facial expressions of fear and anger

How facial masks alter the interaction of gaze direction, head orientation, and emotion recognition

The COVID-19 pandemic has altered the way we interact with each other: mandatory mask-wearing obscures facial information that is crucial for emotion recognition. Whereas the influence of wearing a mask on emotion recognition has been repeatedly investigated, little is known about the impact on interaction effects among emotional signals and other social signals. Therefore, the current study sought to explore how gaze direction, head orientation, and emotional expression interact with respect to emotion perception, and how these interactions are altered by wearing a face mask. In two online experiments, we presented face stimuli from the Radboud Faces Database displaying different facial expressions (anger, fear, happiness, neutral, and sadness), gaze directions (−13°, 0°, and 13°), and head orientations (−45°, 0°, and 45°) – either without (Experiment 1) or with mask (Experiment 2). Participants categorized the displayed emotional expressions. Not surprisingly, masks impaired emotion recognition. Surprisingly, without the mask, emotion recognition was unaffected by averted head orientations and only slightly affected by gaze direction. The mask strongly interfered with this ability. The mask increased the influence of head orientation and gaze direction, in particular for the emotions that were poorly recognized with mask. The results suggest that in case of uncertainty due to ambiguity or absence of signals, we seem to unconsciously factor in extraneous information.

Emotional Gaze: The Effects of Gaze Direction on the Perception of Facial Emotions

Previous research has found that when gaze direction matches the underlying behavioral intent communicated by the expression of a specific emotion, it enhances or facilitates the perception of that emotion; this is called the shared signal hypothesis (SSH). Specifically, a direct gaze shares an approach-orientated signal with the emotions of anger and joy, whereas an averted gaze shares an avoidance-orientated signal with fear and sadness. In this research, we attempted to verify the SSH by using different materials on Asian participants. In Experiment 1 we employed photos of models exhibiting direct and averted gazes for rating tasks, in order to study the effects of gaze direction on participants’ perception of emotion. In Experiment 2 we utilized smiling faces in a similar investigation. The results show that for neutral and smiling faces, a direct gaze (relative to a gaze of avoidance) increased the likelihood of a subject perceiving a happy mood; a gaze of avoidance increased the likelihood that anger and fear would be perceived. The effect of gaze direction on emotional expression perception was verified, but a “facilitating-impairing” pattern was not. The difference between our work and previous research may be attributable to the materials employed (which were more ecological), as well as the participants, who were from a different culture.

Dissociable effects of averted "gaze" on the priming of bodily representations and motor actions

Gaze direction is an important stimulus that signals key details about social (dis)engagement and objects in our physical environment. Here, we explore how gaze direction influences the perceiver's processing of bodily information. Specifically, we examined how averted versus direct gaze modifies the operation of effector-centered representations (i.e., specific fingers) versus movement-centered representations (i.e., finger actions). Study 1 used a stimulus-response compatibility paradigm that tested the priming of a relevant effector or relevant movement, after observing videos of direct or averted gaze. We found a selective priming of relevant effectors, but only after averted gaze videos. Study 2 found similar priming effects with symbolic direction cues (averted arrows). Study 3 found that averted gaze cues do not influence generic spatial compatibility effects, and thus, are specific to body representations. In sum, this research suggests that both human and symbolic averted cues selectively prime relevant body-part representations, highlighting the dynamic interplay between our bodies, minds, and environments.

Turning Heads: The Effects of Face View and Eye Gaze Direction on the Perceived Attractiveness of Expressive Faces

We investigated whether the perceived attractiveness of expressive faces was influenced by head turn and eye gaze towards or away from the observer. In all experiments, happy faces were consistently rated as more attractive than angry faces. A head turn towards the observer, whereby a full-face view was shown, was associated with relatively higher attractiveness ratings when gaze direction was aligned with face view (Experiment 1). However, preference for full-face views of happy faces was not affected by gaze shifts towards or away from the observer (Experiment 2a). In Experiment 3, the relative duration of each face view (front-facing or averted at 15°) during a head turn away or towards the observer was manipulated. There was benefit on attractiveness ratings for happy faces shown for a longer duration from the front view, regardless of the direction of head turn. Our findings support previous studies indicating a preference for positive expressions on attractiveness judgements, which is further enhanced by the front views of faces, whether presented during a head turn or shown statically. In sum, our findings imply a complex interaction between cues of social attention, indicated by the view of the face shown, and reward on attractiveness judgements of unfamiliar faces.

Magnocellular and parvocellular pathway contributions to facial threat cue processing

Human faces evolved to signal emotions, with their meaning contextualized by eye gaze. For instance, a fearful expression paired with averted gaze clearly signals both presence of threat and its probable location. Conversely, direct gaze paired with facial fear leaves the source of the fear-evoking threat ambiguous. Given that visual perception occurs in parallel streams with different processing emphases, our goal was to test a recently developed hypothesis that clear and ambiguous threat cues would differentially engage the magnocellular (M) and parvocellular (P) pathways, respectively. We employed two-tone face images to characterize the neurodynamics evoked by stimuli that were biased toward M or P pathways. Human observers (N = 57) had to identify the expression of fearful or neutral faces with direct or averted gaze while their magnetoencephalogram was recorded. Phase locking between the amygdaloid complex, orbitofrontal cortex (OFC) and fusiform gyrus increased early (0–300 ms) for M-biased clear threat cues (averted-gaze fear) in the β-band (13–30 Hz) while P-biased ambiguous threat cues (direct-gaze fear) evoked increased θ (4–8 Hz) phase locking in connections with OFC of the right hemisphere. We show that M and P pathways are relatively more sensitive toward clear and ambiguous threat processing, respectively, and characterize the neurodynamics underlying emotional face processing in the M and P pathways.

Sex-related differences in behavioral and amygdalar responses to compound facial threat cues

During face perception, we integrate facial expression and eye gaze to take advantage of their shared signals. For example, fear with averted gaze provides a congruent avoidance cue, signaling both threat presence and its location, whereas fear with direct gaze sends an incongruent cue, leaving threat location ambiguous. It has been proposed that the processing of different combinations of threat cues is mediated by dual processing routes: reflexive processing via magnocellular (M) pathway and reflective processing via parvocellular (P) pathway. Because growing evidence has identified a variety of sex differences in emotional perception, here we also investigated how M and P processing of fear and eye gaze might be modulated by observer's sex, focusing on the amygdala, a structure important to threat perception and affective appraisal. We adjusted luminance and color of face stimuli to selectively engage M or P processing and asked observers to identify emotion of the face. Female observers showed more accurate behavioral responses to faces with averted gaze and greater left amygdala reactivity both to fearful and neutral faces. Conversely, males showed greater right amygdala activation only for M-biased averted-gaze fear faces. In addition to functional reactivity differences, females had proportionately greater bilateral amygdala volumes, which positively correlated with behavioral accuracy for M-biased fear. Conversely, in males only the right amygdala volume was positively correlated with accuracy for M-biased fear faces. Our findings suggest that M and P processing of facial threat cues is modulated by functional and structural differences in the amygdalae associated with observer's sex.

Neurodynamics and connectivity during facial fear perception: The role of threat exposure and signal congruity

Fearful faces convey threat cues whose meaning is contextualized by eye gaze: While averted gaze is congruent with facial fear (both signal avoidance), direct gaze (an approach signal) is incongruent with it. We have previously shown using fMRI that the amygdala is engaged more strongly by fear with averted gaze during brief exposures. However, the amygdala also responds more to fear with direct gaze during longer exposures. Here we examined previously unexplored brain oscillatory responses to characterize the neurodynamics and connectivity during brief (~250 ms) and longer (~883 ms) exposures of fearful faces with direct or averted eye gaze. We performed two experiments: one replicating the exposure time by gaze direction interaction in fMRI (N = 23), and another where we confirmed greater early phase locking to averted-gaze fear (congruent threat signal) with MEG (N = 60) in a network of face processing regions, regardless of exposure duration. Phase locking to direct-gaze fear (incongruent threat signal) then increased significantly for brief exposures at ~350 ms, and at ~700 ms for longer exposures. Our results characterize the stages of congruent and incongruent facial threat signal processing and show that stimulus exposure strongly affects the onset and duration of these stages.

Observer's anxiety facilitates magnocellular processing of clear facial threat cues, but impairs parvocellular processing of ambiguous facial threat cues

Facial expression and eye gaze provide a shared signal about threats. While a fear expression with averted gaze clearly points to the source of threat, direct-gaze fear renders the source of threat ambiguous. Separable routes have been proposed to mediate these processes, with preferential attunement of the magnocellular (M) pathway to clear threat, and of the parvocellular (P) pathway to threat ambiguity. Here we investigated how observers’ trait anxiety modulates M- and P-pathway processing of clear and ambiguous threat cues. We scanned subjects (N = 108) widely ranging in trait anxiety while they viewed fearful or neutral faces with averted or directed gaze, with the luminance and color of face stimuli calibrated to selectively engage M- or P-pathways. Higher anxiety facilitated processing of clear threat projected to M-pathway, but impaired perception of ambiguous threat projected to P-pathway. Increased right amygdala reactivity was associated with higher anxiety for M-biased averted-gaze fear, while increased left amygdala reactivity was associated with higher anxiety for P-biased, direct-gaze fear. This lateralization was more pronounced with higher anxiety. Our findings suggest that trait anxiety differentially affects perception of clear (averted-gaze fear) and ambiguous (direct-gaze fear) facial threat cues via selective engagement of M and P pathways and lateralized amygdala reactivity.

The processing of body expressions during emotional scenes: the modulation role of attachment styles

There is broad evidence indicating that contextual information influence the processing of emotional stimuli. However, attachment theory suggests that attachment styles contribute to the ways in which people perceive emotional events. To shed light on whether the processing of body expressions during different emotional scenes is modulated by attachment styles, attachment-related electrophysiological differences were measured using event-related potentials. For avoidantly attached group, our results suggested that larger N170 amplitudes were educed by neutral bodies than angry bodies, which was found only in neutral scene. Moreover, significant differences were found in P300 amplitudes in response to angry bodies compared with neutral ones only during angry scene. However, securely and anxiously attached individuals were associated with larger P300 amplitudes in response to angry bodies versus neutral ones in both emotional scenes. The current study highlights the characteristics of cognitive processing of attachment styles on body expressions during different emotional scenes, with the variation of N170 and P300 amplitude in different emotional scenes as the best example.

Neurocognitive mechanisms of gaze-expression interactions in face processing and social attention

The face conveys a rich source of non-verbal information used during social communication. While research has revealed how specific facial channels such as emotional expression are processed, little is known about the prioritization and integration of multiple cues in the face during dyadic exchanges. Classic models of face perception have emphasized the segregation of dynamic vs. static facial features along independent information processing pathways. Here we review recent behavioral and neuroscientific evidence suggesting that within the dynamic stream, concurrent changes in eye gaze and emotional expression can yield early independent effects on face judgments and covert shifts of visuospatial attention. These effects are partially segregated within initial visual afferent processing volleys, but are subsequently integrated in limbic regions such as the amygdala or via reentrant visual processing volleys. This spatiotemporal pattern may help to resolve otherwise perplexing discrepancies across behavioral studies of emotional influences on gaze-directed attentional cueing. Theoretical explanations of gaze-expression interactions are discussed, with special consideration of speed-of-processing (discriminability) and contextual (ambiguity) accounts. Future research in this area promises to reveal the mental chronometry of face processing and interpersonal attention, with implications for understanding how social referencing develops in infancy and is impaired in autism and other disorders of social cognition.

Amygdala responses to averted vs direct gaze fear vary as a function of presentation speed

We examined whether amygdala responses to rapidly presented fear expressions are preferentially tuned to averted vs direct gaze fear and conversely whether responses to more sustained presentations are preferentially tuned to direct vs averted gaze fear. We conducted three functional magnetic resonance imaging (fMRI) studies to test these predictions including: Study 1: a block design employing sustained presentations (1 s) of averted vs direct gaze fear expressions taken from the Pictures of Facial Affect; Study 2: a block design employing rapid presentations (300 ms) of these same stimuli and Study 3: a direct replication of these studies in the context of a single experiment using stimuli selected from the NimStim Emotional Face Stimuli. Together, these studies provide evidence consistent with an early, reflexive amygdala response tuned to clear threat and a later reflective response tuned to ambiguous threat.