Processing of gaze direction within the N170/M170 time window: A combined EEG/MEG study

The Timing of Gaze Direction Perception: ERP Decoding and Task Modulation

Distinguishing the direction of another person's eye gaze is extremely important in everyday social interaction, as it provides critical information about people's attention and, therefore, intentions. The temporal dynamics of gaze processing have been investigated using event-related potentials (ERPs) recorded with electroencephalography (EEG). However, the moment at which our brain distinguishes the gaze direction (GD), irrespectively of other facial cues, remains unclear. To solve this question, the present study aimed to investigate the time course of gaze direction processing, using an ERP decoding approach, based on the combination of a support vector machine and error-correcting output codes. We recorded EEG in young healthy subjects, 32 of them performing GD detection and 34 conducting face orientation tasks. Both tasks presented 3D realistic faces with five different head and gaze orientations each: 30°, 15° to the left or right, and 0°. While the classical ERP analyses did not show clear GD effects, ERP decoding analyses revealed that discrimination of GD, irrespective of head orientation, started at 140 ms in the GD task and at 120 ms in the face orientation task. GD decoding accuracy was higher in the GD task than in the face orientation task and was the highest for the direct gaze in both tasks. These findings suggest that the decoding of brain patterns is modified by task relevance, which changes the latency and the accuracy of GD decoding.

Aberrant Effective Connectivity During Eye Gaze Processing Is Linked to Social Functioning and Symptoms in Schizophrenia

BACKGROUND

Patients with schizophrenia show abnormal gaze processing, which is associated with social dysfunction. These abnormalities are related to aberrant connectivity among brain regions that are associated with visual processing, social cognition, and cognitive control. In this study, we investigated 1) how effective connectivity during gaze processing is disrupted in schizophrenia and 2) how this may contribute to social dysfunction and clinical symptoms.

METHODS

Thirty-nine patients with schizophrenia/schizoaffective disorder (SZ) and 33 healthy control participants completed an eye gaze processing task during functional magnetic resonance imaging. Participants viewed faces with different gaze angles and performed explicit and implicit gaze processing. Four brain regions-the secondary visual cortex, posterior superior temporal sulcus, inferior parietal lobule, and posterior medial frontal cortex-were identified as nodes for dynamic causal modeling analysis.

RESULTS

Both the SZ and healthy control groups showed similar model structures for general gaze processing. Explicit gaze discrimination led to changes in effective connectivity, including stronger excitatory, bottom-up connections from the secondary visual cortex to the posterior superior temporal sulcus and inferior parietal lobule and inhibitory, top-down connections from the posterior medial frontal cortex to the secondary visual cortex. Group differences in top-down modulation from the posterior medial frontal cortex to the posterior superior temporal sulcus and inferior parietal lobule were noted, such that these inhibitory connections were attenuated in the healthy control group but further strengthened in the SZ group. Connectivity was associated with social dysfunction and symptom severity.

CONCLUSIONS

The SZ group showed notably stronger top-down inhibition during explicit gaze discrimination, which was associated with more social dysfunction but less severe symptoms among patients. These findings help pinpoint neural mechanisms of aberrant gaze processing and may serve as future targets for interventions that combine neuromodulation with social cognitive training.

Association between attachment anxiety and the gaze direction-related N170

Attachment theory suggests that interindividual differences in attachment security versus insecurity (anxiety and avoidance) contribute to the ways in which people perceive social emotional signals, particularly from the human face. Among different facial features, eye gaze conveys crucial information for social interaction, with a straight gaze triggering different cognitive and emotional processes as compared to an averted gaze. It remains unknown, however, how interindividual differences in attachment associate with early face encoding in the context of a straight versus averted gaze. Using electroencephalography (EEG) and recording event-related potentials (ERPs), specifically the N170 component, the present study (N = 50 healthy adults) measured how the characteristics of attachment anxiety and avoidance relate to the encoding of faces with respect to gaze direction and head orientation. Our findings reveal a significant relationship between gaze direction (irrespective of head orientation) and attachment anxiety on the interhemispheric (i.e. right) asymmetry of the N170 and thus provide evidence for an association between attachment anxiety and eye gaze processing during early visual face encoding.

Contrast reversal of the iris and sclera increases the face sensitive N170

Previous research has demonstrated that reversing the contrast of the eye region, which includes the eyebrows, affects the N170 ERP. To selectively assess the impact of just the eyes, the present study evaluated the N170 in response to reversing contrast polarity of just the iris and sclera in upright and inverted face stimuli. Contrast reversal of the eyes increased the amplitude of the N170 for upright faces, but not for inverted faces, suggesting that the contrast of eyes is an important contributor to the N170 ERP.

Dissociation of early and late face-related processes in autism spectrum disorder and Williams syndrome

Background

Williams syndrome (WS) and Autism Spectrum Disorders (ASD) are neurodevelopmental conditions associated with atypical but opposite face-to-face interactions patterns: WS patients overly stare at others, ASD individuals escape eye contact. Whether these behaviors result from dissociable visual processes within the occipito-temporal pathways is unknown.

Using high-density electroencephalography, multivariate signal processing algorithms and a protocol designed to identify and extract evoked activities sensitive to facial cues, we investigated how WS (N = 14), ASD (N = 14) and neurotypical subjects (N = 14) decode the information content of a face stimulus.

Results

We found two neural components in neurotypical participants, both strongest when the eye region was projected onto the subject's fovea, simulating a direct eye contact situation, and weakest over more distant regions, reaching a minimum when the focused region was outside the stimulus face. The first component peaks at 170 ms, an early signal known to be implicated in low-level face features. The second is identified later, 260 ms post-stimulus onset and is implicated in decoding salient face social cues.

Remarkably, both components were found distinctly impaired and preserved in WS and ASD. In WS, we could weakly decode the 170 ms signal based on our regressor relative to facial features, probably due to their relatively poor ability to process faces’ morphology, while the late 260 ms component was highly significant. The reverse pattern was observed in ASD participants who showed neurotypical like early 170 ms evoked activity but impaired late evoked 260 ms signal.

Conclusions

Our study reveals a dissociation between WS and ASD patients and points at different neural origins for their social impairments.

Eye Direction Detection and Perception as Premises of a Social Brain: A Narrative Review of Behavioral and Neural Data

The eyes and the gaze are important stimuli for social interaction in humans. Impaired recognition of facial identity, facial emotions, and inference of the intentions of others may result from difficulties in extracting information relevant to the eye region, mainly the direction of gaze. Therefore, a review of these data is of interest. Behavioral data demonstrating the importance of the eye region and how humans respond to gaze direction are reviewed narratively, and several theoretical models on how visual information on gaze is processed are discussed to propose a unified hypothesis. Several issues that have not yet been investigated are identified. The authors tentatively suggest experiments that might help progress research in this area. The neural aspects are subsequently reviewed to best describe the low-level and higher-level visual information processing stages in the targeted subcortical and cortical areas. A specific neural network is proposed on the basis of the literature. Various gray areas, such as the temporality of the processing of visual information, the question of salience priority, and the coordination between the two hemispheres, remain unclear and require further investigations. Finally, disordered gaze direction detection mechanisms and their consequences on social cognition and behavior are discussed as key deficiencies in several conditions, such as autism spectrum disorder, 22q11.2 deletion, schizophrenia, and social anxiety disorder. This narrative review provides significant additional data showing that the detection and perception of someone's gaze is an essential part of the development of our social brain.

I can see it in your eyes: Perceived gaze direction impacts ERP and behavioural measures of affective theory of mind

Looking at someone's eyes is thought to be important for affective theory of mind (aTOM), our ability to infer their emotional state. However, it is unknown whether an individual's gaze direction influences our aTOM judgements and what the time course of this influence might be. We presented participants with sentences describing individuals in positive, negative or neutral scenarios, followed by direct or averted gaze neutral face pictures of those individuals. Participants made aTOM judgements about each person's mental state, including their affective valence and arousal, and we investigated whether the face gaze direction impacted those judgements. Participants rated that gazers were feeling more positive when they displayed direct gaze as opposed to averted gaze, and that they were feeling more aroused during negative contexts when gaze was averted as opposed to direct. Event-related potentials associated with face perception and affective processing were examined using mass-univariate analyses to track the time-course of this eye-gaze and affective processing interaction at a neural level. Both positive and negative trials were differentiated from neutral trials at many stages of processing. This included the early N200 and EPN components, believed to reflect automatic emotion areas activation and attentional selection respectively. This also included the later P300 and LPP components, thought to reflect elaborative cognitive appraisal of emotional content. Critically, sentence valence and gaze direction interacted over these later components, which may reflect the incorporation of eye-gaze in the cognitive evaluation of another's emotional state. The results suggest that gaze perception directly impacts aTOM processes, and that altered eye-gaze processing in clinical populations may contribute to associated aTOM impairments.

The roles of gaze and head orientation in face categorization during rapid serial visual presentation

Little is known about how perceived gaze direction and head orientation may influence human categorization of visual stimuli as faces. To address this question, a sequence of unsegmented natural images, each containing a random face or a non-face object, was presented in rapid succession (stimulus duration: 91.7 ms per image) during which human observers were instructed to respond immediately to every face presentation. Faces differed in gaze and head orientation in 7 combinations - full-front views with perceived gaze (1) directed to the observer, (2) averted to the left, or (3) averted to the right, left ¾ side views with (4) direct gaze or (5) averted gaze, and right ¾ side views with (6) direct gaze or (7) averted gaze - were presented randomly throughout the sequence. We found highly accurate and rapid behavioural responses to all kinds of faces. Crucially, both perceived gaze direction and head orientation had comparable, non-interactive effects on response times, where direct gaze was responded faster than averted gaze by 48 ms and full-front view faster than ¾ side view also by 48 ms on average. Presentations of full-front faces with direct gaze led to an additive speed advantage of 96 ms to ¾ faces with averted gaze. The results reveal that the effects of perceived gaze direction and head orientation on the speed of face categorization probably depend on the degree of social relevance of the face to the viewer.

From Gaze Perception to Social Cognition: The Shared-Attention System

When two people look at the same object in the environment and are aware of each other's attentional state, they find themselves in a shared-attention episode. This can occur through intentional or incidental signaling and, in either case, causes an exchange of information between the two parties about the environment and each other's mental states. In this article, we give an overview of what is known about the building blocks of shared attention (gaze perception and joint attention) and focus on bringing to bear new findings on the initiation of shared attention that complement knowledge about gaze following and incorporate new insights from research into the sense of agency. We also present a neurocognitive model, incorporating first-, second-, and third-order social cognitive processes (the shared-attention system, or SAS), building on previous models and approaches. The SAS model aims to encompass perceptual, cognitive, and affective processes that contribute to and follow on from the establishment of shared attention. These processes include fundamental components of social cognition such as reward, affective evaluation, agency, empathy, and theory of mind.

Meeting another's gaze shortens subjective time by capturing attention

Gaze directed at the observer (direct gaze) is an important and highly salient social signal with multiple effects on cognitive processes and behavior. It is disputed whether the effect of direct gaze is caused by attentional capture or increased arousal. Time estimation may provide an answer because attentional capture predicts an underestimation of time whereas arousal predicts an overestimation. In a temporal bisection task, observers were required to classify the duration of a stimulus as short or long. Stimulus duration was selected randomly between 988 and 1479 ms. When gaze was directed at the observer, participants underestimated stimulus duration, suggesting that effects of direct gaze are caused by attentional capture, not increased arousal. Critically, this effect was limited to dynamic stimuli where gaze appeared to move toward the participant. The underestimation was present with stimuli showing a full face, but also with stimuli showing only the eye region, inverted faces and high-contrast eye-like stimuli. However, it was absent with static pictures of full faces and dynamic nonfigurative stimuli. Because the effect of direct gaze depended on motion, which is common in naturalistic scenes, more consideration needs to be given to the ecological validity of stimuli in the study of social attention.

Dynamic causal modeling of eye gaze processing in schizophrenia

BACKGROUND

Abnormal eye gaze perception is related to symptoms and social functioning in schizophrenia. However, little is known about the brain network mechanisms underlying these abnormalities. Here, we employed dynamic causal modeling (DCM) of fMRI data to discover aberrant effective connectivity within networks associated with eye gaze processing in schizophrenia.

METHODS

Twenty-seven patients (schizophrenia/schizoaffective disorder, SZ) and 22 healthy controls (HC) completed an eye gaze processing task during fMRI. Participants viewed faces with different gaze angles and performed explicit gaze discrimination (Gaze: "Looking at you?" yes/no) or implicit gaze processing (Gender: "male or female?"). Four brain regions, the secondary visual cortex (Vis), posterior superior temporal sulcus (pSTS), inferior parietal lobule (IPL), and posterior medial frontal cortex (pMFC) were identified as nodes for subsequent DCM analysis.

RESULTS

SZ and HC showed similar generative model structure, but SZ showed altered connectivity for specific self-connections, inter-regional connections during all gaze processing (reduced excitatory bottom-up and enhanced inhibitory top-down connections), and modulation by explicit gaze discrimination (increased frontal inhibition of visual cortex). Altered effective connectivity was significantly associated with poorer social cognition and functioning.

CONCLUSIONS

General gaze processing in SZ is associated with distributed cortical dysfunctions and bidirectional connectivity between regions, while explicit gaze discrimination involves predominantly top-down abnormalities in the visual system. These results suggest plausible neural mechanisms underpinning gaze processing deficits and may serve as bio-markers for intervention.

MEG signatures of long-term effects of agreement and disagreement with the majority

People often change their beliefs by succumbing to an opinion of others. Such changes are often referred to as effects of social influence. While some previous studies have focused on the reinforcement learning mechanisms of social influence or on its internalization, others have reported evidence of changes in sensory processing evoked by social influence of peer groups. In this study, we used magnetoencephalographic (MEG) source imaging to further investigate the long-term effects of agreement and disagreement with the peer group. The study was composed of two sessions. During the first session, participants rated the trustworthiness of faces and subsequently learned group rating of each face. In the first session, a neural marker of an immediate mismatch between individual and group opinions was found in the posterior cingulate cortex, an area involved in conflict-monitoring and reinforcement learning. To identify the neural correlates of the long-lasting effect of the group opinion, we analysed MEG activity while participants rated faces during the second session. We found MEG traces of past disagreement or agreement with the peers at the parietal cortices 230 ms after the face onset. The neural activity of the superior parietal lobule, intraparietal sulcus, and precuneus was significantly stronger when the participant’s rating had previously differed from the ratings of the peers. The early MEG correlates of disagreement with the majority were followed by activity in the orbitofrontal cortex 320 ms after the face onset. Altogether, the results reveal the temporal dynamics of the neural mechanism of long-term effects of disagreement with the peer group: early signatures of modified face processing were followed by later markers of long-term social influence on the valuation process at the ventromedial prefrontal cortex.

Feeling through another's eyes: Perceived gaze direction impacts ERP and behavioural measures of positive and negative affective empathy

Looking at the eyes informs us about the thoughts and emotions of those around us, and impacts our own emotional state. However, it is unknown how perceiving direct and averted gaze impacts our ability to share the gazer's positive and negative emotions, abilities referred to as positive and negative affective empathy. We presented 44 participants with contextual sentences describing positive, negative and neutral events happening to other people (e.g. "Her newborn was saved/killed/fed yesterday afternoon."). These were designed to elicit positive, negative, or little to no empathy, and were followed by direct or averted gaze images of the individuals described. Participants rated their affective empathy for the individual and their own emotional valence on each trial. Event-related potentials time-locked to face-onset and associated with empathy and emotional processing were recorded to investigate whether they were modulated by gaze direction. Relative to averted gaze, direct gaze was associated with increased positive valence in the positive and neutral conditions and with increased positive empathy ratings. A similar pattern was found at the neural level, using robust mass-univariate statistics. The N100, thought to reflect an automatic activation of emotion areas, was modulated by gaze in the affective empathy conditions, with opposite effect directions in positive and negative conditions.. The P200, an ERP component sensitive to positive stimuli, was modulated by gaze direction only in the positive empathy condition. Positive and negative trials were processed similarly at the early N200 processing stage, but later diverged, with only negative trials modulating the EPN, P300 and LPP components. These results suggest that positive and negative affective empathy are associated with distinct time-courses, and that perceived gaze direction uniquely modulates positive empathy, highlighting the importance of studying empathy with face stimuli.

Perceived Gaze Direction Differentially Affects Discrimination of Facial Emotion, Attention, and Gender - An ERP Study

The perception of eye-gaze is thought to be a key component of our everyday social interactions. While the neural correlates of direct and averted gaze processing have been investigated, there is little consensus about how these gaze directions may be processed differently as a function of the task being performed. In a within-subject design, we examined how perception of direct and averted gaze affected performance on tasks requiring participants to use directly available facial cues to infer the individuals' emotional state (emotion discrimination), direction of attention (attention discrimination) and gender (gender discrimination). Neural activity was recorded throughout the three tasks using EEG, and ERPs time-locked to face onset were analyzed. Participants were most accurate at discriminating emotions with direct gaze faces, but most accurate at discriminating attention with averted gaze faces, while gender discrimination was not affected by gaze direction. At the neural level, direct and averted gaze elicited different patterns of activation depending on the task over frontal sites, from approximately 220-290 ms. More positive amplitudes were seen for direct than averted gaze in the emotion discrimination task. In contrast, more positive amplitudes were seen for averted gaze than for direct gaze in the gender discrimination task. These findings are among the first direct evidence that perceived gaze direction modulates neural activity differently depending on task demands, and that at the behavioral level, specific gaze directions functionally overlap with emotion and attention discrimination, precursors to more elaborated theory of mind processes.

Early and late cortical responses to directly gazing faces are task dependent

Gender categorisation of human faces is facilitated when gaze is directed toward the observer (i.e., a direct gaze), compared with situations where gaze is averted or the eyes are closed (Macrae, Hood, Milne, Rowe, & Mason, Psychological Science, 13(5), 460-464, 2002). However, the temporal dynamics underlying this phenomenon remain to some extent unknown. Here, we used electroencephalography (EEG) to assess the neural correlates of this effect, focusing on the event-related potential (ERP) components known to be sensitive to gaze perception (i.e., P1, N170, and P3b). We first replicated the seminal findings of Macrae et al. (2002, Experiment 1) regarding facilitated gender discrimination, and subsequently measured the underlying neural responses. Our data revealed an early preferential processing of direct gaze as compared with averted gaze and closed eyes at the P1, which reverberated at the P3b (Experiment 2). Critically, using the same material, we failed to reproduce these effects when gender categorisation was not required (Experiment 3). Taken together, our data confirm that direct gaze enhances the early P1, as well as later cortical responses to face processing, although the effect appears to be task dependent.