Explicit versus implicit gaze processing assessed by ERPs

Autistic and nonautistic adolescents do not differ in adaptation to gaze direction

Predictive processing accounts of autism posit that autistic individuals' perception is less biased by expectations than nonautistic individuals', perhaps through stronger precision-weighting of prediction errors. Since precision-weighting is fundamental to all information processing, under this theory, the differences between autistic and nonautistic individuals should be domain-general and observable in both behavior and brain responses. This study used EEG, behavioral responses, and eye-tracking co-registration during gaze-direction adaptation, to investigate whether increased precision-weighting of prediction errors is evident through smaller adaptation after-effects in autistic adolescents compared with nonautistic peers. Multilevel modeling showed that autistic and nonautistic adolescents' responses were consistent with behavioral adaptation, with Bayesian statistics providing extremely strong evidence for the absence of a group difference. Cluster-based permutation testing of ERP responses did not show the expected adaptation after-effect but did show habituation to repeated stimulus presentation, and no group difference was detected, a result not consistent with the theoretical account. Combined with the few other available studies, the current findings raise challenges for the theory, suggesting no fundamental difference in precision-weighting of prediction errors in autism.

Realness of face images can be decoded from non-linear modulation of EEG responses

Artificially created human faces play an increasingly important role in our digital world. However, the so-called uncanny valley effect may cause people to perceive highly, yet not perfectly human-like faces as eerie, bringing challenges to the interaction with virtual agents. At the same time, the neurocognitive underpinnings of the uncanny valley effect remain elusive. Here, we utilized an electroencephalography (EEG) dataset of steady-state visual evoked potentials (SSVEP) in which participants were presented with human face images of different stylization levels ranging from simplistic cartoons to actual photographs. Assessing neuronal responses both in frequency and time domain, we found a non-linear relationship between SSVEP amplitudes and stylization level, that is, the most stylized cartoon images and the real photographs evoked stronger responses than images with medium stylization. Moreover, realness of even highly similar stylization levels could be decoded from the EEG data with task-related component analysis (TRCA). Importantly, we also account for confounding factors, such as the size of the stimulus face's eyes, which previously have not been adequately addressed. Together, this study provides a basis for future research and neuronal benchmarking of real-time detection of face realness regarding three aspects: SSVEP-based neural markers, efficient classification methods, and low-level stimulus confounders.

The impact of emotional facial expressions on reflexive attention depends on the aim of dynamic gaze changes: An ERP study

The emotional expression and gaze direction of a face are important cues for human social interactions. However, the interplay of these factors and their neural correlates are only partially understood. In the current study, we investigated ERP correlates of gaze and emotion processing following the initial presentation of faces with different emotional expressions (happy, neutral, angry) and an averted or direct gaze direction as well as following a subsequent change in gaze direction that occurred in half of the trials. We focused on the time course and scalp topography of the N170 and EPN components. The N170 amplitude was larger to averted than direct gaze for the initial face presentation and larger to gaze changes from direct to averted than from averted to direct in response to the gaze change. For the EPN component in response to the initial face presentation, we replicate classic effects of emotion, which did not interact with gaze direction. As a major new finding, changes from direct to averted gaze elicited an EPN-like effect when the face showed a happy expression. No such effect was seen for angry expressions. We conclude that happy faces reflexively attract attention when they look at the observer rather than away from the observer. These results for happy expressions are in line with the shared signal hypothesis that posits a better processing of expressions if their approach or avoidance tendency is consistent with gaze direction. However, the shared signal hypothesis is not supported by the present results for angry faces.

Association between attachment anxiety and the gaze direction-related N170

Attachment theory suggests that interindividual differences in attachment security versus insecurity (anxiety and avoidance) contribute to the ways in which people perceive social emotional signals, particularly from the human face. Among different facial features, eye gaze conveys crucial information for social interaction, with a straight gaze triggering different cognitive and emotional processes as compared to an averted gaze. It remains unknown, however, how interindividual differences in attachment associate with early face encoding in the context of a straight versus averted gaze. Using electroencephalography (EEG) and recording event-related potentials (ERPs), specifically the N170 component, the present study (N = 50 healthy adults) measured how the characteristics of attachment anxiety and avoidance relate to the encoding of faces with respect to gaze direction and head orientation. Our findings reveal a significant relationship between gaze direction (irrespective of head orientation) and attachment anxiety on the interhemispheric (i.e. right) asymmetry of the N170 and thus provide evidence for an association between attachment anxiety and eye gaze processing during early visual face encoding.

Perceived gaze direction affects recollection processes in recognition of concrete and abstract words: electrophysiological evidence

Studies have revealed that memory performance can be affected by perceived gaze direction. However, it remains unclear whether direct gaze promotes or hinders word memory, and the effect of gaze direction on memory of words with different concreteness requires investigation. In the study phase, concrete and abstract words were presented on direct- or averted-gaze faces, and participants were instructed to judge gaze direction and memorize words. In the test phase, participants were asked to discriminate whether a word was old or new. Electroencephalogram recordings were taken in both phases. Behavioral and time-frequency results verified the direct-gaze memory advantage, showing that memory performance was better in the direct-gaze condition than the averted-gaze condition for both concrete and abstract words. Event-related potential results showed that in both direct- and averted-gaze conditions, the early old/new effects (FN400) associated with familiarity were only elicited for concrete words but not abstract words. The late old/new effects (LPC) associated with recollection were elicited in all conditions. More importantly, concrete words elicited greater LPC than abstract words in the direct-gaze condition, whereas there was no such significant LPC difference in the averted-gaze condition. Topographic map analysis found that neural generators between concrete and abstract words differed in the direct-gaze condition but not in the averted-gaze condition. The study supports the hypothesis that direct-gaze promotes memory performance. Furthermore, it is mainly in memory recollection that gaze direction affects words with different concreteness.

I can see it in your eyes: Perceived gaze direction impacts ERP and behavioural measures of affective theory of mind

Looking at someone's eyes is thought to be important for affective theory of mind (aTOM), our ability to infer their emotional state. However, it is unknown whether an individual's gaze direction influences our aTOM judgements and what the time course of this influence might be. We presented participants with sentences describing individuals in positive, negative or neutral scenarios, followed by direct or averted gaze neutral face pictures of those individuals. Participants made aTOM judgements about each person's mental state, including their affective valence and arousal, and we investigated whether the face gaze direction impacted those judgements. Participants rated that gazers were feeling more positive when they displayed direct gaze as opposed to averted gaze, and that they were feeling more aroused during negative contexts when gaze was averted as opposed to direct. Event-related potentials associated with face perception and affective processing were examined using mass-univariate analyses to track the time-course of this eye-gaze and affective processing interaction at a neural level. Both positive and negative trials were differentiated from neutral trials at many stages of processing. This included the early N200 and EPN components, believed to reflect automatic emotion areas activation and attentional selection respectively. This also included the later P300 and LPP components, thought to reflect elaborative cognitive appraisal of emotional content. Critically, sentence valence and gaze direction interacted over these later components, which may reflect the incorporation of eye-gaze in the cognitive evaluation of another's emotional state. The results suggest that gaze perception directly impacts aTOM processes, and that altered eye-gaze processing in clinical populations may contribute to associated aTOM impairments.

The roles of gaze and head orientation in face categorization during rapid serial visual presentation

Little is known about how perceived gaze direction and head orientation may influence human categorization of visual stimuli as faces. To address this question, a sequence of unsegmented natural images, each containing a random face or a non-face object, was presented in rapid succession (stimulus duration: 91.7 ms per image) during which human observers were instructed to respond immediately to every face presentation. Faces differed in gaze and head orientation in 7 combinations - full-front views with perceived gaze (1) directed to the observer, (2) averted to the left, or (3) averted to the right, left ¾ side views with (4) direct gaze or (5) averted gaze, and right ¾ side views with (6) direct gaze or (7) averted gaze - were presented randomly throughout the sequence. We found highly accurate and rapid behavioural responses to all kinds of faces. Crucially, both perceived gaze direction and head orientation had comparable, non-interactive effects on response times, where direct gaze was responded faster than averted gaze by 48 ms and full-front view faster than ¾ side view also by 48 ms on average. Presentations of full-front faces with direct gaze led to an additive speed advantage of 96 ms to ¾ faces with averted gaze. The results reveal that the effects of perceived gaze direction and head orientation on the speed of face categorization probably depend on the degree of social relevance of the face to the viewer.

Neural Oscillatory Abnormalities During Gaze Processing in Schizophrenia: Evidence of Reduced Theta Phase Consistency and Inter-areal Theta-Gamma Coupling

BACKGROUND

Abnormal gaze discrimination in schizophrenia (SZ) is associated with impairment in social functioning, but the neural mechanisms remain unclear. Evidence suggests that local neural oscillations and inter-areal communication through neural synchronization are critical physiological mechanisms supporting basic and complex cognitive processes. The roles of these mechanisms in abnormal gaze processing in SZ have not been investigated. The present study examined local neural oscillations and connectivity between anterior and bilateral posterior brain areas during gaze processing.

METHODS

During electroencephalography recording, 28 participants with SZ and 34 healthy control participants completed a gaze discrimination task. Time-frequency decomposition of electroencephalography data was used to examine neural oscillatory power and intertrial phase consistency at bilateral posterior and midline anterior scalp sites. In addition, connectivity between these anterior and posterior sites, in terms of cross-frequency coupling between theta phase and gamma amplitude, was examined using the Kullback-Leibler Modulation Index.

RESULTS

Participants with SZ showed reduced total power of theta-band activity relative to healthy control participants at all sites examined. This group difference could be accounted for by reduced intertrial phase consistency of theta activity in SZ participants, which was related to reduced gaze discrimination accuracy in SZ. In addition, SZ participants exhibited reduced Kullback-Leibler indexing, both feedforward and feedback connectivity, between the posterior and anterior sites.

CONCLUSIONS

These findings suggest that abnormal theta phase consistency and dysconnection between posterior face processing and anterior areas may underlie gaze processing deficits in SZ.

Feeling through another's eyes: Perceived gaze direction impacts ERP and behavioural measures of positive and negative affective empathy

Looking at the eyes informs us about the thoughts and emotions of those around us, and impacts our own emotional state. However, it is unknown how perceiving direct and averted gaze impacts our ability to share the gazer's positive and negative emotions, abilities referred to as positive and negative affective empathy. We presented 44 participants with contextual sentences describing positive, negative and neutral events happening to other people (e.g. "Her newborn was saved/killed/fed yesterday afternoon."). These were designed to elicit positive, negative, or little to no empathy, and were followed by direct or averted gaze images of the individuals described. Participants rated their affective empathy for the individual and their own emotional valence on each trial. Event-related potentials time-locked to face-onset and associated with empathy and emotional processing were recorded to investigate whether they were modulated by gaze direction. Relative to averted gaze, direct gaze was associated with increased positive valence in the positive and neutral conditions and with increased positive empathy ratings. A similar pattern was found at the neural level, using robust mass-univariate statistics. The N100, thought to reflect an automatic activation of emotion areas, was modulated by gaze in the affective empathy conditions, with opposite effect directions in positive and negative conditions.. The P200, an ERP component sensitive to positive stimuli, was modulated by gaze direction only in the positive empathy condition. Positive and negative trials were processed similarly at the early N200 processing stage, but later diverged, with only negative trials modulating the EPN, P300 and LPP components. These results suggest that positive and negative affective empathy are associated with distinct time-courses, and that perceived gaze direction uniquely modulates positive empathy, highlighting the importance of studying empathy with face stimuli.

The perceived present: What is it, and what is it there for?

It is proposed that the perceived present is not a moment in time, but an information structure comprising an integrated set of products of perceptual processing. All information in the perceived present carries an informational time marker identifying it as "present". This marker is exclusive to information in the perceived present. There are other kinds of time markers, such as ordinality ("this stimulus occurred before that one") and duration ("this stimulus lasted for 50 ms"). These are different from the "present" time marker and may be attached to information regardless of whether it is in the perceived present or not. It is proposed that the perceived present is a very short-term and very high-capacity holding area for perceptual information. The maximum holding time for any given piece of information is ~100 ms: This is affected by the need to balance the value of informational persistence for further processing against the problem of obsolescence of the information. The main function of the perceived present is to facilitate access by other specialized, automatic processes.

Eye contact in active and passive viewing: Event-related brain potential evidence from a combined eye tracking and EEG study

Eye contact is a salient social cue, which is assumed to influence already early neural correlates of face perception. Specifically, the N170 component of the event-related potential (ERP) has often been found to be larger for faces with an averted gaze as compared to faces that directly look at the observer. In most existing ERP studies, effects of eye contact were investigated under comparatively artificial conditions where participants were instructed to maintain a steady fixation while they passively observed gaze changes in the stimulus face. It is therefore unclear to what extent neural correlates of eye contact generalize to more naturalistic situations that involve a continuous interplay between directed and averted gaze between the communication partners. To start bridging this gap, the present study compared the passive viewing of gaze changes to an active condition in which the participant's own gaze (measured online with an eye tracker) interacted with the gaze position of a continuously presented stimulus face. We also investigated whether eye contact effects were modulated by the face's emotional expression. In both the passive and the active viewing condition, N170 amplitudes were larger when the gaze of the stimulus faces was averted rather than directed towards the participant. Furthermore, eye contact decreased P300 amplitudes in both conditions. The emotional expression of the face also modulated the N170, but this effect did not interact with that of gaze direction. We conclude that the neural correlates of gaze perception during active gaze interactions are comparable to those found during passive viewing, encouraging the further study of eye contact effects in more naturalistic settings.

Perceived Gaze Direction Differentially Affects Discrimination of Facial Emotion, Attention, and Gender - An ERP Study

The perception of eye-gaze is thought to be a key component of our everyday social interactions. While the neural correlates of direct and averted gaze processing have been investigated, there is little consensus about how these gaze directions may be processed differently as a function of the task being performed. In a within-subject design, we examined how perception of direct and averted gaze affected performance on tasks requiring participants to use directly available facial cues to infer the individuals' emotional state (emotion discrimination), direction of attention (attention discrimination) and gender (gender discrimination). Neural activity was recorded throughout the three tasks using EEG, and ERPs time-locked to face onset were analyzed. Participants were most accurate at discriminating emotions with direct gaze faces, but most accurate at discriminating attention with averted gaze faces, while gender discrimination was not affected by gaze direction. At the neural level, direct and averted gaze elicited different patterns of activation depending on the task over frontal sites, from approximately 220-290 ms. More positive amplitudes were seen for direct than averted gaze in the emotion discrimination task. In contrast, more positive amplitudes were seen for averted gaze than for direct gaze in the gender discrimination task. These findings are among the first direct evidence that perceived gaze direction modulates neural activity differently depending on task demands, and that at the behavioral level, specific gaze directions functionally overlap with emotion and attention discrimination, precursors to more elaborated theory of mind processes.

ERP evidence on how gaze convergence affects social attention

How people process gaze cues from multiple others is an important topic but rarely studied. Our study investigated this question using an adapted gaze cueing paradigm to examine the cueing effect of multiple gazes and its neural correlates. We manipulated gaze directions from two human avatars to be either convergent, created by the two avatars simultaneously averting their gazes to the same direction, or non-convergent, when only one of the two avatars shifted its gaze. Our results showed faster reaction times and larger target-congruency effects following convergent gazes shared by the avatars, compared with the non-convergent gaze condition. These findings complement previous research to demonstrate that observing shared gazes from as few as two persons is sufficient to enhance gaze cueing. Additionally, ERP analyses revealed that (1) convergent gazes evoked both left and right hemisphere N170, while non-convergent gazes evoked N170 mainly in the hemisphere contralateral to the cueing face; (2) effects of target congruency on target-locked N1 and P3 were modulated by gaze convergence. These findings shed light on temporal features of the processing of multi-gaze cues.

The influence of top-down modulation on the processing of direct gaze

Gaze or eye contact is one of the most important nonverbal social cues, which is fundamental to human social interactions. To achieve real time and dynamic face-to-face communication, our brain needs to process another person's gaze direction rapidly and without explicit instruction. In order to explain the fast and spontaneous processing of direct gaze, the fast-track modulator model was proposed. Here, we review recent developments in gaze processing research in the last decade to extend the fast-track modulator model. In particular, we propose that task demand or top-down modulation could play a more crucial role at gaze processing than formerly assumed. We suggest that under different task demands, top-down modulation can facilitate or interfere with the direct gaze effects for early visual processing. The proposed modification of the model extends the role of task demand and its implication on the direct gaze effect, as well as the need to better control for top-down processing in order to better disentangle the role of top-down and bottom-up processing on the direct gaze effect. This article is categorized under: Cognitive Biology > Evolutionary Roots of Cognition Psychology > Perception and Psychophysics Neuroscience > Cognition.

Early and late cortical responses to directly gazing faces are task dependent

Gender categorisation of human faces is facilitated when gaze is directed toward the observer (i.e., a direct gaze), compared with situations where gaze is averted or the eyes are closed (Macrae, Hood, Milne, Rowe, & Mason, Psychological Science, 13(5), 460-464, 2002). However, the temporal dynamics underlying this phenomenon remain to some extent unknown. Here, we used electroencephalography (EEG) to assess the neural correlates of this effect, focusing on the event-related potential (ERP) components known to be sensitive to gaze perception (i.e., P1, N170, and P3b). We first replicated the seminal findings of Macrae et al. (2002, Experiment 1) regarding facilitated gender discrimination, and subsequently measured the underlying neural responses. Our data revealed an early preferential processing of direct gaze as compared with averted gaze and closed eyes at the P1, which reverberated at the P3b (Experiment 2). Critically, using the same material, we failed to reproduce these effects when gender categorisation was not required (Experiment 3). Taken together, our data confirm that direct gaze enhances the early P1, as well as later cortical responses to face processing, although the effect appears to be task dependent.

Spatiotemporal commonalities of fronto-parietal activation in attentional orienting triggered by supraliminal and subliminal gaze cues: An event-related potential study

Eye gaze triggers attentional shifts with and without conscious awareness. It remains unclear whether the spatiotemporal patterns of electric neural activity are the same for conscious and unconscious attentional shifts. Thus, the present study recorded event-related potentials (ERPs) and evaluated the neural activation involved in attentional orienting induced by subliminal and supraliminal gaze cues. Nonpredictive gaze cues were presented in the central field of vision, and participants were asked to detect a subsequent peripheral target. The mean reaction time was shorter for congruent gaze cues than for incongruent gaze cues under both presentation conditions, indicating that both types of cues reliably trigger attentional orienting. The ERP analysis revealed that averted versus straight gaze induced greater negative deflection in the bilateral fronto-central and temporal regions between 278 and 344 ms under both supraliminal and subliminal presentation conditions. Supraliminal cues, irrespective of gaze direction, induced a greater negative amplitude than did subliminal cues at the right posterior cortices at a peak of approximately 170 ms and in the 200-300 ms. These results suggest that similar spatial and temporal fronto-parietal activity is involved in attentional orienting triggered by both supraliminal and subliminal gaze cues, although inputs from different visual processing routes (cortical and subcortical regions) may trigger activity in the attentional network.

Neural Temporal Dynamics of Social Exclusion Elicited by Averted Gaze: An Event-Related Potentials Study

Eye gaze plays a fundamental role in social communication. The averted eye gaze during social interaction, as the most common form of silent treatment, conveys a signal of social exclusion. In the present study, we examined the time course of brain response to social exclusion by using a modified version of Eye-gaze paradigm. The event-related potentials (ERPs) data and the subjective rating data showed that the frontocentral P200 was positively correlated with negative mood of excluded events, whereas, the centroparietal late positive potential (LPP) was positively correlated with the perceived ostracism intensity. Both the P200 and LPP were more positive-going for excluded events than for included events. These findings suggest that brain responses sensitive to social exclusion can be divided into the early affective processing stage, linking to the early pre-cognitive warning system; and the late higher-order processes stage, demanding attentional resources for elaborate stimuli evaluation and categorization generally not under specific situation.

Eye gaze perception in bipolar disorder: Self-referential bias but intact perceptual sensitivity

OBJECTIVES

Deficits in social cognition predict poor functional outcome in severe mental illnesses such as schizophrenia and autism. However, research findings on social cognition in bipolar disorder (BD) are sparse and inconsistent. This study aimed to characterize a critical social cognitive process-eye gaze perception-and examine its functional correlates in BD to inform psychopathological mechanisms.

METHODS

Thirty participants with BD, 37 healthy controls (HC), and 46 psychiatric controls with schizophrenia (SZ) completed an eye-contact perception task. They viewed faces with varying gaze directions, head orientations, and emotion, and made eye-contact judgments. Psychophysics methods were used to estimate perception thresholds and the slope of the perception curve, which were then compared between the groups and correlated with clinical and functional measures using Bayesian inference.

RESULTS

Compared with HC, patients with BD over-perceived eye contact when gaze direction was ambiguous, and this self-referential bias was similar to that in SZ. Patients with BD had lower thresholds (i.e., needed weaker eye-contact signal to start perceiving gaze as self-directed) but a similar slope compared with HC. Regression analyses showed that steeper slope predicted better socio-emotional functioning in HC and SZ, but not in BD.

CONCLUSIONS

The psychopathology of social dysfunction was fundamentally different between BD and SZ in this modest sample. Eye gaze perception in BD was characterized by a self-referential bias but preserved perceptual sensitivity, the latter of which distinguished BD from SZ. The relationship between gaze perception and broader socio-emotional functioning in SZ and HC was absent in BD.

Altered N170 and mood symptoms in bipolar disorder: An electrophysiological study of configural face processing

OBJECTIVES

Altered social behavior during mood episodes in bipolar disorder often has detrimental and long-lasting interpersonal consequences. Abnormal face processing may play a role in linking brain functions to clinical symptoms and behavior. This study aimed to understand configural face processing in bipolar disorder as a function of basic communicative attributes of the face and mood symptoms using event-related brain potentials (ERPs).

METHODS

Forty-two participants with bipolar I disorder (BP) and 43 healthy controls (HC) viewed face stimuli varying in emotion (neutral or fearful), head orientation (forward or deviated), and gaze direction (direct or averted) while ERPs were recorded. Configural face processing was indexed by the N170 wave.

RESULTS

BP participants had comparable overall N170 amplitude and peak latency to HC, although timing was more variable in the BP group. Abnormal N170 modulations by communicative face attributes were observed in BP: exaggerated sensitivity to emotion (fearful > neutral) in the left hemisphere, and reduced sensitivity to gaze-head incongruency (where N170 is normally larger in response to faces with incongruent than congruent gaze and head direction) in the right hemisphere. The former was not associated with mood symptoms, suggesting a heightened trait-like sensitivity to negative emotions. The latter was correlated with greater manic symptoms, indicating that an impaired perceptual sensitivity to faces with features signaling incongruent social attention may underlie social deficits observed during mania.

CONCLUSIONS

These findings suggest a pathophysiological role of altered configural face processing in the phenomenology of bipolar disorder, and call for further investigations to evaluate its potential as a biomarker and treatment target.

Processing of gaze direction within the N170/M170 time window: A combined EEG/MEG study

Gaze direction is an important social signal for human beings. Beside the role of gaze in attention orienting, direct gaze (that is, gaze directed toward an observer) is a highly relevant biological stimulus that elicits attention capture and increases face encoding. Brain imaging studies have emphasized the role of the superior temporal sulcus (STS) in the coding of gaze direction and in the integration of gaze and head cues of social attention. The dynamics of the processing and integration of these cues remains, however, unclear. In order to address this question, we used deviated and frontal faces with averted and direct gaze in a combined electro- and magneto- encephalography (EEG-MEG) study. We showed distinct effects of gaze direction on the N170 and M170 responses. There was an interaction between gaze direction and head orientation between 134 and 162ms in MEG and a main effect of gaze direction between 171 and 186ms in EEG. These effects involved the posterior and anterior regions of the STS respectively. Both effects also emphasized the sensitivity to direct gaze. These data highlight the central role of the STS in gaze processing.

Event-related potentials index neural response to eye contact

Sensitivity to eye-contact is a foundation upon which social cognition is built. However, there are no known neural markers characterizing response to reciprocal gaze. Using co-registered EEG and eye-tracking, we measured brain activity while participants viewed faces that responded to their looking patterns. Contingent upon participant gaze, onscreen faces opened their eyes or mouths; in this way we measured brain response to reciprocal eye-contact. We identified two ERP components that were largest in response to reciprocal eye-contact: the N170 and the P300. The magnitude of the components' differences between reciprocal eye-contact and mouth movement predicted self-reported social function. Individuals with greater brain response to reciprocal eye-contact reported more normative scores on measures of autistic traits. These results present the first neural markers of eye-contact, revealing that reciprocal eye-contact is identified in less than 500ms. Furthermore, individual differences in brain response to eye-contact predict meaningful variability in self-reports of social performance.

Altered attentional and perceptual processes as indexed by N170 during gaze perception in schizophrenia: Relationship with perceived threat and paranoid delusions

Using gaze information to orient attention and guide behavior is critical to social adaptation. Previous studies have suggested that abnormal gaze perception in schizophrenia (SCZ) may originate in abnormal early attentional and perceptual processes and may be related to paranoid symptoms. Using event-related brain potentials (ERPs), this study investigated altered early attentional and perceptual processes during gaze perception and their relationship to paranoid delusions in SCZ. Twenty-eight individuals with SCZ or schizoaffective disorder and 32 demographically matched healthy controls (HCs) completed a gaze-discrimination task with face stimuli varying in gaze direction (direct, averted), head orientation (forward, deviated), and emotion (neutral, fearful). ERPs were recorded during the task. Participants rated experienced threat from each face after the task. Participants with SCZ were as accurate as, though slower than, HCs on the task. Participants with SCZ displayed enlarged N170 responses over the left hemisphere to averted gaze presented in fearful relative to neutral faces, indicating a heightened encoding sensitivity to faces signaling external threat. This abnormality was correlated with increased perceived threat and paranoid delusions. Participants with SCZ also showed a reduction of N170 modulation by head orientation (normally increased amplitude to deviated faces relative to forward faces), suggesting less integration of contextual cues of head orientation in gaze perception. The psychophysiological deviations observed during gaze discrimination in SCZ underscore the role of early attentional and perceptual abnormalities in social information processing and paranoid symptoms of SCZ.

Early Left Parietal Activity Elicited by Direct Gaze: A High-Density EEG Study

Gaze is one of the most important cues for human communication and social interaction. In particular, gaze contact is the most primary form of social contact and it is thought to capture attention. A very early-differentiated brain response to direct versus averted gaze has been hypothesized. Here, we used high-density electroencephalography to test this hypothesis. Topographical analysis allowed us to uncover a very early topographic modulation (40-80 ms) of event-related responses to faces with direct as compared to averted gaze. This modulation was obtained only in the condition where intact broadband faces-as opposed to high-pass or low-pas filtered faces-were presented. Source estimation indicated that this early modulation involved the posterior parietal region, encompassing the left precuneus and inferior parietal lobule. This supports the idea that it reflected an early orienting response to direct versus averted gaze. Accordingly, in a follow-up behavioural experiment, we found faster response times to the direct gaze than to the averted gaze broadband faces. In addition, classical evoked potential analysis showed that the N170 peak amplitude was larger for averted gaze than for direct gaze. Taken together, these results suggest that direct gaze may be detected at a very early processing stage, involving a parallel route to the ventral occipito-temporal route of face perceptual analysis.

Direct gaze facilitates rapid orienting to faces: Evidence from express saccades and saccadic potentials

•

We investigated the role of direct gaze for rapid orienting to faces.

•

Faster express saccades towards faces occurred only when faces were with direct gaze.

•

Saccade-locked ERPs discriminated between gaze directions from the saccade onset.

Direct gaze is a crucial signal in human social communication, which is known to attract visual attention and modulate a wide range of behaviours. The present study investigated whether direct gaze facilitates rapid orienting to faces, which is important for adaptive on-line communication, and its neural correlates. Fifteen participants performed a rapid orienting task, in which they were instructed to saccade to peripherally presented buildings or faces containing direct or averted gaze as quickly as possible. Electroencephalographic recordings were made during the task. Shorter express saccade latencies were found for faces with direct gaze, compared to averted gaze or buildings, while no significant difference was found between faces with averted gaze and buildings. Furthermore, saccade-locked event-related potential (ERP) amplitudes in parieto-occipital areas discriminated faces with direct gaze from buildings and faces with averted gaze corroborating behavioural results. These results show that detection of direct gaze facilitates rapid orienting to faces.

Eye gaze and head orientation modulate the inhibition of return for faces

The present study used an inhibition of return (IOR) spatial cueing paradigm to examine how gaze direction and head orientation modulate attention capture for human faces. Target response time (RT) was measured after the presentation of a peripheral cue, which was either a face (with front-facing or averted gaze, in either frontal head view or averted head view) or a house (control). Participants fixated on a centered cross at all times and responded via button press to a peripheral target after a variable stimulus onset asynchrony (SOA) from the stimulus cue. At the shortest SOA (150 ms), RTs were shorter for faces than houses, independent of an IOR response, suggesting a cue-based RT advantage elicited by faces. At the longest SOA (2,400 ms), a larger IOR magnitude was found for faces compared to houses. Both the cue-based RT advantage and later IOR responses were modulated by gaze-head congruency; these effects were strongest for frontal gaze faces in frontal head view, and for averted gaze faces in averted head view. Importantly, participants were not given any specific information regarding the stimuli, nor were they told the true purpose of the study. These findings indicate that the congruent combination of head and gaze direction influence the exogenous attention capture of faces during inhibition of return.

Social decisions affect neural activity to perceived dynamic gaze

Gaze direction, a cue of both social and spatial attention, is known to modulate early neural responses to faces e.g. N170. However, findings in the literature have been inconsistent, likely reflecting differences in stimulus characteristics and task requirements. Here, we investigated the effect of task on neural responses to dynamic gaze changes: away and toward transitions (resulting or not in eye contact). Subjects performed, in random order, social (away/toward them) and non-social (left/right) judgment tasks on these stimuli. Overall, in the non-social task, results showed a larger N170 to gaze aversion than gaze motion toward the observer. In the social task, however, this difference was no longer present in the right hemisphere, likely reflecting an enhanced N170 to gaze motion toward the observer. Our behavioral and event-related potential data indicate that performing social judgments enhances saliency of gaze motion toward the observer, even those that did not result in gaze contact. These data and that of previous studies suggest two modes of processing visual information: a 'default mode' that may focus on spatial information; a 'socially aware mode' that might be activated when subjects are required to make social judgments. The exact mechanism that allows switching from one mode to the other remains to be clarified.

The temporal unfolding of face processing in social anxiety disorder--a MEG study

The current study is the first to use magnetoencephalography (MEG) to examine how individuals with social anxiety disorder (SAD) process emotional facial expressions (EFEs). We expected that, compared to healthy controls (HCs), participants with SAD will show an early (<200 ms post-stimulus) over-activation in the insula and the fusiform gyrus (FG, associated with the N170/M170 component), and later (>200 ms post-stimulus) over-activation in the dorsolateral prefrontal cortex (DLPFC). Individuals with SAD (n = 12) and healthy controls (HCs, n = 12) were presented with photographs of facial displays during MEG recording. As compared to the HC group, the SAD group showed a reduced M170 (right FG under-activation around 130–200 ms); early reduced activation in the right insula, and lower insular sensitivity to the type of EFE displayed. In addition, the SAD group showed a late over-activation in the right DLPFC. This unique EFE processing pattern in SAD suggests an early under-activation of cortical areas, possibly related to reduced emphasis on high spatial frequency information and greater early emphasis on low spatial frequency information. The late DLPFC over-activation in the SAD group may correlate to failures of cognitive control in this disorder. The importance of a temporal perspective for the understanding of facial processing in psychopathology is underlined.

•

This study is the first to use MEG to study social anxiety disorder (SAD).

•

SADs and controls viewed emotional facial expressions during MEG.

•

Compared to controls, SADs showed reduced M170 (early fusiform gyrus activity).

•

SADs presented a late over-activation in the right dorsolateral prefrontal cortex.

•

The late frontal over-activity may correlate to failures of cognitive control in SAD.

The N170 observed 'in the wild': robust event-related potentials to faces in cluttered dynamic visual scenes

As a social species in a constantly changing environment, humans rely heavily on the informational richness and communicative capacity of the face. Thus, understanding how the brain processes information about faces in real-time is of paramount importance. The N170 is a high-temporal resolution electrophysiological index of the brain's early response to visual stimuli that is reliably elicited in carefully controlled laboratory-based studies. Although the N170 has often been reported to be of greatest amplitude to faces, there has been debate regarding whether this effect might be an artefact of certain aspects of the controlled experimental stimulation schedules and materials. To investigate whether the N170 can be identified in more realistic conditions with highly variable and cluttered visual images and accompanying auditory stimuli we recorded EEG 'in the wild', while participants watched pop videos. Scene-cuts to faces generated a clear N170 response, and this was larger than the N170 to transitions where the videos cut to non-face stimuli. Within participants, wild-type face N170 amplitudes were moderately correlated to those observed in a typical laboratory experiment. Thus, we demonstrate that the face N170 is a robust and ecologically valid phenomenon and not an artefact arising as an unintended consequence of some property of the more typical laboratory paradigm.

Role of visual integration in gaze perception and emotional intelligence in schizophrenia

BACKGROUND

Individuals with schizophrenia demonstrate a wide range of social cognitive deficits that significantly compromise functioning. Early visual processing is frequently disrupted in schizophrenia, and growing evidence suggests a role of perceptual dysfunctions in socioemotional functioning in the disorder. This study examined visual integration (the ability to effectively integrate individual, local visual features into a holistic representation), a target construct of basic perception identified by the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia initiative, and its relationship with eye- contact perception and emotional intelligence in schizophrenia.

METHODS

Twenty-nine participants with schizophrenia (SCZ) and 23 healthy controls (HC) completed tasks measuring visual integration (Coherent Motion Task, Contour Integration Task), an eye-contact perception task, and a measure of emotional intelligence.

RESULTS

SCZ participants showed compromised visual integration as suggested by poorer performance on the Contour Integration Task relative to HC. Visual integration was a significant predictor of eye-contact perception and emotional intelligence among SCZ. The amounts of variances in these 2 social cognitive areas accounted for by visual integration were comparable to and overlapped with those accounted for by the diagnosis of schizophrenia.

CONCLUSIONS

Individuals with schizophrenia showed compromised visual integration, and this may play a significant role in the observed deficits in higher level processing of social information in the disorder.

Face the hierarchy: ERP and oscillatory brain responses in social rank processing

Recognition of social hierarchy is a key feature that helps us navigate through our complex social environment. Neuroimaging studies have identified brain structures involved in the processing of hierarchical stimuli but the precise temporal dynamics of brain activity associated with such processing remains largely unknown. Here, we used electroencephalography to examine the effect of social hierarchy on neural responses elicited by faces. In contrast to previous studies, the key manipulation was that a hierarchical context was constructed, not by varying facial expressions, but by presenting neutral-expression faces in a game setting. Once the performance-based hierarchy was established, participants were presented with high-rank, middle-rank and low-rank player faces and had to evaluate the rank of each face with respect to their own position. Both event-related potentials and task-related oscillatory activity were investigated. Three main findings emerge from the study. First, the experimental manipulation had no effect on the early N170 component, which may suggest that hierarchy did not modulate the structural encoding of neutral-expression faces. Second, hierarchy significantly modulated the amplitude of the late positive potential (LPP) within a 400-700 ms time-window, with more a prominent LPP occurring when the participants processed the face of the highest-rank player. Third, high-rank faces were associated with the highest reduction of alpha power. Taken together these findings provide novel electrophysiological evidence for enhanced allocation of attentional resource in the presence of high-rank faces. At a broader level, this study brings new insights into the neural processing underlying social categorization.

Effects of Peripheral Eccentricity and Head Orientation on Gaze Discrimination

Visual search tasks support a special role for direct gaze in human cognition, while classic gaze judgment tasks suggest the congruency between head orientation and gaze direction plays a central role in gaze perception. Moreover, whether gaze direction can be accurately discriminated in the periphery using covert attention is unknown. In the present study, individual faces in frontal and in deviated head orientations with a direct or an averted gaze were flashed for 150 ms across the visual field; participants focused on a centred fixation while judging the gaze direction. Gaze discrimination speed and accuracy varied with head orientation and eccentricity. The limit of accurate gaze discrimination was less than ±6° eccentricity. Response times suggested a processing facilitation for direct gaze in fovea, irrespective of head orientation, however, by ±3° eccentricity, head orientation started biasing gaze judgments, and this bias increased with eccentricity. Results also suggested a special processing of frontal heads with direct gaze in central vision, rather than a general congruency effect between eye and head cues. Thus, while both head and eye cues contribute to gaze discrimination, their role differs with eccentricity.

Neural activity in the posterior superior temporal region during eye contact perception correlates with autistic traits

The present study investigated the relationship between neural activity associated with gaze processing and autistic traits in typically developed subjects using magnetoencephalography. Autistic traits in 24 typically developed college students with normal intelligence were assessed using the Autism Spectrum Quotient (AQ). The Minimum Current Estimates method was applied to estimate the cortical sources of magnetic responses to gaze stimuli. These stimuli consisted of apparent motion of the eyes, displaying direct or averted gaze motion. Results revealed gaze-related brain activations in the 150-250 ms time window in the right posterior superior temporal sulcus (pSTS), and in the 150-450 ms time window in medial prefrontal regions. In addition, the mean amplitude in the 150-250 ms time window in the right pSTS region was modulated by gaze direction, and its activity in response to direct gaze stimuli correlated with AQ score. pSTS activation in response to direct gaze is thought to be related to higher-order social processes. Thus, these results suggest that brain activity linking eye contact and social signals is associated with autistic traits in a typical population.

Multiple faces elicit augmented neural activity

How do our brains respond when we are being watched by a group of people?Despite the large volume of literature devoted to face processing, this question has received very little attention. Here we measured the effects on the face-sensitive N170 and other ERPs to viewing displays of one, two and three faces in two experiments. In Experiment 1, overall image brightness and contrast were adjusted to be constant, whereas in Experiment 2 local contrast and brightness of individual faces were not manipulated. A robust positive-negative-positive (P100-N170-P250) ERP complex and an additional late positive ERP, the P400, were elicited to all stimulus types. As the number of faces in the display increased, N170 amplitude increased for both stimulus sets, and latency increased in Experiment 2. P100 latency and P250 amplitude were affected by changes in overall brightness and contrast, but not by the number of faces in the display per se. In Experiment 1 when overall brightness and contrast were adjusted to be constant, later ERP (P250 and P400) latencies showed differences as a function of hemisphere. Hence, our data indicate that N170 increases its magnitude when multiple faces are seen, apparently impervious to basic low-level stimulus features including stimulus size. Outstanding questions remain regarding category-sensitive neural activity that is elicited to viewing multiple items of stimulus categories other than faces.

Is it in the eyes? Dissociating the role of emotion and perceptual features of emotionally expressive faces in modulating orienting to eye gaze

This study investigated the role of the eye region of emotional facial expressions in modulating gaze orienting effects. Eye widening is characteristic of fearful and surprised expressions and may significantly increase the salience of perceived gaze direction. This perceptual bias rather than the emotional valence of certain expressions may drive enhanced gaze orienting effects. In a series of three experiments involving low anxiety participants, different emotional expressions were tested using a gaze-cueing paradigm. Fearful and surprised expressions enhanced the gaze orienting effect compared with happy or angry expressions. Presenting only the eye regions as cueing stimuli eliminated this effect whereas inversion globally reduced it. Both inversion and the use of eyes only attenuated the emotional valence of stimuli without affecting the perceptual salience of the eyes. The findings thus suggest that low-level stimulus features alone are not sufficient to drive gaze orienting modulations by emotion. Rather, they interact with the emotional valence of the expression that appears critical. The study supports the view that rapid processing of fearful and surprised emotional expressions can potentiate orienting to another person's averted gaze in non-anxious people.

Effects of inverting contour and features on processing for static and dynamic face perception: an MEG study

We investigated the effects of inverting facial contour (hair and chin) and features (eyes, nose and mouth) on processing for static and dynamic face perception using magnetoencephalography (MEG). We used apparent motion, in which the first stimulus (S1) was replaced by a second stimulus (S2) with no interstimulus interval and subjects perceived visual motion, and presented three conditions as follows: (1) U&U: Upright contour and Upright features, (2) U&I: Upright contour and Inverted features, and (3) I&I: Inverted contour and Inverted features. In static face perception (S1 onset), the peak latency of the fusiform area's activity, which was related to static face perception, was significantly longer for U&I and I&I than for U&U in the right hemisphere and for U&I than for U&U and I&I in the left. In dynamic face perception (S2 onset), the strength (moment) of the occipitotemporal area's activity, which was related to dynamic face perception, was significantly larger for I&I than for U&U and U&I in the right hemisphere, but not the left. These results can be summarized as follows: (1) in static face perception, the activity of the right fusiform area was more affected by the inversion of features while that of the left fusiform area was more affected by the disruption of the spatial relation between the contour and features, and (2) in dynamic face perception, the activity of the right occipitotemporal area was affected by the inversion of the facial contour.

Does it make a difference if I have an eye contact with you or with your picture? An ERP study

Several recent studies have begun to examine the neurocognitive mechanisms involved in perceiving and responding to eye contact, a salient social signal of interest and readiness for interaction. Laboratory experiments measuring observers' responses to pictorial instead of live eye gaze cues may, however, only vaguely approximate the real-life affective significance of gaze direction cues. To take this into account, we measured event-related brain potentials and subjective affective responses in healthy adults while viewing live faces with a neutral expression through an electronic shutter and faces as pictures on a computer screen. Direct gaze elicited greater face-sensitive N170 amplitudes and early posterior negativity potentials than averted gaze or closed eyes, but only in the live condition. The results show that early-stage processing of facial information is enhanced by another person's direct gaze when the person is faced live. We propose that seeing a live face with a direct gaze is processed more intensely than a face with averted gaze or closed eyes, as the direct gaze is capable of intensifying the feeling of being the target of the other's interest and intentions. These results may have implications for the use of pictorial stimuli in the social cognition studies.

Neural bases of eye and gaze processing: the core of social cognition

Eyes and gaze are very important stimuli for human social interactions. Recent studies suggest that impairments in recognizing face identity, facial emotions or in inferring attention and intentions of others could be linked to difficulties in extracting the relevant information from the eye region including gaze direction. In this review, we address the central role of eyes and gaze in social cognition. We start with behavioral data demonstrating the importance of the eye region and the impact of gaze on the most significant aspects of face processing. We review neuropsychological cases and data from various imaging techniques such as fMRI/PET and ERP/MEG, in an attempt to best describe the spatio-temporal networks underlying these processes. The existence of a neuronal eye detector mechanism is discussed as well as the links between eye gaze and social cognition impairments in autism. We suggest impairments in processing eyes and gaze may represent a core deficiency in several other brain pathologies and may be central to abnormal social cognition.