The human amygdala plays an important role in gaze monitoring. A PET study

Following gaze: gaze-following behavior as a window into social cognition

In general, individuals look where they attend and next intend to act. Many animals, including our own species, use observed gaze as a deictic (“pointing”) cue to guide behavior. Among humans, these responses are reflexive and pervasive: they arise within a fraction of a second, act independently of task relevance, and appear to undergird our initial development of language and theory of mind. Human and nonhuman animals appear to share basic gaze-following behaviors, suggesting the foundations of human social cognition may also be present in nonhuman brains.

Looking the Other Way: The Role of Gaze Direction in the Cross-race Memory Effect

One of the most replicable findings reported in the social psychological literature is the cross-race memory effect. We argue this effect derives from higher-order interactions among social cues that determine the perceived relevance of a face to an observer. The current research tested this hypothesis by examining the combined influences of eye gaze direction and race on face memory. The physical subtlety of eye gaze belies its powerful influence on social perception, and in this case helps specify the relevance of same- versus other-race faces. We found that only in faces making direct eye contact-not those displaying averted eye gaze-was the cross-race memory effect evident. Likewise, only in same-race faces did direct relative to averted gaze enhance face memory. These findings have implications for our general understanding of the combinatorial nature of social perception and help clarify the underlying cause of the cross-race memory effect.

Connectivity analysis reveals a cortical network for eye gaze perception

Haxby et al. (Haxby JV, Hoffman EA, Gobbini MI. 2000. The distributed human neural system for face perception. Trends Cogn Sci. 4:223–233.) proposed that eye gaze processing results from an interaction between a “core” face-specific system involved in visual analysis and an “extended” system involved in spatial attention, more generally. However, the full gaze perception network has remained poorly specified. In the context of a functional magnetic resonance imaging study, we used psychophysiological interactions (PPIs) to identify brain regions that showed differential connectivity (correlation) with core face perception structures (posterior superior temporal sulcus [pSTS] and fusiform gyrus [FG]) when viewing gaze shifts relative to control eye movements (opening/closing the eyes). The PPIs identified altered connectivity between the pSTS and MT/V5, intraparietal sulcus, frontal eye fields, superior temporal gyrus (STG), supramarginal gyrus, and middle frontal gyrus (MFG). The FG showed altered connectivity with the same areas of the STG and MFG, demonstrating the contribution of both dorsal and ventral core face areas to gaze perception. We propose that this network provides an interactive system that alerts us to seen changes in other agents’ gaze direction, makes us aware of their altered focus of spatial attention, and prepares a corresponding shift in our own attention.

Integration of gaze direction and facial expression in patients with unilateral amygdala damage

Affective and social processes play a major role in everyday life, but appropriate methods to assess disturbances in these processes after brain lesions are still lacking. Past studies have shown that amygdala damage can impair recognition of facial expressions, particularly fear, as well as processing of gaze direction; but the mechanisms responsible for these deficits remain debated. Recent accounts of human amygdala function suggest that it is a critical structure involved in self-relevance appraisal. According to such accounts, responses to a given facial expression may vary depending on concomitant gaze direction and perceived social meaning. Here we investigated facial emotion recognition and its interaction with gaze in patients with unilateral amygdala damage (n = 19), compared to healthy controls (n = 10), using computer-generated dynamic face stimuli expressing variable intensities of fear, anger or joy, with different gaze directions (direct versus averted). If emotion perception is influenced by the self-relevance of expression based on gaze direction, a fearful face with averted gaze should be more relevant than the same expression with direct gaze because it signals danger near the observer; whereas anger with direct gaze should be more relevant than with averted gaze because it directly threatens the observer. Our results confirm a critical role for the amygdala in self-relevance appraisal, showing an interaction between gaze and emotion in healthy controls, a trend for such interaction in left-damaged patients but not in right-damaged patients. Impaired expression recognition was generally more severe for fear, but with a greater deficit for right versus left damage. These findings do not only provide new insights on human amygdala function, but may also help design novel neuropsychological tests sensitive to amygdala dysfunction in various patient populations.

The mere perception of eye contact increases arousal during a word-spelling task

Eye contact is a highly salient and fundamentally social signal. This entails that the mere perception of direct gaze may trigger differentiated neurobehavioral responses as compared to other gaze directions. We investigated this issue using a visual word-spelling task where faces under different gaze directions and head orientations were displayed on-screen concomitantly with the words. We show evidence for automatic increase of skin conductance response (SCR), indicative of arousal, associated with the perception of direct gaze as compared to both averted gaze and closed eyes. Moreover, the perception of averted gaze was associated with an increase of electromyographic (EMG) corrugator activity. These effects were observed in two demanding word-spelling tasks, but not in a simple letter decision task. We propose to interpret these findings in terms of the social value of direct and averted gaze and conclude that some circumstances such as the task at hand may be essential for uncovering the neurobehavioral responses associated with the perception of others' gaze.

Wired for her face? Male attentional bias for female faces

Under conditions of inattention or deficits in orienting attention, special classes of stimuli (e.g. faces, bodies) are more likely to be perceived than other stimuli. This suggests that biologically salient visual stimuli automatically recruit attention, even when they are task-irrelevant or ignored. Here we report results from a behavioral experiment with female and male subjects and two magnetoencephalography (MEG) experiments with male subjects only, in which we investigated attentional capture with face and hand stimuli. In both the behavioral and MEG experiments, subjects were required to count the number of gender-specific targets from either face or hand categories within a block of stimuli. In the behavioral experiment, we found that male subjects were significantly more accurate in response to female than male face target blocks. There was no corresponding effect found in response to hand target blocks. Female subjects did not show a gender-based difference in response to face or hand target blocks. MEG results indicated that the male subjects' responses to face stimuli in primary visual cortex (V1) and the face-selective part of the fusiform gyrus (FG) were reduced when male face stimuli were not relevant to the task, whereas female faces maintained a strong response in these areas in both task-relevant and task-irrelevant conditions. These results suggest that within the male brain, female face stimuli are more resilient to suppression than male faces, once attention is drawn to the part of the visual field where the face appears.

Neural circuitry of submissive behavior in social anxiety disorder: A preliminary study of response to direct eye gaze

Fear of eye gaze is common in social anxiety disorder (SAD) and may represent an evolutionarily conserved submissive behavior. SAD subjects and healthy volunteers who underwent functional magnetic resonance imaging showed significant differences in neural activity in amygdala, fusiform, insula, anterior cingulate and prefrontal cortex in response to direct versus averted gaze. Neural response to direct gaze may identify brain regions important in the pathophysiology of SAD.

Amygdala response to faces parallels social behavior in Williams syndrome

Individuals with Williams syndrome (WS), a genetically determined disorder, show relatively strong face-processing abilities despite poor visuospatial skills and depressed intellectual function. Interestingly, beginning early in childhood they also show an unusually high level of interest in face-to-face social interaction. We employed functional magnetic resonance imaging (fMRI) to investigate physiological responses in face-sensitive brain regions, including ventral occipito-temporal cortex and the amygdala, in this unique genetic disorder. Participants included 17 individuals with WS, 17 age- and gender-matched healthy adults (chronological age-matched controls, CA) and 17 typically developing 8- to 9-year-old children (developmental age controls, DA). While engaged in a face discrimination task, WS participants failed to recruit the amygdala, unlike both CA and DA controls. WS fMRI responses in ventral occipito-temporal cortex, however, were comparable to those of DA controls. Given the integral role of the amygdala in social behavior, the failure of WS participants to recruit this region during face processing may be a neural correlate of the abnormally high sociability that characterizes this disorder.

Autism: research into causes and intervention

Autism spectrum conditions are neuro-developmental syndromes with strong heritability. Cognitive theories have had some success in explaining why the cluster of features should co-occur. Empathizing deficits have the potential to make sense of one triad of impairments (social difficulties, communication difficulties and imagining others' minds), and may have a brain basis in the amygdala and left medial frontal cortex. A strong systemizing drive may account for a distinct triad of strengths (good attention to detail, deep, narrow interests and islets of ability). The brain basis of systemizing is yet to be understood. Family genetics studies suggest that these same cognitive dimensions (reduced empathizing alongside a strong drive to systemize) may also characterize the 'broader phenotype' among first-degree relatives. Molecular genetic studies are underway and any candidate genes for autism will ultimately need to be tested in relation to the observed differences in the brain, cognition and behaviour. The ethics of genetic screening or gene therapy should be thought about well ahead of these becoming available, since there is by no means any consensus that these would be desirable given the wide range of phenotypic traits, not all of which are disabling. Future research will need to focus on evaluating the extent to which any form of intervention reduces the triad of impairments whilst supporting the triad of strengths.

Impaired social cognition in mild Alzheimer disease

Abnormal decoding of social information has been associated with the conversion from prodromal Alzheimer's disease (AD) to dementia. Since the distributed neural networks involved in face processing are differentially affected in prodromal and dementia states of AD and in Fronto-Temporal Dementia (FTD), we hypothesized a differential impairment in face processing in these populations. Facial expression, gender and gaze direction decoding abilities were examined in patients with probable amnesic Mild Cognitive Impairment (aMCI, N=10) fulfilling criteria for prodromal AD, in patients with mild and moderate AD (N=10) as well as in FTD patients (N=10) and in a group of age- and sex-matched healthy comparison subjects (N=10). Gender recognition was preserved in all groups. Compared to controls, patients with mild or moderate AD were impaired in expression recognition and FTD patients were impaired in expression and gaze direction determination, whereas MCI patients were not impaired at all.

Neural bases of eye and gaze processing: the core of social cognition

Eyes and gaze are very important stimuli for human social interactions. Recent studies suggest that impairments in recognizing face identity, facial emotions or in inferring attention and intentions of others could be linked to difficulties in extracting the relevant information from the eye region including gaze direction. In this review, we address the central role of eyes and gaze in social cognition. We start with behavioral data demonstrating the importance of the eye region and the impact of gaze on the most significant aspects of face processing. We review neuropsychological cases and data from various imaging techniques such as fMRI/PET and ERP/MEG, in an attempt to best describe the spatio-temporal networks underlying these processes. The existence of a neuronal eye detector mechanism is discussed as well as the links between eye gaze and social cognition impairments in autism. We suggest impairments in processing eyes and gaze may represent a core deficiency in several other brain pathologies and may be central to abnormal social cognition.

Mirroring of attention by neurons in macaque parietal cortex

Macaques, like humans, rapidly orient their attention in the direction other individuals are looking. Both cortical and subcortical pathways have been proposed as neural mediators of social gaze following, but neither pathway has been characterized electrophysiologically in behaving animals. To address this gap, we recorded the activity of single neurons in the lateral intraparietal area (LIP) of rhesus macaques to determine whether and how this area might contribute to gaze following. A subset of LIP neurons mirrored observed attention by firing both when the subject looked in the preferred direction of the neuron, and when observed monkeys looked in the preferred direction of the neuron, despite the irrelevance of the monkey images to the task. Importantly, the timing of these modulations matched the time course of gaze-following behavior. A second population of neurons was suppressed by social gaze cues, possibly subserving task demands by maintaining fixation on the observed face. These observations suggest that LIP contributes to sharing of observed attention and link mirror representations in parietal cortex to a well studied imitative behavior.

Approachability and the amygdala: insights from Williams syndrome

Williams syndrome (WS) is a genetic neurodevelopmental disorder in which hypersociability is a characteristic feature. Given that the amygdala has been identified as an integral component of the neural system underlying sociability, researchers have suggested that the abnormal amygdala volumes found in individuals with WS may play a role in their hypersociability. The aim of this study was to examine the relationship between amygdala volume and hypersociability, as measured by approachability ratings, in 22 individuals with WS and 22 normal controls matched on chronological age, sex, and handedness. We confirmed previous findings of increased amygdala volumes and higher approachability ratings of both 'positive' and 'negative' faces in individuals with WS. A positive relationship between right amygdala volume and approachability ratings was found in individuals with WS, particularly ratings of 'negative' faces. The results unexpectedly revealed that individuals with WS report using features other than the eyes and mouth to determine approachability, particularly when they are younger. These findings support the theory that amygdala dysfunction in WS is related to their hypersociability. Furthermore, we propose that individuals with WS use atypical cognitive strategies compared to controls to determine approachability.

Human social attention: A new look at past, present, and future investigations

The present review examines the neural-behavioral correlates of human social attention, with special regard to the neural mechanisms involved in processing gaze information and the functional impact of gaze direction on the spatial orienting of attention. Our review suggests that there is strong evidence that specific brain systems are preferentially biased toward processing gaze information, yet this specificity is not mirrored by the behavioral data as measured in highly controlled model attention tasks such as the Posner cueing paradigm. In less controlled tasks, however, such as when observers are left free to select what they want to attend, they focus on people and their eyes, consistent with one's intuition and with the neural evidence that eyes are special. We discuss a range of implications of these data, including that much is to be gained by examining brain and behavioral processes to social stimuli as they occur in complex real-world settings.

Asymmetries of the human social brain in the visual, auditory and chemical modalities

Structural and functional asymmetries are present in many regions of the human brain responsible for motor control, sensory and cognitive functions and communication. Here, we focus on hemispheric asymmetries underlying the domain of social perception, broadly conceived as the analysis of information about other individuals based on acoustic, visual and chemical signals. By means of these cues the brain establishes the border between 'self' and 'other', and interprets the surrounding social world in terms of the physical and behavioural characteristics of conspecifics essential for impression formation and for creating bonds and relationships. We show that, considered from the standpoint of single- and multi-modal sensory analysis, the neural substrates of the perception of voices, faces, gestures, smells and pheromones, as evidenced by modern neuroimaging techniques, are characterized by a general pattern of right-hemispheric functional asymmetry that might benefit from other aspects of hemispheric lateralization rather than constituting a true specialization for social information.

Duration matters: dissociating neural correlates of detection and evaluation of social gaze

The interpretation of interpersonal gaze behavior requires the use of complex cognitive processes and guides social interactions. Among a variety of different gaze characteristics, gaze direction and gaze duration modulate crucially the meaning of the "social gaze". Nevertheless, prior neuroimaging studies disregarded the relevance of gaze duration by focusing on gaze direction only. The present functional magnetic resonance imaging (fMRI) study focused on the differentiation of these two gaze parameters. Therefore direct gaze displayed by virtual characters was contrasted with averted gaze and, additionally, systematically varied with respect to gaze duration (i.e., 1, 2.5 or 4 s). Consistent with prior findings, behavioral data showed that likeability was higher for direct than for averted gaze and increased linearly with increasing direct gaze duration. On the neural level, distinct brain regions were associated with the processing of gaze direction and gaze duration: (i) the comparison between direct and averted gaze revealed activations in bilateral occipito-temporal regions including the posterior superior temporal sulcus (pSTS); (ii) whereas increasing duration of direct gaze evoked differential neural responses in the medial prefrontal cortex (MPFC) including orbitofrontal and paracingulate regions. The results suggest two complementary cognitive processes related to different gaze parameters. On the one hand, the recruitment of multimodal sensory regions in the pSTS indicates detection of gaze direction via complex visual analysis. On the other hand, the involvement of the MPFC associated with outcome monitoring and mentalizing indicates higher-order social cognitive processes related to evaluation of the ongoing communicational input conveyed by direct gaze duration.

Neural mechanisms of social attention

Social attention is conveyed primarily by gaze, but also by head and body orientation. These cues not only signal a seen agent's direction of attention but are also used to infer their current goals and intentions. Here, we review recent research showing that different gaze, head and body orientations are represented by distinct neural mechanisms, and show that a statistical summary of recent neuroimaging studies reveals a widespread neural network for gaze processing. We discuss how this network subserves visual analysis of social attention cues, and imitative attention shifts and mental state attributions from these cues. We also review new research indicating that the posterior superior temporal sulcus region responds to the inferred intentionality of social cues, and consider the development of the gaze perception system.

The eye contact effect: mechanisms and development

The 'eye contact effect' is the phenomenon that perceived eye contact with another human face modulates certain aspects of the concurrent and/or immediately following cognitive processing. In addition, functional imaging studies in adults have revealed that eye contact can modulate activity in structures in the social brain network, and developmental studies show evidence for preferential orienting towards, and processing of, faces with direct gaze from early in life. We review different theories of the eye contact effect and advance a 'fast-track modulator' model. Specifically, we hypothesize that perceived eye contact is initially detected by a subcortical route, which then modulates the activation of the social brain as it processes the accompanying detailed sensory information.

Is eye to eye contact really threatening and avoided in social anxiety?--An eye-tracking and psychophysiology study

The effects of direct and averted gaze on autonomic arousal and gaze behavior in social anxiety were investigated using a new paradigm including animated movie stimuli and eye-tracking methodology. While high, medium, and low socially anxious (HSA vs. MSA vs. LSA) women watched animated movie clips, in which faces responded to the gaze of the participants with either direct or averted gaze, their eye movements, heart rate (HR) and skin conductance responses (SCR) were continuously recorded. Groups did not differ in their gaze behavior concerning direct vs. averted gaze, but high socially anxious women tended to fixate the eye region of the presented face longer than MSA and LSA, respectively. Furthermore, they responded to direct gaze with more pronounced cardiac acceleration. This physiological finding indicates that direct gaze may be a fear-relevant feature for socially anxious individuals in social interaction. However, this seems not to result in gaze avoidance. Future studies should examine the role of gaze direction and its interaction with facial expressions in social anxiety and its consequences for avoidance behavior and fear responses. Additionally, further research is needed to clarify the role of gaze perception in social anxiety.

Aberrant brain activation during gaze processing in boys with fragile X syndrome

CONTEXT

Eye contact is a fundamental component of human social behavior. Individuals with fragile X syndrome (fraX), particularly male subjects, avoid eye contact and display other social deficits. To date (to our knowledge), this behavior in fraX has been studied only in female subjects, who show lesser degrees of gaze aversion.

OBJECTIVE

To determine the neural correlates of the perception of direct eye gaze in adolescent boys with fraX using functional magnetic resonance imaging.

DESIGN

Cross-sectional study.

SETTING

Academic medical center.

PARTICIPANTS

Thirteen adolescent boys with fraX, 10 boys with developmental delay, and 13 typically developing control subjects.

MAIN OUTCOME MEASURES

Behavioral performance and brain activation during functional magnetic resonance imaging were evaluated during the presentation of faces with eye gaze directed to or averted away from subjects and during successive presentations of stimuli with eye gaze directed toward the subject. Whole-brain and region of interest analyses and regression analyses with task performance were performed.

RESULTS

Significantly greater activation was observed in prefrontal cortices in controls compared with boys having fraX, who (in contrast) demonstrated elevated left insula activation to direct eye gaze stimuli. Furthermore, compared with controls, boys with fraX showed greater sensitization in the left amygdala with successive exposure to direct gaze.

CONCLUSIONS

Compared with controls, boys with fraX display distinct patterns of brain activation in response to direct eye gaze. These results suggest that aberrant neural processing of direct eye gaze in subjects with fraX may be related to the associated avoidant response.

Amygdala activation during reading of emotional adjectives--an advantage for pleasant content

This event-related functional magnetic resonance imaging (fMRI) study investigated brain activity elicited by emotional adjectives during silent reading without specific processing instructions. Fifteen healthy volunteers were asked to read a set of randomly presented high-arousing emotional (pleasant and unpleasant) and low-arousing neutral adjectives. Silent reading of emotional in contrast to neutral adjectives evoked enhanced activations in visual, limbic and prefrontal brain regions. In particular, reading pleasant adjectives produced a more robust activation pattern in the left amygdala and the left extrastriate visual cortex than did reading unpleasant or neutral adjectives. Moreover, extrastriate visual cortex and amygdala activity were significantly correlated during reading of pleasant adjectives. Furthermore, pleasant adjectives were better remembered than unpleasant and neutral adjectives in a surprise free recall test conducted after scanning. Thus, visual processing was biased towards pleasant words and involved the amygdala, underscoring recent theoretical views of a general role of the human amygdala in relevance detection for both pleasant and unpleasant stimuli. Results indicate preferential processing of pleasant information in healthy young adults and can be accounted for within the framework of appraisal theory.

A developmental examination of amygdala response to facial expressions

Several lines of evidence implicate the amygdala in face-emotion processing, particularly for fearful facial expressions. Related findings suggest that face-emotion processing engages the amygdala within an interconnected circuitry that can be studied using a functional-connectivity approach. Past work also underscores important functional changes in the amygdala during development. Taken together, prior research on amygdala function and development reveals a need for more work examining developmental changes in the amygdala's response to fearful faces and in amygdala functional connectivity during face processing. The present study used event-related functional magnetic resonance imaging to compare 31 adolescents (9-17 years old) and 30 adults (21-40 years old) on activation to fearful faces in the amygdala and other regions implicated in face processing. Moreover, these data were used to compare patterns of amygdala functional connectivity in adolescents and adults. During passive viewing, adolescents demonstrated greater amygdala and fusiform activation to fearful faces than did adults. Functional connectivity analysis revealed stronger connectivity between the amygdala and the hippocampus in adults than in adolescents. Within each group, variability in age did not correlate with amygdala response, and sex-related developmental differences in amygdala response were not found. Eye movement data collected outside of the magnetic resonance imaging scanner using the same task suggested that developmental differences in amygdala activation were not attributable to differences in eye-gaze patterns. Amygdala hyperactivation in response to fearful faces may explain increased vulnerability to affective disorders in adolescence; stronger amygdala-hippocampus connectivity in adults than adolescents may reflect maturation in learning or habituation to facial expressions.

Oxytocin attenuates affective evaluations of conditioned faces and amygdala activity

Social relations between humans critically depend on our affective experiences of others. Oxytocin enhances prosocial behavior, but its effect on humans' affective experience of others is not known. We tested whether oxytocin influences affective ratings, and underlying brain activity, of faces that have been aversively conditioned. Using a standard conditioning procedure, we induced differential negative affective ratings in faces exposed to an aversive conditioning compared with nonconditioning manipulation. This differential negative evaluative effect was abolished by treatment with oxytocin, an effect associated with an attenuation of activity in anterior medial temporal and anterior cingulate cortices. In amygdala and fusiform gyrus, this modulation was stronger for faces with direct gaze, relative to averted gaze, consistent with a relative specificity for socially relevant cues. The data suggest that oxytocin modulates the expression of evaluative conditioning for socially relevant faces via influences on amygdala and fusiform gyrus, an effect that may explain its prosocial effects.

Time course of superior temporal sulcus activity in response to eye gaze: a combined fMRI and MEG study

The human superior temporal sulcus (STS) has been suggested to be involved in gaze processing, but temporal data regarding this issue are lacking. We investigated this topic by combining fMRI and MEG in four normal subjects. Photographs of faces with either averted or straight eye gazes were presented and subjects passively viewed the stimuli. First, we analyzed the brain areas involved using fMRI. A group analysis revealed activation of the STS for averted compared to straight gazes, which was confirmed in all subjects. We then measured brain activity using MEG, and conducted a 3D spatial filter analysis. The STS showed higher activity in response to averted versus straight gazes during the 150-200 ms period, peaking at around 170 ms, after stimulus onset. In contrast, the fusiform gyrus, which was detected by the main effect of stimulus presentations in fMRI analysis, exhibited comparable activity across straight and averted gazes at about 170 ms. These results indicate involvement of the human STS in rapid processing of the eye gaze of another individual.

Eye-Gaze Direction Modulates Race-Related Amygdala Activity

Although previous research has found greater activity in the human amygdala in response to Black male compared with White male targets, the basis of this effect remains unclear. For example, is it race alone that triggers amygdala activity, or do other stimulus cues, in conjunction with racial group membership, also play a critical role in this regard? To address this issue, we used functional magnetic resonance imaging to measure amygdala activity in response to Black and White male targets displaying different eye-gaze directions (i.e. direct or averted gaze), as gaze cues have been shown to influence the socio-emotional aspects of person construal. The results revealed that eye-gaze direction significantly moderates race-related amygdala activity. Specifically, Black targets only generated greater amygdala activity than White targets when the faces bore direct gaze. This finding is noteworthy as it demonstrates the importance of compound stimulus cues in the appraisal of social targets.

Facing the gaze of others

Others' gaze constitutes a rich and essential social signal, which is decoded by taking into account other aspects of the face as well as the social context. Since the perception of averted gaze and that of direct gaze trigger distinct cognitive processes, the studies on gaze perception have focused separately on these two gaze directions. The perception of averted gaze induces orienting of spatial attention in the gazed-at direction as well as joint attention processes while direct gaze or gaze contact signals interest directed at the observer's self and is often the preliminary to interindividual interactions. Studies in cognitive neuroscience have focused first on averted gaze perception. However, recent studies have emphasised the asymmetries in the processing of direct versus averted gaze. This has led to a growing interest in the neural substrates of direct gaze perception. This issue has recently started to be actively addressed in our group using fMRI, MEG, EEG and source reconstruction methods. These studies emphasise that the perception of direct gaze elicits early processes that are related to face and eye movement encoding as well as to emotion and theory-of-mind.

The left amygdala knows fear: laterality in the amygdala response to fearful eyes

The detection of threat is a role that the amygdala plays well, evidenced by its increased response to fearful faces in human neuroimaging studies. A critical element of the fearful face is an increase in eye white area (EWA), hypothesized to be a significant cue in activating the amygdala. However, another important social signal that can increase EWA is a lateral shift in gaze direction, which also serves to orient attention to potential threats. It is unknown how the amygdala differentiates between these increases in EWA and those that are specifically associated with fear. Using functional magnetic resonance imaging, we show that the left amygdala distinguished between fearful eyes and gaze shifts despite similar EWA increases whereas the right amygdala was less discriminatory. Additional analyses also revealed selective hemispheric response patterns in the left fusiform gyrus. Our data show clear hemispheric differences in EWA-based fear activation, suggesting the existence of parallel mechanisms that code for emotional face information.

Learning affective values for faces is expressed in amygdala and fusiform gyrus

To monitor the environment for social threat humans must build affective evaluations of others. These evaluations are malleable and to a high degree shaped by responses engendered by specific social encounters. The precise neuronal mechanism by which these evaluations are constructed is poorly understood. We tested a hypothesis that conjoint activity in amygdala and fusiform gyrus would correlate with acquisition of social stimulus value. We tested this using a reinforcement learning algorithm, Q-learning, that assigned values to faces as a function of a history of pairing, or not pairing, with aversive shocks. Behaviourally, we observed a correlation between conditioning induced changes in skin conductance response (SCR) and subjective ratings for likeability of faces. Activity in both amygdala and fusiform gyrus (FG) correlated with the output of the reinforcement learning algorithm parameterized by these ratings. In amygdala, this effect was greater for averted than direct gaze faces. Furthermore, learning-related activity change in these regions correlated with SCR and subjective ratings. We conclude that amygdala and fusiform encode affective value in a manner that closely approximates a standard computational solution to learning.

Brain activation during eye gaze discrimination in stable schizophrenia

OBJECTIVE

Earlier studies described gaze discrimination impairment in schizophrenia. The purpose of this study was to compare gaze discrimination abilities and associated brain activation in persons with stable schizophrenia and matched controls.

METHODS

13 schizophrenia and 12 healthy participants underwent a gaze discrimination task with face stimuli rotated at 0, 4 and 8 degrees deviation. During fMRI with BOLD imaging, subjects were asked to identify whether a face was making eye contact. Subject-level parameter estimates for BOLD signal change were entered into an orientation by group mixed effect repeated measures ANOVA.

RESULTS

Gaze discrimination performance did not differ between groups. Patients showed decreased activation in areas of bilateral inferior frontal and occipital areas, and select temporo-limbic regions, including amygdala. Groups differed by activation patterns according to gaze deviation. In controls, faces with 4 degrees deviation produced higher activation in frontal and temporal regions. In patients, 0 degrees deviation produced increased activation in amygdala and areas of temporal neocortex.

CONCLUSIONS

Despite similar gaze discrimination abilities, schizophrenia patients exhibit decreased brain activation in areas associated with executive, emotional and visual processing. Controls exhibited increased activation associated with the more difficult task in select frontal and temporal regions. Patients exhibited increased activation associated with direct gaze in temporal regions, which may relate to common symptoms.

Oxytocin increases gaze to the eye region of human faces

BACKGROUND

In nonhuman mammals, oxytocin has a critical role in peer recognition and social approach behavior. In humans, oxytocin has been found to enhance trust and the ability to interpret the emotions of others. It has been suggested that oxytocin may enhance facial processing by increasing focus on the eye region of human faces.

METHODS

In a double-blind, randomized, placebo-controlled, between-subject design, we tracked the eye movements of 52 healthy male volunteers who were presented with 24 neutral human faces after intranasal administration of 24 IU oxytocin or placebo.

RESULTS

Participants given oxytocin showed an increased number of fixations and total gaze time toward the eye region compared with placebo participants.

CONCLUSIONS

Oxytocin increases gaze specifically toward the eye region of human faces. This may be one mechanism by which oxytocin enhances emotion recognition, interpersonal communication, and social approach behavior in humans. Findings suggest a possible role for oxytocin in the treatment of disorders characterized by eye-gaze avoidance and facial processing deficits.

Dissociable Roles of the Superior Temporal Sulcus and the Intraparietal Sulcus in Joint Attention: A Functional Magnetic Resonance Imaging Study

Previous imaging work has shown that the superior temporal sulcus (STS) region and the intraparietal sulcus (IPS) are specifically activated during the passive observation of shifts in eye gaze [Pelphrey, K. A., Singerman, J. D., Allison, T., & McCarthy, G. Brain activation evoked by perception of gaze shifts: The influence of context. Neuropsychologia, 41, 156–170, 2003; Hoffman, E. A., & Haxby, J. V. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nature Neuroscience, 3, 80–84, 2000; Puce, A., Allison, T., Bentin, S., Gore, J. C., & McCarthy, G. Temporal cortex activation in humans viewing eye and mouth movements. Journal of Neuroscience, 18, 2188–2199, 1998; Wicker, B., Michel, F., Henaff, M. A., & Decety, J. Brain regions involved in the perception of gaze: A PET study. Neuroimage, 8, 221–227, 1998]. Are the same brain regions also involved in extracting gaze direction in order to establish joint attention? In an event-related functional magnetic resonance imaging experiment, healthy human subjects actively followed the directional cue provided by the eyes of another person toward an object in space or, in the control condition, used a nondirectional symbolic cue to make an eye movement toward an object in space. Our results show that the posterior part of the STS region and the cuneus are specifically involved in extracting and using detailed directional information from the eyes of another person to redirect one's own gaze and establish joint attention. The IPS, on the other hand, seems to be involved in encoding spatial direction and mediating shifts of spatial attention independent of the type of cue that triggers this process.

Recognizing emotion from facial expressions: psychological and neurological mechanisms

Recognizing emotion from facial expressions draws on diverse psychological processes implemented in a large array of neural structures. Studies using evoked potentials, lesions, and functional imaging have begun to elucidate some of the mechanisms. Early perceptual processing of faces draws on cortices in occipital and temporal lobes that construct detailed representations from the configuration of facial features. Subsequent recognition requires a set of structures, including amygdala and orbitofrontal cortex, that links perceptual representations of the face to the generation of knowledge about the emotion signaled, a complex set of mechanisms using multiple strategies. Although recent studies have provided a wealth of detail regarding these mechanisms in the adult human brain, investigations are also being extended to nonhuman primates, to infants, and to patients with psychiatric disorders.

Explicit versus implicit gaze processing assessed by ERPs

Gaze processing was investigated using event-related potentials in two different tasks in which front-view and 3/4-view faces were presented, with eyes gazing straight ahead or averted. Task alternated between an explicit gaze direction judgment and a judgment on head orientation where gaze was irrelevant. Accuracy and reaction times were affected by the congruency of gaze and head directions in both tasks suggesting gaze was processed implicitly in the head orientation task. In both tasks, larger P1 and N170 were found for 3/4-view faces compared to front-view faces that were not due to the luminance or contrast of the pictures. The N170 was also larger for averted than straight gaze but for front-view faces only. In contrast, larger amplitudes for straight than averted gaze were reliably measured around 400-600 ms regardless of head orientation or task demands, and likely reflected the outcome of gaze processing. The results suggest that head orientation and gaze direction discrimination occur regardless of task demands and interact at the decision making level. Neural markers of head orientation occur before those for gaze direction and the early structural encoding stages of face processing are view-dependant.

Processing the socially relevant parts of faces

Faces are processed by a distributed neural system in the visual as well as in the non-visual cortex [the "core" and the "extended" systems, J.V. Haxby, E.A. Hoffman, M.I. Gobbini, The distributed human neural system for face perception, Trends Cogn. Sci. 4 (2000) 223-233]. Yet, the functions of the different brain regions included in the face processing system are far from clear. On the basis of the case study of a patient unable to recognize fearful faces, Adolphs et al. [R. Adolphs, F. Gosselin, T.W. Buchanan, D. Tranel, P. Schyns, A.R. Damasio, A mechanism for impaired fear recognition after amygdala damage, Nature 433 (2005) 68-72] suggested that the amygdala might play a role in orienting attention towards the eyes, i.e. towards the region of face conveying most information about fear. In a functional magnetic resonance (fMRI) study comparing patterns of activation during observation of whole faces and parts of faces displaying neutral expressions, we evaluated the neural systems for face processing when only partial information is provided, as well as those involved in processing two socially relevant facial areas (the eyes and the mouth). Twenty-four subjects were asked to perform a gender decision task on pictures showing whole faces, upper faces (eyes and eyebrows), and lower faces (mouth). Our results showed that the amygdala was activated more in response to the whole faces than to parts of faces, indicating that the amygdala is involved in orienting attention toward eye and mouth. Processing of parts of faces in isolation was found to activate other regions within both the "core" and the "extended" systems, as well as structures outside this network, thus suggesting that these structures are involved in building up the representation of the whole face from its parts.

3D pattern of brain abnormalities in Williams syndrome visualized using tensor-based morphometry

Williams syndrome (WS) is a neurodevelopmental disorder associated with deletion of approximately 20 contiguous genes in chromosome band 7q11.23. Individuals with WS exhibit mild to moderate mental retardation, but are relatively more proficient in specific language and musical abilities. We used tensor-based morphometry (TBM) to visualize the complex pattern of gray/white matter reductions in WS, based on fluid registration of structural brain images.

METHODS

3D T1-weighted brain MRIs of 41 WS subjects (age [mean+/-SD]: 29.2+/-9.2 years; 23F/18M) and 39 age-matched healthy controls (age: 27.5+/-7.4 years; 23F/16M) were fluidly registered to a minimum deformation target. Fine-scale volumetric differences were mapped between diagnostic groups. Local regions were identified where regional structure volumes were associated with diagnosis, and with intelligence quotient (IQ) scores. Brain asymmetry was also mapped and compared between diagnostic groups.

RESULTS

WS subjects exhibited widely distributed brain volume reductions (approximately 10-15% reduction; P<0.0002, permutation test). After adjusting for total brain volume, the frontal lobes, anterior cingulate, superior temporal gyrus, amygdala, fusiform gyrus and cerebellum were found to be relatively preserved in WS, but parietal and occipital lobes, thalamus and basal ganglia, and midbrain were disproportionally decreased in volume (P<0.0002). These regional volumes also correlated positively with performance IQ in adult WS subjects (age > or = 30 years, P = 0.038).

CONCLUSION

TBM facilitates 3D visualization of brain volume reductions in WS. Reduced parietal/occipital volumes may be associated with visuospatial deficits in WS. By contrast, frontal lobes, amygdala, and cingulate gyrus are relatively preserved or even enlarged, consistent with unusual affect regulation and language production in WS.

Engineering Human Cooperation : Does Involuntary Neural Activation Increase Public Goods Contributions?

In a laboratory experiment, we use a public goods game to examine the hypothesis that human subjects use an involuntary eye-detector mechanism for evaluating the level of privacy. Half of our subjects are "watched" by images of a robot presented on their computer screen. The robot-named Kismet and invented at MIT-is constructed from objects that are obviously not human with the exception of its eyes. In our experiment, Kismet produces a significant difference in behavior that is not consistent with existing economic models of preferences, either self- or other-regarding. Subjects who are "watched" by Kismet contribute 29% more to the public good than do subjects in the same setting without Kismet.

Gender specific processing of eye contact within the human medial temporal lobe

Eye contact is a powerful social stimulus for human and non-human primates. However, it is unclear whether brain mechanisms that interpret eye contact are sensitive to gender. Here we show that human brain responses to eye contact are indeed gender specific. Recording event-related potentials directly from the medial temporal lobes, we found that eye contact elicited specific responses in men only when they saw female faces. Conversely, women responded specifically to eye contact only when they saw pictures of men. Thus, the human medial temporal lobes subserve specifically the processing of eye contact with persons of the opposite gender.

Gaze cueing of attention: visual attention, social cognition, and individual differences

During social interactions, people's eyes convey a wealth of information about their direction of attention and their emotional and mental states. This review aims to provide a comprehensive overview of past and current research into the perception of gaze behavior and its effect on the observer. This encompasses the perception of gaze direction and its influence on perception of the other person, as well as gaze-following behavior such as joint attention, in infant, adult, and clinical populations. Particular focus is given to the gaze-cueing paradigm that has been used to investigate the mechanisms of joint attention. The contribution of this paradigm has been significant and will likely continue to advance knowledge across diverse fields within psychology and neuroscience.