The human amygdala plays an important role in gaze monitoring. A PET study

Rapid and multiple-stage activation of the human amygdala for processing facial signals

Human faces transmit multiple valuable signals, and neuroimaging studies have shown that the amygdala is active in response to facial stimuli. However, little has been known about the temporal profile of amygdala activation during facial signal processing until recently. Here we review three recent studies conducted by our group in which we recorded amygdala intracranial electroencephalography in humans. The subjects were engaged in tasks that required automatic processing of faces, eye gazes and emotional expressions. Time-frequency statistical parametric mapping analyses revealed that the amygdala showed gamma-band activation in response to emotional expressions, gazes and faces, with peak latencies at about 100 ms, 200 ms and 250 ms, respectively. These results suggest that: (1) the amygdala performs multiple-stage processing in response to these facial signals using different visual input routes, and (2) amygdala activation for processing all of these facial signals is rapid, which could be prior to or simultaneous with conscious awareness of faces.

Lateralized interactive social content and valence processing within the human amygdala

In the past, the amygdala has generally been conceptualized as a fear-processing module. Recently, however, it has been proposed to respond to all stimuli that are relevant with respect to the current needs, goals, and values of an individual. This raises the question of whether the human amygdala may differentiate between separate kinds of relevance. A distinction between emotional (vs. neutral) and social (vs. non-social) relevance is supported by previous studies showing that the human amygdala preferentially responds to both emotionally and socially significant information, and these factors might even display interactive encoding properties. However, no investigation has yet probed a full 2 (positive vs. negative valence) × 2 (social vs. non-social content) processing pattern, with neutral images as an additional baseline. Applying such an extended orthogonal factorial design, our fMRI study demonstrates that the human amygdala is (1) more strongly activated for neutral social vs. non-social information, (2) activated at a similar level when viewing social positive or negative images, but (3) displays a valence effect (negative vs. positive) for non-social images. In addition, this encoding pattern is not influenced by cognitive or behavioral emotion regulation mechanisms, and displays a hemispheric lateralization with more pronounced effects on the right side. Finally, the same valence × social content interaction was found in three additional cortical regions, namely the right fusiform gyrus, right anterior superior temporal gyrus, and medial orbitofrontal cortex. Overall, these findings suggest that valence and social content processing represent distinct kinds of relevance that interact within the human amygdala as well as in a more extensive cortical network, likely subserving a key role in relevance detection.

Distinct Contributions to Facial Emotion Perception of Foveated versus Nonfoveated Facial Features

Foveated stimuli receive visual processing that is quantitatively and qualitatively different from nonfoveated stimuli. At normal interpersonal distances, people move their eyes around another’s face so that certain features receive foveal processing; on any given fixation, other features therefore project extrafoveally. Yet little is known about the processing of extrafoveally presented facial features, how informative those extrafoveally presented features are for face perception (e.g., for assessing another’s emotion), or what processes extract task-relevant (e.g., emotion-related) cues from facial features that first appear outside the fovea, and how these processes are implemented in the brain.

Frontolimbic neural circuitry at 6 months predicts individual differences in joint attention at 9 months

Elucidating the neural basis of joint attention in infancy promises to yield important insights into the development of language and social cognition, and directly informs developmental models of autism. We describe a new method for evaluating responding to joint attention performance in infancy that highlights the 9- to 10-month period as a time interval of maximal individual differences. We then demonstrate that fractional anisotropy in the right uncinate fasciculus, a white matter fiber bundle connecting the amygdala to the ventral-medial prefrontal cortex and anterior temporal pole, measured in 6-month-olds predicts individual differences in responding to joint attention at 9 months of age. The white matter microstructure of the right uncinate was not related to receptive language ability at 9 months. These findings suggest that the development of core nonverbal social communication skills in infancy is largely supported by preceding developments within right lateralized frontotemporal brain systems.

Reversible inactivation of pSTS suppresses social gaze following in the macaque (Macaca mulatta)

Humans and other primates shift their attention to follow the gaze of others [gaze following (GF)]. This behavior is a foundational component of joint attention, which is severely disrupted in neurodevelopmental disorders such as autism and schizophrenia. Both cortical and subcortical pathways have been implicated in GF, but their contributions remain largely untested. While the proposed subcortical pathway hinges crucially on the amygdala, the cortical pathway is thought to require perceptual processing by a region in the posterior superior temporal sulcus (pSTS). To determine whether pSTS is necessary for typical GF behavior, we engaged rhesus macaques in a reward discrimination task confounded by leftward- and rightward-facing social distractors following saline or muscimol injections into left pSTS. We found that reversible inactivation of left pSTS with muscimol strongly suppressed GF, as assessed by reduced influence of observed gaze on target choices and saccadic reaction times. These findings demonstrate that activity in pSTS is required for normal GF by primates.

Hard to "tune in": neural mechanisms of live face-to-face interaction with high-functioning autistic spectrum disorder

Persons with autism spectrum disorders (ASD) are known to have difficulty in eye contact (EC). This may make it difficult for their partners during face to face communication with them. To elucidate the neural substrates of live inter-subject interaction of ASD patients and normal subjects, we conducted hyper-scanning functional MRI with 21 subjects with autistic spectrum disorder (ASD) paired with typically-developed (normal) subjects, and with 19 pairs of normal subjects as a control. Baseline EC was maintained while subjects performed real-time joint-attention task. The task-related effects were modeled out, and inter-individual correlation analysis was performed on the residual time-course data. ASD-Normal pairs were less accurate at detecting gaze direction than Normal-Normal pairs. Performance was impaired both in ASD subjects and in their normal partners. The left occipital pole (OP) activation by gaze processing was reduced in ASD subjects, suggesting that deterioration of eye-cue detection in ASD is related to impairment of early visual processing of gaze. On the other hand, their normal partners showed greater activity in the bilateral occipital cortex and the right prefrontal area, indicating a compensatory workload. Inter-brain coherence in the right IFG that was observed in the Normal-Normal pairs (Saito et al., 2010) during EC diminished in ASD-Normal pairs. Intra-brain functional connectivity between the right IFG and right superior temporal sulcus (STS) in normal subjects paired with ASD subjects was reduced compared with in Normal-Normal pairs. This functional connectivity was positively correlated with performance of the normal partners on the eye-cue detection. Considering the integrative role of the right STS in gaze processing, inter-subject synchronization during EC may be a prerequisite for eye cue detection by the normal partner.

The role of the amygdala in atypical gaze on emotional faces in autism spectrum disorders

Reduced focus toward the eyes is a characteristic of atypical gaze on emotional faces in autism spectrum disorders (ASD). Along with the atypical gaze, aberrant amygdala activity during face processing compared with neurotypically developed (NT) participants has been repeatedly reported in ASD. It remains unclear whether the previously reported dysfunctional amygdalar response patterns in ASD support an active avoidance of direct eye contact or rather a lack of social attention. Using a recently introduced emotion classification task, we investigated eye movements and changes in blood oxygen level-dependent (BOLD) signal in the amygdala with a 3T MRI scanner in 16 autistic and 17 control adult human participants. By modulating the initial fixation position on faces, we investigated changes triggered by the eyes compared with the mouth. Between-group interaction effects revealed different patterns of gaze and amygdalar BOLD changes in ASD and NT: Individuals with ASD gazed more often away from than toward the eyes, compared with the NT group, which showed the reversed tendency. An interaction contrast of group and initial fixation position further yielded a significant cluster of amygdala activity. Extracted parameter estimates showed greater response to eyes fixation in ASD, whereas the NT group showed an increase for mouth fixation. The differing patterns of amygdala activity in combination with differing patterns of gaze behavior between groups triggered by direct eye contact and mouth fixation, suggest a dysfunctional profile of the amygdala in ASD involving an interplay of both eye-avoidance processing and reduced orientation.

The effect of gaze direction on three-dimensional face recognition in infants

Eye gaze is an important tool for social contact. In this study, we investigated whether direct gaze facilitates the recognition of three-dimensional face images in infants. We presented artificially produced face images in rotation to 6-8 month-old infants. The eye gaze of the face images was either direct or averted. Sixty-one sequential images of each face were created by rotating the vertical axis of the face from frontal view to ± 30°. The recognition performances of the infants were then compared between faces with direct gaze and faces with averted gaze. Infants showed evidence that they were able to discriminate the novel from familiarized face by 8 months of age and only when gaze is direct. These results suggest that gaze direction may affect three-dimensional face recognition in infants.

Electrophysiological responses in mothers to their own and unfamiliar child's gaze information

An attachment bond between a mother and her child is one of the most intimate human relationships. It is important for a mother to be sensitive to her child's gaze direction because exchanging gaze information plays a vital role in their relationship. Furthermore, recent studies have revealed differential neural activation patterns in mothers when presented the faces of their own children or the unfamiliar child of other people. Based on these findings, in the present study, we investigated whether mothers show differential neural responses to gaze information of their own child compared to that of an unfamiliar child. To this end, event-related-potentials elicited by the faces of one's own or an unfamiliar child with straight or averted gaze directions were measured using an oddball-paradigm. The results showed that peak amplitudes of the N170 component were enlarged by viewing the straight gazes compared to the averted gazes of one's own child, but not of an unfamiliar child. When the gaze was directed straight, the P3 amplitude elicited by one's own child's face is smaller than that elicited by an unfamiliar child's face. P3s elicited in viewing one's own child's face with averted gaze and in viewing an unfamiliar child's face with straight gaze were positively correlated with state-anxiety. These results bolster the hypothesis that processing the gaze information of one's own child elicits differential neural activation compared to the gaze information of an other person's unfamiliar child at both perceptual and evaluative stages of face processing.

Hypersensitivity in borderline personality disorder during mindreading

BACKGROUND

One of the core symptoms of borderline personality disorder (BPD) is the instability in interpersonal relationships. This might be related to existent differences in mindreading between BPD patients and healthy individuals.

METHODS

We examined the behavioural and neurophysiological (fMRI) responses of BPD patients and healthy controls (HC) during performance of the 'Reading the Mind in the Eyes' test (RMET).

RESULTS

Mental state discrimination was significantly better and faster for affective eye gazes in BPD patients than in HC. At the neurophysiological level, this was manifested in a stronger activation of the amygdala and greater activity of the medial frontal gyrus, the left temporal pole and the middle temporal gyrus during affective eye gazes. In contrast, HC subjects showed a greater activation in the insula and the superior temporal gyri.

CONCLUSION

These findings indicate that BPD patients are highly vigilant to social stimuli, maybe because they resonate intuitively with mental states of others.

Perceived eye region and the processing of fearful expressions in mild cognitive impairment patients

BACKGROUND

The aim of the present study was to assess the possibility of compensating early facial expression recognition impairments in amnestic Mild Cognitive Impairment (a-MCI) patients.

METHODS

Twelve patients with a-MCI and 17 healthy participants matched according to age and education participated in the study. The originality of the present study was to cue the recognition of facial expressions (happiness, anger, fear, and neutral) by comparing eye region expressions and entire facial expressions.

RESULTS

A deficit in the recognition of fearful expressions was observed in a-MCI patients relative to the control group, whereas recognition of all the other emotional expressions was spared. Nevertheless, when eye expressions cued the recognition of fearful facial expressions, the performance of normal controls and a-MCI patients was comparable.

CONCLUSION

The present paper indicates a selective impairment in fear recognition in the prodromal state of Alzheimer's disease, and the possibility of compensating this deficit by orienting selective attention on specific facial features.

Effects of a common variant in the CD38 gene on social processing in an oxytocin challenge study: possible links to autism

The intranasal application of oxytocin (OT) has been shown to influence behavioral and neural correlates of social processing. These effects are probably mediated by genetic variations within the OT system. One potential candidate could be the CD38 gene, which codes for a transmembrane protein engaged in OT secretion processes. A common variation in this gene (rs3796863) was recently found to be associated with autism spectrum disorders (ASD). Using an imaging genetics approach, we studied differential effects of an intranasal OT application on neural processing of social stimuli in 55 healthy young men depending on their CD38 gene variant in a double-blind placebo-controlled crossover design. Genotype had a significant influence on both behavioral and neuronal measures of social processing. Homozygotic risk allele carriers showed slower reaction times (RT) and higher activation of left fusiform gyrus during visual processing of social stimuli. Under OT activation differences between genotypes were more evident (though not statistically significantly increased) and RT were accelerated in homozygotic risk allele carriers. According to our data, rs3796863 mainly influences fusiform gyrus activation, an area which has been widely discussed in ASD research. OT seems to modulate this effect by enhancing activation differences between allele groups, which suggests an interaction between genetic makeup and OT availability on fusiform gyrus activation. These results support recent approaches to apply OT as a pharmacological treatment of ASD symptoms.

["Theory of mind" and its neuronal correlates in forensically relevant disorders]

Theory of mind (ToM), the ability to recognize mental states of others, and empathy are crucial cognitive-emotional processes for appropriate social interactions. Deficits in these processes can lead to maladjusted social behavior or even to aggressive or criminal behavior. ToM and empathy deficits have been found in different forensically relevant disorders, such as schizophrenia, pedophilia but especially in autism and psychopathy according to Hare. Most notably, autistic and psychopathic patients differ in their type of deficits and in their neuronal correlates. While autistic individuals lack the ability to take the perspective of others, psychopaths lack empathy. The aim of this article is to provide a better understanding of the pathophysiology of ToM and empathy deficits in forensically relevant disorders by reviewing and discussing the findings of neuroimaging and lesion studies and to highlight crucial implications for neuropsychotherapy according to Grawe.

Right hemispheric dominance and interhemispheric cooperation in gaze-triggered reflexive shift of attention

AIMS

The neural substrate for the processing of gaze remains unknown. The aim of the present study was to clarify which hemisphere dominantly processes and whether bilateral hemispheres cooperate with each other in gaze-triggered reflexive shift of attention.

METHODS

Twenty-eight normal subjects were tested. The non-predictive gaze cues were presented either in unilateral or bilateral visual fields. The subjects localized the target as soon as possible.

RESULTS

Reaction times (RT) were shorter when gaze-cues were congruent toward than away from targets, whichever visual field they were presented in. RT were shorter in left than right visual field presentations. RT in mono-directional bilateral presentations were shorter than both of those in left and right presentations. When bi-directional bilateral cues were presented, RT were faster when valid cues were presented in the left than right visual fields.

CONCLUSION

The right hemisphere appears to be dominant, and there is interhemispheric cooperation in gaze-triggered reflexive shift of attention.

The Social, Emotional and Cultural Life of the Orbitofrontal Cortex

Formal assessment of the impact of frontal-lobe impairment on social and interpersonal function has been somewhat neglected relative to interest in documenting executive impairment on nonsocial tasks. Despite this, experimental evidence is accruing that attests to the role of the orbitomedial frontal lobes, the amygdala and related structures in the appraisal and interpretation of socially relevant stimuli. In this address the evidence regarding the frontal lobes and social cognition, that is, social information processing is reviewed. At least 3 different constructs have been proposed that appear to be mediated by cerebral systems encompassing the orbitomedial frontal cortex: (1) social schema or social knowledge networks that guide judgments and behavioural responses, (2) emotional processing, including both the ability to recognise different affective states in others and the ability to experience affective (somatic) responses to emotionally important environmental stimuli, and (3) theory of mind – the ability to conceptualise what others are thinking and feeling. On the basis of this recent experimental work it is argued that a variety of assessment techniques are becoming available that are both practical and potentially sensitive to psychosocial changes after frontal lobe damage.

Neurocognitive mechanisms of gaze-expression interactions in face processing and social attention

The face conveys a rich source of non-verbal information used during social communication. While research has revealed how specific facial channels such as emotional expression are processed, little is known about the prioritization and integration of multiple cues in the face during dyadic exchanges. Classic models of face perception have emphasized the segregation of dynamic vs. static facial features along independent information processing pathways. Here we review recent behavioral and neuroscientific evidence suggesting that within the dynamic stream, concurrent changes in eye gaze and emotional expression can yield early independent effects on face judgments and covert shifts of visuospatial attention. These effects are partially segregated within initial visual afferent processing volleys, but are subsequently integrated in limbic regions such as the amygdala or via reentrant visual processing volleys. This spatiotemporal pattern may help to resolve otherwise perplexing discrepancies across behavioral studies of emotional influences on gaze-directed attentional cueing. Theoretical explanations of gaze-expression interactions are discussed, with special consideration of speed-of-processing (discriminability) and contextual (ambiguity) accounts. Future research in this area promises to reveal the mental chronometry of face processing and interpersonal attention, with implications for understanding how social referencing develops in infancy and is impaired in autism and other disorders of social cognition.

Understanding the social brain in autism

Autism is an early onset neurodevelopmental disorder characterized by disruption of early social interaction. Although the social disability of autism remains the central defining feature of the condition, mechanisms that might account for this disability remain poorly understood. This paper briefly reviews some aspects of the social deficit in autism focusing on new approaches to characterizing social information processing problems, potential brain mechanisms, and theoretical models of the disorder. It will touch on aspects of specific social processes that appear to develop in unusual ways in autism including facial perception, joint attention, and social information processing. The importance of adopting more ecologically valid methods and for integrating the various approaches in deriving new models for social deficits in autism will be highlighted. Future research should build on the emerging synergy of different aspects of social neuroscience.

Rapid amygdala gamma oscillations in response to eye gaze

Background

The eye gaze of other individuals conveys important social information and can trigger multiple psychological activities; some of which, such as emotional reactions and attention orienting, occur very rapidly. Although some neuroscientific evidence has suggested that the amygdala may be involved in such rapid gaze processing, no evidence has been reported concerning the speed at which the amygdala responds to eye gaze.

Methodology/Principal Findings

To investigate this issue, we recorded electrical activity within the amygdala of six subjects using intracranial electrodes. Subjects observed images of eyes and mosaics pointing in averted and straight directions. The amygdala showed higher gamma-band oscillations for eye gaze than for mosaics, which peaked at 200 ms regardless of the direction of the gaze.

Conclusion

These results indicate that the human amygdala rapidly processes eye gaze.

Eye Can See What You Want: Posterior Intraparietal Sulcus Encodes the Object of an Actor's Gaze

In a social setting, seeing Sally look at a clock means something different to seeing her gaze longingly at a slice of chocolate cake. In both cases, her eyes and face might be turned rightward, but the information conveyed is markedly different, depending on the object of her gaze. Numerous studies have examined brain systems underlying the perception of gaze direction, but less is known about the neural basis of perceiving gaze shifts to specific objects. During fMRI, participants observed an actor look toward one of two objects, each occupying a distinct location. Video stimuli were sequenced to obtain repetition suppression (RS) for object identity, independent of spatial location. In a control condition, a spotlight highlighted one of the objects, but no actor was present. Observation of the human actor's gaze compared with the spotlight engaged frontal, parietal, and temporal cortices, consistent with a broad action observation network. RS for gazed object in the human condition was found in posterior intraparietal sulcus (pIPS). RS for highlighted object in the spotlight condition was found in middle occipital, inferior temporal, medial fusiform gyri, and superior parietal lobule. These results suggest that human pIPS is specifically sensitive to the type object that an observed actor looks at (tool vs. food), irrespective of the observed actor's gaze location (left vs. right). A general attention or lower-level object feature processing mechanism cannot account for the findings because a very different response pattern was seen in the spotlight control condition. Our results suggest that, in addition to spatial orienting, human pIPS has an important role in object-centered social orienting.

Autism spectrum traits predict the neural response to eye gaze in typical individuals

Autism Spectrum Disorders (ASD) are neurodevelopmental disorders characterised by impaired social interaction and communication, restricted interests and repetitive behaviours. The severity of these characteristics are posited to lie on a continuum extending into the typical population, and typical adults' performance on behavioural tasks that are impaired in ASD is correlated with the extent to which they display autistic traits (as measured by Autism Spectrum Quotient, AQ). Individuals with ASD also show structural and functional differences in brain regions involved in social perception. Here we show that variation in AQ in typically developing individuals is associated with altered brain activity in the neural circuit for social attention perception while viewing others' eye gaze. In an fMRI experiment, participants viewed faces looking at variable or constant directions. In control conditions, only the eye region was presented or the heads were shown with eyes closed but oriented at variable or constant directions. The response to faces with variable vs. constant eye gaze direction was associated with AQ scores in a number of regions (posterior superior temporal sulcus, intraparietal sulcus, temporoparietal junction, amygdala, and MT/V5) of the brain network for social attention perception. No such effect was observed for heads with eyes closed or when only the eyes were presented. The results demonstrate a relationship between neurophysiology and autism spectrum traits in the typical (non-ASD) population and suggest that changes in the functioning of the neural circuit for social attention perception is associated with an extended autism spectrum in the typical population.

Fear and happiness in the eyes: an intra-cerebral event-related potential study from the human amygdala

We present the response pattern of intracranial event-related potentials (ERPs) recorded from depth-electrodes in the human amygdala (four patients) to faces or face parts encoding fearful, happy or neutral expressions. The amygdala showed increased amplitude ERPs (from 200 to 400 ms post-stimulus) in response to the eye region of the face compared to whole faces and to the mouth region. In particular, a strong emotional valence effect was observed, both at group and at single-subject level, with a preferential response to fearful eyes respect to every other stimulus category from 200 to 400 ms after stimulus presentation. A preferential response to smiling eyes compared to happy faces and smiling mouths was also observed at group level from 300 to 400 ms post-stimulus presentation. A complementary time-frequency analysis was performed showing that an increase in the theta frequency band (4-7 Hz) accounted for the main event-related band power (ERBP) change during the 200-500 ms post stimulus interval. The analysis of the ERBPs changes according to their emotional valence showed a strong increase in theta ERBP to fearful eyes, which was higher respect to any other facial stimulus. Moreover, theta ERBP increase to "smiling eyes" was larger respect with that evoked by smiling mouths and whole happy faces. Minimal post-stimulus ERBPs changes were evoked by neutral stimuli. These data are consistent with a special role of the amygdala in processing facial signals, both with negative and positive valence, conveyed by the eye region of the face.

Facial Attractiveness Biases the Perception of Eye Contact

Attractive faces are appealing: We like to look at them, and we like to be looked at by them. We presented attractive and unattractive smiling and neutral faces containing identical eye regions with different gaze directions. Participants judged whether or not a face looked directly at them. Overall, attractive faces increased participants' tendency to perceive eye contact, consistent with a self-referential positivity bias. However, attractiveness effects were modulated by facial expression and gender: For female faces, observers more likely perceived eye contact in attractive than unattractive faces, independent of expression. For male faces, attractiveness effects were limited to neutral expressions and were absent in smiling faces. A signal detection analysis elucidated a systematic pattern in which (a) smiling faces, but not highly attractive faces, reduced sensitivity in gaze perception overall, and (b) attractiveness had a more consistent impact on bias than sensitivity measures. We conclude that combined influences of attractiveness, expression, and gender determine the formation of an overall impression when deciding which individual's interest in oneself may be beneficial and should be reciprocated.

Comparing social attention in autism and amygdala lesions: effects of stimulus and task condition

The amygdala plays a critical role in orienting gaze and attention to socially salient stimuli. Previous work has demonstrated that SM a patient with rare bilateral amygdala lesions, fails to fixate and make use of information from the eyes in faces. Amygdala dysfunction has also been implicated as a contributing factor in autism spectrum disorders (ASD), consistent with some reports of reduced eye fixations in ASD. Yet, detailed comparisons between ASD and patients with amygdala lesions have not been undertaken. Here we carried out such a comparison, using eye tracking to complex social scenes that contained faces. We presented participants with three task conditions. In the Neutral task, participants had to determine what kind of room the scene took place in. In the Describe task, participants described the scene. In the Social Attention task, participants inferred where people in the scene were directing their attention. SM spent less time looking at the eyes and much more time looking at the mouths than control subjects, consistent with earlier findings. There was also a trend for the ASD group to spend less time on the eyes, although this depended on the particular image and task. Whereas controls and SM looked more at the eyes when the task required social attention, the ASD group did not. This pattern of impairments suggests that SM looks less at the eyes because of a failure in stimulus-driven attention to social features, whereas individuals with ASD look less at the eyes because they are generally insensitive to socially relevant information and fail to modulate attention as a function of task demands. We conclude that the source of the social attention impairment in ASD may arise upstream from the amygdala, rather than in the amygdala itself.

Neural bases of gaze and emotion processing in children with autism spectrum disorders

Abnormal eye contact is a core symptom of autism spectrum disorders (ASD), though little is understood of the neural bases of gaze processing in ASD. Competing hypotheses suggest that individuals with ASD avoid eye contact due to the anxiety-provoking nature of direct eye gaze or that eye-gaze cues hold less interest or significance to children with ASD. The current study examined the effects of gaze direction on neural processing of emotional faces in typically developing (TD) children and those with ASD. While undergoing functional magnetic resonance imaging (fMRI), 16 high-functioning children and adolescents with ASD and 16 TD controls viewed a series of faces depicting emotional expressions with either direct or averted gaze. Children in both groups showed significant activity in visual-processing regions for both direct and averted gaze trials. However, there was a significant group by gaze interaction such that only TD children showed reliably greater activity in ventrolateral prefrontal cortex for direct versus averted gaze. The ASD group showed no difference between direct and averted gaze in response to faces conveying negative emotions. These results highlight the key role of eye gaze in signaling communicative intent and suggest altered processing of the emotional significance of direct gaze in children with ASD.

Amygdala regulation of fear and emotionality in fragile X syndrome

Fear is a universal response to a threat to one's body or social status. Disruption in the detection and response of the brain's fear system is commonly observed in a variety of neurodevelopmental disorders, including fragile X syndrome (FXS), a brain disorder characterized by variable cognitive impairment and behavioral disturbances such as social avoidance and anxiety. The amygdala is highly involved in mediating fear processing, and increasing evidence supports the idea that inhibitory circuits play a key role in regulating the flow of information associated with fear conditioning in the amygdala. Here, we review the known and potential importance of amygdala fear circuits in FXS, and how developmental studies are critical to understand the formation and function of neuronal circuits that modulate amygdala-based behaviors.

Neurons responsive to face-view in the primate ventrolateral prefrontal cortex

Studies have indicated that temporal and prefrontal brain regions process face and vocal information. Face-selective and vocalization-responsive neurons have been demonstrated in the ventrolateral prefrontal cortex (VLPFC) and some prefrontal cells preferentially respond to combinations of face and corresponding vocalizations. These studies suggest VLPFC in nonhuman primates may play a role in communication that is similar to the role of inferior frontal regions in human language processing. If VLPFC is involved in communication, information about a speaker's face including identity, face-view, gaze, and emotional expression might be encoded by prefrontal neurons. In the following study, we examined the effect of face-view in ventrolateral prefrontal neurons by testing cells with auditory, visual, and a set of human and monkey faces rotated through 0°, 30°, 60°, 90°, and -30°. Prefrontal neurons responded selectively to either the identity of the face presented (human or monkey) or to the specific view of the face/head, or to both identity and face-view. Neurons which were affected by the identity of the face most often showed an increase in firing in the second part of the stimulus period. Neurons that were selective for face-view typically preferred forward face-view stimuli (0° and 30° rotation). The neurons which were selective for forward face-view were also auditory responsive compared to other neurons which responded to other views or were unselective which were not auditory responsive. Our analysis showed that the human forward face (0°) was decoded better and also contained the most information relative to other face-views. Our findings confirm a role for VLPFC in the processing and integration of face and vocalization information and add to the growing body of evidence that the primate ventrolateral prefrontal cortex plays a prominent role in social communication and is an important model in understanding the cellular mechanisms of communication.

Individual and relational contributions to parallel and joint attention in infancy

OBJECTIVE

This study examined the contributions of maternal bids for joint attention, relationship quality, and infant characteristics, to individual differences in infants' parallel and joint attention.

METHOD

Fifty-two 10-month-olds and their mothers were assessed in order to investigate concurrent predictors of infant parallel attention, responding to joint attention, and initiating joint attention.

RESULTS

Parallel attention was predicted by infants' higher mental development, low expression of negative emotionality, and maternal entertaining behaviors. Responding to joint attention was marginally predicted by total maternal bids for joint attention. Initiating joint attention was predicted by the infants' low expression of negative emotionality, as well as marginally predicted by fewer maternal teaching behaviors.

CONCLUSION

These results further the understanding of the factors influencing infant parallel as well as joint attention.

Look at me, I'll remember you: the perception of self-relevant social cues enhances memory and right hippocampal activity

Being looked at by a person enhances the subsequent memorability of her/his identity. Here, we tested the specificity of this effect and its underlying brain processes. We manipulated three social cues displayed by an agent: Gaze Direction (Direct/Averted), Emotional Expression (Anger/Neutral), and Pointing gesture (Presence/Absence). Our behavioral experiment showed that direct as compared with averted gaze perception enhanced subsequent retrieval of face identity. Similar effect of enhanced retrieval was found when pointing finger was absent as compared with present but not for anger as compared with neutral expression. The fMRI results revealed amygdala activity for both Anger and Direct gaze conditions, suggesting emotional arousal. Yet, the right hippocampus, known to play a role in self-relevant memory processes, was only revealed during direct gaze perception. Further investigations suggest that right hippocampal activity was maximal for the most self-relevant social event (i.e. actor expressing anger and pointing toward the participant with direct gaze). Altogether, our results suggest that the perception of self-relevant social cues such as direct gaze automatically prompts "self-relevant memory" processes.

Reprint of: Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention

SM is a patient with complete bilateral amygdala lesions who fails to fixate the eyes in faces and is consequently impaired in recognizing fear (Adolphs et al., 2005). Here we first replicated earlier findings in SM of reduced gaze to the eyes when seen in whole faces. Examination of the time course of fixations revealed that SM's reduced eye contact is particular pronounced in the first fixation to the face, and less abnormal in subsequent fixations. In a second set of experiments, we used a gaze-contingent presentation of faces with real time eye tracking, wherein only a small region of the face is made visible at the center of gaze. In essence, viewers explore the face by moving a small searchlight over the face with their gaze. Under such viewing conditions, SM's fixations to eye region of faces became entirely normalized. We suggest that this effect arises from the absence of bottom-up effects due to the facial features, allowing gaze location to be driven entirely by top-down control. Together with SM's failure to fixate the eyes in whole faces primarily at the very first saccade, the findings suggest that the saliency of the eyes normally attract our gaze in an amygdala-dependent manner. Impaired eye gaze is also a prominent feature of several psychiatric illnesses in which the amygdala has been hypothesized to be dysfunctional, and our findings and experimental manipulation may hold promise for interventions in such populations, including autism and fragile X syndrome.

Modulation of face processing by emotional expression and gaze direction during intracranial recordings in right fusiform cortex

We recorded intracranial local field potentials from structurally intact human visual cortex during several face processing tasks in a patient before brain surgery. Intracranial local field potentials were measured from subdural electrodes implanted in a right fusiform region with face-sensitive activity and a more medial location in posterior parahippocampal gyrus with house-selective activity. This electrode implantation allowed us to compare neural responses with different facial properties within two adjacent but functionally distinct cortical regions. Five experiments were conducted to determine the temporal dynamics of perceptual (Experiments 1 and 5), emotional (Experiments 2 and 3), and social (Experiment 4) effects on face-specific responses in the right fusiform. Our findings showed an early negative deflection (N200) that primarily reflected category-selective perceptual encoding of facial information, whereas higher order effects of face individuation, emotional expression, and gaze direction produced selective modulations in the same face-specific region during a later time period (from 200 to 1000 msec after onset). These results shed new lights on the time course of face recognition mechanisms in human visual cortex and reveal for the first time anatomically overlapping but temporally distinct influences of identity or emotional/social factors on face processing in right fusiform gyrus, which presumably reflect top-down feedback effects from distant brain areas.

Emotionally meaningful targets enhance orienting triggered by a fearful gazing face

Studies investigating the effect of emotional expression on spatial orienting to a gazed-at location have produced mixed results. The present study investigated the role of affective context in the integration of emotion processing and gaze-triggered orienting. In three experiments, a face gazed nonpredictively to the left or right, and then its expression became fearful or happy. Participants identified (Experiments 1 and 2) or detected (Experiment 3) a peripheral target presented 225 or 525 ms after the gaze cue onset. In Experiments 1 and 3 the targets were either threatening (a snarling dog) or nonthreatening (a smiling baby); in Experiment 2 the targets were neutral. With emotionally-valenced targets, the gaze-cuing effect was larger when the face was fearful compared to happy--but only with the longer cue-target interval. With neutral targets, there was no interaction between gaze and expression. Our results indicate that a meaningful context optimizes attentional integration of gaze and expression information.

"Stay tuned": inter-individual neural synchronization during mutual gaze and joint attention

Eye contact provides a communicative link between humans, prompting joint attention. As spontaneous brain activity might have an important role in the coordination of neuronal processing within the brain, their inter-subject synchronization might occur during eye contact. To test this, we conducted simultaneous functional MRI in pairs of adults. Eye contact was maintained at baseline while the subjects engaged in real-time gaze exchange in a joint attention task. Averted gaze activated the bilateral occipital pole extending to the right posterior superior temporal sulcus, the dorso-medial prefrontal cortex, and the bilateral inferior frontal gyrus. Following a partner's gaze toward an object activated the left intraparietal sulcus. After all the task-related effects were modeled out, inter-individual correlation analysis of residual time-courses was performed. Paired subjects showed more prominent correlations than non-paired subjects in the right inferior frontal gyrus, suggesting that this region is involved in sharing intention during eye contact that provides the context for joint attention.

Cortical processing of head- and eye-gaze cues guiding joint social attention

Previous fMRI experiments showed an involvement of the STS in the processing of eye-gaze direction in joint attention. Since head-gaze direction can also be used for the assessment of another person's attentional focus, we compared the mechanisms underlying the processing of head- and eye-gaze direction using a combined psychophysical and fMRI approach. Subjects actively followed the head- or eye-gaze direction of a person in a photograph towards one of seven possible targets by moving their eyes. We showed that the right posterior superior temporal sulcus (STS) as well as the right fusiform gyrus (FSG) were involved in both processing of head- as well as eye-gaze direction. Another finding was a bilateral deactivation of a distinct area in the middle STS (mSTS) as well as the left anterior STS (aSTS), that was stronger when subjects followed eye-gaze direction than when they followed head-gaze direction. We assume that this deactivation is based on an active suppression of information arising from the distracting other directional cue, i.e. head-gaze direction in the eye-gaze direction task and eye-gaze direction in the head-gaze direction task. These results further support the hypothesis that the human equivalent of the gaze sensitive area in monkeys lies in more anterior parts of the STS than previously thought.

Gaze cueing elicited by emotional faces is influenced by affective context

When we observe someone shift their gaze to a peripheral event or object, a corresponding shift in our own attention often follows. This social orienting response, joint attention, has been studied in the laboratory using the gaze cueing paradigm. Here, we investigate the combined influence of the emotional content displayed in two critical components of a joint attention episode: The facial expression of the cue face, and the affective nature of the to-be-localized target object. Hence, we presented participants with happy and disgusted faces as cueing stimuli, and neutral (Experiment 1), pleasant and unpleasant (Experiment 2) pictures as target stimuli. The findings demonstrate an effect of 'emotional context' confined to participants viewing pleasant pictures. Specifically, gaze cueing was boosted when the emotion of the gazing face (i.e., happy) matched that of the targets (pleasant). Demonstrating modulation by emotional context highlights the vital flexibility that a successful joint attention system requires in order to assist our navigation of the social world.

Amygdala involvement in self-blame regret

Regret-related brain activity is dependent on free choice, but it is unclear whether this activity is a function of more subtle differences in the degree of responsibility a decision-maker exerts over a regrettable outcome. In this experiment, we show that trial-by-trial subjective ratings of regret depend on a higher subjective sense of responsibility, as well as being dependent on objective responsibility. Using fMRI we show an enhanced amygdala response to regret-related outcomes when these outcomes are associated with high, as compared to low, responsibility. This enhanced response was maximal in participants who showed a greater level of enhancement in their subjective ratings of regret engendered by an objective increase in responsibility. Orbitofrontal and cingulate cortex showed opposite effects, with an enhanced response for regret-related outcomes when participants were not objectively responsible. The findings indicate that the way the brain processes regret-related outcomes depends on both objective and subjective aspects of responsibility, highlighting the critical importance of the amygdala.

Does it make a difference if I have an eye contact with you or with your picture? An ERP study

Several recent studies have begun to examine the neurocognitive mechanisms involved in perceiving and responding to eye contact, a salient social signal of interest and readiness for interaction. Laboratory experiments measuring observers' responses to pictorial instead of live eye gaze cues may, however, only vaguely approximate the real-life affective significance of gaze direction cues. To take this into account, we measured event-related brain potentials and subjective affective responses in healthy adults while viewing live faces with a neutral expression through an electronic shutter and faces as pictures on a computer screen. Direct gaze elicited greater face-sensitive N170 amplitudes and early posterior negativity potentials than averted gaze or closed eyes, but only in the live condition. The results show that early-stage processing of facial information is enhanced by another person's direct gaze when the person is faced live. We propose that seeing a live face with a direct gaze is processed more intensely than a face with averted gaze or closed eyes, as the direct gaze is capable of intensifying the feeling of being the target of the other's interest and intentions. These results may have implications for the use of pictorial stimuli in the social cognition studies.

Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention

SM is a patient with complete bilateral amygdala lesions who fails to fixate the eyes in faces and is consequently impaired in recognizing fear (Adolphs et al., 2005). Here we first replicated earlier findings in SM of reduced gaze to the eyes when seen in whole faces. Examination of the time course of fixations revealed that SM's reduced eye contact is particular pronounced in the first fixation to the face, and less abnormal in subsequent fixations. In a second set of experiments, we used a gaze-contingent presentation of faces with real time eye tracking, wherein only a small region of the face is made visible at the center of gaze. In essence, viewers explore the face by moving a small searchlight over the face with their gaze. Under such viewing conditions, SM's fixations to eye region of faces became entirely normalized. We suggest that this effect arises from the absence of bottom-up effects due to the facial features, allowing gaze location to be driven entirely by top-down control. Together with SM's failure to fixate the eyes in whole faces primarily at the very first saccade, the findings suggest that the saliency of the eyes normally attract our gaze in an amygdala-dependent manner. Impaired eye gaze is also a prominent feature of several psychiatric illnesses in which the amygdala has been hypothesized to be dysfunctional, and our findings and experimental manipulation may hold promise for interventions in such populations, including autism and fragile X syndrome.

"Artificial humans": Psychology and neuroscience perspectives on embodiment and nonverbal communication

"Artificial humans", so-called "Embodied Conversational Agents" and humanoid robots, are assumed to facilitate human-technology interaction referring to the unique human capacities of interpersonal communication and social information processing. While early research and development in artificial intelligence (AI) focused on processing and production of natural language, the "new AI" has also taken into account the emotional and relational aspects of communication with an emphasis both on understanding and production of nonverbal behavior. This shift in attention in computer science and engineering is reflected in recent developments in psychology and social cognitive neuroscience. This article addresses key challenges which emerge from the goal to equip machines with socio-emotional intelligence and to enable them to interpret subtle nonverbal cues and to respond to social affordances with naturally appearing behavior from both perspectives. In particular, we propose that the creation of credible artificial humans not only defines the ultimate test for our understanding of human communication and social cognition but also provides a unique research tool to improve our knowledge about the underlying psychological processes and neural mechanisms.

Neural correlates to seen gaze-direction and head orientation in the macaque monkey amygdala

Human neuropsychological studies suggest that the amygdala is implicated in social cognition, in which cognition of seen gaze-direction, especially the direct gaze, is essential, and that the perception of gaze direction is modulated by the head orientation of the facial stimuli. However, neural correlates to these issues remain unknown. In the present study, neuronal activity was recorded from the macaque monkey amygdala during performance of a sequential delayed non-matching-to-sample task based on gaze direction. The facial stimuli consisted of two head orientations (frontal; straight to the monkey, profile; 30 degrees rightwards from the front) with different gaze directions (directed toward and averted to the left or right of the monkey). Of the 1091 neurons recorded, 61 responded to more than one facial stimulus. Of these face-responsive neurons, 44 displayed responses selective to the facial stimuli (face neurons). Most amygdalar face neurons discriminated both gaze direction and head orientation, and exhibited a significant interaction between the two types about information. Furthermore, factor analysis on the response magnitudes of the face neurons to the facial stimuli revealed that two factors derived from these facial stimuli were correlated with two head orientations. The overall responses of the face neurons to direct gazes in the profile and frontal faces were significantly larger than that to averted gazes. The results suggest that information of both gaze and head direction is integrated in the amygdala, and that the amygdala is implicated in detection of direct gaze.