The human amygdala plays an important role in gaze monitoring. A PET study

Involvement of medial temporal structures in reflexive attentional shift by gaze

Recent studies have revealed that eye gaze triggers reflexive shift of the observer's visuospatial attention to its direction even if it does not predict any events in the environment. To determine whether medial temporal structures are involved in this reflexive gaze processing, an experiment of the gaze-cuing paradigm was carried out in seven epileptic patients who had undergone unilateral temporal lobectomy and nine age- and IQ-matched epileptic controls who had not undergone any surgical treatments. Gaze cues were presented for 200 ms to the unilateral visual field, after which subjects were required to localize targets as quickly as possible. They were also instructed that gaze directions were not predictive of the location of the targets. When the gaze cues stimulated the intact hemisphere in lobectomized patients or either hemisphere in controls, reaction times for correct responses were significantly shorter when gaze directions were toward the targets than away from the targets. This cuing effect was not manifested following stimulation of the lesioned hemisphere in lobectomized patients. These findings suggest that the medial temporal structures, including the amygdala, play a crucial role in the reflexive shift of attention triggered by another person's gaze direction in humans.

When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing

Direct gaze or eye contact is a strong social signal, which triggers distinct processes as compared to other gaze directions. Thus, direct gaze could be early dissociated from averted gaze during the processing of gaze direction. In order to examine the processing of gaze contact relative to averted gaze, we studied evoked potentials in human adults in response to the apparent motion of gaze. Deviated and frontal faces were presented with a fixed gaze direction, followed by an apparent movement of the eyes either toward the subject or away from him/her. The results showed that the perception of direct relative to averted gaze evoked a greater, later and longer lasting N170, suggesting that gaze contact recruited more resources than averted gaze in the early stage of gaze direction processing. Furthermore, direct and averted motion of gaze elicited distinct ERP components between 160 and 210 ms, initiated over centro-parietal electrodes. Source reconstruction revealed the involvement of the Theory-of-Mind network, including the regions of the superior temporal sulcus, the medial prefrontal and the orbitofrontal cortices, in this early dissociation. In addition, the perception of gaze contact relative to averted gaze yielded increased fronto-central P3a and parieto-occipital P3b. All the results were significant whatever the head orientation. Our findings show that gaze contact, as compared to other gaze directions, is an essential social cue which recruits early specific processes. The dissociation between direct and averted gaze processing occurs as soon as 160 ms, involving the social brain network.

It's all in the eyes: neural responses to socially significant gaze shifts

Gaze direction signals another's focus of social attention. Here, we recorded event-related potentials to a multiface display where a gaze aversion created three different social scenarios involving social attention, mutual gaze exchange, and gaze avoidance. N170 was unaffected by social scenario. P350 latency was the shortest in social attention and mutual gaze exchange, whereas P500 was thelargest for gaze avoidance. Our data suggest that neural activity after 300 ms poststimulus may index processes associated with extracting social meaning, whereas that earlier than 300 ms may index processing of gaze change independent of social context.

Face Cells: Separate Processing of Expression and Gaze in the Amygdala

A neuroimaging study in monkeys has shown that separate regions of the amygdala are responsive to facial expression and gaze/head orientation.

"Far from the heart far from the eye": evidence from the Capgras delusion

INTRODUCTION

Capgras syndrome is characterised by the belief that a significant other has been replaced by an identical-looking impostor. These patients have no difficulties with visual recognition but fail to show a skin conductance response (SCR) to the objects of the delusion. A case of Capgras delusion (YY), specifically characterised by the absence of brain lesions, constituted a good opportunity to test the relationship between SCR hyporesponsiveness and eye movement patterns to familiar and unfamiliar faces.

METHODS

Visual scan path and SCR were recorded for YY and 8 controls during the presentation of family members' photographs matched with unfamiliar faces of the same sex, age, and physical likeness. Eye movement patterns were explored by selecting three specific areas of interest (AOI) involving the eyes, the mouth, and the face regions.

RESULTS

In contrast with controls, YY showed a reduction in number and sum of fixation durations to the eyes (p<.01) and no differential SCRs (p>.05) to familiar vs. nonfamiliar faces. SCR and fixation duration to family members' eyes were significantly correlated (r=.77) in both YY and controls.

CONCLUSIONS

Eye region exploration seems to be related to the autonomic reactivity elicited by the affective valence of familiar faces.

Abnormal activation of the social brain during face perception in autism

ASD involves a fundamental impairment in processing social-communicative information from faces. Several recent studies have challenged earlier findings that individuals with autism spectrum disorder (ASD) have no activation of the fusiform gyrus (fusiform face area, FFA) when viewing faces. In this study, we examined activation to faces in the broader network of face-processing modules that comprise what is known as the social brain. Using 3T functional resonance imaging, we measured BOLD signal changes in 10 ASD subjects and 7 healthy controls passively viewing nonemotional faces. We replicated our original findings of significant activation of face identity-processing areas (FFA and inferior occipital gyrus, IOG) in ASD. However, in addition, we identified hypoactivation in a more widely distributed network of brain areas involved in face processing [including the right amygdala, inferior frontal cortex (IFC), superior temporal sulcus (STS), and face-related somatosensory and premotor cortex]. In ASD, we found functional correlations between a subgroup of areas in the social brain that belong to the mirror neuron system (IFC, STS) and other face-processing areas. The severity of the social symptoms measured by the Autism Diagnostic Observation Schedule was correlated with the right IFC cortical thickness and with functional activation in that area. When viewing faces, adults with ASD show atypical patterns of activation in regions forming the broader face-processing network and social brain, outside the core FFA and IOG regions. These patterns suggest that areas belonging to the mirror neuron system are involved in the face-processing disturbances in ASD.

Facial-expression and gaze-selective responses in the monkey amygdala

The social behavior of both human and nonhuman primates relies on specializations for the recognition of individuals, their facial expressions, and their direction of gaze. A broad network of cortical and subcortical structures has been implicated in face processing, yet it is unclear whether co-occurring dimensions of face stimuli, such as expression and direction of gaze, are processed jointly or independently by anatomically and functionally segregated neural structures. Awake macaques were presented with a set of monkey faces displaying aggressive, neutral, and appeasing expressions with head and eyes either averted or directed. BOLD responses to these faces as compared to Fourier-phase-scrambled images revealed widespread activation of the superior temporal sulcus and inferotemporal cortex and included activity in the amygdala. The different dimensions of the face stimuli elicited distinct activation patterns among the amygdaloid nuclei. The basolateral amygdala, including the lateral, basal, and accessory basal nuclei, produced a stronger response for threatening than appeasing expressions. The central nucleus and bed nucleus of the stria terminalis responded more to averted than directed-gaze faces. Independent behavioral measures confirmed that faces with averted gaze were more arousing, suggesting the activity in the central nucleus may be related to attention and arousal.

Perceiving fear in dynamic body expressions

Characteristic fear behaviour like putting the hands in front of the face and running for cover provides strong fear signals to observers who may not themselves be aware of any danger. Using event-related functional magnetic resonance imaging (fMRI) in humans, we investigated how such dynamic fear signals from the whole body are perceived. A factorial design allowed us to investigate brain activity induced by viewing bodies, bodily expressions of fear and the role of dynamic information in viewing them. Our critical findings are threefold. We find that viewing neutral and fearful body expressions enhances amygdala activity; moreover actions expressing fear activate the temporal pole and lateral orbital cortex more than neutral actions; and finally differences in activations between static and dynamic bodily expressions were larger for actions expressing fear in the STS and premotor cortex compared to neutral actions.

Skin conductance responses to masked conditioned stimuli: Phylogenetic/ontogenetic factors versus direction of threat?

Evolutionarily old threat stimuli are likely to require less conscious information processing than threat stimuli of a more recent date. To test this proposal two differential conditioning experiments, with biological threat stimuli (e.g. snakes) in half the groups and cultural threat stimuli (e.g. guns) in the other half, were conducted. The conditioned (CS+) and the control (CS-) stimuli were backward masked during the extinction phase to prevent conscious recognition. The differential skin conductance responding for both biological and cultural threat stimuli survived the masking procedure when the conditioned stimuli were directed towards the participants (Experiment 1), but for neither type of CS when stimuli were not directed towards the participants (Experiment 2). These findings are discussed in relation to the previous finding by Ohman and co-workers and in relation to imminence of threat.

Genetic influences on the neural basis of social cognition

The neural basis of social cognition has been the subject of intensive research in both human and non-human primates. Exciting, provocative and yet consistent findings are emerging. A major focus of interest is the role of efferent and afferent connectivity between the amygdala and the neocortical brain regions, now believed to be critical for the processing of social and emotional perceptions. One possible component is a subcortical neural pathway, which permits rapid and preconscious processing of potentially threatening stimuli, and it leads from the retina to the superior colliculus, to the pulvinar nucleus of the thalamus and then to the amygdala. This pathway is activated by direct eye contact, one of many classes of potential threat, and may be particularly responsive to the 'whites of the eyes'. In humans, autonomic arousal evoked by this stimulus is associated with the activity in specific cortical regions concerned with processing visual information from faces. The integrated functioning of these pathways is modulated by one or more X-linked genes, yet to be identified. The emotional responsiveness of the amygdala, and its associated circuits, to social threat is also influenced by functional polymorphisms in the promoter of the serotonin transporter gene. We still do not have a clear account of how specific allelic variation, in candidate genes, increases susceptibility to developmental disorders, such as autism, or psychiatric conditions, such as anxiety or depressive illness. However, the regulation of emotional responsiveness to social cues lies at the heart of the problem, and recent research indicates that we may be nearing a deeper and more comprehensive understanding.

Unilateral amygdala lesions hamper attentional orienting triggered by gaze direction

The newly discovered deficit in a bilateral amygdala-damaged case, of not being able to allocate attention to the critical feature of a face (Adolphs R, Gosselin F, Buchanan TW, Tranel D, Schyns P, Damasio AR. 2005. A mechanism for impaired fear recognition after amygdala damage. Nature. 433:68--72.), has opened a new window into the function of the amygdala. This case implies that the amygdala might be essential in detecting potentially relevant social stimuli, and directing attention accordingly. In this study, we have sought to test this implication by investigating the behavioral performance of 5 unilateral amygdala-damaged subjects on spatial cueing tasks. The tasks employed central gaze and arrow direction as cues to trigger attentional orienting in peripheral target detection. Although age-matched normal controls demonstrated a significant congruency effect such that targets presented congruently to cue direction elicited faster detection, amygdala subjects demonstrated no such congruency effect for gaze cues in the face of a significant congruency effect for arrow cues. The results suggest that the social valence of a stimulus is critical for amygdala involvement in visual processing. The results also support the implicated role of the amygdala in detecting and analyzing relevant social stimuli, and orienting attention accordingly.

Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule

Electrophysiological recording in the anterior superior temporal sulcus (STS) of monkeys has demonstrated separate cell populations responsive to direct and averted gaze. Human functional imaging has demonstrated posterior STS activation in gaze processing, particularly in coding the intentions conveyed by gaze, but to date has provided no evidence of dissociable coding of different gaze directions. Because the spatial resolution typical of group-based fMRI studies (approximately 6-10 mm) exceeds the size of cellular patches sensitive to different facial characteristics (1-4 mm in monkeys), a more sensitive technique may be required. We therefore used fMRI adaptation, which is considered to offer superior resolution, to investigate whether the human anterior STS contains representations of different gaze directions, as suggested by non-human primate research. Subjects viewed probe faces gazing left, directly ahead, or right. Adapting to leftward gaze produced a reduction in BOLD response to left relative to right (and direct) gaze probes in the anterior STS and inferior parietal cortex; rightward gaze adaptation produced a corresponding reduction to right gaze probes. Consistent with these findings, averted gaze in the adapted direction was misidentified as direct. Our study provides the first human evidence of dissociable neural systems for left and right gaze.

Seeing the face through the eyes: a developmental perspective on face expertise

Most people are experts in face recognition. We propose that the special status of this particular body part in telling individuals apart is the result of a developmental process that heavily biases human infants and children to attend towards the eyes of others. We review the evidence supporting this proposal, including neuroimaging results and studies in developmental disorders, like autism. We propose that the most likely explanation of infants' bias towards eyes is the fact that eye gaze serves important communicative functions in humans.

Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging

Brain imaging studies in humans have shown that face processing in several areas is modulated by the affective significance of faces, particularly with fearful expressions, but also with other social signals such gaze direction. Here we review haemodynamic and electrical neuroimaging results indicating that activity in the face-selective fusiform cortex may be enhanced by emotional (fearful) expressions, without explicit voluntary control, and presumably through direct feedback connections from the amygdala. fMRI studies show that these increased responses in fusiform cortex to fearful faces are abolished by amygdala damage in the ipsilateral hemisphere, despite preserved effects of voluntary attention on fusiform; whereas emotional increases can still arise despite deficits in attention or awareness following parietal damage, and appear relatively unaffected by pharmacological increases in cholinergic stimulation. Fear-related modulations of face processing driven by amygdala signals may implicate not only fusiform cortex, but also earlier visual areas in occipital cortex (e.g., V1) and other distant regions involved in social, cognitive, or somatic responses (e.g., superior temporal sulcus, cingulate, or parietal areas). In the temporal domain, evoked-potentials show a widespread time-course of emotional face perception, with some increases in the amplitude of responses recorded over both occipital and frontal regions for fearful relative to neutral faces (as well as in the amygdala and orbitofrontal cortex, when using intracranial recordings), but with different latencies post-stimulus onset. Early emotional responses may arise around 120ms, prior to a full visual categorization stage indexed by the face-selective N170 component, possibly reflecting rapid emotion processing based on crude visual cues in faces. Other electrical components arise at later latencies and involve more sustained activities, probably generated in associative or supramodal brain areas, and resulting in part from the modulatory signals received from amygdala. Altogether, these fMRI and ERP results demonstrate that emotion face perception is a complex process that cannot be related to a single neural event taking place in a single brain regions, but rather implicates an interactive network with distributed activity in time and space. Moreover, although traditional models in cognitive neuropsychology have often considered that facial expression and facial identity are processed along two separate pathways, evidence from fMRI and ERPs suggests instead that emotional processing can strongly affect brain systems responsible for face recognition and memory. The functional implications of these interactions remain to be fully explored, but might play an important role in the normal development of face processing skills and in some neuropsychiatric disorders.

Eyes always attract attention but gaze orienting is task-dependent: Evidence from eye movement monitoring

Eyes and gaze are central to social cognition but whether they attract attention differently depending on the task is unknown. Here, the shift in attention towards the eye region and gaze direction of a perceived face was studied in two tasks by monitoring eye movements. The same face stimuli in front- or 3/4-view, with direct or averted gaze, were used in both tasks. In the Gaze task, subjects performed an explicit gaze direction judgment (gaze straight or averted) while in the Head task they performed a head orientation judgment (front- or 3/4-view). Gaze processing was evident in both tasks as shown by longer RTs and lower accuracy when head and gaze directions did not match. In both tasks the eye region was the most attended area but the amount of viewing was task-dependent. Most importantly, approximately 90% of the initial saccades landed in the eye region in the Gaze task but only approximately 50% of them did so in the Head task. These saccades were made in the direction signaled by gaze in the Gaze task but in the direction signaled by head orientation in the Head task. Altogether, these task-modulated behaviors argue against a purely exogenous and automatic orienting-to-gaze mechanism. Based on patient work and neuroimaging studies of gaze processing, we suggest that this task-dependent orienting behavior is rather endogenous and subtended by cortical areas amongst which frontal regions play a central role. We discuss the implications of this finding for clinical populations.

On feeling in control: A biological theory for individual differences in control perception

This review aims to create a cross-disciplinary framework for understanding the perception of control. Although, the personality trait locus of control, the most common measure of control perception, has traditionally been regarded as a product of social learning, it may have biological antecedents as well. It is suggested that control perception follows from the brain's capacity for self regulation, leading to flexible and goal directed behaviours. To this account, a model is presented which spans several levels of analyses. On a behavioural level, control perception may be a corollary of emotion regulation, executive functions, and social cognition. On a neural level, these self-regulatory functions are substantiated in part by the dorsolateral and ventral prefrontal cortex and the anterior cingulate cortex. In addition, a possible role of subcortical-cortical dopamine pathways underlying control perception is discussed.

Right hemispheric dominance in gaze-triggered reflexive shift of attention in humans

Recent findings suggest a right hemispheric dominance in gaze-triggered shifts of attention. The aim of this study was to clarify the dominant hemisphere in the gaze processing that mediates attentional shift. A target localization task, with preceding non-predicative gaze cues presented to each visual field, was undertaken by 44 healthy subjects, measuring reaction time (RT). A face identification task was also given to determine hemispheric dominance in face processing for each subject. RT differences between valid and invalid cues were larger when presented in the left rather than the right visual field. This held true regardless of individual hemispheric dominance in face processing. Together, these results indicate right hemispheric dominance in gaze-triggered reflexive shifts of attention in normal healthy subjects.

Caught in the act: The impact of audience on the neural response to morally and socially inappropriate behavior

We examined the impact of witnesses on the neural response to moral and social transgressions using fMRI. In this study, participants (N=16) read short vignettes describing moral and social transgressions in the presence or absence of an audience. In line with our hypothesis, ventrolateral (BA 47) and dorsomedial (BA 8) frontal cortex showed increased BOLD responses to moral transgressions regardless of audience and to social transgressions in the presence of an audience relative to neutral situations. These findings are consistent with the suggestion that these regions of prefrontal cortex modify behavioral responses in response to social cues. Greater activity was observed in left temporal-parietal junction, medial prefrontal cortex and temporal poles to moral and to a lesser extent social transgressions relative to neutral stories, regardless of audience. These regions have been implicated in the representation of the mental states of others (Theory of Mind). The presence of an audience was associated with increased left amygdala activity across all conditions.

Autism and the development of face processing

Autism is a pervasive developmental condition, characterized by impairments in non-verbal communication, social relationships and stereotypical patterns of behavior. A large body of evidence suggests that several aspects of face processing are impaired in autism, including anomalies in gaze processing, memory for facial identity and recognition of facial expressions of emotion. In search of neural markers of anomalous face processing in autism, much interest has focused on a network of brain regions that are implicated in social cognition and face processing. In this review, we will focus on three such regions, namely the STS for its role in processing gaze and facial movements, the FFA in face detection and identification and the amygdala in processing facial expressions of emotion. Much evidence suggests that a better understanding of the normal development of these specialized regions is essential for discovering the neural bases of face processing anomalies in autism. Thus, we will also examine the available literature on the normal development of face processing. Key unknowns in this research area are the neuro-developmental processes, the role of experience and the interactions among components of the face processing system in shaping each of the specialized regions for processing faces during normal development and in autism.

Eye remember you two: gaze direction modulates face recognition in a developmental study

The effects of gaze direction on memory for faces were studied in children from three different age groups (6-7, 8-9, and 10-11 years old) using a computerized version of a task devised by Hood, Macrae, Cole-Davies and Dias (2003). Participants were presented with a sequence of faces in an encoding phase, and were then required to judge which faces they had previously encountered in a surprise two-alternative forced-choice recognition test. In one condition, stimulus eye gaze was either direct or deviated at the viewing phase, and eyes were closed at the test phase. In another condition, stimulus eyes were closed at the viewing phase, with either direct or deviated gaze at the test phase. Modulation of gaze direction affected hit rates, with participants demonstrating greater accuracy for direct gaze targets compared to deviated gaze targets in both conditions. Reaction times (RT) to correctly recognized stimuli were faster for direct gaze stimuli at the viewing phase, but not at the test phase. The age group of participants differentially affected these measures: there was a greater hit rate advantage for direct gaze stimuli in older children, although RTs were less affected by age. These findings suggest that while the facilitation of face recognition by gaze direction is robust across encoding and recognition stages, the efficiency of the process is affected by the stage at which gaze is modulated.

Fear recognition ability predicts differences in social cognitive and neural functioning in men

By testing the facial fear-recognition ability of 341 men in the general population, we show that 8.8% have deficits akin to those seen with acquired amygdala damage. Using psychological tests and functional magnetic resonance imaging (fMRI) we tested the hypothesis that poor fear recognition would predict deficits in other domains of social cognition and, in response to socially relevant stimuli, abnormal activation in brain regions that putatively reflect engagement of the "social brain." On tests of "theory of mind" ability, 25 "low fear scorers" (LFS) performed significantly worse than 25 age- and IQ-matched "normal (good) fear scorers" (NFS). In fMRI, we compared evoked activity during a gender judgement task to neutral faces portraying different head and eye gaze orientations in 12 NFS and 12 LFS subjects. Despite identical between-group accuracy in gender discrimination, LFS demonstrated significantly reduced activation in amygdala, fusiform gyrus, and anterior superior temporal cortices when viewing faces with direct versus averted gaze. In a functional connectivity analysis, NFS show enhanced connectivity between the amygdala and anterior temporal cortex in the context of direct gaze; this enhanced coupling is absent in LFS. We suggest that important individual differences in social cognitive skills are expressed within the healthy male population, which appear to have a basis in a compromised neural system that underpins social information processing.

Snakes as agents of evolutionary change in primate brains

Current hypotheses that use visually guided reaching and grasping to explain orbital convergence, visual specialization, and brain expansion in primates are open to question now that neurological evidence reveals no correlation between orbital convergence and the visual pathway in the brain that is associated with reaching and grasping. An alternative hypothesis proposed here posits that snakes were ultimately responsible for these defining primate characteristics. Snakes have a long, shared evolutionary existence with crown-group placental mammals and were likely to have been their first predators. Mammals are conservative in the structures of the brain that are involved in vigilance, fear, and learning and memory associated with fearful stimuli, e.g., predators. Some of these areas have expanded in primates and are more strongly connected to visual systems. However, primates vary in the extent of brain expansion. This variation is coincident with variation in evolutionary co-existence with the more recently evolved venomous snakes. Malagasy prosimians have never co-existed with venomous snakes, New World monkeys (platyrrhines) have had interrupted co-existence with venomous snakes, and Old World monkeys and apes (catarrhines) have had continuous co-existence with venomous snakes. The koniocellular visual pathway, arising from the retina and connecting to the lateral geniculate nucleus, the superior colliculus, and the pulvinar, has expanded along with the parvocellular pathway, a visual pathway that is involved with color and object recognition. I suggest that expansion of these pathways co-occurred, with the koniocellular pathway being crucially involved (among other tasks) in pre-attentional visual detection of fearful stimuli, including snakes, and the parvocellular pathway being involved (among other tasks) in protecting the brain from increasingly greater metabolic demands to evolve the neural capacity to detect such stimuli quickly. A diet that included fruits or nectar (though not to the exclusion of arthropods), which provided sugars as a neuroprotectant, may have been a required preadaptation for the expansion of such metabolically active brains. Taxonomic differences in evolutionary exposure to venomous snakes are associated with similar taxonomic differences in rates of evolution in cytochrome oxidase genes and in the metabolic activity of cytochrome oxidase proteins in at least some visual areas in the brains of primates. Raptors that specialize in eating snakes have larger eyes and greater binocularity than more generalized raptors, and provide non-mammalian models for snakes as a selective pressure on primate visual systems. These models, along with evidence from paleobiogeography, neuroscience, ecology, behavior, and immunology, suggest that the evolutionary arms race begun by constrictors early in mammalian evolution continued with venomous snakes. Whereas other mammals responded by evolving physiological resistance to snake venoms, anthropoids responded by enhancing their ability to detect snakes visually before the strike.

Autism, the superior temporal sulcus and social perception

The most common clinical sign of autism spectrum disorders (ASD) is social interaction impairment, which is associated with communication deficits and stereotyped behaviors. Based on recent brain-imaging results, our hypothesis is that abnormalities in the superior temporal sulcus (STS) are highly implicated in ASD. STS abnormalities are characterized by decreased gray matter concentration, rest hypoperfusion and abnormal activation during social tasks. STS anatomical and functional anomalies occurring during early brain development could constitute the first step in the cascade of neural dysfunction underlying ASD. We will focus this review on the STS, which has been highly implicated in social cognition. We will review recent data on the contribution of the STS to normal social cognition and review brain-imaging data implicating this area in ASD. This review is part of the INMED/TINS special issue "Nature and nurture in brain development and neurological disorders", based on presentations at the annual INMED/TINS symposium (http://inmednet.com/).

Functional neuroimaging of emotional learning and autonomic reactions

This article provides a selective overview of the functional neuroimaging literature with an emphasis on emotional activation processes. Emotions are fast and flexible response systems that provide basic tendencies for adaptive action. From the range of involved component functions, we first discuss selected automatic mechanisms that control basic adaptational changes. Second, we illustrate how neuroimaging work has contributed to the mapping of the network components associated with basic emotion families (fear, anger, disgust, happiness), and secondary dimensional concepts that organise the meaning space for subjective experience and verbal labels (emotional valence, activity/intensity, approach/withdrawal, etc.). Third, results and methodological difficulties are discussed in view of own neuroimaging experiments that investigated the component functions involved in emotional learning. The amygdala, prefrontal cortex, and striatum form a network of reciprocal connections that show topographically distinct patterns of activity as a correlate of up and down regulation processes during an emotional episode. Emotional modulations of other brain systems have attracted recent research interests. Emotional neuroimaging calls for more representative designs that highlight the modulatory influences of regulation strategies and socio-cultural factors responsible for inhibitory control and extinction. We conclude by emphasising the relevance of the temporal process dynamics of emotional activations that may provide improved prediction of individual differences in emotionality.

Pupillary contagion: central mechanisms engaged in sadness processing

Empathic responses underlie our ability to share emotions and sensations with others. We investigated whether observed pupil size modulates our perception of other's emotional expressions and examined the central mechanisms modulated by incidental perception of pupil size in emotional facial expressions. We show that diminishing pupil size enhances ratings of emotional intensity and valence for sad, but not happy, angry or neutral facial expressions. This effect was associated with modulation of neural activity within cortical and subcortical regions implicated in social cognition. In an identical context, we show that the observed pupil size was mirrored by the observers' own pupil size. This empathetic contagion engaged the brainstem pupillary control nuclei (Edinger-Westphal) in proportion to individual subject's sensitivity to this effect. These findings provide evidence that perception-action mechanisms extend to non-volitional operations of the autonomic nervous system.

The 'amygdala theory of autism' revisited: linking structure to behavior

The 'amygdala theory of autism' suggests a crucial role for the amygdala in the neurobiological basis of autism spectrum disorders. However, to date evidence is lacking of a direct relationship between amygdala measures and behavioral manifestations of autism in affected individuals. In 17 adult individuals with Asperger syndrome (AS) and 17 well-matched controls we therefore assessed associations between MRI-derived amygdala volume and behavioral variables of emotion recognition and social cognition, as well as with core AS symptomatology. Results revealed that individuals with AS exhibited impairments in emotion recognition and social cognition compared to controls and also showed atypical relationships between amygdala volumes and overall head size. We found positive associations between emotional and social understanding and amygdala volume in the control group, but not in the AS group. In the AS group however, amygdala size was negatively related to diagnostic parameters, with smaller amygdala volumes involving higher levels of restricted-repetitive behavior domains. Our data seem to indicate that in AS the amygdala is not crucially involved in social and emotional understanding. It may, however, be a mediator for narrow interest patterns and the imposition of routines and rituals. Our data, in conjunction with current literature, seem to argue for a modification of the 'amygdala theory of autism'.

Individuals with autism spectrum disorder show normal responses to a fear potential startle paradigm

The present study utilized a fear potentiated startle paradigm to examine amygdala function in individuals with autism spectrum disorder. Two competing hypotheses regarding amygdala dysfunction in autism have been proposed: (1) The amygdala is under-responsive, in which case it would be predicted that, in a fear potentiated startle experiment, individuals with autism would exhibit decreased fear conditioning and/or potentiation, and (2) The amygdala is over responsive, in which case an exaggerated potentiation of the startle response would be predicted. Fourteen adolescents and adults diagnosed with autism spectrum disorder and 14 age, gender, IQ, and anxiety level-matched typical adolescents and adults participated. Both participants with autism and typical participants potentiated the startle response following fear conditioning and no group differences in the latency or amplitude of the potentiated startle response were found. These results suggest that this aspect of amygdala function, namely fear conditioning and potentiation of the startle response, is intact in individuals with autism.

Impaired recognition of negative basic emotions in autism: a test of the amygdala theory

Autism and Asperger Syndrome are autism spectrum conditions (ASC) characterized by deficits in understanding others' minds, an aspect of which involves recognizing emotional expressions. This is thought to be related to atypical function and structure of the amygdala, and performance by people with ASC on emotion recognition tasks resembles that seen in people with acquired amygdala damage. In general, emotion recognition findings in ASC have been inconsistent, which may reflect low numbers of participants, low numbers of stimuli and trials, heterogeneity of symptom severity within ASC groups, and ceiling effects on some tasks. The present study tested 39 male adults with ASC and 39 typical male controls on a task of basic emotion recognition from photographs, in two separate experiments. On a control face discrimination task the group with ASC were not impaired. People with ASC were less accurate on the emotion recognition task compared to controls, but only for the negative basic emotions. This is discussed in the light of similar findings from people with damage to the amygdala.

THE GLOBAL DECEPTION RESEARCH TEAM. A WORLD OF LIES

This article reports two worldwide studies of stereotypes about liars. These studies are carried out in 75 different countries and 43 different languages. In Study 1, participants respond to the open-ended question "How can you tell when people are lying?" In Study 2, participants complete a questionnaire about lying. These two studies reveal a dominant pan-cultural stereotype: that liars avert gaze. The authors identify other common beliefs and offer a social control interpretation.

Atypical visual orienting to gaze- and arrow-cues in adults with high functioning autism

The present study investigates visual orienting to directional cues (arrow or eyes) in adults with high functioning autism (n = 19) and age matched controls (n = 19). A choice reaction time paradigm is used in which eye- or arrow direction correctly (congruent) or incorrectly (incongruent) cues target location. In typically developing participants, the visual orienting reflex is longer for eyes than for arrows. Right side cueing, but not left side cueing, induced a congruence effect for eyes, while this effect was evident for right as well as for left side arrow cues. In participants with autism the overall visual orienting reflex was not different between arrows and eyes and no laterality effect was found for eyes cueing. These findings suggest that, instead of a specific Eye Direction Detector persons with autism might have a general 'Symbol Direction Detector'.

Subcortical face processing

Recent functional imaging, neuropsychological and electrophysiological studies on adults have provided evidence for a fast, low-spatial-frequency, subcortical face-detection pathway that modulates the responses of certain cortical areas to faces and other social stimuli. These findings shed light on an older literature on the face-detection abilities of newborn infants, and the hypothesis that these newborn looking preferences are generated by a subcortical route. Converging lines of evidence indicate that the subcortical face route provides a developmental foundation for what later becomes the adult cortical 'social brain' network, and that disturbances to this pathway might contribute to certain developmental disorders.

Brain Networks Involved in Viewing Angry Hands or Faces

Most neuropsychological research on the perception of emotion concerns the perception of faces. Yet in everyday life, hand actions are also modulated by our affective state, revealing it, in turn, to the observer. We used functional magnetic resonance imaging (fMRI) to identify brain regions engaged during the observation of hand actions performed either in a neutral or an angry way. We also asked whether these are the same regions as those involved in perceiving expressive faces. During the passive observation of emotionally neutral hand movements, the fMRI signal increased significantly in dorsal and ventral premotor cortices, with the exact location of the 'peaks' distinct from those induced by face observation. Various areas in the extrastriate visual cortex were also engaged, overlapping with the face-related activity. When the observed hand action was performed with emotion, additional regions were recruited including the right dorsal premotor, the right medial prefrontal cortex, the left anterior insula and a region in the rostral part of the supramarginal gyrus bilaterally. These regions, except for the supramarginal gyrus, were also activated during the perception of angry faces. These results complement the wealth of studies on the perception of affect from faces and provide further insights into the processes involved in the perception of others underlying, perhaps, social constructs such as empathy.

Cortical regions involved in eye movements, shifts of attention, and gaze perception

Human vision is an active process that involves shifting attention across the visual scene, with or without moving the eyes. Such shifts of attention can be generated at will (endogenously) or be triggered automatically, i.e., generated in response to exogenous stimuli including socially relevant cues such as someone else's gaze. What are the common and distinct brain mechanisms involved in these processes? To address this question, we carried out a quantitative effect-location meta-analysis of 59 brain-imaging experiments whose results were published using standardized coordinates. For each condition of interest, namely voluntary and visually triggered eye movements, voluntary and visually triggered (covert) shifts of attention, and perception of someone else's gaze, we computed activation likelihood estimation (ALE) maps. Those maps represent at each voxel of the brain the probability of reporting a signal change related to the condition of interest. For eye movements, this analysis confirmed the spatial location of the frontal eye fields, supplementary eye fields, and parietal saccade-related regions. The map of covert shifts of attention demonstrated highest similarity with the map of saccadic eye movements. Gaze perception showed common activation likelihood with the other conditions in the right intraparietal sulcus and in the lateral precentral gyrus. It demonstrated more similarity with the reflexive than with the voluntary saccades and shifts of attention. We propose that a core network of frontoparietal and temporal brain regions is recruited when we shift the focus of our attention with or without eye movements in response to the appearance of a visual target, as well as when we see someone else shift his or her gaze.

A deficit in discriminating gaze direction in a case with right superior temporal gyrus lesion

The superior temporal sulcus (STS) region is well recognized as being heavily involved in detecting and discriminating gaze. Lesions confined to this area are quite rare in humans, and so the research has mainly depended on animal studies and functional neuroimaging in normal human subjects. We report one such rare case, a 54-year-old Japanese female with a possible congenital s anomaly who, after a cerebral hemorrhage, demonstrated a lesion almost completely confined to the entire right superior temporal gyrus (STG). In the subacute phase, the patient showed evidence of left hemispatial neglect, from which she gradually recovered. In the chronic phase, she showed a puzzling difficulty in obtaining eye-contact. We have conducted, in conjunction with conventional neuropsychological evaluations, experimental assessment of her ability in gaze cognition. Her performance on neuropsychological testing demonstrated no compromise in intellect, memory, or language skills, and a close-to-full recovery from neglect. On gaze cognition experiments, she was repeatedly shown to perceive left gaze as straight, and to a lesser degree, straight gaze as right. We suggest that the function of the STG in detecting gaze, together with the directional information it receives from earlier visual areas, may be associated, when damaged, with this deficit in detecting contra-directional gaze. We have demonstrated for the first time that a single circumscribed lesion to the STG results in both gaze processing deficit and concurrent aberrant gaze behavior of the victim herself, implicating a mechanism within the STG as an interface between gaze of others and gaze of self.

Rhesus monkeys attribute perceptions to others

Paramount among human cognitive abilities is the capacity to reason about what others think, want, and see--a capacity referred to as a theory of mind (ToM). Despite its importance in human cognition, the extent to which other primates share human ToM capacities has for decades remained a mystery. To date, primates [1, 2] have performed poorly in behavioral tasks that require ToM abilities, despite the fact that some macaques are known to encode social stimuli at the level of single neurons [3-5]. Here, we presented rhesus macaques with a more ecologically relevant ToM task in which subjects could "steal" a contested grape from one of two human competitors. In six experiments, monkeys selectively retrieved the grape from an experimenter who was incapable of seeing the grape rather than an experimenter who was visually aware. These results suggest that rhesus macaques possess an essential component of ToM: the ability to deduce what others perceive on the basis of where they are looking. These results converge with new findings illustrating the importance of competitive paradigms in apes [6]. Moreover, they raise the possibility that, in primates, cortical cells thought to encode where others are looking [7] may encode what those individuals see as well.

A systems approach to appraisal mechanisms in emotion

While artificial neural networks are regularly employed in modeling the perception of facial and vocal emotion expression as well as in automatic expression decoding by artificial agents, this approach is yet to be extended to the modeling of emotion elicitation and differentiation. In part, this may be due to the dominance of discrete and dimensional emotion models, which have not encouraged computational modeling. This situation has changed with the advent of appraisal theories of emotion and a number of attempts to develop rule-based models can be found in the literature. However, most of these models operate at a high level of conceptual abstraction and rarely include the underlying neural architecture. In this contribution, an appraisal-based emotion theory, the Component Process Model (CPM), is described that seems particularly suited to modeling with the help of artificial neural network approaches. This is due to its high degree of specificity in postulating underlying mechanisms including efferent physiological and behavioral manifestations as well as to the possibility of linking the theoretical assumptions to underlying neural architectures and dynamic processes. This paper provides a brief overview of the model, suggests constraints imposed by neural circuits, and provides examples on how the temporal unfolding of emotion can be conceptualized and experimentally tested. In addition, it is shown that the specific characteristics of emotion episodes can be profitably explored with the help of non-linear dynamic systems theory.

Neuroanatomical substrates of social cognition dysfunction in autism

In this review article, we summarize recent progress toward understanding the neural structures and circuitry underlying dysfunctional social cognition in autism. We review selected studies from the growing literature that has used the functional neuroimaging techniques of cognitive neuroscience to map out the neuroanatomical substrates of social cognition in autism. We also draw upon functional neuroimaging studies with neurologically normal individuals and individuals with brain lesions to highlight the insights these studies offer that may help elucidate the search for the neural basis of social cognition deficits in autism. We organize this review around key brain structures that have been implicated in the social cognition deficits in autism: (1) the amygdala, (2) the superior temporal sulcus region, and (3) the fusiform gyrus. We review some of what is known about the contribution of each structure to social cognition and then review autism studies that implicate that particular structure. We conclude with a discussion of several potential future directions in the cognitive neuroscience of social deficits in autism.

A mechanism for impaired fear recognition after amygdala damage

Ten years ago, we reported that SM, a patient with rare bilateral amygdala damage, showed an intriguing impairment in her ability to recognize fear from facial expressions. Since then, the importance of the amygdala in processing information about facial emotions has been borne out by a number of lesion and functional imaging studies. Yet the mechanism by which amygdala damage compromises fear recognition has not been identified. Returning to patient SM, we now show that her impairment stems from an inability to make normal use of information from the eye region of faces when judging emotions, a defect we trace to a lack of spontaneous fixations on the eyes during free viewing of faces. Although SM fails to look normally at the eye region in all facial expressions, her selective impairment in recognizing fear is explained by the fact that the eyes are the most important feature for identifying this emotion. Notably, SM's recognition of fearful faces became entirely normal when she was instructed explicitly to look at the eyes. This finding provides a mechanism to explain the amygdala's role in fear recognition, and points to new approaches for the possible rehabilitation of patients with defective emotion perception.

Cooperation of different neuronal systems during hand sign recognition

Hand signs with symbolic meaning can often be utilized more successfully than words to communicate an intention; however, the underlying brain mechanisms are undefined. The present study using magnetoencephalography (MEG) demonstrates that the primary visual, mirror neuron, social recognition and object recognition systems are involved in hand sign recognition. MEG detected well-orchestrated multiple brain regional electrical activity among these neuronal systems. During the assessment of the meaning of hand signs, the inferior parietal, superior temporal sulcus (STS) and inferior occipitotemporal regions were simultaneously activated. These three regions showed similar time courses in their electrical activity, suggesting that they work together during hand sign recognition by integrating information in the ventral and dorsal pathways through the STS. The results also demonstrated marked right hemispheric predominance, suggesting that hand expression is processed in a manner similar to that in which social signs, such as facial expressions, are processed.

When strangers pass: processing of mutual and averted social gaze in the superior temporal sulcus

Using functional magnetic resonance imaging (fMRI), we investigated brain activity evoked by mutual and averted gaze in a compelling and commonly experienced social encounter. Through virtual-reality goggles, subjects viewed a man who walked toward them and shifted his neutral gaze either toward (mutual gaze) or away (averted gaze) from them. Robust activity was evoked in the superior temporal sulcus (STS) and fusiform gyrus (FFG). For both conditions, STS activity was strongly right lateralized. Mutual gaze evoked greater activity in the STS than did averted gaze, whereas the FFG responded equivalently to mutual and averted gaze. Thus, we show that the STS is involved in processing social information conveyed by shifts in gaze within an overtly social context. This study extends understanding of the role of the STS in social cognition and social perception by demonstrating that it is highly sensitive to the context in which a human action occurs.

Neuroimaging in disorders of social and emotional functioning: what is the question?

Social and emotional processing uses neural systems involving structures ranging from the brain stem to the associational cortex. Neuroimaging research has attempted to identify abnormalities in components of these systems that would underlie the behavioral abnormalities seen in disorders of social and emotional processing, notably autism spectrum disorders, the focus of this review. However, the findings have been variable. The most replicated anatomic finding (a tendency toward large brains) is not modular, and metabolic imaging and functional imaging (although showing substantial atypicality in activation) are not consistent regarding specific anatomic sites. Moreover, autism spectrum disorder demonstrates substantial heterogeneity on multiple levels. Here evidence is marshaled from a review of neuroimaging data to support the claim that abnormalities in social and emotional processing on the autism spectrum are a consequence of systems disruptions in which the behaviors are a final common pathway and the focal findings can be variable, downstream of other pathogenetic mechanisms, and downstream of more pervasive abnormalities.

Differential impairments of facial affect recognition in schizophrenia subtypes and major depression

The goal of this study was to assess facial affect recognition abilities in subjects with various schizophrenia subtypes and subjects with major depression. A total of six disorganized, 21 paranoid and 18 residual subjects with schizophrenia (DSM-IV criteria) were compared with 21 subjects with major depression (DSM-IV criteria) and 30 matched healthy control subjects. Two experimental tasks requiring the sorting and rating of emotional facial expressions were applied. Disorganized and paranoid subjects showed strong impairments in the sorting of emotional facial expressions. Depressive subjects displayed only minor deficits, and residual subjects were unimpaired. Subjects with disorganized schizophrenia rated emotional facial expressions as more aroused, and depressive subjects rated them as less aroused, than the other study groups. Our study demonstrates strong deficits in facial affect recognition in subjects with schizophrenia and pronounced disorganized or psychotic symptoms. Deficits in facial affect recognition are specific to schizophrenia. They may be considered as a state marker of schizophrenia.

The impact of early and late damage to the human amygdala on ‘theory of mind’ reasoning

There is a burgeoning interest in the neural basis of the ability to attribute mental states to others; a capacity referred to as 'theory of mind' (ToM). We examined the effects of lesions of the amygdala which arise at different stages of development on this key aspect of social cognition. Tests of ToM, executive and general neuropsychological function were given to subjects with lesions of the amygdala arising congenitally or in early childhood ('early damage', n = 15), subjects who acquired damage to the amygdala in adulthood ('late damage' n = 11) and matched clinical (n = 14) and healthy comparison groups (n = 38). Subjects with early damage to the amygdala, particularly if the lesion was associated with childhood onset of seizures, were impaired relative to all other groups on more advanced tests of ToM reasoning, such as detecting tactless or ironic comments or interpreting non-literal utterances. These deficits held for subjects with either left or right early amygdala damage and encompassed the understanding of both the beliefs and emotional states of others. In contrast, subjects who acquired damage to the amygdala in adulthood (usually as part of an anterior temporal lobectomy) were not impaired in ToM reasoning relative to both clinical and healthy controls, supporting the position that the amygdala is not part of the neural circuitry mediating the 'on-line' performance of ToM reasoning. In line with theories which claim that ToM is an independent faculty of cognition, we found that the pattern of results held after co-varying for measures of executive function, memory and general intellectual functioning. We discuss the results in the light of recent theories which link early developmental insults to the amygdala with the ToM impairments which are thought to be a core neurocognitive deficit found in disorders such as autism. We conclude that the amygdala may play an important role in the neural systems supporting the normal development of ToM reasoning.

Face to face: visual scanpath evidence for abnormal processing of facial expressions in social phobia

Cognitive models of social phobia propose that cognitive biases and fears regarding negative evaluation by others result in preferential attention to interpersonal sources of threat. These fears may account for the hypervigilance and avoidance of eye contact commonly reported by clinicians. This study provides the first objective examination of threat-related processing in social phobia. It was predicted that hyperscanning (hypervigilance) and eye avoidance would be most apparent in social phobia for overt expressions of threat. An infrared corneal reflection technique was used to record visual scanpaths in response to angry, sad, and happy vs. neutral facial expressions. Twenty-two subjects with social phobia were compared with age- and sex-matched normal controls. As predicted, social phobia subjects displayed hyperscanning, (increased scanpath length) and avoidance (reduced foveal fixations) of the eyes, particularly evident for angry faces. The results could not be explained by either medication or co-morbid depression. These findings are consistent with theories emphasising the role of information processing biases in social phobia, and show promise in the application to treatment evaluation in this disorder.

The amygdala processes the emotional significance of facial expressions: an fMRI investigation using the interaction between expression and face direction

Neuroimaging studies have shown activity in the amygdala in response to facial expressions of emotion, but the specific role of the amygdala remains unknown. We hypothesized that the amygdala is involved in emotional but not basic sensory processing for facial expressions. To test this hypothesis, we manipulated the face directions of emotional expressions in the unilateral visual fields; this manipulation made it possible to alter the emotional significance of the facial expression for the observer without affecting the physical features of the expression. We presented angry/neutral expressions looking toward/away from the subject and depicted brain activity using fMRI. After the image acquisitions, the subject's experience of negative emotion when perceiving each stimulus was also investigated. The left amygdala showed the interaction between emotional expression and face direction, indicating higher activity for angry expressions looking toward the subjects than angry expressions looking away from them. The experienced emotion showed the corresponding interaction. Regression analysis showed a positive relation between the left amygdala activity and experienced emotion. These results suggest that the amygdala is involved in emotional but not visuoperceptual processing for emotional facial expressions, which specifically includes the decoding of emotional significance and elicitation of one's own emotions corresponding to that significance.

Neural correlates of mental state decoding in human adults: an event-related potential study

Successful negotiation of human social interactions rests on having a theory of mind - an understanding of how others' behaviors can be understood in terms of internal mental states, such as beliefs, desires, intentions, and emotions. A core theory-of-mind skill is the ability to decode others' mental states on the basis of observable information, such as facial expressions. Although several recent studies have focused on the neural correlates of reasoning about mental states, no research has addressed the question of what neural systems underlie mental state decoding. We used dense-array event-related potentials (ERP) to show that decoding mental states from pictures of eyes is associated with an N270-400 component over inferior frontal and anterior temporal regions of the right hemisphere. Source estimation procedures suggest that orbitofrontal and medial temporal regions may underlie this ERP effect. These findings suggest that different components of everyday theory-of-mind skills may rely on dissociable neural mechanisms.