Electrophysiological studies of human face perception. II: Response properties of face-specific potentials generated in occipitotemporal cortex

Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs

Using ERPs in a face recognition task, we investigated whether inversion and contrast reversal, which seem to disrupt different aspects of face configuration, differentially affected encoding and memory for faces. Upright, inverted, and negative (contrast-reversed) unknown faces were either immediately repeated (0-lag) or repeated after 1 intervening face (1-lag). The encoding condition (new) consisted of the first presentation of items correctly recognized in the two repeated conditions. 0-lag faces were recognized better and faster than 1-lag faces. Inverted and negative pictures elicited longer reaction times, lower hit rates, and higher false alarm rates than upright faces. ERP analyses revealed that negative and inverted faces affected both early (encoding) and late (recognition) stages of face processing. Early components (N170, VPP) were delayed and enhanced by both inversion and contrast reversal which also affected P1 and P2 components. Amplitudes were higher for inverted faces at frontal and parietal sites from 350 to 600 ms. Priming effects were seen at encoding stages, revealed by shorter latencies and smaller amplitudes of N170 for repeated stimuli, which did not differ depending on face type. Repeated faces yielded more positive amplitudes than new faces from 250 to 450 ms frontally and from 400 to 600 ms parietally. However, ERP differences revealed that the magnitude of this repetition effect was smaller for negative and inverted than upright faces at 0-lag but not at 1-lag condition. Thus, face encoding and recognition processes were affected by inversion and contrast-reversal differently.

Human neural systems for face recognition and social communication

Face perception is mediated by a distributed neural system in humans that consists of multiple, bilateral regions. The functional organization of this system embodies a distinction between the representation of invariant aspects of faces, which is the basis for recognizing individuals, and the representation of changeable aspects, such as eye gaze, expression, and lip movement, which underlies the perception of information that facilitates social communication. The system also has a hierarchical organization. A core system, consisting of occipitotemporal regions in extrastriate visual cortex, mediates the visual analysis of faces. An extended system consists of regions from neural systems for other cognitive functions that can act in concert with the core system to extract meaning from faces. Of regions in the extended system for face perception, the amygdala plays a central role in processing the social relevance of information gleaned from faces, particularly when that information may signal a potential threat.

Magnetoencephalographic evidence of early processing of direction of gaze in humans

Despite the crucial importance of direction of gaze in social interactions, it is only recently that there has been interest in human brain responses to gaze direction. Using full-head magnetoencephalography, we investigated the correlates of direction of gaze in full faces and in eyes-only stimuli, measuring the early face-responsive component between 145 and 225 ms. Faces with eyes forward or averted had larger responses than faces with eyes up or closed. For eyes-only stimuli the shortest latencies were seen for eyes averted and the smallest amplitudes were seen for eyes closed. The data were explained by two dipoles in inferior-temporal regions, which showed greater activation for upright faces than face parts or inverted faces, as well as some sensitivity to direction of gaze in this very early stage of processing.

A neural network reflecting decisions about human faces

Anatomic structures have been linked to the mnemonic component of working memory, but the neural network underlying associated decision processes remains elusive. Here we present an event-related functional magnetic resonance imaging study that measured activity during the decision period of a delayed face recognition task. A double dissociation of activity between anterior cingulate cortex (ACC), and a network including left fusiform face area (FFA) and left dorsolateral prefrontal cortex (DLPFC), reflected whether a probe face matched the remembered face at the time of decision. Greater activity in the left FFA and left DLPFC correlated with probe faces that matched the remembered face; in contrast, activity in ACC was greater when the probe face did not match the remembered face. These results support a model where frontal regions act in concert with stimulus-specific temporal structures to make recognition decisions about visual stimuli.

Face learning and memory: the twins test

The clinical utility of current face recognition tests has been questioned. To evaluate if a new paradigm may measure this type of memory more accurately, the authors created a novel test to examine face learning (previously uninvestigated) and short- and long-term retention. For this initial investigation of test sensitivity to hemisphere of dysfunction, patients with surgical resection from a temporal lobe and healthy subjects were tested. Recognition was evaluated on 3 trials: after a single exposure, after 4 exposures (for learning), and after a 24-hr delay interval. Patients with a right resection performed significantly worse than healthy controls and patients with left resection. There was no difference between patients with a left resection and controls. Classification of individual patients to side of resection based on test results showed higher sensitivity (82%) than published for other tests and maintained good specificity (79%).

Structural encoding of human and schematic faces: holistic and part-based processes

The range of specificity and the response properties of the extrastriate face area were investigated by comparing the N170 event-related potential (ERP) component elicited by photographs of natural faces, realistically painted portraits, sketches of faces, schematic faces, and by nonface meaningful and meaningless visual stimuli. Results showed that the N170 distinguished between faces and nonface stimuli when the concept of a face was clearly rendered by the visual stimulus, but it did not distinguish among different face types: Even a schematic face made from simple line fragments triggered the N170. However, in a second experiment, inversion seemed to have a different effect on natural faces in which face components were available and on the pure gestalt-based schematic faces: The N170 amplitude was enhanced when natural faces were presented upside down but reduced when schematic faces were inverted. Inversion delayed the N170 peak latency for both natural and schematic faces. Together, these results suggest that early face processing in the human brain is subserved by a multiple-component neural system in which both whole-face configurations and face parts are processed. The relative involvement of the two perceptual processes is probably determined by whether the physiognomic value of the stimuli depends upon holistic configuration, or whether the individual components can be associated with faces even when presented outside the face context.

Event-Related Potentials, Configural Encoding, and Feature-Based Encoding in Face Recognition

Abstract The effects of manipulating configural and feature information on the face recognition process were investigated by recording event-related potentials (ERPs) from five electrode sites (Fz, Cz, Pz, T5, T6), while 17 European subjects performed an own-race and other-race face recognition task. A series of upright faces were presented in a study phase, followed by a test phase where subjects indicated whether inverted and upright faces were studied or novel via a button press response. An inversion effect, illustrating the disruption of upright configural information, was reflected in accuracy measures and in greater lateral N2 amplitude to inverted faces, suggesting that structural encoding is harder for inverted faces. An own-race advantage was found, which may reflect the use of configural encoding for the more frequently experienced own-race faces, and feature-based encoding for the less familiar other-race faces, and was reflected in accuracy measures and ERP effects. The midline N2 was larger to configurally encoded faces (i. e., own-race and upright), possibly suggesting configural encoding involves more complex processing than feature-based encoding. An N400-like component was sensitive to feature manipulations, with greater amplitude to other-race than own-race faces and to inverted than upright faces. This effect was interpreted as reflecting increased activation of incompatible representations activated by a feature-based strategy used in processing of other-race and inverted faces. The late positive complex was sensitive to configural manipulation with larger amplitude to other-race than own-race faces, and was interpreted as reflecting the updating of an own-race norm used in face recognition, to incorporate other-race information.

Language in mental retardation: Individual and syndromic differences, and neurogenetic variation 1Based on a keynote presentation at the Third European Conference on Psychological Theory and Research in Mental Retardation, Geneva, September 1st, 2000

Predominantly non-etiological conceptions have dominated the field of mental retardation (MR) since the discovery of the genetic etiology of Down syndrome (DS) in the sixties. However, contemporary approaches are becoming more etiologically oriented. Important differences across MR syndromes of genetic origin are being documented, particularly in the cognition and language domains, differences not explicable in terms of psychometric level, motivation, or other dimensions. This paper highlights the major difficulties observed in the oral language development of individuals with genetic syndromes of mental retardation. The extent of inter- and within-syndrome variability are evaluated. Possible brain underpinnings of the behavioural differences are envisaged. Cases of atypically favourable language development in MR individuals are also summarized and explanatory variables discussed. It is suggested that differences in brain architectures, originating in neurological development and having genetic origins, may largely explain the syndromic as well as the individual within-syndrome variability documented. Lastly, the major implications of the above points for current debates about modularity and developmental connectionism are spelt out.

Sex-related differences in event-related potentials, face recognition, and facial affect processing in prepubertal children

Thirty-five prepubertal children, 17 boys and 18 girls, between the ages of 8 and 11 years, were studied to examine electrophysiological and cognitive sex differences during a face-recognition-memory (FRM) task and a facial-affect-identification task (FAIT). All participants were prepubertal, as determined by J. M. Tanner's (1962) staging and endocrine evaluation. Sex-dependent event-related potential (ERP) amplitude asymmetries were found during FRM. Boys displayed greater right versus left ERP amplitude to auditory tone probes during the task, whereas girls displayed the opposite pattern. In addition, positive correlations were obtained between ERP amplitude during FRM and FAIT accuracy scores for boys, but not for girls. Results suggest that girls and boys may use different neuronal systems in the processing of faces and facial affect. Findings are consistent with developmental theories regarding sex differences in visuospatial processing.

Eyes first! Eye processing develops before face processing in children

Faces and eyes are critical social stimuli which adults process with ease, but how this expertise develops is not yet understood. Neural changes associated with face and eye processing were investigated developmentally using ERPs (N170), in 128 subjects (4-15 year olds and adults). Stimuli included upright faces to assess configural processing, eyes and inverted faces to assess feature-based processing. N170 was present in the youngest children with similar patterns of face sensitivity seen in adults. Development of N170 to upright faces continued until adulthood, suggesting slow maturation of configural processing. In contrast, N170 was shorter latency and much larger to eyes than faces in children and was mature by 11 years, suggesting the early presence of an eye detector, with a rapid maturational course.

The lateral occipital complex and its role in object recognition

Here we review recent findings that reveal the functional properties of extra-striate regions in the human visual cortex that are involved in the representation and perception of objects. We characterize both the invariant and non-invariant properties of these regions and we discuss the correlation between activation of these regions and recognition. Overall, these results indicate that the lateral occipital complex plays an important role in human object recognition.

Event-related potentials for category-specific information during passive viewing of faces and objects

Normal subjects passively viewed an upright or inverted face or objects during recording of event-related potentials. Face inversion augmented N170 amplitude and latency in the temporal region, but only the latency in the parietal region. The same manipulation slowed down the onset of the P220 and caused disappearance of the N300, whereas none of these effects was seen after object inversion. Item-specific processing of objects was observed, namely disappearance of the N190 and the appearance of a P170 wave in the left posterior hemisphere to one object but not the other. These results are concordant with the hypothesis of category-specific processing during the recognition of faces and objects.

Seen Gaze-Direction Modulates Fusiform Activity and Its Coupling with Other Brain Areas during Face Processing

Gaze-contact is often a preliminary to social interaction and so constitutes a signal for the allocation of processing resources to the gazing face. We investigated how gaze direction influences face processing in an fMRI study, where seen gaze and head direction could independently be direct or deviated. Direct relative to averted gaze elicited stronger activation for faces in ventral occipitotemporal cortices around the fusiform gyrus, regardless of head orientation. Moreover, direct gaze led to greater correlation between activity in the fusiform and the amygdala, a region associated with emotional responses and stimulus saliency. By contrast, faces with averted gaze (again, regardless of head orientation) yielded increased correlation between activity in the fusiform and the intraparietal sulcus, a region associated with shifting attention to the periphery.

Direction of gaze effects on early face processing: eyes-only versus full faces

The N170 event-related potential (ERP) reflects an early stage of face processing. We wished to determine if it would also index the intuitively important information provided by direction of gaze. Two studies were run. In one, stimuli included full faces with the eyes looking forward, to the left or closed; in the other study the stimuli included eyes-only, looking forward, left or closed. Gaze direction had no effects on amplitude, but longer latencies were found for faces with eyes closed. With eyes-only stimuli, more marked effects on latencies and borderline effects on amplitudes were seen. We suggest that there can only be limited evidence of gaze-specific sensitivity in ERP studies in humans, without eye movement.

III. Electrophysiological studies of face processing in Williams syndrome

Williams Syndrome (WMS) is a genetically based disorder characterized by pronounced variability in performance across different domains of cognitive functioning. This study examined brain activity linked to face-processing abilities, which are typically spared in individuals with WMS. Subjects watched photographic pairs of upright or inverted faces and indicated if the second face matched or did not match the first face. Results from a previous study with normal adults showed dramatic differences in the timing and distribution of ERP effects linked to recognition of upright and inverted faces. In normal adults, upright faces elicited ERP differences to matched vs. mismatched faces at approximately 320 msec (N320) after the onset of the second stimulus. This "N320" effect was largest over anterior regions of the right hemisphere. In contrast, the mismatch/match effect for inverted faces consisted of a large positive component between 400 and 1000 msec (P500) that was largest over parietal regions and was symmetrical. In contrast to normal adults, WMS subjects showed an N320-mismatch effect for both upright and inverted faces. Additionally, the WMS subjects did not display the N320 right-hemisphere asymmetry observed in the normal adults. WMS subjects also displayed an abnormally small negativity at 100 msec (N100) and an abnormally large negativity at 200 msec (N200) to both upright and inverted faces. This ERP pattern was observed in all subjects with WMS but was not observed in the normal controls. These results may be linked to increased attention to faces in subjects with WMS and might be specific to the disorder. These results were consistent with our ERP studies of language processing in WMS, which suggested abnormal cerebral specialization for spared cognitive functions in individuals with WMS.

Effects of face inversion on the structural encoding and recognition of faces. Evidence from event-related brain potentials

It was investigated how face inversion affects face-specific components of event-related brain potentials (ERPs) which are assumed to reflect the structural encoding and the recognition of faces. ERPs were recorded to upright and inverted photographs of familiar faces, unfamiliar faces, and houses. In Part I, participants had to detect infrequently presented targets (hands), in Part II, attention was either directed towards or away from the pictorial stimuli. When compared with upright unfamiliar faces, upright familiar faces elicited an enhanced negativity between 300 ms and 450 ms ('N400f') and an enhanced positivity between 450 and 650 ms post-stimulus ('P600f'). It is suggested that these ERP modulations are generated by processes involved in the recognition of faces. Face inversion is known to disrupt face recognition processes. Accordingly, 'N400f' and 'P600f' were generally absent in response to inverted familiar and unfamiliar faces. The face-specific N170 component at lateral posterior electrodes was not affected by face familiarity, indicating that it reflects processing stages prior to face identification. N170 was delayed and enhanced for inverted relative to upright faces. While N170 enhancements were also observed for inverted relative to upright houses, the N170 latency shift caused by stimulus inversion was face-specific. Directing attention away from the faces towards a demanding primary visual task resulted in an N170 delay for inverted as well as for upright faces, suggesting that the time course of structural encoding of faces is affected by attentional factors. These results demonstrate that ERPs can be used as electrophysiological markers of specialised brain processes underlying the structural encoding and subsequent recognition of faces.

Hemispheric asymmetries for whole-based and part-based face processing in the human fusiform gyrus

Behavioral studies indicate a right hemisphere advantage for processing a face as a whole and a left hemisphere superiority for processing based on face features. The present PET study identifies the anatomical localization of these effects in well-defined regions of the middle fusiform gyri of both hemispheres. The right middle fusiform gyrus, previously described as a face-specific region, was found to be more activated when matching whole faces than face parts whereas this pattern of activity was reversed in the left homologous region. These lateralized differences appeared to be specific to faces since control objects processed either as wholes or parts did not induce any change of activity within these regions. This double dissociation between two modes of face processing brings new evidence regarding the lateralized localization of face individualization mechanisms in the human brain.

Social perception from visual cues: role of the STS region

Social perception refers to initial stages in the processing of information that culminates in the accurate analysis of the dispositions and intentions of other individuals. Single-cell recordings in monkeys, and neurophysiological and neuroimaging studies in humans, reveal that cerebral cortex in and near the superior temporal sulcus (STS) region is an important component of this perceptual system. In monkeys and humans, the STS region is activated by movements of the eyes, mouth, hands and body, suggesting that it is involved in analysis of biological motion. However, it is also activated by static images of the face and body, suggesting that it is sensitive to implied motion and more generally to stimuli that signal the actions of another individual. Subsequent analysis of socially relevant stimuli is carried out in the amygdala and orbitofrontal cortex, which supports a three-structure model proposed by Brothers. The homology of human and monkey areas involved in social perception, and the functional interrelationships between the STS region and the ventral face area, are unresolved issues.

The distributed human neural system for face perception

Face perception, perhaps the most highly developed visual skill in humans, is mediated by a distributed neural system in humans that is comprised of multiple, bilateral regions. We propose a model for the organization of this system that emphasizes a distinction between the representation of invariant and changeable aspects of faces. The representation of invariant aspects of faces underlies the recognition of individuals, whereas the representation of changeable aspects of faces, such as eye gaze, expression, and lip movement, underlies the perception of information that facilitates social communication. The model is also hierarchical insofar as it is divided into a core system and an extended system. The core system is comprised of occipitotemporal regions in extrastriate visual cortex that mediate the visual analysis of faces. In the core system, the representation of invariant aspects is mediated more by the face-responsive region in the fusiform gyrus, whereas the representation of changeable aspects is mediated more by the face-responsive region in the superior temporal sulcus. The extended system is comprised of regions from neural systems for other cognitive functions that can be recruited to act in concert with the regions in the core system to extract meaning from faces.

Distinct representations of eye gaze and identity in the distributed human neural system for face perception

Face perception requires representation of invariant aspects that underlie identity recognition as well as representation of changeable aspects, such as eye gaze and expression, that facilitate social communication. Using functional magnetic resonance imaging (fMRI), we investigated the perception of face identity and eye gaze in the human brain. Perception of face identity was mediated more by regions in the inferior occipital and fusiform gyri, and perception of eye gaze was mediated more by regions in the superior temporal sulci. Eye-gaze perception also seemed to recruit the spatial cognition system in the intraparietal sulcus to encode the direction of another's gaze and to focus attention in that direction.