The nature of synthetic face adaptation

fMRI evidence that hyper-caricatured faces activate object-selective cortex

Many brain imaging studies have looked at the cortical responses to object categories and faces. A popular way to manipulate face stimuli is by using a "face space," a high dimensional representation of individual face images, with the average face located at the origin. However, how the brain responds to faces that deviate substantially from average has not been much explored. Increasing the distance from the average (leading to increased caricaturing) could increase neural responses in face-selective regions, an idea supported by results from non-human primates. Here, we used a face space based on principal component analysis (PCA) to generate faces ranging from average to heavily caricatured. Using functional magnetic resonance imaging (fMRI), we first independently defined face-, object- and scene-selective areas with a localiser scan and then measured responses to parametrically caricatured faces. We also included conditions in which the images of faces were inverted. Interestingly in the right fusiform face area (FFA), we found that the patterns of fMRI response were more consistent as caricaturing increased. However, we found no consistent effect of either caricature level or facial inversion on the average fMRI response in the FFA or face-selective regions more broadly. In contrast, object-selective regions showed an increase in both the consistency of response pattern and the average fMRI response with increasing caricature level. This shows that caricatured faces recruit processing from regions typically defined as object-selective, possibly through enhancing low-level properties that are characteristic of objects.

Face Adaptation and Face Priming as Tools for Getting Insights Into the Quality of Face Space

During the recognition of faces, the incoming perceptual information is matched against mental representations of familiar faces stored in memory. Face space models describe an abstract concept of face representations and their mental organization, in which facial representations are located on various characteristic dimensions, depending on their specific facial characteristics. However, these models are defined just as incompletely as the general understanding of face recognition. We took two phenomena from face processing to better understand face recognition, and so the nature of face space: face adaptation and face priming. The face literature has mainly focused on face adaptation, largely neglecting face priming when trying to integrate outcomes regarding face recognition into the face space framework. Consequently, the present paper aims to review both phenomena and their contributions to face recognition, representation, and face space.

Adaptation to dynamic faces produces face identity aftereffects

Face aftereffects are well established for static stimuli and have been used extensively as a tool for understanding the neural mechanisms underlying face recognition. It has also been argued that adaptive coding, as demonstrated by face aftereffects, plays a functional role in face recognition by calibrating our face norms to reflect current experience. If aftereffects tap high-level perceptual mechanisms that are critically involved in everyday face recognition then they should also occur for moving faces. Here we asked whether face identity aftereffects can be induced using dynamic adaptors. The face identity aftereffect occurs when adaptation to a particular identity (e.g., Dan) biases subsequent perception toward the opposite identity (e.g., antiDan). We adapted participants to video of real faces that displayed either rigid, non-rigid, or no motion and tested for aftereffects in static antifaces. Adapt and test stimuli differed in size, to minimize low-level adaptation. Aftereffects were found in all conditions, suggesting that face identity aftereffects tap high-level mechanisms important for face recognition. Aftereffects were not significantly reduced in the motion conditions relative to the static condition. Overall, our results support the view that face aftereffects reflect adaptation of high-level mechanisms important for real-world face recognition in which faces are moving.

Bidirectional Gender Face Aftereffects: Evidence Against Normative Facial Coding

Facial appearance can be altered, not just by restyling but also by sensory processes. Exposure to a female face can, for instance, make subsequent faces look more masculine than they would otherwise. Two explanations exist. According to one, exposure to a female face renormalizes face perception, making that female and all other faces look more masculine as a consequence—a unidirectional effect. According to that explanation, exposure to a male face would have the opposite unidirectional effect. Another suggestion is that face gender is subject to contrastive aftereffects. These should make some faces look more masculine than the adaptor and other faces more feminine—a bidirectional effect. Here, we show that face gender aftereffects are bidirectional, as predicted by the latter hypothesis. Images of real faces rated as more and less masculine than adaptors at baseline tended to look even more and less masculine than adaptors post adaptation. This suggests that, rather than mental representations of all faces being recalibrated to better reflect the prevailing statistics of the environment, mental operations exaggerate differences between successive faces, and this can impact facial gender perception.

Face-Space: A Unifying Concept in Face Recognition Research

The concept of a multidimensional psychological space, in which faces can be represented according to their perceived properties, is fundamental to the modern theorist in face processing. Yet the idea was not clearly expressed until 1991. The background that led to the development of face-space is explained, and its continuing influence on theories of face processing is discussed. Research that has explored the properties of the face-space and sought to understand caricature, including facial adaptation paradigms, is reviewed. Face-space as a theoretical framework for understanding the effect of ethnicity and the development of face recognition is evaluated. Finally, two applications of face-space in the forensic setting are discussed. From initially being presented as a model to explain distinctiveness, inversion, and the effect of ethnicity, face-space has become a central pillar in many aspects of face processing. It is currently being developed to help us understand adaptation effects with faces. While being in principle a simple concept, face-space has shaped, and continues to shape, our understanding of face perception.

Long-Term Exposure to American and European Movies and Television Series Facilitates Caucasian Face Perception in Young Chinese Watchers

Most young Chinese people now learn about Caucasian individuals via media, especially American and European movies and television series (AEMT). The current study aimed to explore whether long-term exposure to AEMT facilitates Caucasian face perception in young Chinese watchers. Before the experiment, we created Chinese, Caucasian, and generic average faces (generic average face was created from both Chinese and Caucasian faces) and tested participants’ ability to identify them. In the experiment, we asked AEMT watchers and Chinese movie and television series (CMT) watchers to complete a facial norm detection task. This task was developed recently to detect norms used in facial perception. The results indicated that AEMT watchers coded Caucasian faces relative to a Caucasian face norm better than they did to a generic face norm, whereas no such difference was found among CMT watchers. All watchers coded Chinese faces by referencing a Chinese norm better than they did relative to a generic norm. The results suggested that long-term exposure to AEMT has the same effect as daily other-race face contact in shaping facial perception.

How previous experience shapes perception in different sensory modalities

What has transpired immediately before has a strong influence on how sensory stimuli are processed and perceived. In particular, temporal context can have contrastive effects, repelling perception away from the interpretation of the context stimulus, and attractive effects (TCEs), whereby perception repeats upon successive presentations of the same stimulus. For decades, scientists have documented contrastive and attractive temporal context effects mostly with simple visual stimuli. But both types of effects also occur in other modalities, e.g., audition and touch, and for stimuli of varying complexity, raising the possibility that context effects reflect general computational principles of sensory systems. Neuroimaging shows that contrastive and attractive context effects arise from neural processes in different areas of the cerebral cortex, suggesting two separate operations with distinct functional roles. Bayesian models can provide a functional account of both context effects, whereby prior experience adjusts sensory systems to optimize perception of future stimuli.

The PCA learning effect: An emerging correlate of face memory during childhood

Human adults implicitly learn the prototype and the principal components of the variability distinguishing faces (Gao & Wilson, 2014). Here we measured the implicit learning effect in adults and 9-year-olds, and with a modified child-friendly procedure, in 7-year-olds. All age groups showed the implicit learning effect by falsely recognizing the average (the prototype effect) and the principal component faces as having been seen (the PCA learning effect). The PCA learning effect, but not the prototype effect increased between 9years of age and adulthood and at both ages was the better predictor of memory for the actually studied faces. In contrast, for the 7-year-olds, the better predictor of face memory was the prototype effect. The pattern suggests that there may be a developmental change between ages 7 and 9 in the mechanism underlying memory for faces. We provide the first evidence that children as young as age 7 can extract the most important dimensions of variation represented by principal components among individual faces, a key ability that grows stronger with age and comes to underlie memory for faces.

Category-specific face prototypes are emerging, but not yet mature, in 5-year-old children

Adults' expertise in face recognition has been attributed to norm-based coding. Moreover, adults possess separable norms for a variety of face categories (e.g., race, sex, age) that appear to enhance recognition by reducing redundancy in the information shared by faces and ensuring that only relevant dimensions are used to encode faces from a given category. Although 5-year-old children process own-race faces using norm-based coding, little is known about the organization and refinement of their face space. The current study investigated whether 5-year-olds rely on category-specific norms and whether experience facilitates the development of dissociable face prototypes. In Experiment 1, we examined whether Chinese 5-year-olds show race-contingent opposing aftereffects and the extent to which aftereffects transfer across face race among Caucasian and Chinese 5-year-olds. Both participant races showed partial transfer of aftereffects across face race; however, there was no evidence for race-contingent opposing aftereffects. To examine whether experience facilitates the development of category-specific prototypes, we investigated whether race-contingent aftereffects are present among Caucasian 5-year-olds with abundant exposure to Chinese faces (Experiment 2) and then tested separate groups of 5-year-olds with two other categories with which they have considerable experience: sex (male/female faces) and age (adult/child faces) (Experiment 3). Across all three categories, 5-year-olds showed no category-contingent opposing aftereffects. These results demonstrate that 5 years of age is a stage characterized by minimal separation in the norms and associated coding dimensions used for faces from different categories and suggest that refinement of the mechanisms that underlie expert face processing occurs throughout childhood.

Face identity aftereffects increase monotonically with adaptor extremity over, but not beyond, the range of natural faces

Face identity aftereffects have been used to test theories of the neural coding underlying expert face recognition. Previous studies reported larger aftereffects for adaptors that are morphed further from the average face than for adaptors closer to the average, which appeared to support opponent coding along face-identity dimensions. However, only two levels were tested and it is not clear where they were located relative to the range of naturally occurring faces. This range is of interest given the functional need of the visual system both to produce good discrimination of real everyday faces and to process novel kinds of faces that we may encounter. Here, Experiment 1 establishes the boundary of faces judged as being able to occur in everyday life. Experiment 2 then shows that aftereffects increase with adaptor extremity up to this natural-range boundary, drop significantly immediately outside the boundary, and then remain stable with no drop towards zero even for highly distorted adaptors far beyond the boundary. Computational modelling shows that this unexpected pattern cannot be explained either by a simple opponent or by a classic multichannel model. However, its qualitative features can be captured either by a combination of opponent and multichannel coding (raising the possibility that not all identity-related face dimensions are opponent coded), or by a 3-pool model containing two S-shaped-response channels and a central bell-shaped channel around the average face (raising the possibility of unexpected similarities with coding of eye and head direction).

Seeing a haptically explored face: visual facial-expression aftereffect from haptic adaptation to a face

Current views on face perception assume that the visual system receives only visual facial signals. However, I show that the visual perception of faces is systematically biased by adaptation to a haptically explored face. Recently, face aftereffects (FAEs; the altered perception of faces after adaptation to a face) have been demonstrated not only in visual perception but also in haptic perception; therefore, I combined the two FAEs to examine whether the visual system receives face-related signals from the haptic modality. I found that adaptation to a haptically explored facial expression on a face mask produced a visual FAE for facial expression. This cross-modal FAE was not due to explicitly imaging a face, response bias, or adaptation to local features. Furthermore, FAEs transferred from vision to haptics. These results indicate that visual face processing depends on substrates adapted by haptic faces, which suggests that face processing relies on shared representation underlying cross-modal interactions.

Face adaptation effects: reviewing the impact of adapting information, time, and transfer

The ability to adapt is essential to live and survive in an ever-changing environment such as the human ecosystem. Here we review the literature on adaptation effects of face stimuli to give an overview of existing findings in this area, highlight gaps in its research literature, initiate new directions in face adaptation research, and help to design future adaptation studies. Furthermore, this review should lead to better understanding of the processing characteristics as well as the mental representations of face-relevant information. The review systematizes studies at a behavioral level in respect of a framework which includes three dimensions representing the major characteristics of studies in this field of research. These dimensions comprise (1) the specificity of adapting face information, e.g., identity, gender, or age aspects of the material to be adapted to (2) aspects of timing (e.g., the sustainability of adaptation effects) and (3) transfer relations between face images presented during adaptation and adaptation tests (e.g., images of the same or different identities). The review concludes with options for how to combine findings across different dimensions to demonstrate the relevance of our framework for future studies.

Cross-category adaptation: objects produce gender adaptation in the perception of faces

Adaptation aftereffects have been found for low-level visual features such as colour, motion and shape perception, as well as higher-level features such as gender, race and identity in domains such as faces and biological motion. It is not yet clear if adaptation effects in humans extend beyond this set of higher order features. The aim of this study was to investigate whether objects highly associated with one gender, e.g. high heels for females or electric shavers for males can modulate gender perception of a face. In two separate experiments, we adapted subjects to a series of objects highly associated with one gender and subsequently asked participants to judge the gender of an ambiguous face. Results showed that participants are more likely to perceive an ambiguous face as male after being exposed to objects highly associated to females and vice versa. A gender adaptation aftereffect was obtained despite the adaptor and test stimuli being from different global categories (objects and faces respectively). These findings show that our perception of gender from faces is highly affected by our environment and recent experience. This suggests two possible mechanisms: (a) that perception of the gender associated with an object shares at least some brain areas with those responsible for gender perception of faces and (b) adaptation to gender, which is a high-level concept, can modulate brain areas that are involved in facial gender perception through top-down processes.

Electrophysiological potentials reveal cortical mechanisms for mental imagery, mental simulation, and grounded (embodied) cognition

Grounded cognition theory proposes that cognition, including meaning, is grounded in sensorimotor processing. The mechanism for grounding cognition is mental simulation, which is a type of mental imagery that re-enacts modal processing. To reveal top-down, cortical mechanisms for mental simulation of shape, event-related potentials were recorded to face and object pictures preceded by mental imagery. Mental imagery of the identical face or object picture (congruous condition) facilitated not only categorical perception (VPP/N170) but also later visual knowledge [N3(00) complex] and linguistic knowledge (N400) for faces more than objects, and strategic semantic analysis (late positive complex) between 200 and 700 ms. The later effects resembled semantic congruity effects with pictures. Mental imagery also facilitated category decisions, as a P3 peaked earlier for congruous than incongruous (other category) pictures, resembling the case when identical pictures repeat immediately. Thus mental imagery mimics semantic congruity and immediate repetition priming processes with pictures. Perception control results showed the opposite for faces and were in the same direction for objects: Perceptual repetition adapts (and so impairs) processing of perceived faces from categorical perception onward, but primes processing of objects during categorical perception, visual knowledge processes, and strategic semantic analysis. For both imagery and perception, differences between faces and objects support domain-specificity and indicate that cognition is grounded in modal processing. Altogether, this direct neural evidence reveals that top-down processes of mental imagery sustain an imagistic representation that mimics perception well enough to prime subsequent perception and cognition. Findings also suggest that automatic mental simulation of the visual shape of faces and objects operates between 200 and 400 ms, and strategic mental simulation operates between 400 and 700 ms.

View-adaptation reveals coding of face pose along image, not object, axes

High-level adaptation effects reveal important features of the neural coding of objects and faces. View-adaptation in particular is a highly useful means of characterizing how depth rotation of the face is represented and therefore, how view-invariant recognition of the face may be achieved. In the present study, we used view adaptation to determine the extent to which depth rotations of a face are represented in an image-based or object-based manner. Specifically, we dissociated object-based axes from image-based axes via a 90° planar rotation of the adapting face and observed that participants' responses pre- and post-adaptation are most consistent with an image-based representation of depth rotations of the face. We discuss our data in the context of previous results describing the impact of planar rotation on related aspects of face perception.

FIAEs in Famous Faces are Mediated by Type of Processing

An important question regarding face aftereffects is whether it is based on face-specific or lower-level mechanisms. One method for addressing this is to explore how adaptation in upright or inverted, photographic positive or negative faces transfers to test stimuli that are either upright or inverted and normal or negated. A series of studies are reported in which this is tested using a typical face identity aftereffect paradigm in unfamiliar and famous faces. Results showed that aftereffects were strongest when the adaptor matched the test stimuli. In addition, aftereffects did not transfer from upright adaptors to inverted test images, but did transfer from inverted adaptors to upright test images in famous faces. However, in unfamiliar faces, a different pattern was observed. The results are interpreted in terms of how identity adaptation interacts with low-level adaptation and highlight differences in the representation of famous and unfamiliar faces.

Neural correlates of after-effects caused by adaptation to multiple face displays

Adaptation to a given face leads to face-related, specific after-effects. Recently, this topic has attracted a lot of attention because it clearly shows that adaptation occurs even at the higher stages of visual cortical processing. However, during our every-day life, faces do not appear in isolation, rather they are usually surrounded by other stimuli. Here, we used psychophysical and fMRI adaptation methods to test whether humans adapt to the gender properties of a composite multiple face stimulus as well. As adaptors we used stimuli composed of eight different individual faces, positioned peripherally in a ring around a fixation mark. We found that the gender discrimination of a subsequent centrally presented target face is significantly biased as a result of long-term adaptation to either male or female multiple face stimuli. Similar to our previous results with single-face adaptors (Kovács et al. in Neuroimage 43(1):156-164, 2008), a concurrent functional magnetic resonance imaging adaptation experiment revealed the strongest blood oxygen level-dependent signal adaptation bilaterally in the fusiform face area. Our results suggest that humans extract the statistical features of the multiple face stimulus and this process occurs at the level of occipito-temporal face processing.

Not all face aftereffects are equal

After prolonged exposure to a female face, faces that had previously seemed androgynous are more likely to be judged as male. Similarly, after prolonged exposure to a face with expanded features, faces that had previously seemed normal are more likely to be judged as having contracted features. These facial aftereffects have both been attributed to the impact of adaptation upon a norm-based opponent code, akin to low-level analyses of colour. While a good deal of evidence is consistent with this, some recent data is contradictory, motivating a more rigorous test. In behaviourally matched tasks we compared the characteristics of aftereffects generated by adapting to colour, to expanded or contracted faces, and to male or female faces. In our experiments opponent coding predicted that the appearance of the adapting image should change and that adaptation should induce symmetrical shifts of two category boundaries. This combination of predictions was firmly supported for colour adaptation, somewhat supported for facial distortion aftereffects, but not supported for facial gender aftereffects. Interestingly, the two face aftereffects we tested generated discrepant patterns of response shifts. Our data suggest that superficially similar aftereffects can ensue from mechanisms that differ qualitatively, and therefore that not all high-level categorical face aftereffects can be attributed to a common coding strategy.

Shared or separate mechanisms for self-face and other-face processing? Evidence from adaptation

Evidence that self-face recognition is dissociable from general face recognition has important implications both for models of social cognition and for our understanding of face recognition. In two studies, we examine how adaptation affects the perception of personally familiar faces, and we use a visual adaptation paradigm to investigate whether the neural mechanisms underlying the recognition of one’s own and other faces are shared or separate. In Study 1 we show that the representation of personally familiar faces is rapidly updated by visual experience with unfamiliar faces, so that the perception of one’s own face and a friend’s face is altered by a brief period of adaptation to distorted unfamiliar faces. In Study 2, participants adapted to images of their own and a friend’s face distorted in opposite directions; the contingent aftereffects we observe are indicative of separate neural populations, but we suggest that these reflect coding of facial identity rather than of the categories “self” and “other.”

Adaptation to antifaces and the perception of correct famous identity in an average face

Previous experiments have examined exposure to anti-identities (faces that possess traits opposite to an identity through a population average), finding that exposure to antifaces enhances recognition of the plus-identity images. Here we examine adaptation to antifaces using famous female celebrities. We demonstrate: that exposure to a color and shape transformed antiface of a celebrity increases the likelihood of perceiving the identity from which the antiface was manufactured in a composite face and that the effect shows size invariance (experiment 1), equivalent effects are seen in internet and laboratory-based studies (experiment 2), adaptation to shape-only antifaces has stronger effects on identity recognition than adaptation to color-only antifaces (experiment 3), and exposure to male versions of the antifaces does not influence the perception of female faces (experiment 4). Across these studies we found an effect of order where aftereffects were more pronounced in early than later trials. Overall, our studies delineate several aspects of identity aftereffects and support the proposal that identity is coded relative to other faces with special reference to a relatively sex-specific mean face representation.

BIOLOGICALLY PLAUSIBLE BSDT RECOGNITION OF COMPLEX IMAGES: THE CASE OF HUMAN FACES

On the basis of recent binary signal detection theory (BSDT), optimal recognition algorithms for complex images are constructed and their optimal performance are calculated. A methodology for comparing BSDT predictions and measured human performance is developed and applied to explaining particular face recognition experiment. The BSDT makes possible computer codes with recognition performance better than that in humans, its fundamental discreteness is consistent with the experiment. Related neurobiological and behavioral effects are briefly discussed.

The role of skin texture and facial shape in representations of age and identity

Faces have both shape and skin texture, but the relative importance of the two in face representations is unclear. Our goals were first, to determine the contribution of shape versus texture to aftereffects for facial age and identity and second, to assess whether adaptation transferred between shape and texture, suggesting integration in a single representation. In our first experiment we examined age aftereffects. We obtained young and old images of two celebrities and created hybrid images, one combining the structure of the old face with the skin texture of the young face, the other combining the young structure with the old skin texture. This allowed us to create adaptation contrasts where the two adapting faces had the same facial structure but different skin texture, and vice versa. In the second experiment, we performed a similar study but this time examining identity aftereffects between two people of a similar age. We found that both skin texture and facial shape generated significant age aftereffects, but the contribution was greater from texture than from shape. Both texture and shape also generated significant identity aftereffects, but the contribution was greater from shape than from texture. In the last experiment, we used the normal and hybrid images to determine if adaptation to one property (i.e., texture) could create aftereffects in the perception of age in the other property (i.e., shape). While there was significant within-component adaptation for texture and shape, there was no evidence of cross-component adaptation. We conclude that shape and texture contribute differently to different face representations, with texture dominating for age. The lack of cross-component adaptation transfer suggests independent encoding of shape and texture, at least for age representations.

Anti-voice adaptation suggests prototype-based coding of voice identity

We used perceptual aftereffects induced by adaptation with anti-voice stimuli to investigate voice identity representations. Participants learned a set of voices then were tested on a voice identification task with vowel stimuli morphed between identities, after different conditions of adaptation. In Experiment 1, participants chose the identity opposite to the adapting anti-voice significantly more often than the other two identities (e.g., after being adapted to anti-A, they identified the average voice as A). In Experiment 2, participants showed a bias for identities opposite to the adaptor specifically for anti-voice, but not for non-anti-voice adaptors. These results are strikingly similar to adaptation aftereffects observed for facial identity. They are compatible with a representation of individual voice identities in a multidimensional perceptual voice space referenced on a voice prototype.

Visual adaptation and face perception

The appearance of faces can be strongly affected by the characteristics of faces viewed previously. These perceptual after-effects reflect processes of sensory adaptation that are found throughout the visual system, but which have been considered only relatively recently in the context of higher level perceptual judgements. In this review, we explore the consequences of adaptation for human face perception, and the implications of adaptation for understanding the neural-coding schemes underlying the visual representation of faces. The properties of face after-effects suggest that they, in part, reflect response changes at high and possibly face-specific levels of visual processing. Yet, the form of the after-effects and the norm-based codes that they point to show many parallels with the adaptations and functional organization that are thought to underlie the encoding of perceptual attributes like colour. The nature and basis for human colour vision have been studied extensively, and we draw on ideas and principles that have been developed to account for norms and normalization in colour vision to consider potential similarities and differences in the representation and adaptation of faces.

Position specificity of adaptation-related face aftereffects

It has been shown that prolonged exposure to a human face leads to shape-selective visual aftereffects. It seems that these face-specific aftereffects (FAEs) have multiple components, related to the adaptation of earlier and higher level processing of visual stimuli. The largest magnitude of FAE, using long-term adaptation periods, is usually observed at the retinotopic position of the preceding adaptor stimulus. However, FAE is also detected, to a smaller degree, at other retinal positions in a spatially invariant way and this component depends less on the adaptation duration. Several lines of evidences suggest that while the position-specific FAE involves lower level areas of the ventral processing stream, the position-invariant FAE depends on the activation of higher level face-processing areas and the fusiform gyrus in particular. In the present paper, we summarize the available behavioural, electrophysiological and neuroimaging results regarding the spatial selectivity of FAE and discuss their implications for the visual stability of object representations across saccadic eye movements.

Adaptation and visual coding

Visual coding is a highly dynamic process and continuously adapting to the current viewing context. The perceptual changes that result from adaptation to recently viewed stimuli remain a powerful and popular tool for analyzing sensory mechanisms and plasticity. Over the last decade, the footprints of this adaptation have been tracked to both higher and lower levels of the visual pathway and over a wider range of timescales, revealing that visual processing is much more adaptable than previously thought. This work has also revealed that the pattern of aftereffects is similar across many stimulus dimensions, pointing to common coding principles in which adaptation plays a central role. However, why visual coding adapts has yet to be fully answered.

Electrophysiological correlates of face distortion after-effects

When observers are exposed to a distorted face the perceived configuration of a subsequently presented face is altered, a phenomenon called face distortion after-effect (FDAE). We compared the face-related components of the event-related potential (ERP) after adaptation to noise images—veridical and distorted faces. We found large bilateral adaptation effects on the P100 and N170 components that are related to face detection. Moreover, we found smaller adaptation effects on the N170, recorded over the right hemisphere, which can be related to the behavioural distortion after-effect and to face configurations. Our results suggest that the observed ERP adaptation effects are general for various steps of face processing and that the FDAEs similarly to gender after-effects are related to the early face-specific ERP components.

Identity-specific face adaptation effects: evidence for abstractive face representations

The effects of selective adaptation on familiar face perception were examined. After prolonged exposure to photographs of a celebrity, participants saw a series of ambiguous morphs that were varying mixtures between the face of that person and a different celebrity. Participants judged fewer of the morphs to resemble the celebrity to which they had been adapted, implying that they were now less sensitive to that particular face. Similar results were obtained when the adapting faces were highly dissimilar in viewpoint to the test morphs; when they were presented upside-down; or when they were vertically stretched to three times their normal height. These effects rule out explanations of adaptation effects solely in terms of low-level image-based adaptation. Instead they are consistent with the idea that relatively viewpoint-independent, person-specific adaptation occurred, at the level of either the "Face Recognition Units" or "Person Identity Nodes" in Burton, Bruce and Johnston's (1990) model of face recognition.

Adaptation improves discrimination of face identity

Whether face adaptation confers any advantages to perceptual processing remains an open question. We investigated whether face adaptation can enhance the ability to make fine discriminations in the vicinity of the adapted face. We compared face discrimination thresholds in three adapting conditions: (i) same-face: where adapting and test faces were the same, (ii) different-face: where adapting and test faces differed, and (iii) baseline: where the adapting stimulus was a blank. Discrimination thresholds for morphed identity changes involving the adapted face (same-face) improved compared with those from both the baseline (no-adaptation) and different-face conditions. Since adapting to a face did not alter discrimination performance for other faces, this effect is selective for the facial identity that is adapted. These results indicate a form of gain control to heighten perceptual sensitivity in the vicinity of a currently viewed face, analogous to forms of adaptive gain control at lower levels of the visual system.

Dynamic norm-based encoding for unfamiliar shapes in human visual cortex

Previous studies have argued that faces and other objects are encoded in terms of their deviation from a class prototype or norm. This prototype is associated with a smaller neural population response compared with nonprototype objects. However, it is still unclear (1) whether a norm-based representation can emerge for unfamiliar or novel object classes through visual experience at the time scale of an experiment and (2) whether the results from previous studies are caused by the prototypicality of a stimulus, by the physical properties of individual stimuli independent from the stimulus distribution, and/or by the trial-to-trial adaptation. Here we show with a combined behavioral and event-related fMRI study in humans that a short amount of visual experience with exemplars from novel object classes determines which stimulus is represented as the norm. Prototypicality effects were observed at the behavioral level by behavioral asymmetries during a stimulus comparison task. The fMRI data revealed that class exemplars closest to the prototypes--the perceived average of each class--were associated with a smaller response in the anterior part of the visual object-selective cortex compared with other class exemplars. By dissociating between the physical characteristics and the prototypicality status of the stimuli and by controlling for trial-to-trial adaptation, we can firmly conclude for the first time that high-level visual areas represent the identity of exemplars using a dynamic, norm-based encoding principle.

Probing the face-space of individuals with prosopagnosia

A useful framework for understanding the mental representation of facial identity is face-space (Valentine, 1991), a multi-dimensional cognitive map in which individual faces are coded relative to the average of previously encountered faces, and in which the distance among faces represents their perceived similarity. We examined whether individuals with prosopagnosia, a disorder characterized by an inability to recognize familiar faces despite normal visual acuity and intellectual abilities, evince behavior consistent with this underlying representational schema. To do so, we compared the performance of 6 individuals with congenital prosopagnosia (CP), with a group of age- and gender-matched control participants in a series of experiments involving judgments of facial identity. We used digital images of male and female faces and morphed them to varying degrees relative to an average face, to create caricatures, anti-caricatures, and anti-faces (i.e. faces of the opposite identity). Across 5 behavioral tasks, CP individuals' performance was similar to that of the control group and consistent with the face-space framework. As a test of the sensitivity of our measures in revealing face processing abnormalities, we also tested a single acquired prosopagnosic (AP) individual, whose performance on the same tasks deviated significantly from the control and CP groups. The findings suggest that, despite an inability to recognize individual identities, CPs perceive faces in a manner consistent with norm-based coding of facial identity, although their representation is likely supported by a feature-based strategy. We suggest that the apparently normal posterior cortical regions, including the fusiform face area, serve as the neural substrate for at least a coarse, feature-based face-space map in CP and that their face recognition impairment arises from the disconnection between these regions and more anterior cortical sites.

Perceptual adaptation helps us identify faces

Adaptation is a fundamental property of perceptual processing. In low-level vision, it can calibrate perception to current inputs, increasing coding efficiency and enhancing discrimination around the adapted level. Adaptation also occurs in high-level vision, as illustrated by face aftereffects. However, the functional consequences of face adaptation remain uncertain. Here we investigated whether adaptation can enhance identification performance for faces from an adapted, relative to an unadapted, population. Five minutes of adaptation to an average Asian or Caucasian face reduced identification thresholds for faces from the adapted relative to the unadapted race. We replicated this interaction in two studies, using different participants, faces and adapting procedures. These results suggest that adaptation has a functional role in high-level, as well as low-level, visual processing. We suggest that adaptation to the average of a population may reduce responses to common properties shared by all members of the population, effectively orthogonalizing identity vectors in a multi-dimensional face space and freeing neural resources to code distinctive properties, which are useful for identification.

What shape are the neural response functions underlying opponent coding in face space? A psychophysical investigation

Recent evidence has shown that face space represents facial identity information using two-pool opponent coding. Here we ask whether the shape of the monotonic neural response functions underlying such coding is linear (i.e. face space codes all equal-sized physical changes with equal sensitivity) or nonlinear (e.g. face space shows greater coding sensitivity around the average face). Using adaptation aftereffects and pairwise discrimination tasks, our results for face attributes of eye height and mouth height demonstrate linear shape; including for bizarre faces far outside the normal range. We discuss how linear coding explains some results in the previous literature, including failures to find that adaptation enhances face discrimination, and suggest possible reasons why face space can maintain detailed coding of values far outside the normal range. We also discuss specific nonlinear coding models needed to explain other findings, and conclude face space appears to use a mixture of linear and nonlinear representations.

The gender-specific face aftereffect is based in retinotopic not spatiotopic coordinates across several natural image transformations

In four experiments, we measured the gender-specific face-aftereffect following subject's eye movement, head rotation, or head movement toward the display and following movement of the adapting stimulus itself to a new test location. In all experiments, the face aftereffect was strongest at the retinal position, orientation, and size of the adaptor. There was no advantage for the spatiotopic location in any experiment nor was there an advantage for the location newly occupied by the adapting face after it moved in the final experiment. Nevertheless, the aftereffect showed a broad gradient of transfer across location, orientation and size that, although centered on the retinotopic values of the adapting stimulus, covered ranges far exceeding the tuning bandwidths of neurons in early visual cortices. These results are consistent with a high-level site of adaptation (e.g. FFA) where units of face analysis have modest coverage of visual field, centered in retinotopic coordinates, but relatively broad tolerance for variations in size and orientation.

Evidence that identity-dependent and identity-independent neural populations are recruited in the perception of five basic emotional facial expressions

Major cognitive and neural models of face perception view that the mechanisms underlying the extraction of facial expression and facial identity information involve separable visual systems. Using the visual adaptation paradigm, we explored the sensitivity of happy, sad, angry, disgusted and fearful facial expressions to changes in identity. Contrary to what would be predicted by traditional face perception models, larger expression aftereffects were produced when the identity of the adapting and test stimuli was the same compared to when the identity differed, suggesting the involvement of identity-dependent neurons in processing these expressions. Furthermore, for all five expressions, the aftereffects remained significant when the adapting and test stimuli differed in identity, suggesting the involvement of identity-independent neural populations. The extent to which the aftereffect transferred across changes in identity was the same for all emotional expressions. Consequently, there is no evidence that the processing of individual facial expressions depend on facial identity differentially. Implications of these findings are discussed.

Three-dimensional information in face representations revealed by identity aftereffects

Representations of individual faces evolve with experience to support progressively more robust recognition. Knowledge of three-dimensional face structure is required to predict an image of a face as illumination and viewpoint change. Robust recognition across such transformations can be achieved with representations based on multiple two-dimensional views, three-dimensional structure, or both. We used face-identity adaptation in a familiarization paradigm to address a long-standing controversy concerning the role of two-dimensional versus three-dimensional information in face representations. We reasoned that if three-dimensional information is coded in the representations of familiar faces, then learning a new face using images generated by one three-dimensional transformation should enhance the robustness of the representation to another type of three-dimensional transformation. Familiarization with multiple views of faces enhanced the transfer of face-identity adaptation effects across changes in illumination by compensating for a generalization cost at a novel test viewpoint. This finding demonstrates a role for three-dimensional information in representations of familiar faces.