Probing the Visual Representation of Faces With Adaptation

fMRI evidence that hyper-caricatured faces activate object-selective cortex

Many brain imaging studies have looked at the cortical responses to object categories and faces. A popular way to manipulate face stimuli is by using a "face space," a high dimensional representation of individual face images, with the average face located at the origin. However, how the brain responds to faces that deviate substantially from average has not been much explored. Increasing the distance from the average (leading to increased caricaturing) could increase neural responses in face-selective regions, an idea supported by results from non-human primates. Here, we used a face space based on principal component analysis (PCA) to generate faces ranging from average to heavily caricatured. Using functional magnetic resonance imaging (fMRI), we first independently defined face-, object- and scene-selective areas with a localiser scan and then measured responses to parametrically caricatured faces. We also included conditions in which the images of faces were inverted. Interestingly in the right fusiform face area (FFA), we found that the patterns of fMRI response were more consistent as caricaturing increased. However, we found no consistent effect of either caricature level or facial inversion on the average fMRI response in the FFA or face-selective regions more broadly. In contrast, object-selective regions showed an increase in both the consistency of response pattern and the average fMRI response with increasing caricature level. This shows that caricatured faces recruit processing from regions typically defined as object-selective, possibly through enhancing low-level properties that are characteristic of objects.

Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

Discerning the subtle differences between individuals’ faces is crucial for social functioning. It requires us not only to solve general challenges of object recognition (e.g., invariant recognition over changes in view or lighting) but also to be attuned to the specific ways in which face structure varies. Three-dimensional morphable models based on principal component analyses of real faces provide descriptions of statistical differences between faces, as well as tools to generate novel faces. We rendered large sets of realistic face pairs from such a model and collected similarity and same/different identity judgments. The statistical model predicted human perception as well as state-of-the-art image-computable neural networks. Results underscore the statistical tuning of face encoding.

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component–based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a “face space” based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

A PCA-Based Active Appearance Model for Characterising Modes of Spatiotemporal Variation in Dynamic Facial Behaviours

Faces carry key personal information about individuals, including cues to their identity, social traits, and emotional state. Much research to date has employed static images of faces taken under tightly controlled conditions yet faces in the real world are dynamic and experienced under ambient conditions. A common approach to studying key dimensions of facial variation is the use of facial caricatures. However, such techniques have again typically relied on static images, and the few examples of dynamic caricatures have relied on animating graphical head models. Here, we present a principal component analysis (PCA)-based active appearance model for capturing patterns of spatiotemporal variation in videos of natural dynamic facial behaviours. We demonstrate how this technique can be applied to generate dynamic anti-caricatures of biological motion patterns in facial behaviours. This technique could be extended to caricaturing other facial dimensions, or to more general analyses of spatiotemporal variations in dynamic faces.

Average faces: How does the averaging process change faces physically and perceptually?

Average faces have been used frequently in face recognition studies, either as a theoretical concept (e.g., face norm) or as a tool to manipulate facial attributes (e.g., modifying identity strength). Nonetheless, how the face averaging process- the creation of average faces using an increasing number of faces -changes the resulting averaged faces and our ability to differentiate between them remains to be elucidated. Here we addressed these questions by combining 3D-face averaging, eye-movement tracking, and the computation of image-based face similarity. Participants judged whether two average faces showed the same person while we systematically increased their average level (i.e., number of faces being averaged). Our results showed, with increasing averaging, both a nonlinear increase of the computational similarity between the resulting average faces and a nonlinear decrease of face discrimination performance. Participants' performance dropped from near-ceiling level when two different faces had been averaged together to chance level when 80 faces were mixed. We also found a nonlinear relationship between face similarity and face discrimination performance, which was fitted nicely with an exponential function. Furthermore, when the comparison task became more challenging, participants performed more fixations onto the faces. Nonetheless, the distribution of fixations across facial features (eyes, nose, mouth, and the center area of a face) remained unchanged. These results not only set new constraints on the theoretical characterization of the average face and its role in establishing face norms but also offer practical guidance for creating approximated face norms to manipulate face identity.

Serial dependence of facial identity reflects high-level face coding

Serial dependence of facial identity is a type of bias where the perceived identity of a face is biased towards a previously presented face. There are individual differences in serial dependence strength and tuning (how the strength varies depending on stimuli similarity), and previous research has shown that both stronger and more narrowly tuned serial dependence of facial identity is associated with better face recognition abilities. These results are consistent with the idea that this bias plays a functional role in face perception. It is important, therefore, to determine whether serial dependence of facial identity reflects a high-level face-coding mechanism acting on the identity of a face or instead predominantly reflects a bias in low-level features, which are also subject to serial dependence. We first sought evidence that serial dependence of facial identity survived changes in low-level visual features, by varying face viewpoint between successive stimuli. We found that serial dependence persisted across changes in viewpoint, arguing against an entirely low-level locus for this bias. We next tested whether the bias was affected by inversion, as sensitivity to inversion is argued to be a characteristic of high-level face-selective processing. Serial dependence was stronger and more narrowly tuned for upright than inverted faces. Taken together, our results are consistent with the view that serial dependence of facial identity affects high-level visual representations and may reflect a face-coding mechanism that is operating at the level of facial identity.

Face Adaptation and Face Priming as Tools for Getting Insights Into the Quality of Face Space

During the recognition of faces, the incoming perceptual information is matched against mental representations of familiar faces stored in memory. Face space models describe an abstract concept of face representations and their mental organization, in which facial representations are located on various characteristic dimensions, depending on their specific facial characteristics. However, these models are defined just as incompletely as the general understanding of face recognition. We took two phenomena from face processing to better understand face recognition, and so the nature of face space: face adaptation and face priming. The face literature has mainly focused on face adaptation, largely neglecting face priming when trying to integrate outcomes regarding face recognition into the face space framework. Consequently, the present paper aims to review both phenomena and their contributions to face recognition, representation, and face space.

Face Representations via Tensorfaces of Various Complexities

Neurons selective for faces exist in humans and monkeys. However, characteristics of face cell receptive fields are poorly understood. In this theoretical study, we explore the effects of complexity, defined as algorithmic information (Kolmogorov complexity) and logical depth, on possible ways that face cells may be organized. We use tensor decompositions to decompose faces into a set of components, called tensorfaces, and their associated weights, which can be interpreted as model face cells and their firing rates. These tensorfaces form a high-dimensional representation space in which each tensorface forms an axis of the space. A distinctive feature of the decomposition algorithm is the ability to specify tensorface complexity. We found that low-complexity tensorfaces have blob-like appearances crudely approximating faces, while high-complexity tensorfaces appear clearly face-like. Low-complexity tensorfaces require a larger population to reach a criterion face reconstruction error than medium- or high-complexity tensorfaces, and thus are inefficient by that criterion. Low-complexity tensorfaces, however, generalize better when representing statistically novel faces, which are faces falling beyond the distribution of face description parameters found in the tensorface training set. The degree to which face representations are parts based or global forms a continuum as a function of tensorface complexity, with low and medium tensorfaces being more parts based. Given the computational load imposed in creating high-complexity face cells (in the form of algorithmic information and logical depth) and in the absence of a compelling advantage to using high-complexity cells, we suggest face representations consist of a mixture of low- and medium-complexity face cells.

Understanding individual differences in theory of mind via representation of minds, not mental states

The human ability to make inferences about the minds of conspecifics is remarkable. The majority of work in this area focuses on mental state representation ('theory of mind'), but has had limited success in explaining individual differences in this ability, and is characterized by the lack of a theoretical framework that can account for the effect of variability in the population of minds to which individuals are exposed. We draw analogies between faces and minds as complex social stimuli, and suggest that theoretical and empirical progress on understanding the mechanisms underlying mind representation can be achieved by adopting a 'Mind-space' framework; that minds, like faces, are represented within a multidimensional psychological space. This Mind-space framework can accommodate the representation of whole cognitive systems, and may help to explain individual differences in the consistency and accuracy with which the mental states of others are inferred. Mind-space may also have relevance for understanding human development, intergroup relations, and the atypical social cognition seen in several clinical conditions.

Face perception: A brief journey through recent discoveries and current directions

Faces are a rich source of information about the people around us. Identity, state of mind, emotions, intentions, age, gender, ethnic background, attractiveness and a host of other attributes about an individual can be gleaned from a face. When face perception fails, dramatic psycho-social consequences can follow at the individual level, as in the case of prosopagnosic parents who are unable to recognize their children at school pick-up. At the species level, social interaction patterns are shaped by human face perception abilities. The computational feat of recognizing faces and facial attributes, and the challenges overcome by the human brain to achieve this feat, have fascinated generations of vision researchers. In this paper, we present a brief overview of some of the milestones of discovery as well as outline a selected set of current directions and open questions on this topic.

360 Degrees of Facial Perception: Congruence in Perception of Frontal Portrait, Profile, and Rotation Photographs

Studies in social perception traditionally use as stimuli frontal portrait photographs. It turns out, however, that 2D frontal depiction may not fully capture the entire morphological diversity of facial features. Recently, 3D images started to become increasingly popular, but whether their perception differs from the perception of 2D has not been systematically studied as yet. Here we investigated congruence in the perception of portrait, left profile, and 360° rotation photographs. The photographs were obtained from 45 male athletes under standardized conditions. In two separate studies, each set of images was rated for formidability (portraits by 62, profiles by 60, and 360° rotations by 94 raters) and attractiveness (portraits by 195, profiles by 176, and 360° rotations by 150 raters) on a 7-point scale. The ratings of the stimuli types were highly intercorrelated (for formidability all rs > 0.8, for attractiveness all rs > 0.7). Moreover, we found no differences in the mean ratings between the three types of stimuli, neither in formidability, nor in attractiveness. Overall, our results clearly suggest that different facial views convey highly overlapping information about structural facial elements of an individual. They lead to congruent assessments of formidability and attractiveness, and a single angle view seems sufficient for face perception research.

Adaptation to dynamic faces produces face identity aftereffects

Face aftereffects are well established for static stimuli and have been used extensively as a tool for understanding the neural mechanisms underlying face recognition. It has also been argued that adaptive coding, as demonstrated by face aftereffects, plays a functional role in face recognition by calibrating our face norms to reflect current experience. If aftereffects tap high-level perceptual mechanisms that are critically involved in everyday face recognition then they should also occur for moving faces. Here we asked whether face identity aftereffects can be induced using dynamic adaptors. The face identity aftereffect occurs when adaptation to a particular identity (e.g., Dan) biases subsequent perception toward the opposite identity (e.g., antiDan). We adapted participants to video of real faces that displayed either rigid, non-rigid, or no motion and tested for aftereffects in static antifaces. Adapt and test stimuli differed in size, to minimize low-level adaptation. Aftereffects were found in all conditions, suggesting that face identity aftereffects tap high-level mechanisms important for face recognition. Aftereffects were not significantly reduced in the motion conditions relative to the static condition. Overall, our results support the view that face aftereffects reflect adaptation of high-level mechanisms important for real-world face recognition in which faces are moving.

Asymmetric neural responses for facial expressions and anti-expressions

Face recognition requires identifying both the invariant characteristics that distinguish one individual from another and the variations within the individual that correspond to emotional expressions. Both have been postulated to be represented via a norm-based code, in which identity or expression are represented as deviations from an average or neutral prototype. We used Fast Periodic Visual Stimulation (FPVS) with electroencephalography (EEG) to compare neural responses for neutral faces, expressions and anti-expressions. Anti-expressions are created by projecting an expression (e.g. a happy face) through the neutral face to form the opposite facial shape (anti-happy). Thus expressions and anti-expressions differ from the norm by the same "configural" amount and thus have equivalent but opposite status with regard to their shape, but differ in their ecological validity. We examined whether neural responses to these complementary stimulus pairs were equivalent or asymmetric, and also tested for norm-based coding by comparing whether stronger responses are elicited by expressions and anti-expressions than neutral faces. Observers viewed 20 s sequences of 6 Hz alternations of neutral faces and expressions, neutral faces and anti-expressions, and expressions and anti-expressions. Responses were analyzed in the frequency domain. Significant responses at half the frequency of the presentation rate (3 Hz), indicating asymmetries in responses, were observed for all conditions. Inversion of the images reduced the size of this signal, indicating these asymmetries are not solely due to differences in the low-level properties of the images. While our results do not preclude a norm-based code for expressions, similar to identity, this representation (as measured by the FPVS EEG responses) may also include components sensitive to which configural distortions form meaningful expressions.

The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction

Uncovering the neural dynamics of facial identity processing along with its representational basis outlines a major endeavor in the study of visual processing. To this end, here, we record human electroencephalography (EEG) data associated with viewing face stimuli; then, we exploit spatiotemporal EEG information to determine the neural correlates of facial identity representations and to reconstruct the appearance of the corresponding stimuli. Our findings indicate that multiple temporal intervals support: facial identity classification, face space estimation, visual feature extraction and image reconstruction. In particular, we note that both classification and reconstruction accuracy peak in the proximity of the N170 component. Further, aggregate data from a larger interval (50–650 ms after stimulus onset) support robust reconstruction results, consistent with the availability of distinct visual information over time. Thus, theoretically, our findings shed light on the time course of face processing while, methodologically, they demonstrate the feasibility of EEG-based image reconstruction.

Temporal Contiguity Training Influences Behavioral and Neural Measures of Viewpoint Tolerance

Humans can often recognize faces across viewpoints despite the large changes in low-level image properties a shift in viewpoint introduces. We present a behavioral and an fMRI adaptation experiment to investigate whether this viewpoint tolerance is reflected in the neural visual system and whether it can be manipulated through training. Participants saw training sequences of face images creating the appearance of a rotating head. Half of the sequences showed faces undergoing veridical changes in appearance across the rotation (non-morph condition). The other half were non-veridical: during rotation, the face simultaneously morphed into another face. This procedure should successfully associate frontal face views with side views of the same or a different identity, and, according to the temporal contiguity hypothesis, thus enhance viewpoint tolerance in the non-morph condition and/or break tolerance in the morph condition. Performance on the same/different task in the behavioral experiment (N = 20) was affected by training. There was a significant interaction between training (associated/not associated) and identity (same/different), mostly reflecting a higher confusion of different identities when they were associated during training. In the fMRI study (N = 20), fMRI adaptation effects were found for same-viewpoint images of untrained faces, but no adaptation for untrained faces was present across viewpoints. Only trained faces which were not morphed during training elicited a slight adaptation across viewpoints in face-selective regions. However, both in the behavioral and in the neural data the effects were small and weak from a statistical point of view. Overall, we conclude that the findings are not inconsistent with the proposal that temporal contiguity can influence viewpoint tolerance, with more evidence for tolerance when faces are not morphed during training.

Repetition Suppression and Memory for Faces is Reduced in Adults with Autism Spectrum Conditions

Autism spectrum conditions (ASC) are associated with a number of atypicalities in face processing, including difficulties in face memory. However, the neural mechanisms underlying this difficulty are unclear. In neurotypical individuals, repeated presentation of the same face is associated with a reduction in activity, known as repetition suppression (RS), in the fusiform face area (FFA). However, to date, no studies have investigated RS to faces in individuals with ASC, or the relationship between RS and face memory. Here, we measured RS to faces and geometric shapes in individuals with a clinical diagnosis of an ASC and in age and IQ matched controls. Relative to controls, the ASC group showed reduced RS to faces in bilateral FFA and reduced performance on a standardized test of face memory. By contrast, RS to shapes in object-selective regions and object memory did not differ between groups. Individual variation in face-memory performance was positively correlated with RS in regions of left parietal and prefrontal cortex. These findings suggest difficulties in face memory in ASC may be a consequence of differences in the way faces are stored and/or maintained across a network of regions involved in both visual perception and short-term/working memory.

View-Specific Coding of Face Shape

Monkey and human cortex contain view-specific face neurons, but it remains unclear whether they code face shape. We tested the view specificity of face-shape coding by inducing figural face aftereffects at one viewpoint (3/4 left) and testing generalization to different viewpoints (front view and 3/4 right). The aftereffects were induced by adaptation to consistent figural distortions (contracted or expanded), which shifts the distortion perceived as most normal toward the adapting distortion. The strong aftereffect that was observed at the adapting view was significantly and substantially reduced for both front-view test faces and mirror-image (3/4 right) test faces, indicating view specificity. The limited transfer across mirror views is strong evidence of view specificity, given their figural similarity. The aftereffects survived a size change between adaptation and test faces (Experiment 2), a result that rules out low-level adaptation as an explanation. These results provide strong evidence that face-shape coding is view-specific.

Neural basis of distorted self-face recognition in social anxiety disorder

Background

The observer perspective causes patients with social anxiety disorder (SAD) to excessively inspect their performance and appearance. This study aimed to investigate the neural basis of distorted self-face recognition in non-social situations in patients with SAD.

Methods

Twenty patients with SAD and 20 age- and gender-matched healthy controls participated in this fMRI study. Data were acquired while participants performed a Composite Face Evaluation Task, during which they had to press a button indicating how much they liked a series of self-faces, attractively transformed self-faces, and attractive others' faces.

Results

Patients had a tendency to show more favorable responses to the self-face and unfavorable responses to the others' faces compared with controls, but the two groups' responses to the attractively transformed self-faces did not differ. Significant group differences in regional activity were observed in the middle frontal and supramarginal gyri in the self-face condition (patients < controls); the inferior frontal gyrus in the attractively transformed self-face condition (patients > controls); and the middle frontal, supramarginal, and angular gyri in the attractive others' face condition (patients > controls). Most fronto-parietal activities during observation of the self-face were negatively correlated with preference scores in patients but not in controls.

Conclusion

Patients with SAD have a positive point of view of their own face and experience self-relevance for the attractively transformed self-faces. This distorted cognition may be based on dysfunctions in the frontal and inferior parietal regions. The abnormal engagement of the fronto-parietal attentional network during processing face stimuli in non-social situations may be linked to distorted self-recognition in SAD.

•

We studied neural responses to attractive self-face images in SAD.

•

Patients with SAD showed more favorable responses to the self-face.

•

Distorted self-recognition in SAD was related to fronto-parietal dysfunctions.

•

Patients with SAD had abnormal attentional network-based self-face recognition.

Visual adaptation provides objective electrophysiological evidence of facial identity discrimination

Discrimination of facial identities is a fundamental function of the human brain that is challenging to examine with macroscopic measurements of neural activity, such as those obtained with functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although visual adaptation or repetition suppression (RS) stimulation paradigms have been successfully implemented to this end with such recording techniques, objective evidence of an identity-specific discrimination response due to adaptation at the level of the visual representation is lacking. Here, we addressed this issue with fast periodic visual stimulation (FPVS) and EEG recording combined with a symmetry/asymmetry adaptation paradigm. Adaptation to one facial identity is induced through repeated presentation of that identity at a rate of 6 images per second (6 Hz) over 10 sec. Subsequently, this identity is presented in alternation with another facial identity (i.e., its anti-face, both faces being equidistant from an average face), producing an identity repetition rate of 3 Hz over a 20 sec testing sequence. A clear EEG response at 3 Hz is observed over the right occipito-temporal (ROT) cortex, indexing discrimination between the two facial identities in the absence of an explicit behavioral discrimination measure. This face identity discrimination occurs immediately after adaptation and disappears rapidly within 20 sec. Importantly, this 3 Hz response is not observed in a control condition without the single-identity 10 sec adaptation period. These results indicate that visual adaptation to a given facial identity produces an objective (i.e., at a pre-defined stimulation frequency) electrophysiological index of visual discrimination between that identity and another, and provides a unique behavior-free quantification of the effect of visual adaptation.

How previous experience shapes perception in different sensory modalities

What has transpired immediately before has a strong influence on how sensory stimuli are processed and perceived. In particular, temporal context can have contrastive effects, repelling perception away from the interpretation of the context stimulus, and attractive effects (TCEs), whereby perception repeats upon successive presentations of the same stimulus. For decades, scientists have documented contrastive and attractive temporal context effects mostly with simple visual stimuli. But both types of effects also occur in other modalities, e.g., audition and touch, and for stimuli of varying complexity, raising the possibility that context effects reflect general computational principles of sensory systems. Neuroimaging shows that contrastive and attractive context effects arise from neural processes in different areas of the cerebral cortex, suggesting two separate operations with distinct functional roles. Bayesian models can provide a functional account of both context effects, whereby prior experience adjusts sensory systems to optimize perception of future stimuli.

Single-cell responses to face adaptation in the human medial temporal lobe

We used a face adaptation paradigm to bias the perception of ambiguous images of faces and study how single neurons in the human medial temporal lobe (MTL) respond to the same images eliciting different percepts. The ambiguous images were morphs between the faces of two familiar individuals, chosen because at least one MTL neuron responded selectively to one but not to the other face. We found that the firing of MTL neurons closely followed the subjects' perceptual decisions--i.e., recognizing one person or the other. In most cases, the response to the ambiguous images was similar to the one obtained when showing the pictures without morphing. Altogether, these results show that many neurons in the medial temporal lobe signal the subjects' perceptual decisions rather than the visual features of the stimulus.

Repetition Suppression in Ventral Visual Cortex Is Diminished as a Function of Increasing Autistic Traits

Repeated viewing of a stimulus causes a change in perceptual sensitivity, known as a visual aftereffect. Similarly, in neuroimaging, repetitions of the same stimulus result in a reduction in the neural response, known as repetition suppression (RS). Previous research shows that aftereffects for faces are reduced in both children with autism and in first-degree relatives. With functional magnetic resonance imaging, we found that the magnitude of RS to faces in neurotypical participants was negatively correlated with individual differences in autistic traits. We replicated this finding in a second experiment, while additional experiments showed that autistic traits also negatively predicted RS to images of scenes and simple geometric shapes. These findings suggest that a core aspect of neural function--the brain's response to repetition--is modulated by autistic traits.

Understanding individual face discrimination by means of fast periodic visual stimulation

This paper reviews a fast periodic visual stimulation (FPVS) approach developed recently to make significant progress in understanding visual discrimination of individual faces. Displaying pictures of faces at a periodic frequency rate leads to a high signal-to-noise ratio (SNR) response in the human electroencephalogram, at the exact frequency of stimulation, a so-called steady-state visual evoked potential (SSVEP, Regan in Electroencephalogr Clin Neurophysiol 20:238-248, 1966). For fast periodic frequency rates, i.e., between 3 and 9 Hz, this response is reduced if the exact same face identity is repeated compared to the presentation of different face identities, the largest difference being observed over the right occipito-temporal cortex. A 6-Hz stimulation rate (cycle duration of ~170 ms) provides the largest difference between different and repeated faces, as also evidenced in face-selective areas of the ventral occipito-temporal cortex in functional magnetic resonance imaging. This high-level discrimination response is reduced following inversion and contrast-reversal of the faces and can be isolated without subtraction thanks to a fast periodic oddball paradigm. Overall, FPVS provides a response that is objective (i.e., at an experimentally defined frequency), implicit, has a high SNR and is directly quantifiable in a short amount of time. Although the approach is particularly appealing for understanding face perception, it can be generalized to study visual discrimination of complex visual patterns such as objects and visual scenes. The advantages of the approach make it also particularly well-suited to investigate these functions in populations who cannot provide overt behavioral responses and can only be tested for short durations, such as infants, young children and clinical populations.

Looking at a Blurry Old Family Photo? Zoom Out!

We investigated recognition of blurry faces and whether viewing size affects identification of such severely degraded images. Despite the common belief that face perception relies on middle spatial frequencies, the critical spatial frequency band for face recognition is not fixed but rather depends on size. This is especially pronounced at small sizes, where observers choose to utilize lower, rather than middle, frequencies to identify a face. Here we assessed recognition of identity via a novel use of the face adaptation paradigm. We examined face identity aftereffects of blurry and intact adaptors at two sizes. Intact adaptors induced significant aftereffects regardless of size. Small, but not large, blurry adaptors produced aftereffects despite the fact that both contained exactly the same level of facial detail. This suggests an inability to utilize low-frequency information for perceiving identity in large faces. We conclude that (1) size is a key factor in human face recognition processes and (2) coarse facial images are better recognized at small sizes.

An objective index of individual face discrimination in the right occipito-temporal cortex by means of fast periodic oddball stimulation

We introduce an approach based on fast periodic oddball stimulation that provides objective, high signal-to-noise ratio (SNR), and behavior-free measures of the human brain's discriminative response to complex visual patterns. High-density electroencephalogram (EEG) was recorded for human observers presented with 60s sequences containing a base-face (A) sinusoidally contrast-modulated at a frequency of 5.88 Hz (F), with face size varying every cycle. Different oddball-faces (B, C, D...) were introduced at fixed intervals (every 4 stimuli = F/5 = 1.18 Hz: AAAABAAAACAAAAD...). Individual face discrimination was indexed by responses at this 1.18 Hz oddball frequency. Following only 4 min of recording, significant responses emerged at exactly 1.18 Hz and its harmonics (e.g., 2F/5 = 2.35 Hz, 3F/5 = 3.53 Hz...), with up to a 300% signal increase over the right occipito-temporal cortex. This response was present in all participants, for both color and greyscale faces, providing a robust implicit neural measure of individual face discrimination. Face inversion or contrast-reversal did not affect the basic 5.88 Hz periodic response over medial occipital channels. However, these manipulations substantially reduced the 1.18 Hz oddball discrimination response over the right occipito-temporal region, indicating that this response reflects high-level processes that are partly face-specific. These observations indicate that fast periodic oddball stimulation can be used to rapidly and objectively characterize the discrimination of visual patterns and may become invaluable in characterizing this process in typical adult, developmental, and neuropsychological patient populations.

Seeing a haptically explored face: visual facial-expression aftereffect from haptic adaptation to a face

Current views on face perception assume that the visual system receives only visual facial signals. However, I show that the visual perception of faces is systematically biased by adaptation to a haptically explored face. Recently, face aftereffects (FAEs; the altered perception of faces after adaptation to a face) have been demonstrated not only in visual perception but also in haptic perception; therefore, I combined the two FAEs to examine whether the visual system receives face-related signals from the haptic modality. I found that adaptation to a haptically explored facial expression on a face mask produced a visual FAE for facial expression. This cross-modal FAE was not due to explicitly imaging a face, response bias, or adaptation to local features. Furthermore, FAEs transferred from vision to haptics. These results indicate that visual face processing depends on substrates adapted by haptic faces, which suggests that face processing relies on shared representation underlying cross-modal interactions.

Face adaptation effects: reviewing the impact of adapting information, time, and transfer

The ability to adapt is essential to live and survive in an ever-changing environment such as the human ecosystem. Here we review the literature on adaptation effects of face stimuli to give an overview of existing findings in this area, highlight gaps in its research literature, initiate new directions in face adaptation research, and help to design future adaptation studies. Furthermore, this review should lead to better understanding of the processing characteristics as well as the mental representations of face-relevant information. The review systematizes studies at a behavioral level in respect of a framework which includes three dimensions representing the major characteristics of studies in this field of research. These dimensions comprise (1) the specificity of adapting face information, e.g., identity, gender, or age aspects of the material to be adapted to (2) aspects of timing (e.g., the sustainability of adaptation effects) and (3) transfer relations between face images presented during adaptation and adaptation tests (e.g., images of the same or different identities). The review concludes with options for how to combine findings across different dimensions to demonstrate the relevance of our framework for future studies.

Viewpoint and Pose in Body-Form Adaptation

Faces and bodies are complex structures, perception of which can play important roles in person identification and inference of emotional state. Face representations have been explored using behavioural adaptation: in particular, studies have shown that face aftereffects show relatively broad tuning for viewpoint, consistent with origin in a high-level structural descriptor far removed from the retinal image. Our goals were to determine first, if body aftereffects also showed a degree of viewpoint invariance, and second if they also showed pose invariance, given that changes in pose create even more dramatic changes in the 2-D retinal image. We used a 3-D model of the human body to generate headless body images, whose parameters could be varied to generate different body forms, viewpoints, and poses. In the first experiment, subjects adapted to varying viewpoints of either slim or heavy bodies in a neutral stance, followed by test stimuli that were all front-facing. In the second experiment, we used the same front-facing bodies in neutral stance as test stimuli, but compared adaptation from bodies in the same neutral stance to adaptation with the same bodies in different poses. We found that body aftereffects were obtained over substantial viewpoint changes, with no significant decline in aftereffect magnitude with increasing viewpoint difference between adapting and test images. Aftereffects also showed transfer across one change in pose but not across another. We conclude that body representations may have more viewpoint invariance than faces, and demonstrate at least some transfer across pose, consistent with a high-level structural description.

Holistic Face Processing is Induced by Shape and Texture

There is increasing evidence that shape and texture are integral parts of face identity. However, it is less clear whether face-specific processing mechanisms are triggered by face shape alone, or if texture might play an important role. We address this question by studying mechanisms involved in holistic face processing. Face stimuli were either full-color pictures of real faces (shape and texture) or line drawings of the same faces (shape without texture). In a change detection task subjects judged whether eyes and eyebrows in two otherwise identical, sequentially presented faces were different in size or not. Afterwards, subjects had to identify the just presented face among two distractor faces (forced-choice identification task). The results obtained from the two tasks give rise to the conclusion that face identification and change detection tasks engage different processing strategies, which capture different aspects of holistic processing. Real faces were processed holistically, irrespective of task requirements, whereas line drawings were processed holistically only if face identification was required. On the basis of the data we conclude that face shape is relevant for the initial processing stage and feature binding, whereas face texture seems to be involved in processing of face configuration more specifically. Moreover, results demonstrate considerable flexibility of the face processing systems allowing for goal-directed and task-specific recall of face information.

A brain network processing the age of faces

Age is one of the most salient aspects in faces and of fundamental cognitive and social relevance. Although face processing has been studied extensively, brain regions responsive to age have yet to be localized. Using evocative face morphs and fMRI, we segregate two areas extending beyond the previously established face-sensitive core network, centered on the inferior temporal sulci and angular gyri bilaterally, both of which process changes of facial age. By means of probabilistic tractography, we compare their patterns of functional activation and structural connectivity. The ventral portion of Wernicke's understudied perpendicular association fasciculus is shown to interconnect the two areas, and activation within these clusters is related to the probability of fiber connectivity between them. In addition, post-hoc age-rating competence is found to be associated with high response magnitudes in the left angular gyrus. Our results provide the first evidence that facial age has a distinct representation pattern in the posterior human brain. We propose that particular face-sensitive nodes interact with additional object-unselective quantification modules to obtain individual estimates of facial age. This brain network processing the age of faces differs from the cortical areas that have previously been linked to less developmental but instantly changeable face aspects. Our probabilistic method of associating activations with connectivity patterns reveals an exemplary link that can be used to further study, assess and quantify structure-function relationships.

A steady-state visual evoked potential approach to individual face perception: effect of inversion, contrast-reversal and temporal dynamics

Presentation of a face stimulus for several seconds at a periodic frequency rate leads to a right occipito-temporal evoked steady-state visual potential (SSVEP) confined to the stimulation frequency band. According to recent evidence (Rossion and Boremanse, 2011), this face-related SSVEP is largely reduced in amplitude when the exact same face is repeated at every stimulation cycle as compared to the presentation of different individual faces. Here this SSVEP individual face repetition effect was tested in 20 participants stimulated with faces at a 4 Hz rate for 84 s, in 4 conditions: faces upright or inverted, normal or contrast-reversed (2×2 design). To study the temporal dynamics of this effect, all stimulation sequences started with 15s of identical faces, after which, in half of the sequences, different faces were introduced. A larger response to different than identical faces at the fundamental (4 Hz) and second harmonic (8 Hz) components was observed for upright faces over the right occipito-temporal cortex. Weaker effects were found for inverted and contrast-reversed faces, two stimulus manipulations that are known to greatly affect the perception of facial identity. Addition of the two manipulations further decreased the effect. The phase of the fundamental frequency SSVEP response was delayed for inverted and contrast-reversed faces, to the same extent as the latency delay observed at the peak of the face-sensitive N170 component observed at stimulation sequence onset. Time-course analysis of the entire sequence of stimulation showed an immediate increase of 4Hz amplitude at the onset (16th second) of different face presentation, indicating a fast, large and frequency-specific release to individual face adaptation in the human brain. Altogether, these observations increase our understanding of the characteristics of the human steady-state face potential response and provide further support for the interest of this approach in the study of the neurofunctional mechanisms of face perception.

FIAEs in Famous Faces are Mediated by Type of Processing

An important question regarding face aftereffects is whether it is based on face-specific or lower-level mechanisms. One method for addressing this is to explore how adaptation in upright or inverted, photographic positive or negative faces transfers to test stimuli that are either upright or inverted and normal or negated. A series of studies are reported in which this is tested using a typical face identity aftereffect paradigm in unfamiliar and famous faces. Results showed that aftereffects were strongest when the adaptor matched the test stimuli. In addition, aftereffects did not transfer from upright adaptors to inverted test images, but did transfer from inverted adaptors to upright test images in famous faces. However, in unfamiliar faces, a different pattern was observed. The results are interpreted in terms of how identity adaptation interacts with low-level adaptation and highlight differences in the representation of famous and unfamiliar faces.

Shared or separate mechanisms for self-face and other-face processing? Evidence from adaptation

Evidence that self-face recognition is dissociable from general face recognition has important implications both for models of social cognition and for our understanding of face recognition. In two studies, we examine how adaptation affects the perception of personally familiar faces, and we use a visual adaptation paradigm to investigate whether the neural mechanisms underlying the recognition of one’s own and other faces are shared or separate. In Study 1 we show that the representation of personally familiar faces is rapidly updated by visual experience with unfamiliar faces, so that the perception of one’s own face and a friend’s face is altered by a brief period of adaptation to distorted unfamiliar faces. In Study 2, participants adapted to images of their own and a friend’s face distorted in opposite directions; the contingent aftereffects we observe are indicative of separate neural populations, but we suggest that these reflect coding of facial identity rather than of the categories “self” and “other.”

Developmental prosopagnosia and super-recognition: no special role for surface reflectance processing

Face recognition by normal subjects depends in roughly equal proportions on shape and surface reflectance cues, while object recognition depends predominantly on shape cues. It is possible that developmental prosopagnosics are deficient not in their ability to recognize faces per se, but rather in their ability to use reflectance cues. Similarly, super-recognizers' exceptional ability with face recognition may be a result of superior surface reflectance perception and memory. We tested this possibility by administering tests of face perception and face recognition in which only shape or reflectance cues are available to developmental prosopagnosics, super-recognizers, and control subjects. Face recognition ability and the relative use of shape and pigmentation were unrelated in all the tests. Subjects who were better at using shape or reflectance cues were also better at using the other type of cue. These results do not support the proposal that variation in surface reflectance perception ability is the underlying cause of variation in face recognition ability. Instead, these findings support the idea that face recognition ability is related to neural circuits using representations that integrate shape and pigmentation information.

The role of skin texture and facial shape in representations of age and identity

Faces have both shape and skin texture, but the relative importance of the two in face representations is unclear. Our goals were first, to determine the contribution of shape versus texture to aftereffects for facial age and identity and second, to assess whether adaptation transferred between shape and texture, suggesting integration in a single representation. In our first experiment we examined age aftereffects. We obtained young and old images of two celebrities and created hybrid images, one combining the structure of the old face with the skin texture of the young face, the other combining the young structure with the old skin texture. This allowed us to create adaptation contrasts where the two adapting faces had the same facial structure but different skin texture, and vice versa. In the second experiment, we performed a similar study but this time examining identity aftereffects between two people of a similar age. We found that both skin texture and facial shape generated significant age aftereffects, but the contribution was greater from texture than from shape. Both texture and shape also generated significant identity aftereffects, but the contribution was greater from shape than from texture. In the last experiment, we used the normal and hybrid images to determine if adaptation to one property (i.e., texture) could create aftereffects in the perception of age in the other property (i.e., shape). While there was significant within-component adaptation for texture and shape, there was no evidence of cross-component adaptation. We conclude that shape and texture contribute differently to different face representations, with texture dominating for age. The lack of cross-component adaptation transfer suggests independent encoding of shape and texture, at least for age representations.

Visual adaptation and face perception

The appearance of faces can be strongly affected by the characteristics of faces viewed previously. These perceptual after-effects reflect processes of sensory adaptation that are found throughout the visual system, but which have been considered only relatively recently in the context of higher level perceptual judgements. In this review, we explore the consequences of adaptation for human face perception, and the implications of adaptation for understanding the neural-coding schemes underlying the visual representation of faces. The properties of face after-effects suggest that they, in part, reflect response changes at high and possibly face-specific levels of visual processing. Yet, the form of the after-effects and the norm-based codes that they point to show many parallels with the adaptations and functional organization that are thought to underlie the encoding of perceptual attributes like colour. The nature and basis for human colour vision have been studied extensively, and we draw on ideas and principles that have been developed to account for norms and normalization in colour vision to consider potential similarities and differences in the representation and adaptation of faces.

Adaptation and visual coding

Visual coding is a highly dynamic process and continuously adapting to the current viewing context. The perceptual changes that result from adaptation to recently viewed stimuli remain a powerful and popular tool for analyzing sensory mechanisms and plasticity. Over the last decade, the footprints of this adaptation have been tracked to both higher and lower levels of the visual pathway and over a wider range of timescales, revealing that visual processing is much more adaptable than previously thought. This work has also revealed that the pattern of aftereffects is similar across many stimulus dimensions, pointing to common coding principles in which adaptation plays a central role. However, why visual coding adapts has yet to be fully answered.

Adaptation and the perception of facial age

We examined how the perceived age of adult faces is affected by adaptation to younger or older adult faces. Observers viewed images of a synthetic male face simulating ageing over a modelled range from 15 to 65 years. Age was varied by changing shape cues or textural cues. Age level was varied in a staircase to find the observer's subjective category boundary between "old" and "young". These boundaries were strongly biased by adaptation to the young or old face, with significant aftereffects induced by either shape or textural cues. A further experiment demonstrated comparable aftereffects for photorealistic images of average older or younger adult faces, and found that aftereffects showed some selectivity for a change in gender but also strongly transferred across gender. This transfer shows that adaptation can adjust to the attribute of age somewhat independently of other facial attributes. These findings suggest that perceived age, like many other natural facial dimensions, is highly susceptible to adaptation, and that this adaptation can be carried by both the structural and textural changes that normally accompany facial ageing.

Identity-specific face adaptation effects: evidence for abstractive face representations

The effects of selective adaptation on familiar face perception were examined. After prolonged exposure to photographs of a celebrity, participants saw a series of ambiguous morphs that were varying mixtures between the face of that person and a different celebrity. Participants judged fewer of the morphs to resemble the celebrity to which they had been adapted, implying that they were now less sensitive to that particular face. Similar results were obtained when the adapting faces were highly dissimilar in viewpoint to the test morphs; when they were presented upside-down; or when they were vertically stretched to three times their normal height. These effects rule out explanations of adaptation effects solely in terms of low-level image-based adaptation. Instead they are consistent with the idea that relatively viewpoint-independent, person-specific adaptation occurred, at the level of either the "Face Recognition Units" or "Person Identity Nodes" in Burton, Bruce and Johnston's (1990) model of face recognition.

Adaptation improves discrimination of face identity

Whether face adaptation confers any advantages to perceptual processing remains an open question. We investigated whether face adaptation can enhance the ability to make fine discriminations in the vicinity of the adapted face. We compared face discrimination thresholds in three adapting conditions: (i) same-face: where adapting and test faces were the same, (ii) different-face: where adapting and test faces differed, and (iii) baseline: where the adapting stimulus was a blank. Discrimination thresholds for morphed identity changes involving the adapted face (same-face) improved compared with those from both the baseline (no-adaptation) and different-face conditions. Since adapting to a face did not alter discrimination performance for other faces, this effect is selective for the facial identity that is adapted. These results indicate a form of gain control to heighten perceptual sensitivity in the vicinity of a currently viewed face, analogous to forms of adaptive gain control at lower levels of the visual system.