Orientation-sensitivity of face identity aftereffects

Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling?

Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.

Neural timing of the other-race effect across the lifespan: A review

Face race influences the way we process faces, so that faces of a different ethnic group are processed for identity less efficiently than faces of one's ethnic group - a phenomenon known as the Other-Race Effect (ORE). Although widely replicated, the ORE is still poorly characterized in terms of its development and the underlying mechanisms. In the last two decades, the Event-Related Potential (ERP) technique has brought insight into the mechanisms underlying the ORE and has demonstrated potential to clarify its development. Here, we review the ERP evidence for a differential neural processing of own-race and other-race faces throughout the lifespan. In infants, race-related processing differences emerged at the N290 and P400 (structural encoding) stages. In children, race affected the P100 (early processing, attention) perceptual stage and was implicitly encoded at the N400 (semantic processing) stage. In adults, processing difficulties for other-race faces emerged at the N170 (structural encoding), P200 (configuration processing) and N250 (accessing individual representations) perceptual stages. Early in processing, race was implicitly encoded from other-race faces (N100, P200 attentional biases) and in-depth processing preferentially applied to own-race faces (N200 attentional bias). Encoding appeared less efficient (Dm effects) and retrieval less recollection-based (old/new effects) for other-race faces. Evidence admits the contribution of perceptual, attentional, and motivational processes to the development and functioning of the ORE, offering no conclusive support for perceptual or socio-cognitive accounts. Cross-racial and non-cross-racial studies provided convergent evidence. Future research would need to include less represented ethnic populations and the developmental population.

Serial dependence of facial identity reflects high-level face coding

Serial dependence of facial identity is a type of bias where the perceived identity of a face is biased towards a previously presented face. There are individual differences in serial dependence strength and tuning (how the strength varies depending on stimuli similarity), and previous research has shown that both stronger and more narrowly tuned serial dependence of facial identity is associated with better face recognition abilities. These results are consistent with the idea that this bias plays a functional role in face perception. It is important, therefore, to determine whether serial dependence of facial identity reflects a high-level face-coding mechanism acting on the identity of a face or instead predominantly reflects a bias in low-level features, which are also subject to serial dependence. We first sought evidence that serial dependence of facial identity survived changes in low-level visual features, by varying face viewpoint between successive stimuli. We found that serial dependence persisted across changes in viewpoint, arguing against an entirely low-level locus for this bias. We next tested whether the bias was affected by inversion, as sensitivity to inversion is argued to be a characteristic of high-level face-selective processing. Serial dependence was stronger and more narrowly tuned for upright than inverted faces. Taken together, our results are consistent with the view that serial dependence of facial identity affects high-level visual representations and may reflect a face-coding mechanism that is operating at the level of facial identity.

Individual Differences in Serial Dependence of Facial Identity are Associated with Face Recognition Abilities

Serial dependence is a perceptual bias where current perception is biased towards prior visual input. This bias occurs when perceiving visual attributes, such as facial identity, and has been argued to play an important functional role in vision, stabilising the perception of objects through integration. In face identity recognition, this bias could assist in building stable representations of facial identity. If so, then individual variation in serial dependence could contribute to face recognition ability. To investigate this possibility, we measured both the strength of serial dependence and the range over which individuals showed this bias (the tuning) in 219 adults, using a new measure of serial dependence of facial identity. We found that better face recognition was associated with stronger serial dependence and narrower tuning, that is, showing serial dependence primarily when sequential faces were highly similar. Serial dependence tuning was further found to be a significant predictor of face recognition abilities independently of both object recognition and face identity aftereffects. These findings suggest that the extent to which serial dependence is used selectively for similar faces is important to face recognition. Our results are consistent with the view that serial dependence plays a functional role in face recognition.

Rapid Adaptation to the Timbre of Natural Sounds

Timbre, the unique quality of a sound that points to its source, allows us to quickly identify a loved one's voice in a crowd and distinguish a buzzy, bright trumpet from a warm cello. Despite its importance for perceiving the richness of auditory objects, timbre is a relatively poorly understood feature of sounds. Here we demonstrate for the first time that listeners adapt to the timbre of a wide variety of natural sounds. For each of several sound classes, participants were repeatedly exposed to two sounds (e.g., clarinet and oboe, male and female voice) that formed the endpoints of a morphed continuum. Adaptation to timbre resulted in consistent perceptual aftereffects, such that hearing sound A significantly altered perception of a neutral morph between A and B, making it sound more like B. Furthermore, these aftereffects were robust to moderate pitch changes, suggesting that adaptation to timbral features used for object identification drives these effects, analogous to face adaptation in vision.

The high-level basis of body adaptation

Prolonged visual exposure, or 'adaptation', to thin (wide) bodies causes a perceptual aftereffect such that subsequently seen bodies appear wider (thinner) than they actually are. Here, we conducted two experiments investigating the effect of rotating the orientation of the test stimuli by 90° from that of the adaptor. Aftereffects were maximal when adapting and test bodies had the same orientation. When they differed, the axis of the perceived distortion changed with the orientation of the body. Experiment 1 demonstrated a 58% transfer of the aftereffect across orientations. Experiment 2 demonstrated an even greater degree of aftereffect transfer when the influence of low-level mechanisms was reduced further by using adaptation and test stimuli with different sizes. These results indicate that the body aftereffect is mediated primarily by high-level object-based processes, with low-level retinotopic mechanisms playing only a minor role. The influence of these low-level processes is further reduced when test stimuli differ in size from adaptation stimuli.

Adaptive Coding and Face Recognition

Face adaptation generates striking face aftereffects, but is this adaptation useful? The answer appears to be yes, with several lines of evidence suggesting that it contributes to our face-recognition ability. Adaptation to face identity is reduced in a variety of clinical populations with impaired face recognition. In addition, individual differences in face adaptation are linked to face-recognition ability in typical adults. People who adapt more readily to new faces are better at recognizing faces. This link between adaptation and recognition holds for both identity and expression recognition. Adaptation updates face norms, which represent the typical or average properties of the faces we experience. By using these norms to code how faces differ from average, the visual system can make explicit the distinctive information that we need to recognize faces. Thus, adaptive norm-based coding may help us to discriminate and recognize faces despite their similarity as visual patterns.

Visual adaptation reveals an objective electrophysiological measure of high-level individual face discrimination

The ability to individualize faces is a fundamental human brain function. Following visual adaptation to one individual face, the suppressed neural response to this identity becomes discriminable from an unadapted facial identity at a neural population level. Here, we investigate a simple and objective measure of individual face discrimination with electroencephalographic (EEG) frequency tagging following adaptation. In a first condition, (1) two facial identities are presented in alternation at a rate of six images per second (6 Hz; 3 Hz identity repetition rate) for a 20 s testing sequence, following 10-s adaptation to one of the facial identities; this results in a significant identity discrimination response at 3 Hz in the frequency domain of the EEG over right occipito-temporal channels, replicating our previous findings. Such a 3 Hz response is absent for two novel conditions, in which (2) the faces are inverted and (3) an identity physically equidistant from the two faces is adapted. These results indicate that low-level visual features present in inverted or unspecific facial identities are not sufficient to produce the adaptation effect found for upright facial stimuli, which appears to truly reflect identity-specific perceptual representations in the human brain.

Saccadic remapping of object-selective information

Saccadic remapping, a presaccadic increase in neural activity when a saccade is about to bring an object into a neuron's receptive field, may be crucial for our perception of a stable world. Studies of perception and saccadic remapping, like ours, focus on the presaccadic acquisition of information from the saccade target, with no direct reference to underlying physiology. While information is known to be acquired prior to a saccade, it is unclear whether object-selective or feature-specific information is remapped. To test this, we performed a series of psychophysical experiments in which we presented a peripheral, nonfoveated face as a presaccadic target. The target face disappeared at saccade onset. After making a saccade to the location of the peripheral target face (which was no longer visible), subjects misperceived the expression of a subsequent, foveally presented neutral face as being repelled away from the peripheral presaccadic face target. This effect was similar to a sequential shape contrast or negative aftereffect but required a saccade, because covert attention was not sufficient to generate the illusion. Additional experiments further revealed that inverting the faces disrupted the illusion, suggesting that presaccadic remapping is object-selective and not based on low-level features. Our results demonstrate that saccadic remapping can be an object-selective process, spatially tuned to the target of the saccade and distinct from covert attention in the absence of a saccade.

Reduced adaptability, but no fundamental disruption, of norm-based face coding following early visual deprivation from congenital cataracts

Faces are adaptively coded relative to visual norms that are updated by experience, and this adaptive coding is linked to face recognition ability. Here we investigated whether adaptive coding of faces is disrupted in individuals (adolescents and adults) who experience face recognition difficulties following visual deprivation from congenital cataracts in infancy. We measured adaptive coding using face identity aftereffects, where smaller aftereffects indicate less adaptive updating of face-coding mechanisms by experience. We also examined whether the aftereffects increase with adaptor identity strength, consistent with norm-based coding of identity, as in typical populations, or whether they show a different pattern indicating some more fundamental disruption of face-coding mechanisms. Cataract-reversal patients showed significantly smaller face identity aftereffects than did controls (Experiments 1 and 2). However, their aftereffects increased significantly with adaptor strength, consistent with norm-based coding (Experiment 2). Thus we found reduced adaptability but no fundamental disruption of norm-based face-coding mechanisms in cataract-reversal patients. Our results suggest that early visual experience is important for the normal development of adaptive face-coding mechanisms.

Reduced adaptability, but no fundamental disruption, of norm-based face-coding mechanisms in cognitively able children and adolescents with autism

Faces are adaptively coded relative to visual norms that are updated by experience. This coding is compromised in autism and the broader autism phenotype, suggesting that atypical adaptive coding of faces may be an endophenotype for autism. Here we investigate the nature of this atypicality, asking whether adaptive face-coding mechanisms are fundamentally altered, or simply less responsive to experience, in autism. We measured adaptive coding, using face identity aftereffects, in cognitively able children and adolescents with autism and neurotypical age- and ability-matched participants. We asked whether these aftereffects increase with adaptor identity strength as in neurotypical populations, or whether they show a different pattern indicating a more fundamental alteration in face-coding mechanisms. As expected, face identity aftereffects were reduced in the autism group, but they nevertheless increased with adaptor strength, like those of our neurotypical participants, consistent with norm-based coding of face identity. Moreover, their aftereffects correlated positively with face recognition ability, consistent with an intact functional role for adaptive coding in face recognition ability. We conclude that adaptive norm-based face-coding mechanisms are basically intact in autism, but are less readily calibrated by experience.

Decision-dependent aftereffects for faces

Adaptation-related aftereffects (AEs) have been found in the perception of face identity, in that perception of an ambiguous face is typically biased away from the identity of a preceding unambiguous adaptor face. In previous studies, we could show that both perceptual ambiguity and physical similarity play a role in determining perceived face identity AEs, Cortex 49 (2013) 1963-1977, Plos One 8 (2013) e70525. Here, we tested further the role of ambiguity by manipulating participants' task such that the very same target stimuli were either ambiguous or unambiguous regarding stimulus classification. We created two partially overlapping continua spanning three unfamiliar face identities each, by morphing identity A via B to C, and B via C to D. In a first session, participants were familiarised with faces A and C and asked to classify faces of the A-B-C continuum as either identity A or C in an AE paradigm. Following adaptation to A or C, we observed contrastive AEs for the ambiguous identity B, but not for the unambiguous identities A or C. In a second session, the same participants were familiarised with faces B and D, followed by tests of AEs for the B-C-D continuum now involving a B-D classification task. We again observed contrastive AEs but only for target identity C (ambiguous for the decision) and not for B or D (unambiguous). Our results suggest that perceptual ambiguity, as given by the task-context, determines whether or not AEs are induced.

Face identity aftereffects increase monotonically with adaptor extremity over, but not beyond, the range of natural faces

Face identity aftereffects have been used to test theories of the neural coding underlying expert face recognition. Previous studies reported larger aftereffects for adaptors that are morphed further from the average face than for adaptors closer to the average, which appeared to support opponent coding along face-identity dimensions. However, only two levels were tested and it is not clear where they were located relative to the range of naturally occurring faces. This range is of interest given the functional need of the visual system both to produce good discrimination of real everyday faces and to process novel kinds of faces that we may encounter. Here, Experiment 1 establishes the boundary of faces judged as being able to occur in everyday life. Experiment 2 then shows that aftereffects increase with adaptor extremity up to this natural-range boundary, drop significantly immediately outside the boundary, and then remain stable with no drop towards zero even for highly distorted adaptors far beyond the boundary. Computational modelling shows that this unexpected pattern cannot be explained either by a simple opponent or by a classic multichannel model. However, its qualitative features can be captured either by a combination of opponent and multichannel coding (raising the possibility that not all identity-related face dimensions are opponent coded), or by a 3-pool model containing two S-shaped-response channels and a central bell-shaped channel around the average face (raising the possibility of unexpected similarities with coding of eye and head direction).

Seeing a haptically explored face: visual facial-expression aftereffect from haptic adaptation to a face

Current views on face perception assume that the visual system receives only visual facial signals. However, I show that the visual perception of faces is systematically biased by adaptation to a haptically explored face. Recently, face aftereffects (FAEs; the altered perception of faces after adaptation to a face) have been demonstrated not only in visual perception but also in haptic perception; therefore, I combined the two FAEs to examine whether the visual system receives face-related signals from the haptic modality. I found that adaptation to a haptically explored facial expression on a face mask produced a visual FAE for facial expression. This cross-modal FAE was not due to explicitly imaging a face, response bias, or adaptation to local features. Furthermore, FAEs transferred from vision to haptics. These results indicate that visual face processing depends on substrates adapted by haptic faces, which suggests that face processing relies on shared representation underlying cross-modal interactions.

Bodies adapt orientation-independent face representations

Faces and bodies share a great number of semantic attributes, such as gender, emotional expressiveness, and identity. Recent studies demonstrate that bodies can activate and modulate face perception. However, the nature of the face representation that is activated by bodies remains unknown. In particular, face and body representations have previously been shown to have a degree of orientation specificity. Here we use body-face adaptation aftereffects to test whether bodies activate face representations in an orientation-dependent manner. Specifically, we used a two-by-two design to examine the magnitude of the body-face aftereffect using upright and inverted body adaptors and upright and inverted face targets. All four conditions showed significant body-face adaptation. We found neither a main effect of body orientation nor an interaction between body and face orientation. There was a main effect of target face orientation, with inverted target faces showing larger aftereffects than upright target faces, consistent with traditional face-face adaptation. Taken together, these results suggest that bodies adapt and activate a relatively orientation-independent representation of faces.

Face adaptation effects: reviewing the impact of adapting information, time, and transfer

The ability to adapt is essential to live and survive in an ever-changing environment such as the human ecosystem. Here we review the literature on adaptation effects of face stimuli to give an overview of existing findings in this area, highlight gaps in its research literature, initiate new directions in face adaptation research, and help to design future adaptation studies. Furthermore, this review should lead to better understanding of the processing characteristics as well as the mental representations of face-relevant information. The review systematizes studies at a behavioral level in respect of a framework which includes three dimensions representing the major characteristics of studies in this field of research. These dimensions comprise (1) the specificity of adapting face information, e.g., identity, gender, or age aspects of the material to be adapted to (2) aspects of timing (e.g., the sustainability of adaptation effects) and (3) transfer relations between face images presented during adaptation and adaptation tests (e.g., images of the same or different identities). The review concludes with options for how to combine findings across different dimensions to demonstrate the relevance of our framework for future studies.

Atypical updating of face representations with experience in children with autism

Face identity aftereffects are significantly diminished in children with autism relative to typical children, which may reflect reduced perceptual updating with experience. Here, we investigated whether this atypicality also extends to non-face stimulus categories, which might signal a pervasive visual processing difference in individuals with autism. We used a figural aftereffect task to measure directly perceptual updating following exposure to distorted upright faces, inverted faces and cars, in typical children and children with autism. A size-change between study and test stimuli limited the likelihood that any processing atypicalities reflected group differences in adaptation to low-level features of the stimuli. Results indicated that, relative to typical children, figural aftereffects for upright faces, but not inverted faces or cars, were significantly attenuated in children with autism. Moreover, the group difference was amplified when we isolated the 'face-selective' component of the aftereffect, by partialling out the mid-level shape adaptation common to upright and inverted face stimuli. Notably, the aftereffects of typical children were disproportionately larger for upright faces than for inverted faces and cars, but the magnitude of aftereffects of autistic children was not similarly modulated according to stimulus category. These findings are inconsistent with a pervasive adaptive coding atypicality relative to typical children, and suggest that reduced perceptual updating may constitute a high-level, and possibly face-selective, visual processing difference in children with autism.

Haptic face aftereffect

Face aftereffects (FAEs) are generally thought of as being a visual phenomenon. However, recent studies have shown that people can haptically recognize a face. Here, I report a haptic, rather than visual, FAE. By using three-dimensional facemasks, I found that haptic exploration of the facial expression of the facemask causes a subsequently touched neutral facemask to be perceived as having the opposite facial expression. The results thus suggest that FAEs can also occur in haptic perception of faces.

Adaptive face space coding in congenital prosopagnosia: typical figural aftereffects but abnormal identity aftereffects

People with congenital prosopagnosia (CP) report difficulty recognising faces in everyday life and perform poorly on face recognition tests. Here, we investigate whether impaired adaptive face space coding might contribute to poor face recognition in CP. To pinpoint how adaptation may affect face processing, a group of CPs and matched controls completed two complementary face adaptation tasks: the figural aftereffect, which reflects adaptation to general distortions of shape, and the identity aftereffect, which directly taps the mechanisms involved in the discrimination of different face identities. CPs displayed a typical figural aftereffect, consistent with evidence that they are able to process some shape-based information from faces, e.g., cues to discriminate sex. CPs also demonstrated a significant identity aftereffect. However, unlike controls, CPs impression of the identity of the neutral average face was not significantly shifted by adaptation, suggesting that adaptive coding of identity is abnormal in CP. In sum, CPs show reduced aftereffects but only when the task directly taps the use of face norms used to code individual identity. This finding of a reduced face identity aftereffect in individuals with severe face recognition problems is consistent with suggestions that adaptive coding may have a functional role in face recognition.

Enhanced attention amplifies face adaptation

Perceptual adaptation not only produces striking perceptual aftereffects, but also enhances coding efficiency and discrimination by calibrating coding mechanisms to prevailing inputs. Attention to simple stimuli increases adaptation, potentially enhancing its functional benefits. Here we show that attention also increases adaptation to faces. In Experiment 1, face identity aftereffects increased when attention to adapting faces was increased using a change detection task. In Experiment 2, figural (distortion) face aftereffects increased when attention was increased using a snap game (detecting immediate repeats) during adaptation. Both were large effects. Contributions of low-level adaptation were reduced using free viewing (both experiments) and a size change between adapt and test faces (Experiment 2). We suggest that attention may enhance adaptation throughout the entire cortical visual pathway, with functional benefits well beyond the immediate advantages of selective processing of potentially important stimuli. These results highlight the potential to facilitate adaptive updating of face-coding mechanisms by strategic deployment of attentional resources.

Visual adaptation and face perception

The appearance of faces can be strongly affected by the characteristics of faces viewed previously. These perceptual after-effects reflect processes of sensory adaptation that are found throughout the visual system, but which have been considered only relatively recently in the context of higher level perceptual judgements. In this review, we explore the consequences of adaptation for human face perception, and the implications of adaptation for understanding the neural-coding schemes underlying the visual representation of faces. The properties of face after-effects suggest that they, in part, reflect response changes at high and possibly face-specific levels of visual processing. Yet, the form of the after-effects and the norm-based codes that they point to show many parallels with the adaptations and functional organization that are thought to underlie the encoding of perceptual attributes like colour. The nature and basis for human colour vision have been studied extensively, and we draw on ideas and principles that have been developed to account for norms and normalization in colour vision to consider potential similarities and differences in the representation and adaptation of faces.

The nature of upright and inverted face representations: an adaptation-transfer study of configuration

It is considered that whole-face processing of spatial structure may only be possible in upright faces, with only local feature processing in inverted faces. We asked whether this was due to impoverished representations of inverted faces. We performed two experiments. In the first, we divided faces into segments to create 'exploded' faces with disrupted second-order structures, and 'scrambled' faces with altered first-order relations; in the second we shifted features within intact facial outlines to create equivalent disruptions of spatial structure. In both we assessed the transfer of adaptation between faces with altered structure and intact faces. Scrambled adaptors did not adapt upright or inverted intact faces, indicating that a whole-face configuration is required at either orientation. Both upright and inverted faces showed a similar decline in aftereffect magnitude when adapting faces had altered second-order structure, implying that this structure is present in both upright and inverted face representations. We conclude that inverted faces are not represented simply as a collection of features, but have a whole-face configuration with second-order structure, similar to upright faces. Thus the qualitative impairments induced by inversion are not due to degraded inverted facial representations, but may reflect limitations in perceptual mechanisms.

Perceptual adaptation helps us identify faces

Adaptation is a fundamental property of perceptual processing. In low-level vision, it can calibrate perception to current inputs, increasing coding efficiency and enhancing discrimination around the adapted level. Adaptation also occurs in high-level vision, as illustrated by face aftereffects. However, the functional consequences of face adaptation remain uncertain. Here we investigated whether adaptation can enhance identification performance for faces from an adapted, relative to an unadapted, population. Five minutes of adaptation to an average Asian or Caucasian face reduced identification thresholds for faces from the adapted relative to the unadapted race. We replicated this interaction in two studies, using different participants, faces and adapting procedures. These results suggest that adaptation has a functional role in high-level, as well as low-level, visual processing. We suggest that adaptation to the average of a population may reduce responses to common properties shared by all members of the population, effectively orthogonalizing identity vectors in a multi-dimensional face space and freeing neural resources to code distinctive properties, which are useful for identification.