Opposite Aftereffects for Chinese and Caucasian Faces are Selective for Social Category Information and not Just Physical Face Differences

Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency

While the human perceptual system constantly adapts to the environment, some of the underlying mechanisms are still poorly understood. For instance, although previous research demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to these effects is unclear. In two experiments, we used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Participants adapted to voices containing emotion-specific information in either F0 or timbre, with all other parameters kept constant at an intermediate 50% morph level. Full emotional voices and ambiguous voices were used as reference conditions. All adaptor stimuli were either of the same (Experiment 1) or opposite speaker gender (Experiment 2) of subsequently presented target voices. In Experiment 1, we found consistent aftereffects in all adaptation conditions. Crucially, aftereffects following timbre adaptation were much larger than following F0 adaptation and were only marginally smaller than those following full adaptation. In Experiment 2, adaptation aftereffects appeared massively and proportionally reduced, with differences between morph types being no longer significant. These results suggest that timbre plays a larger role than F0 in vocal emotion adaptation, and that vocal emotion adaptation is compromised by eliminating gender-correspondence between adaptor and target stimuli. Our findings also add to mounting evidence suggesting a major role of timbre in auditory adaptation.

Serial dependence of facial identity for own- and other-race faces

It is well established that individuals are better at recognising faces of their own-race compared with other-races; however, there is ongoing debate regarding the perceptual mechanisms that may be involved and therefore sensitive to face-race. Here, we ask whether serial dependence of facial identity, a bias where the perception of a face's identity is biased towards a previously presented face, shows an other-race effect. Serial dependence is associated with face recognition ability and appears to operate on high-level, face-selective representations, like other candidate mechanisms (e.g., holistic processing). We therefore expected to find an other-race effect for serial dependence for our Caucasian and Asian participants. While participants showed robust effects of serial dependence for all faces, only Caucasian participants showed stronger serial dependence for own-race faces. Intriguingly, we found that individual variation in own-race, but not other-race, serial dependence was significantly associated with face recognition abilities. Preliminary evidence also suggested that other-race contact is associated with other-race serial dependence. In conclusion, though we did not find an overall difference in serial dependence for own- versus other-race faces in both participant groups, our results highlight that this bias may be functionally different for own- versus other-race faces and sensitive to racial experience.

The Role of Racial and Developmental Experience on Emotional Adaptive Coding in Autism Spectrum Disorder

Sensitivity to emotional face aids in rapid detection and evaluation of others, such that by school-age, children and youth exhibit adult-like patterns when the prolonged viewing of an emotional face distorts the perception of a subsequent face. However, the developmental considerations of this phenomenon (known as emotional adaptive coding) are unclear given ongoing maturational and experiential changes, including the influence of own-race experiences or the lack of face expertise, as is evident in autism spectrum disorder (ASD). This study addressed whether emotional adaptive coding is sensitive to factors of face perception expertise, specifically self-race and developmental experience, in adults (age 19-28 years) and youth (age 10-16 years). Emotional adaptive coding was not influenced by race expertise (i.e., other versus same race identity) in White and Asian adults. Emotional adaptation coding during childhood and adolescence is consistent with adults, though youth with ASD exhibited stronger adaptor after-effects in response to other-race faces, relative to TD youth and adults. By extending prior work to examine the integration of race and emotional adaptive coding in ASD, we discovered that the strength of response in ASD is atypical when viewing other-race faces, which clarifies the role of racial and facial experience on emotional face adaption.

Asymmetric neural responses for facial expressions and anti-expressions

Face recognition requires identifying both the invariant characteristics that distinguish one individual from another and the variations within the individual that correspond to emotional expressions. Both have been postulated to be represented via a norm-based code, in which identity or expression are represented as deviations from an average or neutral prototype. We used Fast Periodic Visual Stimulation (FPVS) with electroencephalography (EEG) to compare neural responses for neutral faces, expressions and anti-expressions. Anti-expressions are created by projecting an expression (e.g. a happy face) through the neutral face to form the opposite facial shape (anti-happy). Thus expressions and anti-expressions differ from the norm by the same "configural" amount and thus have equivalent but opposite status with regard to their shape, but differ in their ecological validity. We examined whether neural responses to these complementary stimulus pairs were equivalent or asymmetric, and also tested for norm-based coding by comparing whether stronger responses are elicited by expressions and anti-expressions than neutral faces. Observers viewed 20 s sequences of 6 Hz alternations of neutral faces and expressions, neutral faces and anti-expressions, and expressions and anti-expressions. Responses were analyzed in the frequency domain. Significant responses at half the frequency of the presentation rate (3 Hz), indicating asymmetries in responses, were observed for all conditions. Inversion of the images reduced the size of this signal, indicating these asymmetries are not solely due to differences in the low-level properties of the images. While our results do not preclude a norm-based code for expressions, similar to identity, this representation (as measured by the FPVS EEG responses) may also include components sensitive to which configural distortions form meaningful expressions.

Social Categories Alone Are Insufficient to Elicit an In-Group Advantage in Perceptions of Within-Person Variability

Within-person variability affects identity perception of other-race faces more than own-race faces; when participants sort images into piles representing different identities, they sort photographs of two other-race identities into more piles than two own-race identities. These results have been interpreted in terms of perceptual expertise, such that lack of experience with other-race faces leads to reduced ability to extract identity-relevant information across images. However, an alternative explanation is that sociocognitive factors (e.g., cognitive disregard for out-group faces) lead to differences in the number of perceived identities. Here, we examined whether social factors alone elicit an in-group advantage in perceptions of within-person variability. Caucasian participants sorted 40 photographs of two unfamiliar Caucasian identities (20 photographs/model) into piles based on the number of identities they believed were present. Half of the participants were told that the images were of students attending their university (in-group), whereas half were told that the images were of students attending a rival university (out-group). Participants sorted the photographs into a comparable number of identities for in- and out-group faces. This lack of an in-group advantage suggests that sociocognitive factors alone cannot account for differences in the number of perceived identities across faces from two categories.

Bidirectional Gender Face Aftereffects: Evidence Against Normative Facial Coding

Facial appearance can be altered, not just by restyling but also by sensory processes. Exposure to a female face can, for instance, make subsequent faces look more masculine than they would otherwise. Two explanations exist. According to one, exposure to a female face renormalizes face perception, making that female and all other faces look more masculine as a consequence—a unidirectional effect. According to that explanation, exposure to a male face would have the opposite unidirectional effect. Another suggestion is that face gender is subject to contrastive aftereffects. These should make some faces look more masculine than the adaptor and other faces more feminine—a bidirectional effect. Here, we show that face gender aftereffects are bidirectional, as predicted by the latter hypothesis. Images of real faces rated as more and less masculine than adaptors at baseline tended to look even more and less masculine than adaptors post adaptation. This suggests that, rather than mental representations of all faces being recalibrated to better reflect the prevailing statistics of the environment, mental operations exaggerate differences between successive faces, and this can impact facial gender perception.

Adults with autism spectrum disorder show evidence of figural aftereffects with male and female faces

The norm-based coding model of face perception posits that face perception involves an implicit comparison of observed faces to a representation of an average face (prototype) that is shaped by experience. Using some methods, observers with autism spectrum disorder (ASD) have shown atypical face perception, but other methods suggest preserved face perception. Here, we used a figural aftereffects paradigm to test whether adults with ASD showed evidence of norm-based coding of faces, and whether they encode separate prototypes for male and female faces, as typical observers do. Following prolonged exposure to distorted faces that differ from their stored prototype, neurotypical adults show aftereffects: their prototype shifts in the direction of the adapting face. We measured aftereffects following adaptation to one distorted gender. There were no significant group differences in the size or direction of the aftereffects; both groups showed sex-selective aftereffects after adapting to expanded female faces but showed aftereffects for both sexes after adapting to contracted face of either sex, demonstrating that adults with and without ASD show evidence of partially dissociable male and female face prototypes. This is the first study to examine sex-selective prototypes using figural aftereffects in adults with ASD and replicates the findings of previous studies examining aftereffects in adults with ASD. The results contrast with studies reporting diminished adaptation in children with ASD.

Category-specific face prototypes are emerging, but not yet mature, in 5-year-old children

Adults' expertise in face recognition has been attributed to norm-based coding. Moreover, adults possess separable norms for a variety of face categories (e.g., race, sex, age) that appear to enhance recognition by reducing redundancy in the information shared by faces and ensuring that only relevant dimensions are used to encode faces from a given category. Although 5-year-old children process own-race faces using norm-based coding, little is known about the organization and refinement of their face space. The current study investigated whether 5-year-olds rely on category-specific norms and whether experience facilitates the development of dissociable face prototypes. In Experiment 1, we examined whether Chinese 5-year-olds show race-contingent opposing aftereffects and the extent to which aftereffects transfer across face race among Caucasian and Chinese 5-year-olds. Both participant races showed partial transfer of aftereffects across face race; however, there was no evidence for race-contingent opposing aftereffects. To examine whether experience facilitates the development of category-specific prototypes, we investigated whether race-contingent aftereffects are present among Caucasian 5-year-olds with abundant exposure to Chinese faces (Experiment 2) and then tested separate groups of 5-year-olds with two other categories with which they have considerable experience: sex (male/female faces) and age (adult/child faces) (Experiment 3). Across all three categories, 5-year-olds showed no category-contingent opposing aftereffects. These results demonstrate that 5 years of age is a stage characterized by minimal separation in the norms and associated coding dimensions used for faces from different categories and suggest that refinement of the mechanisms that underlie expert face processing occurs throughout childhood.

Adaptor identity modulates adaptation effects in familiar face identification and their neural correlates

Adaptation-related aftereffects (AEs) show how face perception can be altered by recent perceptual experiences. Along with contrastive behavioural biases, modulations of the early event-related potentials (ERPs) were typically reported on categorical levels. Nevertheless, the role of the adaptor stimulus per se for face identity-specific AEs is not completely understood and was therefore investigated in the present study. Participants were adapted to faces (S1s) varying systematically on a morphing continuum between pairs of famous identities (identities A and B), or to Fourier phase-randomized faces, and had to match the subsequently presented ambiguous faces (S2s; 50/50% identity A/B) to one of the respective original faces. We found that S1s identical with or near to the original identities led to strong contrastive biases with more identity B responses following A adaptation and vice versa. In addition, the closer S1s were to the 50/50% S2 on the morphing continuum, the smaller the magnitude of the AE was. The relation between S1s and AE was, however, not linear. Additionally, stronger AEs were accompanied by faster reaction times. Analyses of the simultaneously recorded ERPs revealed categorical adaptation effects starting at 100 ms post-stimulus onset, that were most pronounced at around 125-240 ms for occipito-temporal sites over both hemispheres. S1-specific amplitude modulations were found at around 300-400 ms. Response-specific analyses of ERPs showed reduced voltages starting at around 125 ms when the S1 biased perception in a contrastive way as compared to when it did not. Our results suggest that face identity AEs do not only depend on physical differences between S1 and S2, but also on perceptual factors, such as the ambiguity of S1. Furthermore, short-term plasticity of face identity processing might work in parallel to object-category processing, and is reflected in the first 400 ms of the ERP.

Face adaptation effects: reviewing the impact of adapting information, time, and transfer

The ability to adapt is essential to live and survive in an ever-changing environment such as the human ecosystem. Here we review the literature on adaptation effects of face stimuli to give an overview of existing findings in this area, highlight gaps in its research literature, initiate new directions in face adaptation research, and help to design future adaptation studies. Furthermore, this review should lead to better understanding of the processing characteristics as well as the mental representations of face-relevant information. The review systematizes studies at a behavioral level in respect of a framework which includes three dimensions representing the major characteristics of studies in this field of research. These dimensions comprise (1) the specificity of adapting face information, e.g., identity, gender, or age aspects of the material to be adapted to (2) aspects of timing (e.g., the sustainability of adaptation effects) and (3) transfer relations between face images presented during adaptation and adaptation tests (e.g., images of the same or different identities). The review concludes with options for how to combine findings across different dimensions to demonstrate the relevance of our framework for future studies.

Ignored faces produce figural face aftereffects

Following adaptation to faces with contracted (or expanded) internal features, faces previously perceived as normal appear distorted in the opposite direction. This figural face aftereffect suggests face-coding mechanisms adapt to changes in the spatial relations of features and/or the global structure of faces. Here, we investigated whether the figural aftereffect requires spatial attention. Participants ignored a distorted adapting face and performed a highly demanding letter-count task. Before and after adaptation, participants rated the normality of morphed distorted faces ranging from 50% contracted through undistorted to 50% expanded. A robust aftereffect was observed. These results suggest that the figural face aftereffect can occur in the absence of spatial attention, even when the attentional demands of the relevant task are high.

Neural correlates of after-effects caused by adaptation to multiple face displays

Adaptation to a given face leads to face-related, specific after-effects. Recently, this topic has attracted a lot of attention because it clearly shows that adaptation occurs even at the higher stages of visual cortical processing. However, during our every-day life, faces do not appear in isolation, rather they are usually surrounded by other stimuli. Here, we used psychophysical and fMRI adaptation methods to test whether humans adapt to the gender properties of a composite multiple face stimulus as well. As adaptors we used stimuli composed of eight different individual faces, positioned peripherally in a ring around a fixation mark. We found that the gender discrimination of a subsequent centrally presented target face is significantly biased as a result of long-term adaptation to either male or female multiple face stimuli. Similar to our previous results with single-face adaptors (Kovács et al. in Neuroimage 43(1):156-164, 2008), a concurrent functional magnetic resonance imaging adaptation experiment revealed the strongest blood oxygen level-dependent signal adaptation bilaterally in the fusiform face area. Our results suggest that humans extract the statistical features of the multiple face stimulus and this process occurs at the level of occipito-temporal face processing.

Shared or separate mechanisms for self-face and other-face processing? Evidence from adaptation

Evidence that self-face recognition is dissociable from general face recognition has important implications both for models of social cognition and for our understanding of face recognition. In two studies, we examine how adaptation affects the perception of personally familiar faces, and we use a visual adaptation paradigm to investigate whether the neural mechanisms underlying the recognition of one’s own and other faces are shared or separate. In Study 1 we show that the representation of personally familiar faces is rapidly updated by visual experience with unfamiliar faces, so that the perception of one’s own face and a friend’s face is altered by a brief period of adaptation to distorted unfamiliar faces. In Study 2, participants adapted to images of their own and a friend’s face distorted in opposite directions; the contingent aftereffects we observe are indicative of separate neural populations, but we suggest that these reflect coding of facial identity rather than of the categories “self” and “other.”

A critical review of the development of face recognition: experience is less important than previously believed

Historically, it has been argued that face individuation develops very slowly, not reaching adult levels until adolescence, with experience being the driving force behind this protracted improvement. Here, we challenge this view based on extensive review of behavioural and neural findings. Results demonstrate qualitative presence of all key phenomena related to face individuation (encoding of novel faces, holistic processing effects, face-space effects, face-selective responses in neuroimaging) at the earliest ages tested, typically 3-5 years of age and in many cases even infancy. Results further argue for quantitative maturity by early childhood, based on an increasing number of behavioural studies that have avoided the common methodological problem of restriction of range, as well as event-related potential (ERP), but not functional magnetic resonance imaging (fMRI) studies. We raise a new possibility that could account for the discrepant fMRI findings-namely, the use of adult-sized head coils on child-sized heads. We review genetic and innate contributions to face individuation (twin studies, neonates, visually deprived monkeys, critical periods, perceptual narrowing). We conclude that the role of experience in the development of the mechanisms of face identification has been overestimated. The emerging picture is that the mechanisms supporting face individuation are mature early, consistent with the social needs of children for reliable person identification in everyday life, and are also driven to an important extent by our evolutionary history.

Selectivity of face aftereffects for expressions and anti-expressions

Adapting to a facial expression can alter the perceived expression of subsequently viewed faces. However, it remains unclear whether this adaptation affects each expression independently or transfers from one expression to another, and whether this transfer impedes or enhances responses to a different expression. To test for these interactions, we probed the basic expressions of anger, fear, happiness, sadness, surprise, and disgust, adapting to one expression and then testing on all six. Each expression was varied in strength by morphing it with a common neutral facial expression. Observers determined the threshold level required to correctly identify each expression, before or after adapting to a face with a neutral or intense expression. The adaptation was strongly selective for the adapting category; responses to the adapting expression were reduced, while other categories showed little consistent evidence of either suppression or facilitation. In a second experiment we instead compared adaptation to each expression and its anti-expression. The latter are defined by the physically complementary facial configuration, yet appear much more ambiguous as expressions. In this case, for most expressions the opposing faces produced aftereffects of opposite sign in the perceived expression. These biases suggest that the adaptation acts in part by shifting the perceived neutral point for the facial configuration. This is consistent with the pattern of renormalization suggested for adaptation to other facial attributes, and thus may reflect a generic level of configural coding. However, for most categories aftereffects were stronger for expressions than anti-expressions, pointing to the possible influence of an additional component of the adaptation at sites that explicitly represent facial expressions. At either level our results are consistent with other recent work in suggesting that the six expressions are defined by dimensions that are largely independently normalized by adaptation, possibly because the facial configurations conveying different expressions vary in independent ways.

Perception and Motivation in Face Recognition

Although humans possess well-developed face processing expertise, face processing is nevertheless subject to a variety of biases. Perhaps the best known of these biases is the Cross-Race Effect—the tendency to have more accurate recognition for same-race than cross-race faces. The current work reviews the evidence for and provides a critical review of theories of the Cross-Race Effect, including perceptual expertise and social cognitive accounts of the bias. The authors conclude that recent hybrid models of the Cross-Race Effect, which combine elements of both perceptual expertise and social cognitive frameworks, provide an opportunity for theoretical synthesis and advancement not afforded by independent expertise or social cognitive models. Finally, the authors suggest future research directions intended to further develop a comprehensive and integrative understanding of biases in face recognition.

Position specificity of adaptation-related face aftereffects

It has been shown that prolonged exposure to a human face leads to shape-selective visual aftereffects. It seems that these face-specific aftereffects (FAEs) have multiple components, related to the adaptation of earlier and higher level processing of visual stimuli. The largest magnitude of FAE, using long-term adaptation periods, is usually observed at the retinotopic position of the preceding adaptor stimulus. However, FAE is also detected, to a smaller degree, at other retinal positions in a spatially invariant way and this component depends less on the adaptation duration. Several lines of evidences suggest that while the position-specific FAE involves lower level areas of the ventral processing stream, the position-invariant FAE depends on the activation of higher level face-processing areas and the fusiform gyrus in particular. In the present paper, we summarize the available behavioural, electrophysiological and neuroimaging results regarding the spatial selectivity of FAE and discuss their implications for the visual stability of object representations across saccadic eye movements.

Simultaneous shape repulsion and global assimilation in the perception of aspect ratio

Although local interactions involving orientation and spatial frequency are well understood, less is known about spatial interactions involving higher level pattern features. We examined interactive coding of aspect ratio, a prevalent two-dimensional feature. We measured perception of two simultaneously flashed ellipses by randomly post-cueing one of them and having observers indicate its aspect ratio. Aspect ratios interacted in two ways. One manifested as an aspect-ratio-repulsion effect. For example, when a slightly tall ellipse and a taller ellipse were simultaneously flashed, the less tall ellipse appeared flatter and the taller ellipse appeared even taller. This repulsive interaction was long range, occurring even when the ellipses were presented in different visual hemifields. The other interaction manifested as a global assimilation effect. An ellipse appeared taller when it was a part of a global vertical organization than when it was a part of a global horizontal organization. The repulsion and assimilation effects temporally dissociated as the former slightly strengthened, and the latter disappeared when the ellipse-to-mask stimulus onset asynchrony was increased from 40 to 140 ms. These results are consistent with the idea that shape perception emerges from rapid lateral and hierarchical neural interactions.

Adaptation to different mouth shapes influences visual perception of ambiguous lip speech

We investigated the effects of adaptation to mouth shapes associated with different spoken sounds (sustained /m/ or /u/) on visual perception of lip speech. Participants were significantly more likely to label ambiguous faces on an /m/-to-/u/ continuum as saying /u/ following adaptation to /m/ mouth shapes than they were in a preadaptation test. By contrast, participants were significantly less likely to label the ambiguous faces as saying /u/ following adaptation to /u/ mouth shapes than they were in a preadaptation test. The magnitude of these aftereffects was equivalent when the same individual was shown in the adaptation and test phases of the experiment and when different individuals were presented in the adaptation and test phases. These findings present novel evidence that adaptation to natural variations in facial appearance influences face perception, and they extend previous research on face aftereffects to visual perception of lip speech.

Young without plastic surgery: perceptual adaptation to the age of female and male faces

Adaptation influences perception not only of simple stimulus qualities such as motion or colour, but also of complex stimuli such as faces. Here we demonstrate contrasting aftereffects of adaptation to facial age. In Experiment 1, participants adapted to either young or old faces, and subsequently estimated the age of morphed test faces with interpolated ages of 30, 40, 50 or 60 years. Following adaptation to old adaptors, test faces were classified as much younger when compared to classifications of the same test faces following adaptation to young faces, which in turn caused subjective test face "aging". These aftereffects were reduced but remained clear even when facial gender changed between adaptor and test faces. In Experiment 2, we induced simultaneous opposite age aftereffects for female and male faces. Overall, these results demonstrate interactions in the perception of facial age and gender, and support dissociable neuronal coding of male and female faces.

Crossing the 'uncanny valley': adaptation to cartoon faces can influence perception of human faces

In this study we assessed whether there is a single face space common to both human and cartoon faces by testing whether adaptation to cartoon faces can affect perception of human faces. Participants were shown Japanese animation cartoon videos containing faces with abnormally large eyes. The use of animated videos eliminated the possibility of position-dependent retinotopic adaptation (because the faces appear at many different locations) and more closely simulated naturalistic exposure. Adaptation to cartoon faces with large eyes significantly shifted preferences for human faces toward larger eyes, consistent with a common, non-retinotopic representation for both cartoon and human faces. This supports the possibility that there are representations that are specific to faces yet common to all kinds of faces.

Perceptual adaptation helps us identify faces

Adaptation is a fundamental property of perceptual processing. In low-level vision, it can calibrate perception to current inputs, increasing coding efficiency and enhancing discrimination around the adapted level. Adaptation also occurs in high-level vision, as illustrated by face aftereffects. However, the functional consequences of face adaptation remain uncertain. Here we investigated whether adaptation can enhance identification performance for faces from an adapted, relative to an unadapted, population. Five minutes of adaptation to an average Asian or Caucasian face reduced identification thresholds for faces from the adapted relative to the unadapted race. We replicated this interaction in two studies, using different participants, faces and adapting procedures. These results suggest that adaptation has a functional role in high-level, as well as low-level, visual processing. We suggest that adaptation to the average of a population may reduce responses to common properties shared by all members of the population, effectively orthogonalizing identity vectors in a multi-dimensional face space and freeing neural resources to code distinctive properties, which are useful for identification.

The Importance of Social Factors is a Matter of Perception

Caucasian female faces were randomly assigned to participants' in- or out-group following false feedback on a personality inventory. Participants recognised in-group faces better than out-group faces, but they showed no evidence of category-contingent opposing aftereffects. We conclude that the explanatory power of social-cognitive models will be enhanced when they are integrated with models of perceptual expertise.

View-Contingent Aftereffects Suggest Joint Coding of Face Shape and View

While it is well established that different neural populations code different face views, behavioural evidence that these neurons also code other aspects of face shape is equivocal. For example, previous studies have interpreted the partial transfer of face aftereffects across different viewpoints as evidence for either view-specific coding of face shape or that the locus of adaptation is in face-coding mechanisms that are relatively robust to changes in face view. Here we show that it is possible to simultaneously induce aftereffects in opposite directions for 3/4 and front views of upright faces with manipulated mouth position (experiment 1). For example, simultaneous adaptation to 3/4 views with raised mouth position and front views with lowered mouth position caused raised mouth position to appear more normal for 3/4 views of novel faces, but less normal for front views. View-contingent adaptation did not occur for inverted faces, however (experiment 2). Dissociable aftereffects for different views of upright faces, but not for different views of inverted faces, suggest that neurons that code face view can also code other aspects of face shape.