Adaptation improves discrimination of face identity

Visual adaptation changes the susceptibility to the fission illusion

Sound-induced flash illusion (SiFI) is the illusion that participants perceive incorrectly that the number of visual flashes is equal to the number of auditory beeps when presented within 100 ms. Although previous studies found that repetition suppression can reduce an individual's perceptual sensitivity to the SiFI, there is not yet a consensus as to how visual adaptation affects the SiFI. In the present study, we added prolonged adapting visual stimuli prior to the presentation of audiovisual stimuli to investigate whether the bottom-up factor of adaptation affects the SiFI. The adapting visual stimuli consisted of one or two of the same visual stimuli that lasted for 2 minutes in succession, followed by the audiovisual stimuli. Both adaptation conditions showed SiFI effects. The accuracy of adapting double-flashes was significantly lower than that of in adapting a single flash for the fission illusion. Our analyses indicated that such a pattern could be attributed to a lower d' in adapting double-flashes than in adapting a single flash. However, the accuracy, discriminability and criterion were not significantly different between the two adaptation conditions because of the instability of the fusion illusion. Thus, the present study indicated that the reduced perceptual sensitivity based on visual adaptation could enhance the fission illusion in multisensory integration.

Contrast Adaptation in Face Perception Revealed Through EEG and Behavior

Exposure to a face can produce biases in the perception of subsequent faces. Typically, these face aftereffects are studied by adapting to an individual face or category (e.g., faces of a given gender) and can result in renormalization of perceptions such that the adapting face appears more neutral. These shifts are analogous to chromatic adaptation, where a renormalization for the average adapting color occurs. However, in color vision, adaptation can also adjust to the variance or range of colors in the distribution. We examined whether this variance or contrast adaptation also occurs for faces, using an objective EEG measure to assess response changes following adaptation. An average female face was contracted or expanded along the horizontal or vertical axis to form four images. Observers viewed a 20 s sequence of the four images presented in a fixed order at a rate of 6 Hz, while responses to the faces were recorded with EEG. A 6 Hz signal was observed over right occipito-temporal channels, indicating symmetric responses to the four images. This test sequence was repeated after 20 s adaptation to alternations between two of the faces (e.g., horizontal contracted and expanded). This adaptation resulted in an additional signal at 3 Hz, consistent with asymmetric responses to adapted and non-adapted test faces. Adapting pairs have the same mean (undistorted) as the test sequence and thus should not bias responses driven only by the mean. Instead, the results are consistent with selective adaptation to the distortion axis. A 3 Hz signal was also observed after adapting to face pairs selected to induce a mean bias (e.g., expanded vertical and expanded horizontal), and this signal was not significantly different from that observed following adaption to a single image that did not form part of the test sequence (e.g., a single image expanded both vertically and horizontally). In a further experiment, we found that this variance adaptation can also be observed behaviorally. Our results suggest that adaptation calibrates face perception not only for the average characteristics of the faces we experience but also for the gamut of faces to which we are exposed.

Face perception: A brief journey through recent discoveries and current directions

Faces are a rich source of information about the people around us. Identity, state of mind, emotions, intentions, age, gender, ethnic background, attractiveness and a host of other attributes about an individual can be gleaned from a face. When face perception fails, dramatic psycho-social consequences can follow at the individual level, as in the case of prosopagnosic parents who are unable to recognize their children at school pick-up. At the species level, social interaction patterns are shaped by human face perception abilities. The computational feat of recognizing faces and facial attributes, and the challenges overcome by the human brain to achieve this feat, have fascinated generations of vision researchers. In this paper, we present a brief overview of some of the milestones of discovery as well as outline a selected set of current directions and open questions on this topic.

Adaptation decorrelates shape representations

Perception and neural responses are modulated by sensory history. Visual adaptation, an example of such an effect, has been hypothesized to improve stimulus discrimination by decorrelating responses across a set of neural units. While a central theoretical model, behavioral and neural evidence for this theory is limited and inconclusive. Here, we use a parametric 3D shape-space to test whether adaptation decorrelates shape representations in humans. In a behavioral experiment with 20 subjects, we find that adaptation to a shape class improves discrimination of subsequently presented stimuli with similar features. In a BOLD fMRI experiment with 10 subjects, we observe that adaptation to a shape class decorrelates the multivariate representations of subsequently presented stimuli with similar features in object-selective cortex. These results support the long-standing proposal that adaptation improves perceptual discrimination and decorrelates neural representations, offering insights into potential underlying mechanisms.

Adaptation is thought to improve discrimination by pulling neural representations of similar stimuli farther apart. Here, the authors separately show that adaptation to a 3D shape class leads to better discrimination performance on similar shapes, and activity patterns diverge in object selective cortical areas.

Visualizing Visual Adaptation

Many techniques have been developed to visualize how an image would appear to an individual with a different visual sensitivity: e.g., because of optical or age differences, or a color deficiency or disease. This protocol describes a technique for incorporating sensory adaptation into the simulations. The protocol is illustrated with the example of color vision, but is generally applicable to any form of visual adaptation. The protocol uses a simple model of human color vision based on standard and plausible assumptions about the retinal and cortical mechanisms encoding color and how these adjust their sensitivity to both the average color and range of color in the prevailing stimulus. The gains of the mechanisms are adapted so that their mean response under one context is equated for a different context. The simulations help reveal the theoretical limits of adaptation and generate "adapted images" that are optimally matched to a specific environment or observer. They also provide a common metric for exploring the effects of adaptation within different observers or different environments. Characterizing visual perception and performance with these images provides a novel tool for studying the functions and consequences of long-term adaptation in vision or other sensory systems.

Exploring the Use of Sensorial LTP/LTD-Like Stimulation to Modulate Human Performance for Complex Visual Stimuli

Is it possible to passively induce visual learning/unlearning in humans for complex stimuli such as faces? We addressed this question in a series of behavioral studies using passive visual stimulation (flickering of faces at specific temporal frequencies) inspired by well-known synaptic mechanisms of learning: long-term potentiation (LTP) vs long-term depression (LTD). We administered a face identity change detection task before and after a passive stimulation protocol to test for potential changes in visual performance. First, with bilateral stimulation, subjects undergoing high-frequency LTP-like stimulation outperformed those submitted to low-frequency LTD-like stimulation despite equivalent baseline performance (exp. 1). Second, unilateral stimulation replicated the differential modulation of performance, but in a hemifield-specific way (exp. 2). Third, for both stimulation groups, a sudden temporary drop in performance on the stimulated side immediately after the stimulation, followed by progressive recovering, can suggest either 'visual fatigue' or 'face adaptation' effects due to the stimulation. Fourth, we tested the life-time of these modulatory effects, revealing they vanish after one hour delay (exp. 3). Fifth, a control study (exp. 4) using low-level visual stimuli also failed to show longer-term effects of sensory stimulation, despite reports of strong effects in the literature. Future studies should determine the necessary and sufficient conditions enabling robust long-term modulation of visual performance using this technique. This step is required to consider further use in fundamental research (e.g., to study neural circuits involved in selective visual processing) and potential educational or clinical applications (e.g., inhibiting socially-irrelevant aspects of face processing in autism).

Visual adaptation provides objective electrophysiological evidence of facial identity discrimination

Discrimination of facial identities is a fundamental function of the human brain that is challenging to examine with macroscopic measurements of neural activity, such as those obtained with functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although visual adaptation or repetition suppression (RS) stimulation paradigms have been successfully implemented to this end with such recording techniques, objective evidence of an identity-specific discrimination response due to adaptation at the level of the visual representation is lacking. Here, we addressed this issue with fast periodic visual stimulation (FPVS) and EEG recording combined with a symmetry/asymmetry adaptation paradigm. Adaptation to one facial identity is induced through repeated presentation of that identity at a rate of 6 images per second (6 Hz) over 10 sec. Subsequently, this identity is presented in alternation with another facial identity (i.e., its anti-face, both faces being equidistant from an average face), producing an identity repetition rate of 3 Hz over a 20 sec testing sequence. A clear EEG response at 3 Hz is observed over the right occipito-temporal (ROT) cortex, indexing discrimination between the two facial identities in the absence of an explicit behavioral discrimination measure. This face identity discrimination occurs immediately after adaptation and disappears rapidly within 20 sec. Importantly, this 3 Hz response is not observed in a control condition without the single-identity 10 sec adaptation period. These results indicate that visual adaptation to a given facial identity produces an objective (i.e., at a pre-defined stimulation frequency) electrophysiological index of visual discrimination between that identity and another, and provides a unique behavior-free quantification of the effect of visual adaptation.

Deciding on race: a diffusion model analysis of race-categorisation

It has long been known that a person's race can affect their decisions about people of another race; an observation that clearly taps into some deep societal issues. However, in order to behave differently in response to someone else's race, you must first categorise that person as other-race. The current study investigates the process of race-categorisation. Two groups of participants, Asian and Caucasian, rapidly classified facial images that varied from strongly Asian, through racially intermediate, to strongly Caucasian. In agreement with previous findings, there was a difference in category boundary between the two groups. Asian participants more frequently judged intermediate images as Caucasian and vice versa. We fitted a decision model, the Ratcliff diffusion model, to our two choice reaction time data. This model provides an account of the processes thought to underlie binary choice decisions. Within its architecture it has two components that could reasonably lead to a difference in race category boundary, these being evidence accumulation rate and a priori bias. The latter is the expectation or prior belief that a participant brings to the task, whilst the former indexes sensitivity to race-dependent perceptual cues. Whilst we find no good evidence for a difference in a priori bias between our two groups, we do find evidence for a difference in evidence accumulation rate. Our Asian participants were more sensitive to Caucasian cues within the images than were our Caucasian participants (and vice versa). These results support the idea that differences in perceptual sensitivity to race-defining visual characteristics drive differences in race categorisation. We propose that our findings fit with a wider view in which perceptual adaptation plays a central role in the visual processing of own and other race.

Task-irrelevant emotion facilitates face discrimination learning

We understand poorly how the ability to discriminate faces from one another is shaped by visual experience. The purpose of the present study is to determine whether face discrimination learning can be facilitated by facial emotions. To answer this question, we used a task-irrelevant perceptual learning paradigm because it closely mimics the learning processes that, in daily life, occur without a conscious intention to learn and without an attentional focus on specific facial features. We measured face discrimination thresholds before and after training. During the training phase (4 days), participants performed a contrast discrimination task on face images. They were not informed that we introduced (task-irrelevant) subtle variations in the face images from trial to trial. For the Identity group, the task-irrelevant features were variations along a morphing continuum of facial identity. For the Emotion group, the task-irrelevant features were variations along an emotional expression morphing continuum. The Control group did not undergo contrast discrimination learning and only performed the pre-training and post-training tests, with the same temporal gap between them as the other two groups. Results indicate that face discrimination improved, but only for the Emotion group. Participants in the Emotion group, moreover, showed face discrimination improvements also for stimulus variations along the facial identity dimension, even if these (task-irrelevant) stimulus features had not been presented during training. The present results highlight the importance of emotions for face discrimination learning.

Erasing the face after-effect

Perceptual after-effects decay over time at a rate that depends on several factors, such as the duration of adaptation and the duration of the test stimuli. Whether this decay is accelerated by exposure to other faces after adaptation is not known. Our goal was to determine if the appearance of other faces during a delay period after adaptation affected the face identity after-effect. In the first experiment we investigated whether, in the perception of ambiguous stimuli created by morphing between two faces, the repulsive after-effects from adaptation to one face were reduced by brief presentation of the second face in a delay period. We found no effect; however, this may have been confounded by a small attractive after-effect from the interference face. In the second experiment, the interference stimuli were faces unrelated to those used as adaptation stimuli, and we examined after-effects at three different delay periods. This showed a decline in after-effects as the time since adaptation increased, and an enhancement of this decline by the presentation of intervening faces. An exponential model estimated that the intervening faces caused an 85% reduction in the time constant of the after-effect decay. In conclusion, we confirm that face after-effects decline rapidly after adaptation and that exposure to other faces hastens the re-setting of the system.

Dissociating the neural bases of repetition-priming and adaptation in the human brain for faces

The repetition of a given stimulus leads to the attenuation of the functional magnetic resonance imaging (fMRI) signal compared with unrepeated stimuli, a phenomenon called fMRI adaptation or repetition suppression (RS). Previous studies have related RS of the fMRI signal behaviorally both to improved performance for the repeated stimulus (priming) and to shifts of perception away from the first stimulus (adaptation-related aftereffects). Here we used identical task (sex discrimination), trial structure [ stimulus 1 (S1): 3,000 ms, interstimulus interval: 600 ms, stimulus 2 (S2): 300 ms], and S2 stimuli (androgynous faces) to test how RS of the face-specific areas of the occipito-temporal cortex relates to priming and aftereffects. By varying S1, we could induce priming (significantly faster reaction times when S1 and S2 were identical compared with different images) as well as sex-specific aftereffect [an increased ratio of male responses if S1 was a female face compared with ambiguous faces or to Fourier-randomized noise (FOU) images]. Presenting any face as S1 led to significant RS of the blood oxygen level-dependent signal in the fusiform and occipital face areas as well as in the lateral occipital cortex of both hemispheres compared with FOU, reflecting stimulus category-specific encoding. Additionally, while sex-specific adaptation effects were only observed in occipital face areas, primed trials led to a signal reduction in both face-selective regions. Altogether, these results suggest the differential neural mechanisms of adaptation and repetition priming.

Adaptor identity modulates adaptation effects in familiar face identification and their neural correlates

Adaptation-related aftereffects (AEs) show how face perception can be altered by recent perceptual experiences. Along with contrastive behavioural biases, modulations of the early event-related potentials (ERPs) were typically reported on categorical levels. Nevertheless, the role of the adaptor stimulus per se for face identity-specific AEs is not completely understood and was therefore investigated in the present study. Participants were adapted to faces (S1s) varying systematically on a morphing continuum between pairs of famous identities (identities A and B), or to Fourier phase-randomized faces, and had to match the subsequently presented ambiguous faces (S2s; 50/50% identity A/B) to one of the respective original faces. We found that S1s identical with or near to the original identities led to strong contrastive biases with more identity B responses following A adaptation and vice versa. In addition, the closer S1s were to the 50/50% S2 on the morphing continuum, the smaller the magnitude of the AE was. The relation between S1s and AE was, however, not linear. Additionally, stronger AEs were accompanied by faster reaction times. Analyses of the simultaneously recorded ERPs revealed categorical adaptation effects starting at 100 ms post-stimulus onset, that were most pronounced at around 125-240 ms for occipito-temporal sites over both hemispheres. S1-specific amplitude modulations were found at around 300-400 ms. Response-specific analyses of ERPs showed reduced voltages starting at around 125 ms when the S1 biased perception in a contrastive way as compared to when it did not. Our results suggest that face identity AEs do not only depend on physical differences between S1 and S2, but also on perceptual factors, such as the ambiguity of S1. Furthermore, short-term plasticity of face identity processing might work in parallel to object-category processing, and is reflected in the first 400 ms of the ERP.

Atypical updating of face representations with experience in children with autism

Face identity aftereffects are significantly diminished in children with autism relative to typical children, which may reflect reduced perceptual updating with experience. Here, we investigated whether this atypicality also extends to non-face stimulus categories, which might signal a pervasive visual processing difference in individuals with autism. We used a figural aftereffect task to measure directly perceptual updating following exposure to distorted upright faces, inverted faces and cars, in typical children and children with autism. A size-change between study and test stimuli limited the likelihood that any processing atypicalities reflected group differences in adaptation to low-level features of the stimuli. Results indicated that, relative to typical children, figural aftereffects for upright faces, but not inverted faces or cars, were significantly attenuated in children with autism. Moreover, the group difference was amplified when we isolated the 'face-selective' component of the aftereffect, by partialling out the mid-level shape adaptation common to upright and inverted face stimuli. Notably, the aftereffects of typical children were disproportionately larger for upright faces than for inverted faces and cars, but the magnitude of aftereffects of autistic children was not similarly modulated according to stimulus category. These findings are inconsistent with a pervasive adaptive coding atypicality relative to typical children, and suggest that reduced perceptual updating may constitute a high-level, and possibly face-selective, visual processing difference in children with autism.

Adaptive face space coding in congenital prosopagnosia: typical figural aftereffects but abnormal identity aftereffects

People with congenital prosopagnosia (CP) report difficulty recognising faces in everyday life and perform poorly on face recognition tests. Here, we investigate whether impaired adaptive face space coding might contribute to poor face recognition in CP. To pinpoint how adaptation may affect face processing, a group of CPs and matched controls completed two complementary face adaptation tasks: the figural aftereffect, which reflects adaptation to general distortions of shape, and the identity aftereffect, which directly taps the mechanisms involved in the discrimination of different face identities. CPs displayed a typical figural aftereffect, consistent with evidence that they are able to process some shape-based information from faces, e.g., cues to discriminate sex. CPs also demonstrated a significant identity aftereffect. However, unlike controls, CPs impression of the identity of the neutral average face was not significantly shifted by adaptation, suggesting that adaptive coding of identity is abnormal in CP. In sum, CPs show reduced aftereffects but only when the task directly taps the use of face norms used to code individual identity. This finding of a reduced face identity aftereffect in individuals with severe face recognition problems is consistent with suggestions that adaptive coding may have a functional role in face recognition.

Enhanced attention amplifies face adaptation

Perceptual adaptation not only produces striking perceptual aftereffects, but also enhances coding efficiency and discrimination by calibrating coding mechanisms to prevailing inputs. Attention to simple stimuli increases adaptation, potentially enhancing its functional benefits. Here we show that attention also increases adaptation to faces. In Experiment 1, face identity aftereffects increased when attention to adapting faces was increased using a change detection task. In Experiment 2, figural (distortion) face aftereffects increased when attention was increased using a snap game (detecting immediate repeats) during adaptation. Both were large effects. Contributions of low-level adaptation were reduced using free viewing (both experiments) and a size change between adapt and test faces (Experiment 2). We suggest that attention may enhance adaptation throughout the entire cortical visual pathway, with functional benefits well beyond the immediate advantages of selective processing of potentially important stimuli. These results highlight the potential to facilitate adaptive updating of face-coding mechanisms by strategic deployment of attentional resources.

The nature of upright and inverted face representations: an adaptation-transfer study of configuration

It is considered that whole-face processing of spatial structure may only be possible in upright faces, with only local feature processing in inverted faces. We asked whether this was due to impoverished representations of inverted faces. We performed two experiments. In the first, we divided faces into segments to create 'exploded' faces with disrupted second-order structures, and 'scrambled' faces with altered first-order relations; in the second we shifted features within intact facial outlines to create equivalent disruptions of spatial structure. In both we assessed the transfer of adaptation between faces with altered structure and intact faces. Scrambled adaptors did not adapt upright or inverted intact faces, indicating that a whole-face configuration is required at either orientation. Both upright and inverted faces showed a similar decline in aftereffect magnitude when adapting faces had altered second-order structure, implying that this structure is present in both upright and inverted face representations. We conclude that inverted faces are not represented simply as a collection of features, but have a whole-face configuration with second-order structure, similar to upright faces. Thus the qualitative impairments induced by inversion are not due to degraded inverted facial representations, but may reflect limitations in perceptual mechanisms.