Position specificity of adaptation-related face aftereffects

A preference to look closer to the eyes is associated with a position-invariant face neural code

When looking at faces, humans invariably move their eyes to a consistent preferred first fixation location on the face. While most people have the preferred fixation location just below the eyes, a minority have it between the nose-tip and mouth. Not much is known about whether these long-term differences in the preferred fixation location are associated with distinct neural representations of faces. To study this, we used a gaze-contingent face adaptation aftereffect paradigm to test in two groups of observers, one with their mean preferred fixation location closer to the eyes (upper lookers) and the other closer to the mouth (lower lookers). In this task, participants were required to maintain their gaze at either their own group's mean preferred fixation location or that of the other group during adaptation and testing. The two possible fixation locations were 3.6° apart on the face. We measured the face adaptation aftereffects when the adaptation and testing happened while participants maintained fixation at either the same or different locations on the face. Both groups showed equally strong adaptation effects when the adaptation and testing happened at the same fixation location. Crucially, only the upper lookers showed a partial transfer of the FAE across the two fixation locations, when adaptation occurred at the eyes. Lower lookers showed no spatial transfer of the FAE irrespective of the adaptation position. Given the classic finding that neural tuning is increasingly position invariant as one moves higher in the visual hierarchy, this result suggests that differences in the preferred fixation location are associated with distinct neural representations of faces.

Modality-specific sensory and decisional carryover effects in duration perception

BACKGROUND

The brain uses recent history when forming perceptual decisions. This results in carryover effects in perception. Although separate sensory and decisional carryover effects have been shown in many perceptual tasks, their existence and nature in temporal processing are unclear. Here, we investigated whether and how previous stimuli and previous choices affect subsequent duration perception, in vision and audition.

RESULTS

In a series of three experiments, participants were asked to classify visual or auditory stimuli into "shorter" or "longer" duration categories. In experiment 1, visual and auditory stimuli were presented in separate blocks. Results showed that current duration estimates were repelled away from the previous trial's stimulus duration, but attracted towards the previous choice, in both vision and audition. In experiment 2, visual and auditory stimuli were pseudorandomly presented in one block. We found that sensory and decisional carryover effects occurred only when previous and current stimuli were from the same modality. Experiment 3 further investigated the stimulus dependence of carryover effects within each modality. In this experiment, visual stimuli with different shape topologies (or auditory stimuli with different audio frequencies) were pseudorandomly presented in one visual (or auditory) block. Results demonstrated sensory carryover (within each modality) despite task-irrelevant differences in visual shape topology or audio frequency. By contrast, decisional carryover was reduced (but still present) across different visual topologies and completely absent across different audio frequencies.

CONCLUSIONS

These results suggest that serial dependence in duration perception is modality-specific. Moreover, repulsive sensory carryover effects generalize within each modality, whereas attractive decisional carryover effects are contingent on contextual details.

Face identity and facial expression representations with adaptation paradigms: New directions for potential applications

Adaptation and aftereffect are well-known procedures for exploring our neural representation of visual stimuli. It has been reported that they occur in face identity, facial expressions, and low-level visual features. This method has two primary advantages. One is to reveal the common or shared process of faces, that is, the overlapped or discrete representation of face identities or facial expressions. The other is to investigate the coding system or theory of face processing that underlies the ability to recognize faces. This study aims to organize recent research to guide the reader into the field of face adaptation and its aftereffect and to suggest possible future expansions in the use of this paradigm. To achieve this, we reviewed the behavioral short-term aftereffect studies on face identity (i.e., who it is) and facial expressions (i.e., what expressions such as happiness and anger are expressed), and summarized their findings about the neural representation of faces. First, we summarize the basic characteristics of face aftereffects compared to simple visual features to clarify that facial aftereffects occur at a different stage and are not inherited or combinations of low-level visual features. Next, we introduce the norm-based coding hypothesis, which is one of the theories used to represent face identity and facial expressions, and adaptation is a commonly used procedure to examine this. Subsequently, we reviewed studies that applied this paradigm to immature or impaired face recognition (i.e., children and individuals with autism spectrum disorder or prosopagnosia) and examined the relationships between their poor recognition performance and representations. Moreover, we reviewed studies dealing with the representation of non-presented faces and social signals conveyed via faces and discussed that the face adaptation paradigm is also appropriate for these types of examinations. Finally, we summarize the research conducted to date and propose a new direction for the face adaptation paradigm.

Spatial selectivity in adaptation to gaze direction

A person's focus of attention is conveyed by the direction of their eyes and face, providing a simple visual cue fundamental to social interaction. A growing body of research examines the visual mechanisms that encode the direction of another person's gaze as we observe them. Here we investigate the spatial receptive field properties of these mechanisms, by testing the spatial selectivity of sensory adaptation to gaze direction. Human observers were adapted to faces with averted gaze presented in one visual hemifield, then tested in their perception of gaze direction for faces presented in the same or opposite hemifield. Adaptation caused strong, repulsive perceptual aftereffects, but only for faces presented in the same hemifield as the adapter. This occurred even though adapting and test stimuli were in the same external location across saccades. Hence, there was clear evidence for retinotopic adaptation and a relative lack of either spatiotopic or spatially invariant adaptation. These results indicate that adaptable representations of gaze direction in the human visual system have retinotopic spatial receptive fields. This strategy of coding others' direction of gaze with positional specificity relative to one's own eye position may facilitate key functions of gaze perception, such as socially cued shifts in visual attention.

Parallel Adaptation to Spatially Distinct Distortions

Optical distortions as a visual disturbance are inherent in many optical devices such as spectacles or virtual reality headsets. In such devices, distortions vary spatially across the visual field. In progressive addition lenses, for example, the left and right regions of the lens skew the peripheral parts of the wearers visual field in opposing directions. The human visual system adapts to homogeneous distortions and the respective aftereffects are transferred to non-retinotopic locations. This study investigates simultaneous adaptation to two opposing distortions at different retinotopic locations. Two oppositely skewed natural image sequences were presented to 10 subjects as adaptation stimuli at two distinct locations in the visual field. To do so, subjects were instructed to keep fixation on a target. Eye tracking was used for gaze control. Change of perceived motion direction was measured in a direction identification task. The point of subjective equality (PSE), that is, the angle at which a group of coherently moving dots was perceived as moving horizontal, was determined for both retinal locations. The shift of perceived motion direction was evaluated by comparing PSE before and after adaptation. A significant shift at both retinal locations in the direction of the skew distortion of the corresponding adaptation stimulus is demonstrated. Consequently, parallel adaptation to two opposing distortions in a retinotopic reference frame was confirmed by this study.

Object responses are highly malleable, rather than invariant, with changes in object appearance

Theoretical frameworks of human vision argue that object responses remain stable, or 'invariant', despite changes in viewing conditions that can alter object appearance but not identity. Here, in a major departure from previous approaches that have relied on two-dimensional (2-D) images to study object processing, we demonstrate that changes in an object's appearance, but not its identity, can lead to striking shifts in behavioral responses to objects. We used inverse multidimensional scaling (MDS) to measure the extent to which arrangements of objects in a sorting task were similar or different when the stimuli were displayed as scaled 2-D images, three-dimensional (3-D) augmented reality (AR) projections, or real-world solids. We were especially interested in whether sorting behavior in each display format was based on conceptual (e.g., typical location) versus physical object characteristics. We found that 2-D images of objects were arranged according to conceptual (typical location), but not physical, properties. AR projections, conversely, were arranged primarily according to physical properties such as real-world size, elongation and weight, but not conceptual properties. Real-world solid objects, unlike both 2-D and 3-D images, were arranged using multidimensional criteria that incorporated both conceptual and physical object characteristics. Our results suggest that object responses can be strikingly malleable, rather than invariant, with changes in the visual characteristics of the stimulus. The findings raise important questions about limits of invariance in object processing, and underscore the importance of studying responses to richer stimuli that more closely resemble those we encounter in real-world environments.

Position selectivity in face-sensitive visual cortex to facial and nonfacial stimuli: an fMRI study

BACKGROUND

Evidence for position sensitivity in object-selective visual areas has been building. On one hand, most of the relevant studies have utilized stimuli for which the areas are optimally selective and examine small sections of cortex. On the other hand, visual field maps established with nonspecific stimuli have been found in increasingly large areas of visual cortex, though generally not in areas primarily responsive to faces.

METHODS

fMRI was used to study the position sensitivity of the occipital face area (OFA) and the fusiform face area (FFA) to both standard rotating wedge retinotopic mapping stimuli and quadrant presentations of synthetic facial stimuli. Analysis methods utilized were both typical, that is, mean univariate BOLD signals and multivoxel pattern analysis (MVPA), and novel, that is, distribution of voxels to pattern classifiers and use of responses to nonfacial retinotopic mapping stimuli to classify responses to facial stimuli.

RESULTS

Polar angle sensitivity was exhibited to standard retinotopic mapping stimuli with a stronger contralateral bias in OFA than in FFA, a stronger bias toward the vertical meridian in FFA than in OFA, and a bias across both areas toward the inferior visual field. Contralateral hemispheric lateralization of both areas was again shown using synthetic face stimuli based on univariate BOLD signals, MVPA, and the biased contribution of voxels toward multivariate classifiers discriminating the contralateral visual field. Classifiers based on polar angle responsivity were used to classify the patterns of activation above chance levels to face stimuli in the OFA but not in the FFA.

CONCLUSIONS

Both the OFA and FFA exhibit quadrant sensitivity to face stimuli, though the OFA exhibits greater position responsivity across stimuli than the FFA and includes overlap in the response pattern to the disparate stimulus types. Such biases are consistent with varying position sensitivity along different surfaces of occipito-temporal cortex.

The Role of Familiarity on Viewpoint Adaptation for Self-Face and Other-Face Images

An adaptation method was used to investigate whether self-face processing is dissociable from general face processing. We explored the viewpoint aftereffect with face images having different degrees of familiarity (never-before-seen faces, recently familiarized faces, personally familiar faces, and the participant’s own face). A face viewpoint aftereffect occurs after prolonged viewing of a face viewed from one side, with the result that the perceived viewing direction of a subsequently presented face image shown near the frontal view is biased in a direction which is the opposite of the adapting orientation. We found that (1) the magnitude of the viewpoint aftereffect depends on the level of familiarity of the adapting and test faces, (2) a cross-identity transfer of the viewpoint aftereffect is found between all categories of faces, but not between an unfamiliar adaptor face and the self-face test, and (3) learning affects the processing of the self-face in greater measure than any other category of faces. These results highlight the importance of familiarity on the face aftereffects, but they also suggest the possibility of separate representations for the self-face, on the one side, and for highly familiar faces, on the other.

Adaptation Duration Dissociates Category-, Image-, and Person-Specific Processes on Face-Evoked Event-Related Potentials

Several studies demonstrated that face perception is biased by the prior presentation of another face, a phenomenon termed as face-related after-effect (FAE). FAE is linked to a neural signal-reduction at occipito-temporal areas and it can be observed in the amplitude modulation of the early event-related potential (ERP) components. Recently, macaque single-cell recording studies suggested that manipulating the duration of the adaptor makes the selective adaptation of different visual motion processing steps possible. To date, however, only a few studies tested the effects of adaptor duration on the electrophysiological correlates of human face processing directly. The goal of the current study was to test the effect of adaptor duration on the image-, identity-, and generic category-specific face processing steps. To this end, in a two-alternative forced choice familiarity decision task we used five adaptor durations (ranging from 200-5000 ms) and four adaptor categories: adaptor and test were identical images-Repetition Suppression (RS); adaptor and test were different images of the Same Identity (SameID); adaptor and test images depicted Different Identities (DiffID); the adaptor was a Fourier phase-randomized image (No). Behaviorally, a strong priming effect was observed in both accuracy and response times for RS compared with both DiffID and No. The electrophysiological results suggest that rapid adaptation leads to a category-specific modulation of P100, N170, and N250. In addition, both identity and image-specific processes affected the N250 component during rapid adaptation. On the other hand, prolonged (5000 ms) adaptation enhanced, and extended category-specific adaptation processes over all tested ERP components. Additionally, prolonged adaptation led to the emergence of image-, and identity-specific modulations on the N170 and P2 components as well. In other words, there was a clear dissociation among category, identity-, and image-specific processing steps in the case of longer (3500 and 5000 ms) but not for shorter durations (< 3500 ms), reflected in the gradual reduction of N170 and enhancement of P2 in the No, DiffID, SameID, and RS conditions. Our findings imply that by manipulating adaptation duration one can dissociate the various steps of human face processing, reflected in the ERP response.

Visual duration aftereffect is position invariant

Adaptation to relatively long or short sensory events leads to a negative aftereffect, such that the durations of the subsequent events within a certain range appear to be contracted or expanded. The distortion in perceived duration is presumed to arise from the adaptation of duration detectors. Here, we focus on the positional sensitivity of those visual duration detectors by exploring whether the duration aftereffect may be constrained by the visual location of stimuli. We adopted two different paradigms, one that tests for transfer across visual hemifields, and the other that tests for simultaneous selectivity between visual hemifields. By employing these experimental designs, we show that the duration aftereffect strongly transfers across visual hemifields and is not contingent on them. The lack of position specificity suggests that duration detectors in the visual system may operate at a relatively later stage of sensory processing.

Encoding-Stage Crosstalk Between Object- and Spatial Property-Based Scene Processing Pathways

Scene categorization draws on 2 information sources: The identities of objects scenes contain and scenes' intrinsic spatial properties. Because these resources are formally independent, it is possible for them to leads to conflicting judgments of scene category. We tested the hypothesis that the potential for such conflicts is mitigated by a system of "crosstalk" between object- and spatial layout-processing pathways, under which the encoded spatial properties of scenes are biased by scenes' object contents. Specifically, we show that the presence of objects strongly associated with a given scene category can bias the encoded spatial properties of scenes containing them toward the average of that category, an effect which is evident both in behavioral measures of scenes' perceived spatial properties and in scene-evoked multivoxel patterns recorded with functional magnetic resonance imaging from the parahippocampal place area (PPA), a region associated with the processing of scenes' spatial properties. These results indicate that harmonization of object- and spatial property-based estimates of scene identity begins when spatial properties are encoded, and that the PPA plays a central role in this process.

Time course of shape and category selectivity revealed by EEG rapid adaptation

A hallmark of human cognition is the ability to rapidly assign meaning to sensory stimuli. It has been suggested that this fast visual object categorization ability is accomplished by a feedforward processing hierarchy consisting of shape-selective neurons in occipito-temporal cortex that feed into task circuits in frontal cortex computing conceptual category membership. We performed an EEG rapid adaptation study to test this hypothesis. Participants were trained to categorize novel stimuli generated with a morphing system that precisely controlled both stimulus shape and category membership. We subsequently performed EEG recordings while participants performed a category matching task on pairs of successively presented stimuli. We used space-time cluster analysis to identify channels and latencies exhibiting selective neural responses. Neural signals before 200 msec on posterior channels demonstrated a release from adaptation for shape changes, irrespective of category membership, compatible with a shape- but not explicitly category-selective neural representation. A subsequent cluster with anterior topography appeared after 200 msec and exhibited release from adaptation consistent with explicit categorization. These signals were subsequently modulated by perceptual uncertainty starting around 300 msec. The degree of category selectivity of the anterior signals was strongly predictive of behavioral performance. We also observed a posterior category-selective signal after 300 msec exhibiting significant functional connectivity with the initial anterior category-selective signal. In summary, our study supports the proposition that perceptual categorization is accomplished by the brain within a quarter second through a largely feedforward process culminating in frontal areas, followed by later category-selective signals in posterior regions.

Spontaneous recovery of motion and face aftereffects

The ability of the visual system to rapidly adjust to changing environmental conditions is one of its key characteristics. Environmental changes can occur over a variety of timescales, however, and it remains unknown how the visual system adapts to these. Does a single mechanism control adaptation across all timescales, or is adaptation subserved by multiple mechanisms, each of which is tuned to its preferred duration? To address this question, we conducted three experiments in which subjects viewed motion (Exp. 1 and 2) or faces (Exp. 3) in a sequence designed to produce opposing aftereffects. A first adapter was presented for a relatively long duration, while a second one was presented only long enough to extinguish the effects of the initial adapter. Continued measurement of aftereffects revealed a spontaneous recovery of adaptation caused by the initial, longer-lasting adapter in all three experiments. This pattern of results suggests that adaptation in the visual system generally reflects a combination of multiple temporally-tuned mechanisms.

A reinvestigation of the reference frame of the tilt-adaptation aftereffect

The tilt-adaptation aftereffect (TAE) is the phenomenon that prolonged perception of a tilted ‘adapter’ stimulus affects the perceived tilt of a subsequent ‘tester’ stimulus. Although it is clear that TAE is strongest when adapter and tester are presented at the same location, the reference frame of the effect is debated. Some authors have reported that TAE is spatiotopic (world centred): It occurs when adapter and tester are presented at the same display location, even when this corresponds to different retinal locations. Others have reported that TAE is exclusively retinotopic (eye centred): It occurs only when adapter and tester are presented at the same retinal location, even when this corresponds to different display locations. Because this issue is crucial for models of transsaccadic perception, we reinvestigated the reference frame of TAE. We report that TAE is exclusively retinotopic, supporting the notion that there is no transsaccadic integration of low-level visual information.

Neural correlates of after-effects caused by adaptation to multiple face displays

Adaptation to a given face leads to face-related, specific after-effects. Recently, this topic has attracted a lot of attention because it clearly shows that adaptation occurs even at the higher stages of visual cortical processing. However, during our every-day life, faces do not appear in isolation, rather they are usually surrounded by other stimuli. Here, we used psychophysical and fMRI adaptation methods to test whether humans adapt to the gender properties of a composite multiple face stimulus as well. As adaptors we used stimuli composed of eight different individual faces, positioned peripherally in a ring around a fixation mark. We found that the gender discrimination of a subsequent centrally presented target face is significantly biased as a result of long-term adaptation to either male or female multiple face stimuli. Similar to our previous results with single-face adaptors (Kovács et al. in Neuroimage 43(1):156-164, 2008), a concurrent functional magnetic resonance imaging adaptation experiment revealed the strongest blood oxygen level-dependent signal adaptation bilaterally in the fusiform face area. Our results suggest that humans extract the statistical features of the multiple face stimulus and this process occurs at the level of occipito-temporal face processing.

Visual stability

Our vision remains stable even though the movements of our eyes, head and bodies create a motion pattern on the retina. One of the most important, yet basic, feats of the visual system is to correctly determine whether this retinal motion is owing to real movement in the world or rather our own self-movement. This problem has occupied many great thinkers, such as Descartes and Helmholtz, at least since the time of Alhazen. This theme issue brings together leading researchers from animal neurophysiology, clinical neurology, psychophysics and cognitive neuroscience to summarize the state of the art in the study of visual stability. Recently, there has been significant progress in understanding the limits of visual stability in humans and in identifying many of the brain circuits involved in maintaining a stable percept of the world. Clinical studies and new experimental methods, such as transcranial magnetic stimulation, now make it possible to test the causal role of different brain regions in creating visual stability and also allow us to measure the consequences when the mechanisms of visual stability break down.

Spatiotopic coding and remapping in humans

How our perceptual experience of the world remains stable and continuous in the face of continuous rapid eye movements still remains a mystery. This review discusses some recent progress towards understanding the neural and psychophysical processes that accompany these eye movements. We firstly report recent evidence from imaging studies in humans showing that many brain regions are tuned in spatiotopic coordinates, but only for items that are actively attended. We then describe a series of experiments measuring the spatial and temporal phenomena that occur around the time of saccades, and discuss how these could be related to visual stability. Finally, we introduce the concept of the spatio-temporal receptive field to describe the local spatiotopicity exhibited by many neurons when the eyes move.