Rapidly acquired shape and face aftereffects are retinotopic and local in origin

Inactivation of face-selective neurons alters eye movements when free viewing faces

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face-encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of the inferior temporal (IT) cortex in two macaque monkeys using an functional magnetic resonance imaging (fMRI) localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face-selective neurons altered the pattern of eye movements on faces: The monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

Inactivation of face selective neurons alters eye movements when free viewing faces

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of inferior temporal (IT) cortex in two macaque monkeys using an fMRI localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face selective neurons altered the pattern of eye movements on faces: the monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

Significance

It has been shown, for more than half a century, that eye movements follow distinctive patterns when free viewing faces. This suggests causal involvement of the face-encoding visual neurons in the eye movements. However, the literature is scant of evidence for this possibility and has focused mostly on the link between low-level image saliency and eye movements. Here, for the first time, we bring causal evidence showing how face-selective neurons in inferior temporal cortex inform and steer eye movements when free viewing faces.

Expectation violations enhance neuronal encoding of sensory information in mouse primary visual cortex

The response of cortical neurons to sensory stimuli is shaped both by past events (adaptation) and the expectation of future events (prediction). Here we employed a visual stimulus paradigm with different levels of predictability to characterise how expectation influences orientation selectivity in the primary visual cortex (V1) of male mice. We recorded neuronal activity using two-photon calcium imaging (GCaMP6f) while animals viewed sequences of grating stimuli which either varied randomly in their orientations or rotated predictably with occasional transitions to an unexpected orientation. For single neurons and the population, there was significant enhancement in the gain of orientation-selective responses to unexpected gratings. This gain-enhancement for unexpected stimuli was prominent in both awake and anaesthetised mice. We implemented a computational model to demonstrate how trial-to-trial variability in neuronal responses were best characterised when adaptation and expectation effects were combined.

Li and Atick's theory of efficient binocular coding: A tutorial and mini-review

Li and Atick (Network: Computation in Neural Systems 5 (1994) 157-174) presented a theory of efficient binocular encoding that explains a number of experimental findings. A binocular neuron is conventionally described in terms of two channels: the left and right eyes. Li and Atick's theory instead describes the neuron in terms of two alternative channels: the binocular sum and difference. The advantage of the latter description is that, unlike the left and right eye channels, the summation and differencing channels are usually uncorrelated; this means that each channel can be optimised independently of the other. The theory shows how to derive optimal receptive fields for the binocular summation and differencing channels; from these, it is easy to derive the neuron's optimal left and right eye receptive fields. The functional reality of the summation and differencing channels is demonstrated by a series of adaptation studies that confirm some counterintuitive predictions of the theory. Here we provide an accessible account of the theory, and review the evidence supporting it.

Lateral inhibition between banks of orientation selective channels predicts shape context effects: A tilt-illusion field

The perceived shapes of almost circular paths are modified by concentrically placed context paths. These induced changes have previously been attributed to curvature masking. This paper shows that, instead, they can be explained by the impacts of local tilt illusions. First, the tilt-illusion was measured over the full range of orientation differences between short test and context lines and it was shown that the resulting function can be predicted by a model based on a vectorial population response of a bank of orientation selective channels, provided lateral inhibition between channels with the same orientation selectivity and adjacent receptive fields was postulated. Subsequently, it was demonstrated that, if the perceived shape of a test path were modified to accommodate the predicted local tilt-illusion, then this could account for previously reported changes in the detectability of a path sinusoidally modulated in radius. Further, we measured points of subjective vertical in test lines and points of subjective circularity in test paths when surrounded by modulated context paths. The tilt required to null the tilt-illusion approximated the maximum orientation difference from circular measured in the modulated paths at their point of subjective circularity, supporting the proposal that the illusory shape change is due to local changes in the position of the path arising from a response to local tilt illusions induced by the orientation context. An important corollary to this result is that such effects will generalize to all paths which are adjacent.

Analysis of shape uses local apparent position rather than physical position

Objects are often identified by the shapes of their boundaries. Here, by measuring threshold amplitudes for detection of sinusoidal modulation of local position, orientation and centrifugal speed in a closed path of Gabor patches, we show that the positions of such boundaries are misperceived to accommodate local illusions of orientation context and motion induced positional bias. These two types of illusion are shown to occur independently, but the misperception of position is additive. We conclude that, in the analysis of shape, the visual system uses the apparent rather than the veridical boundary conformation.

Spontaneous recovery of adaptation aftereffects of natural facial categories

Adaptation to a natural face attribute such as a happy face can bias the perception of a subsequent face in this dimension such as a neutral face. Such face adaptation aftereffects have been widely found in many natural facial categories. However, how temporally tuned mechanisms could control the temporal dynamics of natural face adaptation aftereffects remains unknown. To address the question, we used a deadaptation paradigm to examine whether the spontaneous recovery of natural facial aftereffects would emerge in four natural facial categories including variable categories (emotional expressions in Experiment 1 and eye gaze in Experiment 2) and invariable categories (facial gender in Experiment 3 and facial identity in Experiment 4). In the deadaptation paradigm, participants adapted to a face with an extreme attribute (such as a 100% angry face in Experiment 1) for a relatively long duration, and then deadapted to a face with an opposite extreme attribute (such as a 100% happy face in Experiment 1) for a relatively short duration. The time courses of face adaptation aftereffects were measured using a top-up manner. Deadaptation only masked the effects of initial longer-lasting adaptation, and the spontaneous recovery of adaptation aftereffects was observed at the post-test stage for all four natural facial categories. These results likely indicate that the temporal dynamics of adaptation aftereffects of natural facial categories may be controlled by multiple temporally tuned mechanisms.

Are expression aftereffects fully explained by tilt adaptation?

Facial expressions are used as critical social cues in everyday life. Adaptation to expressions causes expression aftereffects. These aftereffects are thought to reflect the operation of face-selective neural mechanisms, and are used by researchers to investigate the nature of those mechanisms. However, recent evidence suggests that expression aftereffects could be at least partially explained by the inheritance of lower-level tilt adaptation through the visual hierarchy. We investigated whether expression aftereffects could be entirely explained by tilt adaptation. Participants completed an expression adaptation task in which we controlled for the influence of tilt by changing the orientation of the adaptor relative to the test stimuli. Although tilt adaptation appeared to make some contribution to the expression aftereffect, robust expression aftereffects still remained after minimizing tilt inheritance, indicating that expression aftereffects cannot be fully explained by tilt adaptation. There was also significant reduction in the expression aftereffects after inverting the adapting face, providing evidence that face-selective processing is involved in these aftereffects.

The Flashed Face Distortion Effect Does Not Depend on Face-Specific Mechanisms

When normal faces are rapidly presented in the visual periphery, they are perceived as grotesque and distorted. This phenomenon, “The flashed-face distortion effect” (FFDE) is a powerful illusion that may reveal important properties of how faces are coded in peripheral vision. Despite the strength of the illusion (and its popularity), there has been almost no follow-up work to examine what governs the strength of the illusion or to develop a clear account of its phenomenology. Presently, our goal was to address this by manipulating aspects of facial appearance and spatial/temporal properties of the flashed-face stimulus to determine what factors modulate the illusion’s strength. In three experiments, we investigated the extent to which local contrast (operationalized by the presence or absence of makeup), image eccentricity, image size, face inversion, and presentation rate of images within the sequence each contributed to the strength of the FFDE. We found that some of these factors (eccentricity and presentation rate) mattered a great deal, while others (makeup, face inversion and image size) made little contribution to the strength of the FFDE. We discuss the implications of these results for a mechanistic account of the FFDE, and suggest several avenues for future research based on this compelling visual illusion.

Linear Summation of Repulsive and Attractive Serial Dependencies: Orientation and Motion Dependencies Sum in Motion Perception

Recent work from several groups has shown that perception of various visual attributes in human observers at a given moment is biased toward what was recently seen. This positive serial dependency is a kind of temporal averaging that exploits short-term correlations in visual scenes to reduce noise and stabilize perception. To date, this stabilizing "continuity field" has been demonstrated on stable visual attributes such as orientation and face identity, yet it would be counterproductive to apply it to dynamic attributes in which change sensitivity is needed. Here, we tested this using motion direction discrimination and predict a negative perceptual dependency: a contrastive relationship that enhances sensitivity to change. Surprisingly, our data showed a cubic-like pattern of dependencies with positive and negative components. By interleaving various stimulus combinations, we separated the components and isolated a positive perceptual dependency for motion and a negative dependency for orientation. A weighted linear sum of the separate dependencies described the original cubic pattern well. The positive dependency for motion shows an integrative perceptual effect and was unexpected, although it is consistent with work on motion priming. These findings suggest that a perception-stabilizing continuity field occurs pervasively, occurring even when it obscures sensitivity to dynamic stimuli.SIGNIFICANCE STATEMENT Recent studies show that visual perception at a given moment is not entirely veridical, but rather biased toward recently seen stimuli: a positive serial dependency. This temporal smoothing process helps perceptual continuity by preserving stable aspects of the visual scene over time, yet, for dynamic stimuli, temporal smoothing would blur dynamics and reduce sensitivity to change. We tested whether this process is selective for stable attributes by examining dependencies in motion perception. We found a clear positive dependency for motion, suggesting that positive perceptual dependencies are pervasive. We also found a concurrent negative (contrastive) dependency for orientation. Both dependencies combined linearly to determine perception, showing that the brain can calculate contrastive and integrative dependencies simultaneously from recent stimulus history when making perceptual decisions.

fMRI adaptation revisited

Adaptation has been widely used in functional magnetic imaging (fMRI) studies to infer neuronal response properties in human cortex. fMRI adaptation has been criticized because of the complex relationship between fMRI adaptation effects and the multiple neuronal effects that could underlie them. Many of the longstanding concerns about fMRI adaptation have received empirical support from neurophysiological studies over the last decade. We review these studies here, and also consider neuroimaging studies that have investigated how fMRI adaptation effects are influenced by high-level perceptual processes. The results of these studies further emphasize the need to interpret fMRI adaptation results with caution, but they also provide helpful guidance for more accurate interpretation and better experimental design. In addition, we argue that rather than being used as a proxy for measurements of neuronal stimulus selectivity, fMRI adaptation may be most useful for studying population-level adaptation effects across cortical processing hierarchies.

Tilt aftereffect due to adaptation to natural stimuli

The human visual system continuously adjusts to the current environment. To investigate these adjustments, biases in observers' perceptions owing to changes in the visual environment are measured (visual aftereffects). Typically, the stimuli used are synthetic and are composed of oriented patterns such as lines or gratings. These patterns are known to activate individual neurons in the visual cortex, but cover only a small subset of actual visual stimulations. To overcome this drawback, recent research has focused on synthetic patterns that mimic several aspects of natural stimulation. However, the aftereffects of natural stimulation per-se remain largely unexplored. Here, we interleaved presentations of unmodified natural image adaptors, selected according to criteria favoring content at a particular orientation, with presentations of targets that test a perceived orientation. This allowed us to measure the change in the perceived orientation, namely the tilt aftereffect (TAE), which resulted from repeated image presentations. Results show a close to standard TAE with adaptor durations around 500ms, which is reduced with longer presentations. Importantly, our method can be generalized to investigate other aftereffects by selecting images differently.

On the hierarchical inheritance of aftereffects in the visual system

The emotion perceived in a face can be influenced by prior exposure to a face expressing a different emotion. Here we show that displacement along a particular emotional axis, that encoding happiness and sadness, can be effected solely by a systematic change in the angle, at the center of the mouth, between the left and right halves of the mouth. We then demonstrate that adaptation to a face with the mouth distorted to change this angle, such that the face expresses an emotion on this axis, causes a face with a neutral expression to be perceived as having the opposite expression. By abstracting the mouths from the faces and examining the magnitude of the angle aftereffects in the mouths alone and in an unfamiliar orientation, we show that the magnitudes of the angle aftereffects are sufficient to account for the changes in perceived emotion in the faces. Further, by applying the distortion to the mouths asymmetrically so that the distortion is manifested by a change in orientation of the mouth stimulus rather than a change in angle, we show that the magnitude of the aftereffect can be predicted by the local tilt aftereffect. We argue, therefore, that the aftereffects of emotion are due to misperception of morphology of the face and that the misperception is due to the local change in perceived orientation due to the systematic application of the tilt aftereffect in a tilt aftereffect field. All adaptation experiments were performed using stimuli that were either high-pass or low-pass filtered for spatial frequency. Results showed that the spatial frequency specificity of the aftereffects was the same for the face, angled mouth, and oriented mouth stimuli, lending further support to the hypothesis that the aftereffects are instantiated in processes early in the visual cortex and that the aftereffects assumed to be higher level are, in fact, inherited.

Detecting global form: separate processes required for Glass and radial frequency patterns

Global processing of form information has been studied extensively using both Glass and radial frequency (RF) patterns. Models, with common early stages, have been proposed for the detection of properties of both pattern types but human performance has not been examined to determine whether the two pattern types interact in the manner this would suggest. The experiments here investigated whether low RF patterns and concentric Glass patterns, which are thought to tap the same level of processing in form-vision, are detected by a common mechanism. Six observers participated in two series of masking experiments. First: sensitivity to the presence of either coherent structure, or contour deformation, was assessed. The computational model predicted that detection of one pattern would be masked by the other. Second: a further experiment examined position coding. The model predicted that localizing the center of form in a Glass pattern would be affected by the presence of an RF pattern: sensitivity to a change of location should be reduced and the apparent location should be drawn toward the center of the masking pattern. However, the results observed in all experiments were inconsistent with the interaction predicted by the models, suggesting that separate neural mechanisms for global processing of signal are required to process these two patterns, and also indicating that the models need to be altered to preclude the interactions that were predicted but not obtained.