Orthogonal motion after-effect illusion predicted by a model of cortical motion processing

Audiovisual interactions outside of visual awareness during motion adaptation

Motion aftereffects (MAEs), illusory motion experienced in a direction opposed to real motion experienced during prior adaptation, have been used to assess audiovisual interactions. In a previous study from our laboratory, we demonstrated that a congruent direction of auditory motion presented concurrently with visual motion during adaptation strengthened the consequent visual MAE, compared to when auditory motion was incongruent in direction. Those judgments of MAE strength, however, could have been influenced by expectations or response bias from mere knowledge of the state of audiovisual congruity during adaptation. To prevent such knowledge, we now employed continuous flash suppression to render visual motion perceptually invisible during adaptation, ensuring that observers were completely unaware of visual adapting motion and only aware of the motion direction of the sound they were hearing. We found a small but statistically significant congruence effect of sound on adaptation strength produced by invisible adaptation motion. After considering alternative explanations for this finding, we conclude that auditory motion can impact the strength of visual processing produced by translational visual motion even when that motion transpires outside of awareness.

A review of visual aftereffects in schizophrenia

Psychosis-a cardinal symptom of schizophrenia-has been associated with a failure to appropriately create or use stored regularities about past states of the world to guide the interpretation of incoming information, which leads to abnormal perceptions and beliefs. The visual system provides a test bed for investigating the role of prior experience and prediction, as accumulated knowledge of the world informs our current perception. More specifically, the strength of visual aftereffects, illusory percepts that arise after prolonged viewing of a visual stimulus, can serve as a valuable measure of the influence of prior experience on current visual processing. In this paper, we review findings from a largely older body of work on visual aftereffects in schizophrenia, attempt to reconcile discrepant findings, highlight the role of antipsychotic medication, consider mechanistic interpretations for behavioral effects, and propose directions for future research.

Low-level mediation of directionally specific motion aftereffects: Motion perception is not necessary

Previous psychophysical experiments with normal human observers have shown that adaptation to a moving dot stream causes directionally specific repulsion in the perceived angle of a subsequently viewed moving probe. In this study, we used a two-alternative forced choice task with roving pedestals to determine the conditions that are necessary and sufficient for producing directionally specific repulsion with compound adaptors, each of which contains two oppositely moving, differently colored component streams. Experiment 1 provided a demonstration of repulsion between single-component adaptors and probes moving at approximately 90° or 270°. In Experiment 2, oppositely moving dots in the adaptor were paired to preclude the appearance of motion. Nonetheless, repulsion remained strong when the angle between each probe stream and one component was approximately 30°. In Experiment 3, adapting dot pairs were kept stationary during their limited lifetimes. Their orientation content alone proved insufficient for producing repulsion. In Experiments 4–6, the angle between the probe and both adapting components was approximately 90° or 270°. Directional repulsion was found when observers were asked to visually track one of the adapting components (Exp. 6), but not when they were asked to attentionally track it (Exp. 5), nor while they passively viewed the adaptor (Exp. 4). Our results are consistent with a low-level mechanism for motion adaptation. This mechanism is not selective for stimulus color and is not susceptible to attentional modulation. The most likely cortical locus of adaptation is area V1.

Mentalizing Another's Visual World-A Novel Exploration via Motion Aftereffect

Past research on level 2 visual perspective-taking (VPT) has mostly focused on understanding the mental rotation involved when one adopts others' perspective; the mechanisms underlying how the visual world of others is mentally represented remain unclear. In three studies, we addressed this question by adopting a novel VPT task with motion stimuli and exploring the aftereffect on motion discrimination from the self-perspective. Overall the results showed a facilitation aftereffect when participants were instructed to take the avatar's perspective. Meanwhile, participants' self-reported perspective-taking tendencies correlated with the aftereffect for both instructed and spontaneous VPT tasks, when the “to-be-adopted” perspective required the participants to mentally transform their self-body clockwise. Specifically, while facilitation was induced for participants with low self-reported perspective-taking tendencies (e.g., viewing a leftward motion stimulus under another's perspective enhanced subsequent perception of leftward motion from the self-perspective), those with high self-reported perspective-taking tendencies showed an adaptation aftereffect (e.g., viewing a leftward motion stimulus under another's perspective weakened subsequent perception of leftward motion from the self-perspective). For these individuals, the adaptation effect indicated the engagement of direction-selective neurons in processing of the subsequent congruent-direction motion from self's perspective. These findings suggest that motion perception from different perspectives (self vs. another) may share the same direction-selective neural circuitry, and this possibility depends on observers' general perspective-taking tendencies.

Recurrent network dynamics reconciles visual motion segmentation and integration

In sensory systems, a range of computational rules are presumed to be implemented by neuronal subpopulations with different tuning functions. For instance, in primate cortical area MT, different classes of direction-selective cells have been identified and related either to motion integration, segmentation or transparency. Still, how such different tuning properties are constructed is unclear. The dominant theoretical viewpoint based on a linear-nonlinear feed-forward cascade does not account for their complex temporal dynamics and their versatility when facing different input statistics. Here, we demonstrate that a recurrent network model of visual motion processing can reconcile these different properties. Using a ring network, we show how excitatory and inhibitory interactions can implement different computational rules such as vector averaging, winner-take-all or superposition. The model also captures ordered temporal transitions between these behaviors. In particular, depending on the inhibition regime the network can switch from motion integration to segmentation, thus being able to compute either a single pattern motion or to superpose multiple inputs as in motion transparency. We thus demonstrate that recurrent architectures can adaptively give rise to different cortical computational regimes depending upon the input statistics, from sensory flow integration to segmentation.

Engaging and disengaging recurrent inhibition coincides with sensing and unsensing of a sensory stimulus

Even simple sensory stimuli evoke neural responses that are dynamic and complex. Are the temporally patterned neural activities important for controlling the behavioral output? Here, we investigated this issue. Our results reveal that in the insect antennal lobe, due to circuit interactions, distinct neural ensembles are activated during and immediately following the termination of every odorant. Such non-overlapping response patterns are not observed even when the stimulus intensity or identities were changed. In addition, we find that ON and OFF ensemble neural activities differ in their ability to recruit recurrent inhibition, entrain field-potential oscillations and more importantly in their relevance to behaviour (initiate versus reset conditioned responses). Notably, we find that a strikingly similar strategy is also used for encoding sound onsets and offsets in the marmoset auditory cortex. In sum, our results suggest a general approach where recurrent inhibition is associated with stimulus ‘recognition' and ‘derecognition'.

Sensory stimuli evoke temporally dynamic responses. Here the authors report that responses to odour onset and offset are orthogonally represented in the locust antennal lobe, differentially entrain oscillations, and propose a model in which they are necessary for initiation and termination of behaviour.

Evidence and Counterevidence in Motion Perception

Sensory neurons gather evidence in favor of the specific stimuli to which they are tuned, but they could improve their sensitivity by also taking counterevidence into account. The Bours-Lankheet model for motion detection uses counterevidence that relies on a specific combination of the ON and OFF channels in the early visual system. Specifically, the model detects pairs of flashes that occur separated in space and time. If the flashes have the same contrast polarity, they are interpreted as evidence in favor of the corresponding motion. But if they have opposite contrasts, they are interpreted as evidence against it. This mechanism provides an explanation for reverse-phi (the perceived reversal of an apparent motion stimulus due to periodic contrast-inversions) that is a conceptual departure from the standard explanations of the effect. Here, we investigate this counterevidence mechanism by measuring directional tuning curves of neurons in the primary visual and middle temporal cortex areas of awake, behaving macaques using constant-contrast and inverting-contrast moving dot stimuli. Our electrophysiological data support the Bours-Lankheet model and suggest that the counterevidence computation occurs at an early stage of neural processing not captured by the standard models.

How Does the Cerebral Cortex Work? Development, Learning, Attention, and 3-D Vision by Laminar Circuits of Visual Cortex

A key goal of behavioral and cognitive neuroscience is to link brain mechanisms to behavioral functions. The present article describes recent progress toward explaining how the visual cortex sees. Visual cortex, like many parts of perceptual and cognitive neocortex, is organized into six main layers of cells, as well as characteristic sublamina. Here it is proposed how these layered circuits help to realize processes of development, learning, perceptual grouping, attention, and 3-D vision through a combination of bottom-up, horizontal, and top-down interactions. A main theme is that the mechanisms which enable development and learning to occur in a stable way imply properties of adult behavior. These results thus begin to unify three fields: infant cortical development, adult cortical neurophysiology and anatomy, and adult visual perception. The identified cortical mechanisms promise to generalize to explain how other perceptual and cognitive processes work.

How previous experience shapes perception in different sensory modalities

What has transpired immediately before has a strong influence on how sensory stimuli are processed and perceived. In particular, temporal context can have contrastive effects, repelling perception away from the interpretation of the context stimulus, and attractive effects (TCEs), whereby perception repeats upon successive presentations of the same stimulus. For decades, scientists have documented contrastive and attractive temporal context effects mostly with simple visual stimuli. But both types of effects also occur in other modalities, e.g., audition and touch, and for stimuli of varying complexity, raising the possibility that context effects reflect general computational principles of sensory systems. Neuroimaging shows that contrastive and attractive context effects arise from neural processes in different areas of the cerebral cortex, suggesting two separate operations with distinct functional roles. Bayesian models can provide a functional account of both context effects, whereby prior experience adjusts sensory systems to optimize perception of future stimuli.

Decoding the motion aftereffect in human visual cortex

In the motion aftereffect (MAE), adapting to a moving stimulus causes a subsequently presented stationary stimulus to appear to move in the opposite direction. Recently, the neural basis of the motion aftereffect has received considerable interest, and a number of brain areas have been implicated in the generation of the illusory motion. Here, we use functional magnetic resonance imaging in combination with multivariate pattern classification to directly compare the neural activity evoked during the observation of both real and illusory motions. We show that the perceived illusory motion is not encoded in the same way as real motion in the same direction. Instead, suppression of the adapted direction of motion results in a shift of the population response of motion sensitive neurons in area MT+, resulting in activation patterns that are in fact more similar to real motion in orthogonal, rather than opposite directions. Although robust motion selectivity was observed in visual areas V1, V2, V3, and V4, this MAE-specific modulation of the population response was only observed in area MT+. Implications for our understanding of the motion aftereffect, and models of motion perception in general, are discussed.

The motion aftereffect

The motion aftereffect is a powerful illusion of motion in the visual image caused by prior exposure to motion in the opposite direction. For example, when one looks at the rocks beside a waterfall they may appear to drift upwards after one has viewed the flowing water for a short period-perhaps 60 seconds. The illusion almost certainly originates in the visual cortex, and arises from selective adaptation in cells tuned to respond to movement direction. Cells responding to the movement of the water suffer a reduction in responsiveness, so that during competitive interactions between detector outputs, false motion signals arise. The result is the appearance of motion in the opposite direction when one later gazes at the rocks. The adaptation is not confined to just one population of cells, but probably occurs at several cortical sites, reflecting the multiple levels of processing involved in visual motion analysis. The effect is unlikely to be caused by neural fatigue; more likely, the MAE and similar adaptation effects provide a form of error-correction or coding optimization, or both.

Bifurcation analysis applied to a model of motion integration with a multistable stimulus

A computational study into the motion perception dynamics of a multistable psychophysics stimulus is presented. A diagonally drifting grating viewed through a square aperture is perceived as moving in the actual grating direction or in line with the aperture edges (horizontally or vertically). The different percepts are the product of interplay between ambiguous contour cues and specific terminator cues. We present a dynamical model of motion integration that performs direction selection for such a stimulus and link the different percepts to coexisting steady states of the underlying equations. We apply the powerful tools of bifurcation analysis and numerical continuation to study changes to the model’s solution structure under the variation of parameters. Indeed, we apply these tools in a systematic way, taking into account biological and mathematical constraints, in order to fix model parameters. A region of parameter space is identified for which the model reproduces the qualitative behaviour observed in experiments. The temporal dynamics of motion integration are studied within this region; specifically, the effect of varying the stimulus gain is studied, which allows for qualitative predictions to be made.

Neural population representation hypothesis of visual flow and its illusory after effect in the brain: psychophysics, neurophysiology and computational approaches

The neural representation of motion aftereffects induced by various visual flows (translational, rotational, motion-in-depth, and translational transparent flows) was studied under the hypothesis that the imbalances in discharge activities would occur in favor in the direction opposite to the adapting stimulation in the monkey MST cells (cells in the medial superior temporal area) which can discriminate the mode (i.e., translational, rotational, or motion-in-depth) of the given flow. In single-unit recording experiments conducted on anaesthetized monkeys, we found that the rate of spontaneous discharge and the sensitivity to a test stimulus moving in the preferred direction decreased after receiving an adapting stimulation moving in the preferred direction, whereas they increased after receiving an adapting stimulation moving in the null direction. To consistently explain the bidirectional perception of a transparent visual flow and its unidirectional motion aftereffect by the same hypothesis, we need to assume the existence of two subtypes of MST D cells which show directionally selective responses to a translational flow: component cells and integration cells. Our physiological investigation revealed that the MST D cells could be divided into two types: one responded to a transparent flow by two peaks at the instances when the direction of one of the component flow matched the preferred direction of the cell, and the other responded by a single peak at the instance when the direction of the integrated motion matched the preferred direction. In psychophysical experiments on human subjects, we found evidence for the existence of component and integration representations in the human brain. To explain the different motion perceptions, i.e., two transparent flows during presentation of the flows and a single flow in the opposite direction to the integrated flows after stopping the flow stimuli, we suggest that the pattern-discrimination system can select the motion representation that is consistent with the perception of the pattern from two motion representations. We discuss the computational aspects related to the integration of component motion fields.

Motion aftereffect duration is not changed by perceptual learning: evidence against the representation modification hypothesis

The representation modification hypothesis of perceptual learning attributes the practice-induced improvements in sensitivity and/or discriminability to changes in the early visual areas. We used motion aftereffects (MAE) to probe the representations of motion direction. In two experiments, four practice sessions on a fine direction-discrimination task caused large stimulus-specific improvements in d' but no significant stimulus-specific changes in either static or dynamic MAE duration at posttest relative to a pretest. Power analysis indicated that the data were approximately 100 times more likely given the hypothesis of no MAE change than the hypothesis of a 10% relative change. In light of converging evidence in the MAE literature, this suggests that little or no change occurred in the cortical representations of visual motion up to and including area MT. The task specificity of the learning effect challenges the representation modification hypothesis and supports an alternative-selective reweighting.

Spatially localized time shifts of the perceptual stream

Visual events trigger representations in different locations and times in the brain. In experience, however, these various neural responses refer to a single unified cause. To investigate how representations might be brought into temporal alignment, we attempted to locally manipulate neural processing in such a way that identical, simultaneous sequences would appear temporally misaligned. After adaptation to a 20 Hz sequentially expanding and contracting concentric grating, a running clock presented in the adapted region of the visual field appeared advanced relative to an identical clock presented simultaneously in an unadapted region. No such effect was observed following 5-Hz adaptation. Clock time reports following an exogenous cue showed the same effect of adaptation on perceived time, demonstrating that the apparent temporal misalignment was not mediated by differences in target selection or allocation of attention. This effect was not mediated by the apparent speed of the adapted clock: a clock in a 20-Hz-adapted spatial location appeared slower than a clock in a 5-Hz-adapted location, rather than faster. Furthermore, reaction times for a clock-hand orientation discrimination task were the same following 5- and 20-Hz adaptation, indicating that neural processing latencies were not differentially affected. Altogether, these findings suggest that the fragmented perceptual stream might be actively brought into temporal alignment through adaptive local mechanisms operating in spatially segregated regions of the visual field.

Human middle temporal cortex, perceptual bias, and perceptual memory for ambiguous three-dimensional motion

When faced with inconclusive or conflicting visual input human observers experience one of multiple possible perceptions. One factor that determines perception of such an ambiguous stimulus is how the same stimulus was perceived on previous occasions, a phenomenon called perceptual memory. We examined perceptual memory of an ambiguous motion stimulus while applying transcranial magnetic stimulation (TMS) to the motion-sensitive areas of the middle temporal cortex (hMT+). TMS increased the predominance of whichever perceptual interpretation was most commonly reported by a given observer at baseline, with reduced perception of the less favored interpretation. This increased incidence of the preferred percept indicates impaired long-term buildup of perceptual memory traces that normally act against individual percept biases. We observed no effect on short-term memory traces acting from one presentation to the next. Our results indicate that hMT+ is important for the long-term buildup of perceptual memory for ambiguous motion stimuli.

Motion fading and the motion aftereffect share a common process of neural adaptation

After prolonged viewing of a slowly drifting or rotating pattern under strict fixation, the pattern appears to slow down and then momentarily stop. Here, we show that this motion fading occurs not only for slowly moving stimuli, but also for stimuli moving at high speed; after prolonged viewing of high-speed stimuli, the stimuli appear to slow down but not to stop. We report psychophysical evidence that the same neural adaptation process likely gives rise to motion fading and to the motion aftereffect.

Modeling the simulated real-world optic flow motion aftereffect

We investigated the simulated real-world optic flow motion aftereffect (MAE) (illusory sense of moving backward following adaptation to expansive optic flow). In Experiment 1, adaptation duration was either 30, 120, 240, or 480 s.

RESULTS

duration of the MAE grew with increasing adaptation duration. In Experiment 2, the MAE was measured across different combinations of values of global optical flow rate and optical edge rate.

RESULTS

the aftereffect was selective for global optical flow rate, suggesting that the aftereffect reflects gain changes at processing levels where a sense of self-motion is generated. RESULTS were used in a computational model of this MAE, which was a modified framework by van de Grind et al. [Vision Res.44, 2269 (2004)].

Sensitivity for reverse-phi motion

Low-level contrast information in the primary visual pathway is represented in two different channels. ON-center cells signal positive contrasts and OFF-center cells signal negative contrasts. In this study we address the question whether initial motion analysis is performed separately in these two channels, or also through combination of signals from ON and OFF cells. We quantitatively compared motion coherence detection for regular and for reverse-phi motion stimuli. In reverse-phi motion the contrast of a pattern flips during displacements. Sensitivity is therefore based on correlating positive and negative contrasts, whereas for regular motion it is based on correlating similar contrasts. We compared tuning curves for step size and temporal interval for stimuli in which motion information was limited to a single combination of step size and interval. Tuning for step size and temporal interval was highly similar for the two types of motion. Moreover, minimal coherence thresholds for both types of motion matched quantitatively, irrespective of dot density. We also measured sensitivity for so-called no-phi motion stimuli, in which the contrast of displaced dots was set to zero. Sensitivity for no-phi motion was low for stimuli containing only black or only white dots. When both dot polarities were present in the stimulus, sensitivity was absent. Thus, motion information based on separate contrasts was effectively cancelled by a component based on different contrasts. Together these results show equal efficiency in correlating dots of opposite contrast and of similar contrast, which strongly suggests efficient detection of correlations across ON and OFF channels.

Effects of adaptation to Glass pattern structure and to path of optic flow

The effects of adaptation to radial and rotational polar optic flow on the classification of noisy test patterns are shown to be very similar to the effects of adaptation to polar Glass patterns on the classification of noisy Glass patterns (Clifford, C. W. G. & Weston, E. (2005). Aftereffect of adaptation to Glass patterns. Vision Research, 45 1355-1363.). In both cases there is a large shift in the signal strength at which test patterns are classified as radial or rotational with equal probability. Two asymmetries were discovered: (1) adaptation to optic flow alters the classification of Glass patterns, but the reverse is not true; and (2) adaptation to Glass patterns decreases detectability of patterns of the same type, but adaptation to optic flow has little effect on the detectability of patterns of any type. We conclude that the mechanisms that detect radial and rotational Glass patterns are independent and independently adaptable, but that the mechanisms that detect the path of optic flow, when directional effects are cancelled out, are linked in an opponent, push-pull fashion.

A single system explains human speed perception

Motion is fully described by a direction and a speed. The processing of direction information by the visual system has been extensively studied; much less is known, however, about the processing of speed. Although it is generally accepted that the direction of motion is processed by a single motion system, no such consensus exists for speed. Psychophysical data from humans suggest two separate systems processing luminance-based fast and slow speeds, whereas neurophysiological recordings in monkeys generally show continuous speed representation, hinting at a single system. Although the neurophysiological findings hint at a single system, they remain inconclusive as only a limited amount of cells can be measured per study and, possibly, the putative different motion systems are anatomically separate. In three psychophysical motion adaptation experiments, we show that predictions on the basis of the two-motion system hypothesis are not met. Instead, concurrent modeling showed that both here-presented and previous data are consistent with a single system subserving human speed perception. These findings have important implications for computational models of motion processing and the low-level organization of the process.

Reduction in the motion coherence threshold for the same direction as that perceived during adaptation

When a stimulus of equally spaced parallel lines is displaced slightly in a direction perpendicular to the lines, low-speed motion toward the displacement direction can be perceived. Such a stimulus embodies both a low-speed and a high-speed component in opposite directions. The dominance of the former would result in the perception of low-speed motion. To see how the unperceived high-speed component is processed by the visual system, I measured coherence thresholds for random-dot test-motion with and without prior adaptation to the low-speed motion of the equally spaced parallel lines. The results depended on the test speeds. At low speeds, the coherence thresholds for the same direction as that perceived during the adaptation phase increased and the coherence thresholds for the opposite direction decreased. At high speeds, the same adaptation resulted in an opposite effect. The threshold reduction for high-speed motion in the same direction as that perceived during the adaptation phase and the threshold elevation in the opposite direction might be due to adaptation of a high-speed processing channel to the high-speed component that was not perceived but was nevertheless detected.

Effects of feature-based attention on the motion aftereffect at remote locations

Previous studies have shown that attention to a particular stimulus feature, such as direction of motion or color, enhances neuronal responses to unattended stimuli sharing that feature. We studied this effect psychophysically by measuring the strength of the motion aftereffect (MAE) induced by an unattended stimulus when attention was directed to one of two overlapping fields of moving dots in a different spatial location. When attention was directed to the same direction of motion as the unattended stimulus, the unattended stimulus induced a stronger MAE than when attention was directed to the opposite direction. Also, when the unattended location contained either uncorrelated motion or had no stimulus at all an MAE was induced in the opposite direction to the attended direction of motion. The strength of the MAE was similar regardless of whether subjects attended to the speed or luminance of the attended dots. These results provide further support for a global feature-based mechanism of attention, and show that the effect spreads across all features of an attended object, and to all locations of visual space.

Dynamics of directional selectivity in MT receptive field centre and surround

We studied receptive field organization of motion-sensitive neurons in macaque middle temporal cortical area (MT), by mapping direction selectivity in space and in time. Stimuli consisted of pseudorandom sequences of single motion steps presented simultaneously at many different receptive field locations. Spatio-temporal receptive field profiles were constructed by cross-correlating stimuli and spikes. The resulting spike-triggered averages revealed centre-surround organization. The temporal dynamics of the receptive fields were generally biphasic with increased probability for the preferred direction at short latency (50-70 ms) and decreased probability at longer latency (80-100 ms). The response latency of the receptive field surround was on average 16 ms longer than that of the centre. Our results show that surround input and biphasic behaviour reflect two different mechanisms, which make MT cells specifically sensitive to motion contrast in space and time.

The integration of multiple stimulus features by V1 neurons

We investigated how V1 neurons integrate two stimulus features by presenting stimuli from a stimulus set made up of all combinations of eight different directions of motion and nine binocular disparities. We investigated the occurrence and shape of the resulting joint tuning function. Among V1 neurons, approximately 80% were jointly tuned for disparity with orientation or direction. The joint tuning function of all jointly tuned neurons was separable into distinct tuning for disparity on the one hand, and orientation or direction tuning on the other. The degree of separability and the mutual information between the stimulus and the firing rates were strongly correlated. The mutual information of jointly tuned neurons when both features were decoded together was highly correlated with the mutual information when the two features were decoded separately, and the information was then summed. Jointly tuned neurons were just as good at representing information about single features as neurons tuned for only a single feature. The tuning properties of most jointly tuned neurons did not dynamically evolve over time, nor did jointly tuned neurons respond earlier than neurons tuned for only a single feature. The response selectivity of V1 neurons is low and decreases the information that a neuron represents about a stimulus. Together these results suggest that distinct stimulus features are integrated very early in visual processing. Furthermore, V1 generates a distributed representation through low response selectivity that avoids the curse of dimensionality by using separable joint tuning functions.

Motion mechanisms in macaque MT

The macaque middle temporal area (MT) is exquisitely sensitive to visual motion and there is a large amount of evidence that neural activity in MT is tightly correlated with the perception of motion. The mechanisms by which MT neurons achieve their directional selectivity, however, have received considerably less attention. We investigated the motion-energy model as a description of motion mechanisms in macaque MT. We first confirmed one of the predictions of the motion-energy model; macaques-just like humans-perceive a reversed direction of motion when a stimulus reverses contrast with every displacement (reverse-phi). This reversal of perceived direction had a clear correlate in the neural responses of MT cells, which were predictive of the monkey's behavioral decisions. Second, we investigated how multiple motion-energy components are combined. Psychophysical data have been used to argue that motion-energy components representing opposite directions are subtracted from each other. Our data show, however, that the interactions among motion-energy components are more complex. In particular, we found that the influence of a given component on the response to a stimulus consisting of multiple components depends on factors other than the response to that component alone. This suggests that there are subthreshold nonlinear interactions among multiple motion-energy components; these could take place within MT or in earlier stages of the motion network such as V1. We propose a model that captures the complexity of these component interactions by means of a competitive interaction among the components. This provides a better description of the MT responses than the subtractive motion opponency envisaged in the motion-energy model, even when the latter is combined with a gain-control mechanism. The competitive interaction extends the dynamic range of the cells and allows them to provide information on more subtle changes in motion patterns, including changes that are not purely directional.

Storage for free: a surprising property of a simple gain-control model of motion aftereffects

If a motion aftereffect (MAE) for given adaptation conditions has a duration T s, and the eyes are closed after adaptation during a waiting period tw=T s before testing, an unexpected MAE of a 'residual' duration TrT s is experienced. This effect is called 'storage' and it is often quantified by a storage factor sigma=TrT/T, which can reach values up to about 0.7-0.8. The phenomenon and its name have invited explanations in terms of inhibition of recovery during darkness. We present a model based on the opposite idea, that an effective test stimulus quickens recovery relative to darkness or other ineffective test stimuli. The model is worked out in mathematical detail and proves to explain 'storage' data from the literature, on the static MAE (sMAE: an MAE experienced for static test stimuli). We also present results of a psychophysical experiment with moving random pixel arrays, quantifying storage phenomena both for the sMAE and the dynamic MAE (dMAE: an MAE experienced for a random dynamic noise test stimulus). Storage factors for the dMAE are lower than for the sMAE. Our model also gives an excellent description of these new data on storage of the dMAE. The term 'storage' might therefore be a misnomer. If an effective test stimulus influences all direction tuned motion sensors indiscriminately and thus speeds up equalization of gains, one gets the storage phenomenon for free.

Velocity dependence of the interocular transfer of dynamic motion aftereffects

It is well established that motion aftereffects (MAEs) can show interocular transfer (IOT); that is, motion adaptation in one eye can give a MAE in the other eye. Different quantification methods and different test stimuli have been shown to give different IOT magnitudes, varying from no to almost full IOT. In this study, we examine to what extent IOT of the dynamic MAE (dMAE), that is the MAE seen with a dynamic noise test pattern, varies with velocity of the adaptation stimulus. We measured strength of dMAE by a nulling method. The aftereffect induced by adaptation to a moving random-pixel array was compensated (nulled), during a brief dynamic test period, by the same kind of motion stimulus of variable luminance signal-to-noise ratio (LSNR). The LSNR nulling value was determined in a Quest-staircase procedure. We found that velocity has a strong effect on the magnitude of IOT for the dMAE. For increasing speeds from 1.5 deg s(-1) to 24 deg s(-1) average IOT values increased about linearly from 18% to 63% or from 32% to 83%, depending on IOT definition. The finding that dMAEs transfer to an increasing extent as speed increases, suggests that binocular cells play a more dominant role at higher speeds.

Unconscious adaptation: a new illusion of depth induced by stimulus features without depth

Here, we show a new illusion of depth induced by psychophysical adaptation to dynamic random-dot stereograms (RDS) that are interocularly anticorrelated (i.e., in which the images for the two eyes have reversed contrast polarity with each other). After prolonged viewing of anticorrelated RDS, the presentation of uncorrelated RDS (i.e., in which two images are mutually independent random-dot patterns) produces the sensation of depth, although both anticorrelated and uncorrelated RDSs are perceptually rivalrous with no consistent depth by themselves. Contrary to other aftereffects demonstrated in a number of visual dimensions, including motion, orientation, and disparity, this illusion results from unconscious adaptation; observers are not aware of what they are being adapted to during the process of adaptation. We further demonstrate that this illusion can be predicted from the simulated responses of disparity-selective neurons based on a local filtering model. Model simulations indicate that the inspection of anticorrelated RDS causes the adaptation of all disparity detectors except one sensitive to its disparity; therefore, those selectively unadapted detectors show relatively strong activation in response to the subsequent presentation of uncorrelated RDS and produce depth perception.

Neuronal adaptation to visual motion in area MT of the macaque

The responsivity of primary sensory cortical neurons is reduced following prolonged adaptation, but such adaptation has been little studied in higher sensory areas. Adaptation to visual motion has strong perceptual effects, so we studied the effect of prolonged stimulation on neuronal responsivity in the macaque's area MT, a cortical area whose importance to visual motion perception is well established. We adapted MT neurons with sinusoidal gratings drifting in the preferred or null direction. Preferred adaptation reduced the responsiveness of MT cells, primarily by changing their contrast gain, and this effect was spatially specific within the receptive field. Null adaptation reduced the ability of null gratings to inhibit the response to a simultaneously presented preferred stimulus. While both preferred and null adaptation alter MT responses, these effects probably do not occur in MT neurons but are likely to reflect adaptation-induced changes in contrast gain earlier in the visual pathway.

A gain-control model relating nulling results to the duration of dynamic motion aftereffects

Strength of the motion aftereffect (MAE) is most often quantified by its duration, a high-variance and rather 'subjective' measure. With the help of an automatic gain-control model we quantitatively relate nulling-thresholds, adaptation strength, direction discrimination threshold, and duration of the dynamic MAE (dMAE). This shows how the nulling threshold, a more objective two-alternative forced-choice measure, relates to the same system property as MAE-durations. Two psychophysical experiments to test the model use moving random-pixel-arrays with an adjustable luminance signal-to-noise ratio. We measure MAE-duration as a function of adaptation strength and compare the results to the model prediction. We then do the same for nulling-thresholds. Model predictions are strongly supported by the psychophysical findings. In a third experiment we test formulae coupling nulling threshold, MAE-duration, and direction-discrimination thresholds, by measuring these quantities as a function of speed. For the medium-to-high speed range of these experiments we found that nulling thresholds increase and dMAE-durations decrease about linearly, whereas direction discrimination thresholds increase exponentially with speed. The model description then suggests that the motion-gain decreases, while the noise-gain and model's threshold increase with speed.

Activity patterns in human motion-sensitive areas depend on the interpretation of global motion

Numerous imaging studies have contributed to the localization of motion-sensitive areas in the human brain. It is, however, still unclear how these areas contribute to global motion perception. Here, we investigate with functional MRI whether the motion-sensitive area hMT+/V5 is involved in perceptual segmentation and integration of motion signals. Stimuli were overlapping moving gratings that can be perceived either as two independently moving, transparent surfaces or as a single surface moving in an intermediate direction. We examined whether motion-sensitive area hMT+/V5 is involved in mediating the switches between the two percepts. The data show differential activation of hMT+/V5 with perceptual switches, suggesting that these are associated with a reconfiguration of cell assemblies in this area.

Multiple uses of visual motion. The case for stability in sensory cortex

The problem of 'readout' from sensory maps has received considerable attention recently. Specifically, many experiments in different systems have suggested that the routing of sensory signals from cortical maps can be impressively flexible. In this review, we discuss many of the experiments addressing readout of motion signals from the middle temporal area (also known as V5) in the macaque monkey. We focus on two different types of output: perceptual reports (categorical decisions, usually) and motion-guided eye movements. We specifically consider situations in which multiple-motion vectors present in the stimulus are combined, as well as those in which one or more of the vectors in the stimulus is selected for output. The results of these studies suggest that in some situations multiple motions are vector averaged, while in others multiple vectors can be maintained. Interestingly, in most of the experiments producing a single (often average) vector, the output is a movement. However, many perceptual experiments involve the simultaneous processing of multiple-stimulus motions. One prosaic explanation for this pattern of apparently discrepant results is that different downstream structures impose different rules, in parallel, on the output from sensory maps such as the one in the middle temporal area. We also specifically discuss the case of motion opponency, a specific readout rule that has been posited to explain perceptual phenomena such as the waterfall illusion (motion aftereffect). We present evidence from a recent experiment showing that an opponent step must occur downstream from the middle temporal area itself. This observation is consistent with our proposal that significant processing need occur downstream from sensory structures. If a single output is to be used for multiple purposes, often at once, this necessitates a degree of task invariance on the sensory information present even at a relatively high level of cortical processing.

Neural correlates of structure-from-motion perception in macaque V1 and MT

Structure-from-motion (SFM) is the perception of three-dimensional shape from motion cues. We used a bistable SFM stimulus, which can be perceived in one of two different ways, to study how neural activity in cortical areas V1 and MT is related to SFM perception. Monkeys performed a depth-order task, where they indicated in which direction the front surface of a rotating SFM cylinder display was moving. To prevent contamination of the neural data because of eye position effects, all experiments with significant effects of radius, vergence, and velocity were excluded. As expected, the activity of approximately 50% of neurons in V1 and approximately 80% of neurons in MT is affected by the stimulus. Furthermore, the activity of 20% of neurons in area V1 is modulated with the percept. This proportion is higher in MT, where the activity of >60% of neurons is modulated with the percept. In both areas, this perceptual modulation occurs only in neurons with activity that is also affected by the stimulus. The perceptual modulation is not correlated with neural tuning properties in area V1, but it is in area MT. Together, these results suggest that V1 is not directly involved in the generation of the SFM percept, whereas MT is. The perceptual modulation in V1 may be attributable to top-down feedback from MT.

Motion integration during motion aftereffects

The perceived global motion of a stimulus depends on how its different local motion-direction vectors are distributed in space and time. When they are explicitly co-localized, as in the case of locally paired motion, competitive motion integration mechanisms produce a unitary global motion direction determined by their vector average. During motion aftereffects induced by simultaneous adaptation to multiple motion directions, just as in the case of locally paired motion, different directional signals originate simultaneously from exactly the same position in space. Therefore, the perceived global motion direction during motion aftereffects results from local vector averaging of the co-localized motion-direction signals induced by adaptation.

Temporal dynamics of binocular disparity processing with corticogeniculate interactions

A neural model is developed to probe how corticogeniculate feedback may contribute to the dynamics of binocular vision. Feedforward and feedback interactions among retinal, lateral geniculate, and cortical simple and complex cells are used to simulate psychophysical and neurobiological data concerning the dynamics of binocular disparity processing, including correct registration of disparity in response to dynamically changing stimuli, binocular summation of weak stimuli, and fusion of anticorrelated stimuli when they are delayed, but not when they are simultaneous. The model exploits dynamic rebounds between opponent ON and OFF cells that are due to imbalances in habituative transmitter gates. It shows how corticogeniculate feedback can carry out a top-down matching process that inhibits incorrect disparity responses and reduces persistence of previously correct responses to dynamically changing displays.

Neuronal basis of the motion aftereffect reconsidered

Several fMRI studies have reported MT+ response increases correlated with perception of the motion aftereffect (MAE). However, attention can strongly affect MT+ responses, and subjects may naturally attend more to the MAE than control trials without MAE. We found that requiring subjects to attend to motion on both MAE and control trials produced equal levels of MT+ response, suggesting that attention may have confounded the interpretation of previous experiments; in our data, attention accounts for the entire effect. After eliminating this confound, we observed that direction-selective motion adaptation produced a direction-selective imbalance in MT+ responses (and earlier visual areas), and yielded a corresponding asymmetry in speed discrimination thresholds. These findings provide physiological evidence that population level response imbalances underlie the MAE, and quantify the relative proportions of direction-selective neurons across human visual areas.

The motion aftereffect: more than area V5/MT? Evidence from 15O-butanol PET studies

The motion aftereffect is a perceptual phenomenon which has been extensively investigated both psychologically and physiologically. Neuroimaging techniques have recently demonstrated that area V5/MT is activated during the perception of this illusion. The aim of this study was to test the hypothesis if a more broadly distributed network of brain regions subserves the motion aftereffect. To identify the neuronal structures involved in the perception of the motion aftereffect, regional cerebral blood flow (rCBF) measurements with positron emission tomography were performed in six normal volunteers. Data were analysed using SPM96. The motion-sensitive visual areas including area V5/MT were activated in both hemispheres. Additionally, the lateral parietal cortex bilaterally, the right dorsolateral prefrontal cortex, the anterior cingulate cortex and the left cerebellum showed significant increases in rCBF values during the experience of the waterfall illusion. In a further reference condition with identical attentional demand but no perception of a motion aftereffect elevated rCBF were found in these regions as well. In conclusion, our findings support the notion that the perceptual illusion of motion arises exclusively in the motion-sensitive visual area V5/MT. In addition, a more widespread network of brain regions including the prefrontal and parietal cortex is activated during the waterfall illusion which represents a non-motion aftereffect-specific subset of brain areas but is involved in more basic attentional processing and cognition.

Slow and fast visual motion channels have independent binocular-rivalry stages

We have previously reported a transparent motion after-effect indicating that the human visual system comprises separate slow and fast motion channels. Here, we report that the presentation of a fast motion in one eye and a slow motion in the other eye does not result in binocular rivalry but in a clear percept of transparent motion. We call this new visual phenomenon 'dichoptic motion transparency' (DMT). So far only the DMT phenomenon and the two motion after-effects (the 'classical' motion after-effect, seen after motion adaptation on a static test pattern, and the dynamic motion after-effect, seen on a dynamic-noise test pattern) appear to isolate the channels completely. The speed ranges of the slow and fast channels overlap strongly and are observer dependent. A model is presented that links after-effect durations of an observer to the probability of rivalry or DMT as a function of dichoptic velocity combinations. Model results support the assumption of two highly independent channels showing only within-channel rivalry, and no rivalry or after-effect interactions between the channels. The finding of two independent motion vision channels, each with a separate rivalry stage and a private line to conscious perception, might be helpful in visualizing or analysing pathways to consciousness.

Seeing multiple directions of motion-physiology and psychophysics

Dot patterns sliding transparently across one another are normally perceived as independently moving surfaces. Recordings from direction-selective neurons in area MT of the macaque suggested that this perceptual segregation did not depend on the presence of two peaks in the population activity. Rather, the visual system seemed to use overall shape of the population response to determine the number and directions of motion components. This approach explained a number of perceptual phenomena, including susceptibility of the motion system to direction metamers, motion patterns combining three or five directions incorrectly perceived by subjects as comprising only two directions. Our findings offer insights into the coding of multi-valued sensory signals and provide constraints for biologically based computational models.

Revisiting motion repulsion: evidence for a general phenomenon?

Previous studies have found large misperceptions when subjects are reporting the perceived angle between two directions of motion moving transparently at an acute angle, the so called motion repulsion. While these errors have been assumed to be caused by interactions between the two directions present, we reassessed these earlier measurements taking into account recent findings about directional misperceptions affecting the perception of single motion (reference repulsion). While our measurements confirm that errors in directional judgments of transparent motions can indeed be as big as 22 degrees we find that motion repulsion, i.e. the interaction between two directions, contributes at most about 7 degrees to these errors. This value is comparable to similar repulsion effects in orientation perception and stereoscopic depth perception, suggesting that they share a common neural basis. Our data further suggest that fast time scale adaptation and/or more general interactions between neurons contribute to motion repulsion while tracking eye movements play little or no role. These findings should serve as important constraints for models of motion perception.

Enhanced motion aftereffect for complex motions

We measured the magnitude of the motion after effect (MAE) elicited by gratings viewed through four spatial apertures symmetrically positioned around fixation. The gratings were identical except for their orientations, which were varied to form patterns of global motion corresponding to radiation, rotation or translation. MAE magnitude was estimated by three methods: the duration of the MAE; the contrast required to null the MAE and the threshold elevation for detecting an abrupt jump. All three techniques showed that MAEs for radiation and rotation were greater than those for translation. The greater adaptability of radiation and rotation over translation also was observed in areas of the display where no adapting stimulus had been presented. We also found that adaptation to motion in one direction had equal effects on sensitivity to motion in the same and opposite directions.

Aftereffect of high-speed motion

A visual illusion known as the motion aftereffect is considered to be the perceptual manifestation of motion sensors that are recovering from adaptation. This aftereffect can be obtained for a specific range of adaptation speeds with its magnitude generally peaking for speeds around 3 deg s-1. The classic motion aftereffect is usually measured with a static test pattern. Here, we measured the magnitude of the motion aftereffect for a large range of velocities covering also higher speeds, using both static and dynamic test patterns. The results suggest that at least two (sub)populations of motion-sensitive neurons underlie these motion aftereffects. One population shows itself under static test conditions and is dominant for low adaptation speeds, and the other is prevalent under dynamic test conditions after adaptation to high speeds. The dynamic motion aftereffect can be perceived for adaptation speeds up to three times as fast as the static motion aftereffect. We tested predictions that follow from the hypothesised division in neuronal substrates. We found that for exactly the same adaptation conditions (oppositely directed transparent motion with different speeds), the aftereffect direction differs by 180 degrees depending on the test pattern. The motion aftereffect is opposite to the pattern moving at low speed when the test pattern is static, and opposite to the high-speed pattern for a dynamic test pattern. The determining factor is the combination of adaptation speed and type of test pattern.

Integration after adaptation to transparent motion: static and dynamic test patterns result in different aftereffect directions

One of the many interesting questions in motion aftereffect (MAE) research is concerned with the location(s) along the pathway of visual processing at which certain perceptual manifestations of this illusory motion originate. One such manifestation is the unidirectionality of the MAE after adaptation to moving plaids or transparent motion. This unidirectionality has led to the suggestion that the origin of this MAE might be a single source (gain control) located at, or beyond areas that are believed to be responsible for the integration of motion signals. In this report we present evidence against this suggestion using a simple experiment. For the same adaptation pattern, which consisted of two orthogonally moving transparent patterns with different speeds, we show that the direction of the resulting unidirectional MAE depends on the nature of the test stimulus. We used two kinds of test patterns: static and dynamic. For exactly the same adaptation conditions, the difference in MAE direction between testing with static and dynamic patterns can be as large as 50 degrees. This finding suggests that this MAE is not just a perceptual manifestation of a passive recovery of adapted motion sensors but an active integrative process using the output of different gain controls. A process which takes place after adaptation. These findings are in line with the idea that there are several sites of adaptation along the pathway of visual motion processing and that the nature of the test pattern determines the fate of our perceptual experience of the MAE.

Recovery of fMRI activation in motion area MT following storage of the motion aftereffect

We used functional magnetic resonance imaging (fMRI) during storage of the motion aftereffect (MAE) to examine the relationship between motion perception and neural activity in the human cortical motion complex MT+ (including area MT and adjacent motion-selective cortex). MT+ responds not only to physical motion but also to illusory motion, as in the MAE when subjects who have adapted to continuous motion report that a subsequent stationary test stimulus appears to move in the opposite direction. In the phenomenon of storage, the total decay time of the MAE is extended by inserting a dark period between adaptation and test phases. That is, when the static test pattern is presented after a storage period equal in duration to the normal MAE, the illusory motion reappears for almost as long as the original effect despite the delay. We examined fMRI activation in MT+ during and after storage. Seven subjects viewed continuous motion, followed either by an undelayed stationary test (immediate MAE) or by a completely dark storage interval preceding the test (stored MAE). Like the perceptual effect, activity in MT+ dropped during the storage interval then rebounded to reach a level much higher than after the same delay without storage. Although MT+ activity was slightly enhanced during the storage period following adaptation to continuous motion (compared with a control sequence in which the adaptation grating oscillated and no MAE was perceived), this enhancement was much less than that observed during the perceptual phenomenon. These results indicate that following adaptation, activity in MT+ is pronounced only with the presentation of an appropriate visual stimulus, during which the MAE is perceived.

Motion after-effect due to binocular sum of adaptation to linear motion

The motion after-effect (MAE) can be elicited by adapting observers to global motion of randomly distributed dots before they view a display containing dots moving in random directions, but no global motion. Experiments by others have shown that if the adaptation stimulus contains two directions of motion, the MAE points opposite to the vector sum of the adapting directions. The present study investigated whether such vector addition in the MAE could also occur if the two directions of motion were presented to separate eyes. Observers were adapted to different, but not opposite, directions of motion in the two eyes. Either the left eye, the right eye, or both eyes were tested. Observers reported the direction of perceived motion during the test. When they saw the test stimulus with both eyes, observers reported seeing motion in the direction opposite that of the vector sum of the adaptation directions. In the monocular test conditions observers reported MAE directions opposite to the corresponding monocular adaptation directions. In a second experiment we verified that subjects had interocular transfer of the MAE. Together these results are consistent with a model in which (1) addition of adaptation directions occurs at a binocular site; (2) directional adaptation occurs at a monocular site; and (3) monocular adaptation is able to change the threshold for obtaining an MAE at the binocular site, thus acting like binocular adaptation in interocular transfer of the MAE.