Interaction of retinal image and eye velocity in motion perception

Changes in visual speed perception induced by anticipatory smooth eye movements

Expectations about forthcoming visual motion shaped by observers' experiences are known to induce anticipatory smooth eye movements (ASEM) and changes in visual perception. Previous studies have demonstrated discrete effects of expectations on the control of ASEM and perception. However, the tasks designed in these studies were not able to segregate the effects of expectations and execution of ASEM itself on perception. In the current study, we attempted to directly examine the effect of ASEM itself on visual speed perception using a two-alternative forced-choice task (2AFC task), in which observers were asked to track a pair of sequentially presented visual motion stimuli with their eyes and to judge whether the second stimulus (test stimulus) was faster or slower than the first (reference stimulus). Our results showed that observers' visual speed perception, quantified by a psychometric function, shifted according to ASEM velocity. This was the case, even though there was no difference in the steady-state eye velocity. Further analyses revealed that the observers' perceptual decisions could be explained by a difference in the magnitude of retinal slip velocity in the initial phase of ocular tracking when the reference and test stimuli were presented, rather than in the steady-state phase. Our results provide psychophysical evidence of the importance of initial ocular tracking in visual speed perception and the strong impact of ASEM.

Virtual stick balancing: sensorimotor uncertainties related to angular displacement and velocity

Sensory uncertainties and imperfections in motor control play important roles in neural control and Bayesian approaches to neural encoding. However, it is difficult to estimate these uncertainties experimentally. Here, we show that magnitude of the uncertainties during the generation of motor control force can be measured for a virtual stick balancing task by varying the feedback delay, τ. It is shown that the shortest stick length that human subjects are able to balance is proportional to τ  2. The proportionality constant can be related to a combined effect of the sensory uncertainties and the error in the realization of the control force, based on a delayed proportional-derivative (PD) feedback model of the balancing task. The neural reaction delay of the human subjects was measured by standard reaction time tests and by visual blank-out tests. Experimental observations provide an estimate for the upper boundary of the average sensorimotor uncertainty associated either with angular position or with angular velocity. Comparison of balancing trials with 27 human subjects to the delayed PD model suggests that the average uncertainty in the control force associated purely with the angular position is at most 14% while that associated purely with the angular velocity is at most 40%. In the general case when both uncertainties are present, the calculations suggest that the allowed uncertainty in angular velocity will always be greater than that in angular position.

Accounting for direction and speed of eye motion in planning visually guided manual tracking

Accurate motor planning in a dynamic environment is a critical skill for humans because we are often required to react quickly and adequately to the visual motion of objects. Moreover, we are often in motion ourselves, and this complicates motor planning. Indeed, the retinal and spatial motions of an object are different because of the retinal motion component induced by self-motion. Many studies have investigated motion perception during smooth pursuit and concluded that eye velocity is partially taken into account by the brain. Here we investigate whether the eye velocity during ongoing smooth pursuit is taken into account for the planning of visually guided manual tracking. We had 10 human participants manually track a target while in steady-state smooth pursuit toward another target such that the difference between the retinal and spatial target motion directions could be large, depending on both the direction and the speed of the eye. We used a measure of initial arm movement direction to quantify whether motor planning occurred in retinal coordinates (not accounting for eye motion) or was spatially correct (incorporating eye velocity). Results showed that the eye velocity was nearly fully taken into account by the neuronal areas involved in the visuomotor velocity transformation (between 75% and 102%). In particular, these neuronal pathways accounted for the nonlinear effects due to the relative velocity between the target and the eye. In conclusion, the brain network transforming visual motion into a motor plan for manual tracking adequately uses extraretinal signals about eye velocity.

Action and attentional load can influence aperture effects on motion perception

When a moving featureless contour is viewed through a stationary circular aperture that occludes the contour's endpoints and the contour moves in a direction non-parallel to its orientation, observers report the contour's direction of motion as perpendicular to the contour's orientation regardless of its actual direction. In typical studies of this aperture effect on motion perception, observers made perceptual judgments of the line's motion. The aperture effect was not measured when observers actively controlled the line's motion. In addition, effects of attentional load on the aperture effect were not measured. Here, we demonstrated that attentional load influenced the aperture effect. Active control reduced the aperture effect, but did not eliminate it. Results have theoretical implications for motion perception and practical implications for the design of technologies that limit an observer's field-of-view such as surgical cameras.

Humans do not have direct access to retinal flow during walking

Perceived visual speed has been reported to be reduced during walking. This reduction has been attributed to a partial subtraction of walking speed from visual speed (F. H. Durgin & K. Gigone, 2007; F. H. Durgin, K. Gigone, & R. Scott, 2005). We tested whether observers still have access to the retinal flow before subtraction takes place. Observers performed a 2IFC visual speed discrimination task while walking on a treadmill. In one condition, walking speed was identical in the two intervals, while in a second condition walking speed differed between intervals. If observers have access to the retinal flow before subtraction, any changes in walking speed across intervals should not affect their ability to discriminate retinal flow speed. Contrary to this "direct access hypothesis," we found that observers were worse at discrimination when walking speed differed between intervals. The results therefore suggest that observers do not have access to retinal flow before subtraction. We also found that the amount of subtraction depended on the visual speed presented, suggesting that the interaction between the processing of visual input and of self-motion is more complex than previously proposed.

The time-delayed inverted pendulum: implications for human balance control

The inverted pendulum is frequently used as a starting point for discussions of how human balance is maintained during standing and locomotion. Here we examine three experimental paradigms of time-delayed balance control: (1) mechanical inverted time-delayed pendulum, (2) stick balancing at the fingertip, and (3) human postural sway during quiet standing. Measurements of the transfer function (mechanical stick balancing) and the two-point correlation function (Hurst exponent) for the movements of the fingertip (real stick balancing) and the fluctuations in the center of pressure (postural sway) demonstrate that the upright fixed point is unstable in all three paradigms. These observations imply that the balanced state represents a more complex and bounded time-dependent state than a fixed-point attractor. Although mathematical models indicate that a sufficient condition for instability is for the time delay to make a corrective movement, tau(n), be greater than a critical delay tau(c) that is proportional to the length of the pendulum, this condition is satisfied only in the case of human stick balancing at the fingertip. Thus it is suggested that a common cause of instability in all three paradigms stems from the difficulty of controlling both the angle of the inverted pendulum and the position of the controller simultaneously using time-delayed feedback. Considerations of the problematic nature of control in the presence of delay and random perturbations ("noise") suggest that neural control for the upright position likely resembles an adaptive-type controller in which the displacement angle is allowed to drift for small displacements with active corrections made only when theta exceeds a threshold. This mechanism draws attention to an overlooked type of passive control that arises from the interplay between retarded variables and noise.

Human cortical areas involved in sustaining perceptual stability during smooth pursuit eye movements

Because both, eye movements and object movements induce an image motion on the retina, eye movements must be compensated to allow a coherent and stable perception of our surroundings. The inferential theory of perception postulates that retinal image motion is compared with an internal reference signal related to eye movements. This mechanism allows to distinguish between the potential sources producing retinal image motion. Referring to this theory, we investigated referential calculation during smooth pursuit eye movements (SPEM) in humans using event-related functional magnetic resonance imaging (fMRI). The blood oxygenation level dependent (BOLD) response related to SPEM in front of a stable background was measured for different parametric steps of preceding motion stimuli and therefore assumed for different states of the referential system. To achieve optimally accurate anatomy and more detectable fMRI signal changes in group analysis, we applied cortex-based statistics both to all brain volumes and to defined regions of interest. Our analysis revealed that the activity in a temporal region as well as the posterior parietal cortex (PPC) depended on the velocity of the preceding stimuli. Previous single-cell recordings in monkeys demonstrated that the visual posterior sylvian area (VPS) is relevant for perceptual stability. The activation apparent in our study thus may represent a human analogue of this area. The PPC is known as being strongly related to goal-directed eye movements. In conclusion, temporal and parietal cortical areas may be involved in referential calculation and thereby in sustaining visual perceptual stability during eye movements.

Simultaneous adaptation of retinal and extra-retinal motion signals

A number of models of motion perception include estimates of eye velocity to help compensate for the incidental retinal motion produced by smooth pursuit. The 'classical' model uses extra-retinal motor command signals to obtain the estimate. More recent 'reference-signal' models use retinal motion information to enhance the extra-retinal signal. The consequence of simultaneously adapting to pursuit and retinal motion is thought to favour the reference-signal model, largely because the perception of motion during pursuit ('perceived stability') changes despite the absence of a standard motion aftereffect. The current experiments investigated whether the classical model could also account for these findings. Experiment 1 replicated the changes to perceived stability and then showed how simultaneous motion adaptation changes perceived retinal speed (a velocity aftereffect). Contrary to claims made by proponents of the reference-signal model, adapting simultaneously to pursuit and retinal motion therefore alters the retinal motion inputs to the stability computation. Experiment 2 tested the idea that simultaneous motion adaptation sets up a competitive interaction between two types of velocity aftereffect, one retinal and one extra-retinal. The results showed that pursuit adaptation by itself drove perceived stability in one direction and that adding adapting retinal motion drove perceived stability in the other. Moreover, perceived stability changed in conditions that contained no mismatch between adapting pursuit and adapting retinal motion, contrary to the reference-signal account. Experiment 3 investigated whether the effects of simultaneous motion adaptation were directionally tuned. Surprisingly no tuning was found, but this was true for both perceived stability and retinal velocity aftereffect. The three experiments suggest that simultaneous motion adaptation alters perceived stability based on separable changes to retinal and extra-retinal inputs. Possible mechanisms underlying the extra-retinal velocity aftereffect are discussed.

Natural vision reveals regional specialization to local motion and to contrast-invariant, global flow in the human brain

Visual changes in feature movies, like in real-live, can be partitioned into global flow due to self/camera motion, local/differential flow due to object motion, and residuals, for example, due to illumination changes. We correlated these measures with brain responses of human volunteers viewing movies in an fMRI scanner. Early visual areas responded only to residual changes, thus lacking responses to equally large motion-induced changes, consistent with predictive coding. Motion activated V5+ (MT+), V3A, medial posterior parietal cortex (mPPC) and, weakly, lateral occipital cortex (LOC). V5+ responded to local/differential motion and depended on visual contrast, whereas mPPC responded to global flow spanning the whole visual field and was contrast independent. mPPC thus codes for flow compatible with unbiased heading estimation in natural scenes and for the comparison of visual flow with nonretinal, multimodal motion cues in it or downstream. mPPC was functionally connected to anterior portions of V5+, whereas laterally neighboring putative homologue of lateral intraparietal area (LIP) connected with frontal eye fields. Our results demonstrate a progression of selectivity from local and contrast-dependent motion processing in V5+ toward global and contrast-independent motion processing in mPPC. The function, connectivity, and anatomical neighborhood of mPPC imply several parallels to monkey ventral intraparietal area (VIP).

Representation of head-centric flow in the human motion complex

Recent neuroimaging studies have identified putative homologs of macaque middle temporal area (area MT) and medial superior temporal area (area MST) in humans. Little is known about the integration of visual and nonvisual signals in human motion areas compared with monkeys. Through extra-retinal signals, the brain can factor out the components of visual flow on the retina that are induced by eye-in-head and head-in-space rotations and achieve a representation of flow relative to the head (head-centric flow) or body (body-centric flow). Here, we used functional magnetic resonance imaging to test whether extra-retinal eye-movement signals modulate responses to visual flow in the human MT+ complex. We distinguished between MT and MST and tested whether subdivisions of these areas may transform the retinal flow into head-centric flow. We report that interactions between eye-movement signals and visual flow are not evenly distributed across MT+. Pursuit hardly influenced the response of MT to flow, whereas the responses in MST to the same retinal stimuli were stronger during pursuit than during fixation. We also identified two subregions in which the flow-related responses were boosted significantly by pursuit, one overlapping part of MST. In addition, we found evidence of a metric relation between rotational flow relative to the head and fMRI signals in a subregion of MST. The latter findings provide an important advance over published single-cell recordings in monkey MST. A visual representation of the rotation of the head in the world derived from head-centric flow may supplement semicircular canals signals and is appropriate for cross-calibrating vestibular and visual signals.

Frame of reference transformations in motion perception during smooth pursuit eye movements

Smooth pursuit eye movements change the retinal image velocity of objects in the visual field. In order to change from a retinocentric frame of reference into a head-centric one, the visual system has to take the eye movements into account. Studies on motion perception during smooth pursuit eye movements have measured either perceived speed or perceived direction during smooth pursuit to investigate this frame of reference transformation, but never both at the same time. We devised a new velocity matching task, in which participants matched both perceived speed and direction during fixation to that during pursuit. In Experiment 1, the velocity matches were determined for a range of stimulus directions, with the head-centric stimulus speed kept constant. In Experiment 2, the retinal stimulus speed was kept approximately constant, with the same range of stimulus directions. In both experiments, the velocity matches for all directions were shifted against the pursuit direction, suggesting an incomplete transformation of the frame of reference. The degree of compensation was approximately constant across stimulus direction. We fitted the classical linear model, the model of Turano and Massof (2001) and that of Freeman (2001) to the velocity matches. The model of Turano and Massof fitted the velocity matches best, but the differences between de model fits were quite small. Evaluation of the models and comparison to a few alternatives suggests that further specification of the potential effect of retinal image characteristics on the eye movement signal is needed.

An fMRI study of optokinetic nystagmus and smooth-pursuit eye movements in humans

Both optokinetic nystagmus (OKN) and smooth-pursuit eye movements (SPEM) are subclasses of so-called slow eye movements. However, optokinetic responses are reflexive whereas smooth pursuit requires the voluntary tracking of a moving target. We used functional magnetic resonance imaging (fMRI) to determine the neural basis of OKN and SPEM, and to uncover whether the two underlying neural systems overlap or are independent at the cortical level. The results showed a largely overlapping neural circuitry. A direct comparison between activity during the execution of OKN and SPEM yielded no oculomotor-related area exclusively dedicated to one or the other eye movement type. Furthermore, the performance of SPEM evoked a bilateral deactivation of the human equivalent of the parietoinsular vestibular cortex. This finding might indicate that the reciprocally inhibitory visual-vestibular interaction involves not only OKN but also SPEM, which are both linked with the encoding of object-motion and self-motion. Moreover, we could show differential activation patterns elicited by look-nystagmus and stare-nystagmus. Look-nystagmus is characterized by large amplitudes and low-frequency resetting eye movements rather resembling SPEM. Look-nystagmus evoked activity in cortical oculomotor centers. By contrast, stare-nystagmus is usually characterized as being more reflexive in nature and as showing smaller amplitudes and higher frequency resetting eye movements. Stare-nystagmus failed to elicit significant signal changes in the same regions as look-nystagmus/SPEM. Thus, less reflexive eye movements correlated with more pronounced signal intensity. Finally, on the basis of a general investigation of slow eye movements, we were interested in a cortical differentiation between subtypes of SPEM. We compared activity associated with predictable and unpredictable SPEM as indicated by appropriate visual cues. In general, predictable and unpredictable SPEM share the same neural network, yet information about the direction of an upcoming target movement reduced the cerebral activity level.

Perceived motion direction during smooth pursuit eye movements

Although many studies have been devoted to motion perception during smooth pursuit eye movements, relatively little attention has been paid to the question of whether the compensation for the effects of these eye movements is the same across different stimulus directions. The few studies that have addressed this issue provide conflicting conclusions. We measured the perceived motion direction of a stimulus dot during horizontal ocular pursuit for stimulus directions spanning the entire range of 360 degrees. The stimulus moved at either 3 or 8 degrees/s. Constancy of the degree of compensation was assessed by fitting the classical linear model of motion perception during pursuit. According to this model, the perceived velocity is the result of adding an eye movement signal that estimates the eye velocity to the retinal signal that estimates the retinal image velocity for a given stimulus object. The perceived direction depends on the gain ratio of the two signals, which is assumed to be constant across stimulus directions. The model provided a good fit to the data, suggesting that compensation is indeed constant across stimulus direction. Moreover, the gain ratio was lower for the higher stimulus speed, explaining differences in results in the literature.

Vertical object motion during horizontal ocular pursuit: compensation for eye movements increases with presentation duration

Smooth pursuit eye movements change the retinal image motion of objects in the visual field. To enable an observer to perceive the motion of these objects veridically, the visual system has to compensate for the effects of the eye movements. The occurrence of the Filehne-illusion (illusory motion of a stationary object during smooth pursuit) shows that this compensation is not always perfect. The amplitude of the illusion appears to decrease with increasing presentation durations of the stationary object. In this study we investigated whether presentation duration has the same effect when an observer views a vertically moving object during horizontal pursuit. In this case, the pursuit eye movements cause the perceived motion path to be oblique instead of vertical; this error in perceived motion direction should decrease with higher presentation durations. In Experiment 1, we found that the error in perceived motion direction indeed decreased with increasing presentation duration, especially for higher pursuit velocities. The results of Experiment 2 showed that the error in perceived motion direction did not depend on the moment during pursuit at which the stimulus was presented, suggesting that the degree of compensation for eye movements is constant throughout pursuit. The results suggest that longer presentation durations cause the eye movement signal that is used by the visual system to increase more than the retinal signal.

The aperture problem in egocentric motion

When only a featureless straight contour of a moving object is visible, one cannot tell its true velocity and the object seems to be moving perpendicularly to its orientation. Using psychophysics and brain imaging, Goltz et al. have now demonstrated that this aperture problem also occurs in visual representations in egocentric coordinates. An afterimage of inclined lines that is perceived to move with smooth-pursuit eye movements appears to move perpendicularly to the lines rather than in the tracking direction.