Cancellation of self-induced retinal image motion during smooth pursuit eye movements

Bio-mimetic high-speed target localization with fused frame and event vision for edge application

Evolution has honed predatory skills in the natural world where localizing and intercepting fast-moving prey is required. The current generation of robotic systems mimics these biological systems using deep learning. High-speed processing of the camera frames using convolutional neural networks (CNN) (frame pipeline) on such constrained aerial edge-robots gets resource-limited. Adding more compute resources also eventually limits the throughput at the frame rate of the camera as frame-only traditional systems fail to capture the detailed temporal dynamics of the environment. Bio-inspired event cameras and spiking neural networks (SNN) provide an asynchronous sensor-processor pair (event pipeline) capturing the continuous temporal details of the scene for high-speed but lag in terms of accuracy. In this work, we propose a target localization system combining event-camera and SNN-based high-speed target estimation and frame-based camera and CNN-driven reliable object detection by fusing complementary spatio-temporal prowess of event and frame pipelines. One of our main contributions involves the design of an SNN filter that borrows from the neural mechanism for ego-motion cancelation in houseflies. It fuses the vestibular sensors with the vision to cancel the activity corresponding to the predator's self-motion. We also integrate the neuro-inspired multi-pipeline processing with task-optimized multi-neuronal pathway structure in primates and insects. The system is validated to outperform CNN-only processing using prey-predator drone simulations in realistic 3D virtual environments. The system is then demonstrated in a real-world multi-drone set-up with emulated event data. Subsequently, we use recorded actual sensory data from multi-camera and inertial measurement unit (IMU) assembly to show desired working while tolerating the realistic noise in vision and IMU sensors. We analyze the design space to identify optimal parameters for spiking neurons, CNN models, and for checking their effect on the performance metrics of the fused system. Finally, we map the throughput controlling SNN and fusion network on edge-compatible Zynq-7000 FPGA to show a potential 264 outputs per second even at constrained resource availability. This work may open new research directions by coupling multiple sensing and processing modalities inspired by discoveries in neuroscience to break fundamental trade-offs in frame-based computer vision.

Neural mechanisms of background and velocity effects in smooth pursuit eye movements

Smooth pursuit eye movements (SPEM) are essential to guide behaviour in complex visual environments. SPEM accuracy is known to be degraded by the presence of a structured visual background and at higher target velocities. The aim of this preregistered study was to investigate the neural mechanisms of these robust behavioural effects. N = 33 participants performed a SPEM task with two background conditions (present and absent) at two target velocities (0.4 and 0.6 Hz). Eye movement and BOLD data were collected simultaneously. Both the presence of a structured background and faster target velocity decreased pursuit gain and increased catch-up saccade rate. Faster targets additionally increased position error. Higher BOLD response with background was found in extensive clusters in visual, parietal, and frontal areas (including the medial frontal eye fields; FEF) partially overlapping with the known SPEM network. Faster targets were associated with higher BOLD response in visual cortex and left lateral FEF. Task-based functional connectivity analyses (psychophysiological interactions; PPI) largely replicated previous results in the basic SPEM network but did not yield additional information regarding the neural underpinnings of the background and velocity effects. The results show that the presentation of visual background stimuli during SPEM induces activity in a widespread visuo-parieto-frontal network including areas contributing to cognitive aspects of oculomotor control such as medial FEF, whereas the response to higher target velocity involves visual and motor areas such as lateral FEF. Therefore, we were able to propose for the first time different functions of the medial and lateral FEF during SPEM.

Tracking and perceiving diverse motion signals: Directional biases in human smooth pursuit and perception

Human smooth pursuit eye movements and motion perception behave similarly when observers track and judge the motion of simple objects, such as dots. But moving objects in our natural environment are complex and contain internal motion. We ask how pursuit and perception integrate the motion of objects with motion that is internal to the object. Observers (n = 20) tracked a moving random-dot kinematogram with their eyes and reported the object’s perceived direction. Objects moved horizontally with vertical shifts of 0, ±3, ±6, or ±9° and contained internal dots that were static or moved ±90° up/down. Results show that whereas pursuit direction was consistently biased in the direction of the internal dot motion, perceptual biases differed between observers. Interestingly, the perceptual bias was related to the magnitude of the pursuit bias (r = 0.75): perceptual and pursuit biases were directionally aligned in observers that showed a large pursuit bias, but went in opposite directions in observers with a smaller pursuit bias. Dissociations between perception and pursuit might reflect different functional demands of the two systems. Pursuit integrates all available motion signals in order to maximize the ability to monitor and collect information from the whole scene. Perception needs to recognize and classify visual information, thus segregating the target from its context. Ambiguity in whether internal motion is part of the scene or contributes to object motion might have resulted in individual differences in perception. The perception-pursuit correlation suggests shared early-stage motion processing or perception-pursuit interactions.

Replicability and reliability of the background and target velocity effects in smooth pursuit eye movements

When we follow a slowly moving target with our eyes, we perform smooth pursuit eye movements (SPEM). Previous investigations point to significantly and robustly reduced SPEM performance in the presence of a stationary background and at higher compared to lower target velocities. However, the reliability of these background and target velocity effects has not yet been investigated systematically. To address this issue, 45 healthy participants (17 m, 28 f) took part in two experimental sessions 7 days apart. In each session, participants were instructed to follow a horizontal SPEM target moving sinusoidally between ±7.89° at three different target velocities, corresponding to frequencies of 0.2, 0.4 and 0.6 Hz. Each target velocity was presented once with and once without a stationary background, resulting in six blocks. The blocks were presented twice per session in order to additionally explore potential task length effects. To assess SPEM performance, velocity gain was calculated as the ratio of eye to target velocity. In line with previous research, detrimental background and target velocity effects were replicated robustly in both sessions with large effect sizes. Good to excellent test-retest reliabilities were obtained at higher target velocities and in the presence of a stationary background, whereas lower reliabilities occurred with slower targets and in the absence of background stimuli. Target velocity and background effects resulted in largely good to excellent reliabilities. These findings not only replicated robust experimental effects of background and target velocity at group level, but also revealed that these effects can be translated into reliable individual difference measures.

Ocular tracking of occluded ballistic trajectories: Effects of visual context and of target law of motion

In tracking a moving target, the visual context may provide cues for an observer to interpret the causal nature of the target motion and extract features to which the visual system is weakly sensitive, such as target acceleration. This information could be critical when vision of the target is temporarily impeded, requiring visual motion extrapolation processes. Here we investigated how visual context influences ocular tracking of motion either congruent or not with natural gravity. To this end, 28 subjects tracked computer-simulated ballistic trajectories either perturbed in the descending segment with altered gravity effects (0g/2g) or retaining natural-like motion (1g). Shortly after the perturbation (550 ms), targets disappeared for either 450 or 650 ms and became visible again until landing. Target motion occurred with either quasi-realistic pictorial cues or a uniform background, presented in counterbalanced order. We analyzed saccadic and pursuit movements after 0g and 2g target-motion perturbations and for corresponding intervals of unperturbed 1g trajectories, as well as after corresponding occlusions. Moreover, we considered the eye-to-target distance at target reappearance. Tracking parameters differed significantly between scenarios: With a neutral background, eye movements did not depend consistently on target motion, whereas with pictorial background they showed significant dependence, denoting better tracking of accelerated targets. These results suggest that oculomotor control is tuned to realistic properties of the visual scene.

Motion integration is anisotropic during smooth pursuit eye movements

Smooth pursuit eye movements (pursuit) are used to minimize the retinal motion of moving objects. During pursuit, the pattern of motion on the retina carries not only information about the object movement but also reafferent information about the eye movement itself. The latter arises from the retinal flow of the stationary world in the direction opposite to the eye movement. To extract the global direction of motion of the tracked object and stationary world, the visual system needs to integrate ambiguous local motion measurements (i.e., the aperture problem). Unlike the tracked object, the stationary world's global motion is entirely determined by the eye movement and thus can be approximately derived from motor commands sent to the eye (i.e., from an efference copy). Because retinal motion opposite to the eye movement is dominant during pursuit, different motion integration mechanisms might be used for retinal motion in the same direction and opposite to pursuit. To investigate motion integration during pursuit, we tested direction discrimination of a brief change in global object motion. The global motion stimulus was a circular array of small static apertures within which one-dimensional gratings moved. We found increased coherence thresholds and a qualitatively different reflexive ocular tracking for global motion opposite to pursuit. Both effects suggest reduced sampling of motion opposite to pursuit, which results in an impaired ability to extract coherence in motion signals in the reafferent direction. We suggest that anisotropic motion integration is an adaptation to asymmetric retinal motion patterns experienced during pursuit eye movements. NEW & NOTEWORTHY This study provides a new understanding of how the visual system achieves coherent perception of an object's motion while the eyes themselves are moving. The visual system integrates local motion measurements to create a coherent percept of object motion. An analysis of perceptual judgments and reflexive eye movements to a brief change in an object's global motion confirms that the visual and oculomotor systems pick fewer samples to extract global motion opposite to the eye movement.

Spontaneous pre-stimulus oscillatory activity shapes the way we look: A concurrent imaging and eye-movement study

Previous behavioural studies have accrued evidence that response time plays a critical role in determining whether selection is influenced by stimulus saliency or target template. In the present work, we investigated to what extent the variations in timing and consequent oculomotor controls are influenced by spontaneous variations in pre-stimulus alpha oscillations. We recorded simultaneously brain activity using magnetoencephalography (MEG) and eye movements while participants performed a visual search task. Our results show that slower saccadic reaction times were predicted by an overall stronger alpha power in the 500 ms time window preceding the stimulus onset, while weaker alpha power was a signature of faster responses. When looking separately at performance for fast and slow responses, we found evidence for two specific sources of alpha activity predicting correct versus incorrect responses. When saccades were quickly elicited, errors were predicted by stronger alpha activity in posterior areas, comprising the angular gyrus in the temporal-parietal junction (TPJ) and possibly the lateral intraparietal area (LIP). Instead, when participants were slower in responding, an increase of alpha power in frontal eye fields (FEF), supplementary eye fields (SEF) and dorsolateral pre-frontal cortex (DLPFC) predicted erroneous saccades. In other words, oculomotor accuracy in fast responses was predicted by alpha power differences in more posterior areas, while the accuracy in slow responses was predicted by alpha power differences in frontal areas, in line with the idea that these areas may be differentially related to stimulus-driven and goal-driven control of selection.

Using Smooth Pursuit Calibration for Difficult-to-Calibrate Participants

Although the 45-dots calibration routine of a previous study ( 2) provided very good accuracy, it requires intense mental effort and the routine proved to be unsuccessful for young children who struggle to maintain concentration. The calibration procedures that are normally used for difficult-to-calibrate participants, such as autistic children and infants, do not suffice since they are not accurate enough and the reliability of research results might be jeopardised. Smooth pursuit has been used before for calibration and is applied in this paper as an alternative routine for participants who are difficult to calibrate with conventional routines. Gaze data is captured at regular intervals and many calibration targets are generated while the eyes are following a moving target. The procedure could take anything between 30 s and 60 s to complete, but since an interesting target and/or a conscious task may be used, participants are assisted to maintain concentration. It was proven that the accuracy that can be attained through calibration with a moving target along an even horizontal path is not significantly worse than the accura-cy that can be attained with a standard method of watching dots appearing in random order. The routine was applied successfully for a group of children with ADD, ADHD and learning abilities. This result is important as it provides for easier calibration - especially in the case of participants who struggle to keep their gaze focused and stable on a stationary target for long enough.

Stationary Objects Flashed Periodically Appear to Move During Smooth Pursuit Eye Movement

We discovered that a white disc flashed twice at the same location appears to move during smooth pursuit eye tracking in the direction opposite to that of the eye movement. We called this novel phenomenon movement-induced apparent motion (MIAM). Using the method of constant stimuli, we measured the required displacement of the second appearance of the disc in the pursuit direction to null the effect during the closed-loop stage of smooth pursuit eye tracking. We observed a strong linear relationship between the points of subjective stationarity and the inter-stimuli intervals for four smooth pursuit eye movement speeds. The slopes and y-intercepts of these linear fits were well predicted by the hypothesis according to which subjects saw illusory motion from the first to the second retinal projections of the flashed disc during smooth pursuit eye movement, with no extra-retinal signal compensation.

Context effects on smooth pursuit and manual interception of a disappearing target

In our natural environment, we interact with moving objects that are surrounded by richly textured, dynamic visual contexts. Yet most laboratory studies on vision and movement show visual objects in front of uniform gray backgrounds. Context effects on eye movements have been widely studied, but it is less well known how visual contexts affect hand movements. Here we ask whether eye and hand movements integrate motion signals from target and context similarly or differently, and whether context effects on eye and hand change over time. We developed a track-intercept task requiring participants to track the initial launch of a moving object ("ball") with smooth pursuit eye movements. The ball disappeared after a brief presentation, and participants had to intercept it in a designated "hit zone." In two experiments (n = 18 human observers each), the ball was shown in front of a uniform or a textured background that either was stationary or moved along with the target. Eye and hand movement latencies and speeds were similarly affected by the visual context, but eye and hand interception (eye position at time of interception, and hand interception timing error) did not differ significantly between context conditions. Eye and hand interception timing errors were strongly correlated on a trial-by-trial basis across all context conditions, highlighting the close relation between these responses in manual interception tasks. Our results indicate that visual contexts similarly affect eye and hand movements but that these effects may be short-lasting, affecting movement trajectories more than movement end points.NEW & NOTEWORTHY In a novel track-intercept paradigm, human observers tracked a briefly shown object moving across a textured, dynamic context and intercepted it with their finger after it had disappeared. Context motion significantly affected eye and hand movement latency and speed, but not interception accuracy; eye and hand position at interception were correlated on a trial-by-trial basis. Visual context effects may be short-lasting, affecting movement trajectories more than movement end points.

Adaptive reliance on the most stable sensory predictions enhances perceptual feature extraction of moving stimuli

The prediction of the sensory outcomes of action is thought to be useful for distinguishing self- vs. externally generated sensations, correcting movements when sensory feedback is delayed, and learning predictive models for motor behavior. Here, we show that aspects of another fundamental function—perception—are enhanced when they entail the contribution of predicted sensory outcomes and that this enhancement relies on the adaptive use of the most stable predictions available. We combined a motor-learning paradigm that imposes new sensory predictions with a dynamic visual search task to first show that perceptual feature extraction of a moving stimulus is poorer when it is based on sensory feedback that is misaligned with those predictions. This was possible because our novel experimental design allowed us to override the “natural” sensory predictions present when any action is performed and separately examine the influence of these two sources on perceptual feature extraction. We then show that if the new predictions induced via motor learning are unreliable, rather than just relying on sensory information for perceptual judgments, as is conventionally thought, then subjects adaptively transition to using other stable sensory predictions to maintain greater accuracy in their perceptual judgments. Finally, we show that when sensory predictions are not modified at all, these judgments are sharper when subjects combine their natural predictions with sensory feedback. Collectively, our results highlight the crucial contribution of sensory predictions to perception and also suggest that the brain intelligently integrates the most stable predictions available with sensory information to maintain high fidelity in perceptual decisions.

Saccadic compensation for reflexive optokinetic nystagmus just as good as compensation for volitional pursuit

The natural viewing behavior of moving observers ideally requires target-selecting saccades to be coordinated with automatic gaze-stabilizing eye movements such as optokinetic nystagmus. However, it is unknown whether saccade plans can compensate for reflexive movement of the eye during the variable saccade latency period, and it is unclear whether reflexive nystagmus is even accompanied by extraretinal signals carrying the eye movement information that could potentially underpin such compensation. We show that saccades do partially compensate for optokinetic nystagmus that displaces the eye during the saccade latency period. Moreover, this compensation is as good as for displacements due to voluntary smooth pursuit. In other words, the saccade system appears to be as well coordinated with reflexive nystagmus as it is with volitional pursuit, which in turn implies that extraretinal signals accompany nystagmus and are just as informative as those accompanying pursuit.

Ocular tracking responses to background motion gated by feature-based attention

Involuntary ocular tracking responses to background motion offer a window on the dynamics of motion computations. In contrast to spatial attention, we know little about the role of feature-based attention in determining this ocular response. To probe feature-based effects of background motion on involuntary eye movements, we presented human observers with a balanced background perturbation. Two clouds of dots moved in opposite vertical directions while observers tracked a target moving in horizontal direction. Additionally, they had to discriminate a change in the direction of motion (±10° from vertical) of one of the clouds. A vertical ocular following response occurred in response to the motion of the attended cloud. When motion selection was based on motion direction and color of the dots, the peak velocity of the tracking response was 30% of the tracking response elicited in a single task with only one direction of background motion. In two other experiments, we tested the effect of the perturbation when motion selection was based on color, by having motion direction vary unpredictably, or on motion direction alone. Although the gain of pursuit in the horizontal direction was significantly reduced in all experiments, indicating a trade-off between perceptual and oculomotor tasks, ocular responses to perturbations were only observed when selection was based on both motion direction and color. It appears that selection by motion direction can only be effective for driving ocular tracking when the relevant elements can be segregated before motion onset.

On the effect of scene motion on color constancy

A  series of experiments with human subjects have shown that color constancy improves when an object moves. It has been hypothesized that this effect is due to some kind of influence of high-level motion processing. We have built a computational model for color perception which replicates the results qualitatively which have been obtained with human subjects. We show that input from high-level motion processing is not required. In our model, the dependence is an effect of eye movement in combination with neural processing. Depending on the type of stimulus used, the eye either tracks the object or the background. When the object moves but is tracked by the observer, the background appears to move when considering the stimulus with respect to eye coordinates. Hence, the retinal input is different for the two conditions leading to a difference in color constancy performance.

Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets

How does the brain coordinate saccadic and smooth pursuit eye movements to track objects that move in unpredictable directions and speeds? Saccadic eye movements rapidly foveate peripheral visual or auditory targets, and smooth pursuit eye movements keep the fovea pointed toward an attended moving target. Analyses of tracking data in monkeys and humans reveal systematic deviations from predictions of the simplest model of saccade-pursuit interactions, which would use no interactions other than common target selection and recruitment of shared motoneurons. Instead, saccadic and smooth pursuit movements cooperate to cancel errors of gaze position and velocity, and thus to maximize target visibility through time. How are these two systems coordinated to promote visual localization and identification of moving targets? How are saccades calibrated to correctly foveate a target despite its continued motion during the saccade? The neural model proposed here answers these questions. Modeled interactions encompass motion processing areas MT, MST, FPA, DLPN and NRTP; saccade planning and execution areas FEF, LIP, and SC; the saccadic generator in the brain stem; and the cerebellum. Simulations illustrate the model's ability to functionally explain and quantitatively simulate anatomical, neurophysiological and behavioral data about coordinated saccade-pursuit tracking.

Inhibition of Steady-State Smooth Pursuit and Catch-Up Saccades by Abrupt Visual and Auditory Onsets

It is known that visual transients prolong saccadic latency and reduce saccadic frequency. The latter effect was attributed to subcortical structures because it occurred only 60–70 ms after stimulus onset. We examined the effects of large task-irrelevant transients on steady-state pursuit and the generation of catch-up saccades. Two screen-wide stripes of equal contrast (4, 20, or 100%) were briefly flashed at equal eccentricities (3, 6, or 12°) from the pursuit target. About 100 ms after flash onset, we observed that pursuit gain dropped by 6–12% and catch-up saccades were entirely suppressed. The relatively long latency of the inhibition suggests that it results from cortical mechanisms that may act by promoting fixation or the deployment of attention over the visual field. In addition, we show that a loud irrelevant sound is able to generate the same inhibition of saccades as visual transients, whereas it only induces a weak modulation of pursuit gain, indicating a privileged access of acoustic information to the saccadic system. Finally, irrelevant changes in motion direction orthogonal to pursuit had a smaller and later inhibitory effect.

Ocular Responses to Brief Motion of Textured Backgrounds During Smooth Pursuit in Humans

We studied the effects of horizontal smooth pursuit on the ocular responses to brief vertical perturbations of textured backgrounds in humans. When the subject was fixating on a stationary target, a brief vertical perturbation of the background elicited a small tracking response. When the subject was pursuing a target moving horizontally, the same background perturbation elicited a larger response: that is, the response to vertical background perturbations was enhanced during pursuit (pursuit-related enhancement). On the other hand, the dependencies of the ocular responses on spatial frequency, temporal frequency, and stimulus contrast were similar regardless of the ongoing behavior of the subject. We also found that a low-level energy-based mechanism underlies the ocular responses to vertical perturbations of the background during fixation and smooth pursuit. We conclude that the pursuit-related enhancement is independent of the properties of visual processing for cross-axis motion of backgrounds, which suggests that this enhancement results from uniform facilitation of the visual system and/or from facilitation of visuomotor transmission downstream of the visual processing.

Updating visual memory across eye movements for ocular and arm motor control

Remembered object locations are stored in an eye-fixed reference frame, so that every time the eyes move, spatial representations must be updated for the arm-motor system to reflect the target's new relative position. To date, studies have not investigated how the brain updates these spatial representations during other types of eye movements, such as smooth-pursuit. Further, it is unclear what information is used in spatial updating. To address these questions we investigated whether remembered locations of pointing targets are updated following smooth-pursuit eye movements, as they are following saccades, and also investigated the role of visual information in estimating eye-movement amplitude for updating spatial memory. Misestimates of eye-movement amplitude were induced when participants visually tracked stimuli presented with a background that moved in either the same or opposite direction of the eye before pointing or looking back to the remembered target location. We found that gaze-dependent pointing errors were similar following saccades and smooth-pursuit and that incongruent background motion did result in a misestimate of eye-movement amplitude. However, the background motion had no effect on spatial updating for pointing, but did when subjects made a return saccade, suggesting that the oculomotor and arm-motor systems may rely on different sources of information for spatial updating.

Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans

To understand how the CNS uses past experiences to generate movements that accommodate minute-by-minute environmental changes, we studied the trial-by-trial updating of the gain for initiating smooth pursuit eye movements and how this relates to the history of previous trials. Ocular responses in humans elicited by a small perturbing motion presented 300 ms after appearance of a target were used as a measure of the gain of visuomotor transmission. After the perturbation, the target was either moved horizontally (pursuit trial) or remained in a stationary position (fixation trial). The trial sequence randomly included pursuit and fixation. The amplitude of the response to the perturbation was modulated in a trial-by-trial manner based on the immediately preceding trial, with preceding fixation and pursuit trials decreasing and increasing the gain, respectively. The effect of the previous trial was larger with shorter intertrial intervals, but did not diminish for at least 2,000 ms. A time-series analysis showed that the response amplitude was significantly correlated with the past few trials, with dynamics that could be approximated by a first-order linear system. The results suggest that the CNS integrates recent experiences to set the gain in preparation for upcoming tracking movements in a changing environment.

Contrast and Assimilation in Motion Perception and Smooth Pursuit Eye Movements

The analysis of visual motion serves many different functions ranging from object motion perception to the control of self-motion. The perception of visual motion and the oculomotor tracking of a moving object are known to be closely related and are assumed to be controlled by shared brain areas. We compared perceived velocity and the velocity of smooth pursuit eye movements in human observers in a paradigm that required the segmentation of target object motion from context motion. In each trial, a pursuit target and a visual context were independently perturbed simultaneously to briefly increase or decrease in speed. Observers had to accurately track the target and estimate target speed during the perturbation interval. Here we show that the same motion signals are processed in fundamentally different ways for perception and steady-state smooth pursuit eye movements. For the computation of perceived velocity, motion of the context was subtracted from target motion (motion contrast), whereas pursuit velocity was determined by the motion average (motion assimilation). We conclude that the human motion system uses these computations to optimally accomplish different functions: image segmentation for object motion perception and velocity estimation for the control of smooth pursuit eye movements.

Contrast sensitivity during the initiation of smooth pursuit eye movements

Eye movements challenge the perception of a stable world by inducing retinal image displacement. During saccadic eye movements visual stability is accompanied by a remapping of visual receptive fields, a compression of visual space and perceptual suppression. Here we explore whether a similar suppression changes the perception of briefly presented low contrast targets during the initiation of smooth pursuit eye movements. In a 2AFC design we investigated the contrast sensitivity for threshold-level stimuli during the initiation of smooth pursuit and during saccades. Pursuit was elicited by horizontal step-ramp and ramp stimuli. At any time from 200 ms before to 500 ms after pursuit stimulus onset, a blurred 0.3 deg wide horizontal line with low contrast just above detection threshold appeared for 10 ms either 2 deg above or below the pursuit trajectory. Observers had to pursue the moving stimulus and to indicate whether the target line appeared above or below the pursuit trajectory. In contrast to perceptual suppression effects during saccades, no pronounced suppression was found at pursuit onset for step-ramp motion. When pursuit was elicited by a ramp stimulus, pursuit initiation was accompanied by catch-up saccades, which caused saccadic suppression. Additionally, contrast sensitivity was attenuated at the time of pursuit or saccade stimulus onset. This attenuation might be due to an attentional deficit, because the stimulus required the focus of attention during the programming of the following eye movement.

Contextual Effects on Smooth-Pursuit Eye Movements

Segregating a moving object from its visual context is particularly relevant for the control of smooth-pursuit eye movements. We examined the interaction between a moving object and a stationary or moving visual context to determine the role of the context motion signal in driving pursuit. Eye movements were recorded from human observers to a medium-contrast Gaussian dot that moved horizontally at constant velocity. A peripheral context consisted of two vertically oriented sinusoidal gratings, one above and one below the stimulus trajectory, that were either stationary or drifted into the same or opposite direction as that of the target at different velocities. We found that a stationary context impaired pursuit acceleration and velocity and prolonged pursuit latency. A drifting context enhanced pursuit performance, irrespective of its motion direction. This effect was modulated by context contrast and orientation. When a context was briefly perturbed to move faster or slower eye velocity changed accordingly, but only when the context was drifting along with the target. Perturbing a context into the direction orthogonal to target motion evoked a deviation of the eye opposite to the perturbation direction. We therefore provide evidence for the use of absolute and relative motion cues, or motion assimilation and motion contrast, for the control of smooth-pursuit eye movements.

Cerebrocerebellar circuits for the perceptual cancellation of eye-movement-induced retinal image motion

Despite smooth pursuit eye movements, we are unaware of resultant retinal image motion. This example of perceptual invariance is achieved by comparing retinal image slip with an internal reference signal predicting the sensory consequences of the eye movement. This prediction can be manipulated experimentally, allowing one to vary the amount of self-induced image motion for which the reference signal compensates and, accordingly, the resulting percept of motion. Here we were able to map regions in CRUS I within the lateral cerebellar hemispheres that exhibited a significant correlation between functional magnetic resonance imaging signal amplitudes and the amount of motion predicted by the reference signal. The fact that these cerebellar regions were found to be functionally coupled with the left parieto-insular cortex and the supplementary eye fields points to these cortical areas as the sites of interaction between predicted and experienced sensory events, ultimately giving rise to the perception of a stable world despite self-induced retinal motion.

Suppression of optokinesis during smooth pursuit eye movements revisited: The role of extra-retinal information

When our eyes track objects that are moving in a richly structured environment, the retinal image of the stationary visual scene inevitably moves over the retina in a direction opposite to the eye movement. Such self-motion-induced global retinal slip usually provides an ideal stimulus for the optokinetic reflex. This reflex operates to compensate for global image flow. However, during smooth pursuit eye movements it must be shut down so that the reflex does not counteract the voluntary pursuit of moving targets. Here, we asked if retinal information is sufficient for this cancellation of the optokinetic reflex during smooth pursuit eye movements. In a series of experiments, we show that neither the eye movement-induced retinal image motion per se nor the relative motion between the pursuit target and the background are sufficient for suppression of optokinesis. We, therefore, conclude that extra-retinal information about smooth pursuit eye movements is required for the cancellation of the optokinetic reflex.

Spatial deployment of attention influences both saccadic and pursuit tracking

We examined the effects of changing spatial aspects of attention during oculomotor tracking. Human subjects were instructed to make a discrimination on either the small (0.8 degrees ) central or the large (8 degrees ) peripheral part of a compound stimulus (two counter-rotating concentric rings) while the stimulus either translated across the screen or was stationary. During this period, a transient perturbation with either step or ramp movement profile occurred. For perturbations leading to a change in position larger than the small ring, saccades occurred more frequently and had much shorter latencies (by 135 ms) when attention was directed to the small ring than when attention was directed to the large ring. These latency differences were sufficiently great that from a single saccade one can identify the attentional instruction with 94% accuracy. However, with target steps as small as the small ring, saccade latencies differed less. For pursuit, ramp perturbations caused larger changes in eye velocity with little change in latency when attention was directed to the small ring. Finally, when only the motion of the non-attended ring was perturbed, most subjects showed stronger saccadic responses to perturbations of the small than the large ring, and stronger pursuit responses to perturbations of the large than the small ring. By fitting the saccade latency distributions with the Reddi and Carpenter LATER model, we found that our subjects apparently employed at least two distinct strategies for changing latency when attending large vs. small. We propose that the timing of the saccade decision process depends on both the size of the attended object and the magnitude of the perturbation.

Visual motion due to eye movements helps guide the hand

Movement of the body, head, or eyes with respect to the world creates one of the most common yet complex situations in which the visuomotor system must localize objects. In this situation, vestibular, proprioceptive, and extra-retinal information contribute to accurate visuomotor control. The utility of retinal motion information, on the other hand, is questionable, since a single pattern of retinal motion can be produced by any number of head or eye movements. Here we investigated whether retinal motion during a smooth pursuit eye movement contributes to visuomotor control. When subjects pursued a moving object with their eyes and reached to the remembered location of a separate stationary target, the presence of a moving background significantly altered the endpoints of their reaching movements. A background that moved with the pursuit, creating a retinally stationary image (no retinal slip), caused the endpoints of the reaching movements to deviate in the direction of pursuit, overshooting the target. A physically stationary background pattern, however, producing retinal image motion opposite to the direction of pursuit, caused reaching movements to become more accurate. The results indicate that background retinal motion is used by the visuomotor system in the control of visually guided action.

Neuroophthalmology: a brief Vademecum

The stunning, intricate interaction between the visual, vestibular and optomotor systems--each a miracle on its own--ensures maintenance of orientation in space as well as visual recognition and target selection despite a host of sensory conflicts and adversary disturbances. Their main goals are to keep a target of interest on the fovea by either maintaining or shifting the direction of gaze in order to produce an accurate internal representation of the visual surroundings, in particular the selected target, and to continuously mirror the spatial relationship between these various visual elements and the self. Not surprising, the implementation of this host of elaborate neural networks encompasses almost every part of the brain, including the brainstem, cerebellum, extrapyramidal system and many areas of the cerebral cortex. Thus far, these systems are among the best investigated in brain research; and enormous knowledge was amassed over the last century employing a variety of techniques, including single cell recordings, eye movement studies, functional imaging and neuropsychological observations. In addition, this prolific line of research has enlightened many fundamental principles of neural and neuronal processing, which have subsequently enriched other fields of brain research as well as computational neuroscience, e.g. the discovery of receptive fields, which have now become a ubiquitous concept in many other areas of neurophysiology. This (improperly) brief, fractional and undoubtedly biased Vademecum is meant to accompany the reader into this marvellous field of neurophysiology and neurology. In particular, it stresses the clinical application of its functional neuroanatomy at the bedside, which, in many respects, is superior to other means of investigating a patient.

Ocular Tracking of Moving Targets: Effects of Perturbing the Background

Primates are able to track a moving target with their eyes, even when the target is seen against a stationary textured background. In this situation, the tracking eye movement induces motion of the background images on the retina (reafference) that competes with the motion of the target's retinal image, potentially disrupting the tracking of the target. Previous work on humans reported that brief perturbations of the background in the opposite direction to pursuit were much less disruptive than perturbations in the same direction as pursuit. Furthermore, if the background moved together with the pursuit target—so as to effectively eliminate the reafference—then the effects of a subsequent background perturbation showed less dependence on direction. This suggested that the direction selectivity to background perturbations during pursuit against a stationary background was due, at least in part, to the prior motion of the background secondary to the pursuit. We now report similar findings in monkeys, and in addition, have investigated the effect of moving the background while the animal was fixating a stationary target. In this situation, the ocular tracking responses to subsequent brief perturbations of the moving background were weaker when the perturbations were in the same direction as the prior background motion than when in the opposite direction. This suggests that the selective insensitivity to the reafferent visual input associated with pursuit across a stationary background is, at least in part, independent of pursuit per se and attributable to a progressive reduction in the sensitivity to sustained background motion.

Recasting the Smooth Pursuit Eye Movement System

Primates use a combination of smooth pursuit and saccadic eye movements to stabilize the retinal image of selected objects within the high-acuity region near the fovea. Pursuit has traditionally been viewed as a relatively automatic behavior, driven by visual motion signals and mediated by pathways that connect visual areas in the cerebral cortex to motor regions in the cerebellum. However, recent findings indicate that this view needs to be reconsidered. Rather than being controlled primarily by areas in extrastriate cortex specialized for processing visual motion, pursuit involves an extended network of cortical areas, and, of these, the pursuit-related region in the frontal eye fields appears to exert the most direct influence. The traditional pathways through the cerebellum are important, but there are also newly identified routes involving structures previously associated with the control of saccades, including the basal ganglia, the superior colliculus, and nuclei in the brain stem reticular formation. These recent findings suggest that the pursuit system has a functional architecture very similar to that of the saccadic system. This viewpoint provides a new perspective on the processing steps that occur as descending control signals interact with circuits in the brain stem and cerebellum responsible for gating and executing voluntary eye movements. Although the traditional view describes pursuit and saccades as two distinct neural systems, it may be more accurate to consider the two movements as different outcomes from a shared cascade of sensory–motor functions.

Commentary: smooth pursuit eye movements: from low-level to high-level vision

If an object of great interest moves in our environment, we are able to elicit smooth pursuit eye movements that keep the image of the moving object stationary on our fovea. The processing of visual motion underlying the execution of smooth pursuit eye movements is very similar to the processing underlying the perception of visual motion. During initiation of smooth pursuit, an averaging across all available motion information occurs. Cognitive factors including attention, prediction and learning are able to influence the execution of smooth pursuit. The pursuit target trajectory in space is represented in the discharge rates of neurons in the posterior parietal cortex of rhesus monkeys.