A 'tachometer' feedback model of smooth pursuit eye movements

A sensory-motor decoder that transforms neural responses in extrastriate area MT into smooth pursuit eye movements

Visual motion drives smooth pursuit eye movements through a sensory-motor decoder that uses multiple parallel neural pathways to transform the population response in extrastriate area MT into movement. We evaluated the decoder by challenging pursuit in monkeys with reduced motion reliability created by reducing coherence of motion in patches of dots. Our strategy was to determine how reduced dot coherence changes the population response in MT. We then predicted the properties of a decoder that transforms the MT population response into dot coherence-induced deficits in the initiation of pursuit and steady-state tracking. During pursuit initiation, decreased dot coherence reduces MT population response amplitude without changing the preferred speed at its peak. The successful decoder reproduces the measured eye movements by multiplication of 1) the estimate of target speed from the peak of the population response with 2) visual-motor gain based on the amplitude of the population response. During steady-state tracking, the decoder that worked for pursuit initiation failed to reproduce the paradox that steady-state eye speeds do not accelerate to the target speed despite persistent image motion. It predicted eye acceleration to target speed even when monkeys' eye speeds were steady at well below the target speed. To account for the effect of dot coherence on steady-state eye speed, we postulate that the decoder uses sensory-motor gain to modulate the eye velocity positive feedback that normally sustains perfect steady-state tracking. Then, poor steady-state tracking persists because of balance between eye deceleration caused by low positive feedback gain and acceleration driven by MT.NEW & NOTEWORTHY By challenging a sensory-motor system with degraded sensory stimuli, we reveal how the sensory-motor decoder transforms the population response in extrastriate area MT into commands for the initiation and steady-state behavior of smooth pursuit eye movements. Conclusions are based on measuring population responses in MT for multiple target speeds and different levels of motion reliability and evaluating a decoder with a biologically motivated architecture to determine the decoder properties that create the measured eye movements.

A sensory-motor decoder that transforms neural responses in extrastriate area MT into smooth pursuit eye movements

Visual motion drives smooth pursuit eye movements through a sensory-motor decoder that uses multiple parallel components and neural pathways to transform the population response in extrastriate area MT into movement. We evaluated the decoder by challenging pursuit in monkeys with reduced motion reliability created by reducing coherence of motion in patches of dots. Reduced dot coherence caused deficits in both the initiation of pursuit and steady-state tracking, revealing the paradox of steady-state eye speeds that fail to accelerate to target speed in spite of persistent image motion. We recorded neural responses to reduced dot coherence in MT and found a decoder that transforms MT population responses into eye movements. During pursuit initiation, decreased dot coherence reduces MT population response amplitude without changing the preferred speed at the peak of the population response. The successful decoder reproduces the measured eye movements by multiplication of (i) the estimate of target speed from the peak of the population response with (ii) visual-motor gain based on the amplitude of the population response. During steady-state tracking, the decoder that worked for pursuit initiation failed. It predicted eye acceleration to target speed even when monkeys' eye speeds were steady at a level well below target speed. We can account for the effect of dot coherence on steady-state eye speed if sensorymotor gain also modulates the eye velocity positive feedback that normally sustains perfect steadystate tracking. Then, poor steady-state tracking persists because of balance between deceleration caused by low positive feedback gain and acceleration driven by MT.

Toward a Biomimetic Neural Circuit Model of Sensory-Motor Processing

Computational models have been a mainstay of research on smooth pursuit eye movements in monkeys. Pursuit is a sensory-motor system that is driven by the visual motion of small targets. It creates a smooth eye movement that accelerates up to target speed and tracks the moving target essentially perfectly. In this review of my laboratory's research, I trace the development of computational models of pursuit eye movements from the early control-theory models to the most recent neural circuit models. I outline a combined experimental and computational plan to move the models to the next level. Finally, I explain why research on nonhuman primates is so critical to the development of the neural circuit models I think we need.

Control of the gain of visual-motor transmission occurs in visual coordinates for smooth pursuit eye movements

Sensory inputs control motor behavior with a strength, or gain, that can be modulated according to the movement conditions. In smooth pursuit eye movements, the response to a brief perturbation of target motion is larger during pursuit of a moving target than during fixation of a stationary target. As a step toward identifying the locus and mechanism of gain modulation, we test whether it acts on signals that are in visual or motor coordinates. Monkeys tracked targets that moved at 15°/s in one of eight directions, including left, right, up, down, and the four oblique directions. In eight-ninths of the trials, the target underwent a brief perturbation that consisted of a single cycle of a 10 Hz sine wave of amplitude ±5°/s in one of the same eight directions. Even for oblique directions of baseline target motion, the magnitude of the eye velocity response to the perturbation was largest for a perturbation near the axis of target motion and smallest for a perturbation along the orthogonal axis. Computational modeling reveals that our data are reproduced when the strength of visual-motor transmission is modulated in sensory coordinates, and there is a static motor bias that favors horizontal eye movements. A network model shows how the output from the smooth eye movement region of the frontal eye fields (FEF(SEM)) could implement gain control by shifting the peak of a visual population response along the axes of preferred image speed and direction.

Incorporating prediction in models for two-dimensional smooth pursuit

A predictive component can contribute to the command signal for smooth pursuit. This is readily demonstrated by the fact that low frequency sinusoidal target motion can be tracked with zero time delay or even with a small lead. The objective of this study was to characterize the predictive contributions to pursuit tracking more precisely by developing analytical models for predictive smooth pursuit. Subjects tracked a small target moving in two dimensions. In the simplest case, the periodic target motion was composed of the sums of two sinusoidal motions (SS), along both the horizontal and the vertical axes. Motions following the same or similar paths, but having a richer spectral composition, were produced by having the target follow the same path but at a constant speed (CS), and by combining the horizontal SS velocity with the vertical CS velocity and vice versa. Several different quantitative models were evaluated. The predictive contribution to the eye tracking command signal could be modeled as a low-pass filtered target acceleration signal with a time delay. This predictive signal, when combined with retinal image velocity at the same time delay, as in classical models for the initiation of pursuit, gave a good fit to the data. The weighting of the predictive acceleration component was different in different experimental conditions, being largest when target motion was simplest, following the SS velocity profiles.

Dynamics of smooth pursuit maintenance

Smooth pursuit eye movements allow the approximate stabilization of a moving visual target on the retina. To study the dynamics of smooth pursuit, we measured eye velocity during the visual tracking of a Gabor target moving at a constant velocity plus a noisy perturbation term. The optimal linear filter linking fluctuations in target velocity to evoked fluctuations in eye velocity was computed. These filters predicted eye velocity to novel stimuli in the 0- to 15-Hz band with good accuracy, showing that pursuit maintenance is approximately linear under these conditions. The shape of the filters were indicative of fast dynamics, with pure delays of merely approximately 67 ms, times-to-peak of approximately 115 ms, and effective integration times of approximately 45 ms. The gain of the system, reflected in the amplitude of the filters, was inversely proportional to the size of the velocity fluctuations and independent of the target mean speed. A modest slow-down of the dynamics was observed as the contrast of the target decreased. Finally, the temporal filters recovered during fixation and pursuit were similar in shape, supporting the notion that they might share a common underlying circuitry. These findings show that the visual tracking of moving objects by the human eye includes a reflexive-like pathway with high contrast sensitivity and fast dynamics.

A theory of the dual pathways for smooth pursuit based on dynamic gain control

The smooth pursuit eye movement (SPEM) system is much more sensitive to target motion perturbations during pursuit than during fixation. This sensitivity is commonly attributed to a dynamic gain control mechanism. Neither the neural substrate nor the functional architecture for this gain control has been fully revealed. There are at least two cortical areas that crucially contribute to smooth pursuit and are therefore eligible sites for dynamic gain control: the medial superior temporal area (MST) and the pursuit area of the frontal eye fields (FEFs), which both project to brain stem premotor structures via parallel pathways. The aim of this study was to develop a model of smooth pursuit based on behavioral, anatomical, and neurophysiological results to account for nonlinear dynamic gain control. Using a behavioral paradigm in humans consisting of a sinusoidal oscillation (4 Hz, +/-8 degrees/s) superimposed on a constant velocity target motion (0-24 degrees/s), we were able to identify relevant gain control parameters in the model. A salient feature of our model is the emergence of two parallel pathways from higher visual cortical to lower motor areas in the brain stem that correspond to the MST and FEF pathways. Detailed analysis of the model revealed that one pathway mainly carries eye velocity related signals, whereas the other is associated mostly with eye acceleration. From comparison with known neurophysiological results we conclude that the dynamic gain control can be attributed to the FEF pathway, whereas the MST pathway serves as the basic circuit for maintaining an ongoing SPEM.

Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans

To understand how the CNS uses past experiences to generate movements that accommodate minute-by-minute environmental changes, we studied the trial-by-trial updating of the gain for initiating smooth pursuit eye movements and how this relates to the history of previous trials. Ocular responses in humans elicited by a small perturbing motion presented 300 ms after appearance of a target were used as a measure of the gain of visuomotor transmission. After the perturbation, the target was either moved horizontally (pursuit trial) or remained in a stationary position (fixation trial). The trial sequence randomly included pursuit and fixation. The amplitude of the response to the perturbation was modulated in a trial-by-trial manner based on the immediately preceding trial, with preceding fixation and pursuit trials decreasing and increasing the gain, respectively. The effect of the previous trial was larger with shorter intertrial intervals, but did not diminish for at least 2,000 ms. A time-series analysis showed that the response amplitude was significantly correlated with the past few trials, with dynamics that could be approximated by a first-order linear system. The results suggest that the CNS integrates recent experiences to set the gain in preparation for upcoming tracking movements in a changing environment.

A quantitative synchronization model for smooth pursuit target tracking

We propose a quantitative model for human smooth pursuit tracking of a continuously moving visual target which is based on synchronization of an internal expectancy model of the target position coupled to the retinal target signal. The model predictions are tested in a smooth circular pursuit eye tracking experiment with transient target blanking of variable duration. In subjects with a high tracking accuracy, the model accounts for smooth pursuit and repeatedly reproduces quantitatively characteristic patterns of the eye dynamics during target blanking. In its simplest form, the model has only one free parameter, a coupling constant. An extended model with a second parameter, a time delay or memory term, accounts for predictive smooth pursuit eye movements which advance the target. The model constitutes an example of synchronization of a complex biological system with perceived sensory signals.

Evidence for object permanence in the smooth-pursuit eye movements of monkeys

We recorded the smooth-pursuit eye movements of monkeys in response to targets that were extinguished (blinked) for 200 ms in mid-trajectory. Eye velocity declined considerably during the target blinks, even when the blinks were completely predictable in time and space. Eye velocity declined whether blinks were presented during steady-state pursuit of a constant-velocity target, during initiation of pursuit before target velocity was reached, or during eye accelerations induced by a change in target velocity. When a physical occluder covered the trajectory of the target during blinks, creating the impression that the target moved behind it, the decline in eye velocity was reduced or abolished. If the target was occluded once the eye had reached target velocity, pursuit was only slightly poorer than normal, uninterrupted pursuit. In contrast, if the target was occluded during the initiation of pursuit, while the eye was accelerating toward target velocity, pursuit during occlusion was very different from normal pursuit. Eye velocity remained relatively stable during target occlusion, showing much less acceleration than normal pursuit and much less of a decline than was produced by a target blink. Anticipatory or predictive eye acceleration was typically observed just prior to the reappearance of the target. Computer simulations show that these results are best understood by assuming that a mechanism of eye-velocity memory remains engaged during target occlusion but is disengaged during target blinks.

Shifts in the population response in the middle temporal visual area parallel perceptual and motor illusions produced by apparent motion

We recorded behavioral, perceptual, and neural responses to targets that provided apparent visual motion consisting of a sequence of stationary flashes. Increasing the flash separation degrades the quality of motion, but for some separations evoked larger smooth pursuit responses from both humans and monkeys than did smooth motion. The same flash separations also produced an increase in perceived speed in humans. Recordings from single neurons in the middle temporal visual area (MT) of awake monkeys revealed a potential basis for the illusion in the population response. Apparent motion produced diminished neural responses relative to smooth motion. However, neurons with slow preferred speeds were more affected than were those with fast preferred speeds. Increasing the flash separation thus caused the population response to become diminished in amplitude and to shift so that the most active neurons had higher preferred speeds. The entire constellation of effects of apparent motion on the magnitude and latency of the initial pursuit response was accounted for if the MT population response was decoded by (1) creating an opponent motion signal for each neuron by treating its preferred and opposite direction responses as those of a pair of oppositely tuned neurons and (2) computing the vector average of these opponent motion signals. Other ways of decoding the population response recorded in MT failed to account for one or more aspects of behavior. We conclude that the effects of apparent motion on both pursuit and perception can be accounted for if target speed is estimated from the MT population response by a neural computation that implements a vector average based on opponent motion.

Experimental and computational analysis of monkey smooth pursuit eye movements

Smooth pursuit eye movements are guided by visual feedback and are surprisingly accurate despite the time delay between visual input and motor output. Previous models have reproduced the accuracy of pursuit either by using elaborate visual signals or by adding sources of motor feedback. Our goal was to constrain what types of signals drive pursuit by obtaining data that would discriminate between these two modeling approaches, represented by the "image motion model" and the "tachometer feedback" model. Our first set of experiments probed the visual properties of pursuit with brief square-pulse and sine-wave perturbations of target velocity. Responses to pulse perturbations increased almost linearly with pulse amplitude, while responses to sine wave perturbations showed strong saturation with increasing stimulus amplitude. The response to sine wave perturbations was strongly dependent on the baseline image velocity at the time of the perturbation. Responses were much smaller if baseline image velocity was naturally large, or was artificially increased by superimposing sine waves on pulse perturbations. The image motion model, but not the tachometer feedback model, could reproduce these features of pursuit. We used a revision of the image motion model that was, like the original, sensitive to both image velocity and image acceleration. Due to a saturating nonlinearity, the sensitivity to image acceleration declined with increasing image velocity. Inclusion of this nonlinearity was motivated by our experimental results, was critical in accounting for the responses to perturbations, and provided an explanation for the unexpected stability of pursuit in the presence of perturbations near the resonant frequency. As an emergent property, the revised image motion model was able to reproduce the frequency and damping of oscillations recorded during artificial feedback delays. Our second set of experiments replicated prior recordings of pursuit responses to multiple-cycle sine wave perturbations, presented over a range of frequencies. The image motion model was able to reproduce the responses to sine wave perturbations across all frequencies, while the tachometer feedback model failed at high frequencies. These failures resulted from the absence of image acceleration signals in the tachometer model. We conclude that visual signals related to image acceleration are important in driving pursuit eye movements and that the nonlinearity of these signals provides stability. Smooth pursuit thus illustrates that a plausible neural strategy for combating natural delays in sensory feedback is to employ information about the derivative of the sensory input.

Visual motion integration for perception and pursuit

To examine the relationship between visual motion processing for perception and pursuit, we measured the pursuit eye-movement and perceptual responses to the same complex-motion stimuli. We show that humans can both perceive and pursue the motion of line-figure objects, even when partial occlusion makes the resulting image motion vastly different from the underlying object motion. Our results show that both perception and pursuit can perform largely accurate motion integration, i.e. the selective combination of local motion signals across the visual field to derive global object motion. Furthermore, because we manipulated perceived motion while keeping image motion identical, the observed parallel changes in perception and pursuit show that the motion signals driving steady-state pursuit and perception are linked. These findings disprove current pursuit models whose control strategy is to minimize retinal image motion, and suggest a new framework for the interplay between visual cortex and cerebellum in visuomotor control.

A neural model of smooth pursuit control and motion perception by cortical area MST

Smooth pursuit eye movements (SPEMs) are eye rotations that are used to maintain fixation on a moving target. Such rotations complicate the interpretation of the retinal image, because they nullify the retinal motion of the target, while generating retinal motion of stationary objects in the background. This poses a problem for the oculomotor system, which must track the stabilized target image while suppressing the optokinetic reflex, which would move the eye in the direction of the retinal background motion (opposite to the direction in which the target is moving). Similarly, the perceptual system must estimate the actual direction and speed of moving objects in spite of the confounding effects of the eye rotation. This paper proposes a neural model to account for the ability of primates to accomplish these tasks. The model simulates the neurophysiological properties of cell types found in the superior temporal sulcus of the macaque monkey, specifically the medial superior temporal (MST) region. These cells process signals related to target motion, background motion, and receive an efference copy of eye velocity during pursuit movements. The model focuses on the interactions between cells in the ventral and dorsal subdivisions of MST, which are hypothesized to process target velocity and background motion, respectively. The model explains how these signals can be combined to explain behavioral data about pursuit maintenance and perceptual data from human studies, including the Aubert--Fleischl phenomenon and the Filehne Illusion, thereby clarifying the functional significance of neurophysiological data about these MST cell properties. It is suggested that the connectivity used in the model may represent a general strategy used by the brain in analyzing the visual world.

Apparent motion produces multiple deficits in visually guided smooth pursuit eye movements of monkeys

We used apparent motion targets to explore how degraded visual motion alters smooth pursuit eye movements. Apparent motion targets consisted of brief stationary flashes with a spatial separation (Deltax), temporal separation (Deltat), and apparent target velocity equal to Deltax/Deltat. Changes in pursuit initiation were readily observed when holding target velocity constant and increasing the flash separation. As flash separation increased, the first deficit observed was an increase in the latency to peak eye acceleration. Also seen was a paradoxical increase in initial eye acceleration. Further increases in the flash separation produced larger increases in latency and resulted in decreased eye acceleration. By varying target velocity, we were able to discern that the visual inputs driving pursuit initiation show both temporal and spatial limits. For target velocities above 4-8 degrees /s, deficits in the initiation of pursuit were seen when Deltax exceeded 0.2-0.5 degrees, even when Deltat was small. For target velocities below 4-8 degrees /s, deficits appeared when Deltat exceeded 32-64 ms, even when Deltax was small. Further experiments were designed to determine whether the spatial limit varied as retinal and extra-retinal factors changed. Varying the initial retinal position of the target for motion at 18 degrees /s revealed that the spatial limit increased as a function of retinal eccentricity. We then employed targets that increased velocity twice, once from fixation and again during pursuit. These experiments revealed that, as expected, the spatial limit is expressed in terms of the flash separation on the retina. The spatial limit is uninfluenced by either eye velocity or the absolute velocity of the target. These experiments also demonstrate that "initiation" deficits can be observed during ongoing pursuit, and are thus not deficits in initiation per se. We conclude that such deficits result from degradation of the retino-centric motion signals that drive pursuit eye acceleration. For large flash separations, we also observed deficits in the maintenance of pursuit: sustained eye velocity failed to match the constant apparent target velocity. Deficits in the maintenance of pursuit depended on both target velocity and Deltat and did not result simply from a failure of degraded image motion signals to drive eye acceleration. We argue that such deficits result from a low gain in the eye velocity memory that normally supports the maintenance of pursuit. This low gain may appear because visual inputs are so degraded that the transition from fixation to tracking is incomplete.

Visual motion analysis for pursuit eye movements in area MT of macaque monkeys

We asked whether the dynamics of target motion are represented in visual area MT and how information about image velocity and acceleration might be extracted from the population responses in area MT for use in motor control. The time course of MT neuron responses was recorded in anesthetized macaque monkeys during target motions that covered the range of dynamics normally seen during smooth pursuit eye movements. When the target motion provided steps of target speed, MT neurons showed a continuum from purely tonic responses to those with large transient pulses of firing at the onset of motion. Cells with large transient responses for steps of target speed also had larger responses for smooth accelerations than for decelerations through the same range of target speeds. Condition-test experiments with pairs of 64 msec pulses of target speed revealed response attenuation at short interpulse intervals in cells with large transient responses. For sinusoidal modulation of target speed, MT neuron responses were strongly modulated for frequencies up to, but not higher than, 8 Hz. The phase of the responses was consistent with a 90 msec time delay between target velocity and firing rate. We created a model that reproduced the dynamic responses of MT cells using divisive gain control, used the model to visualize the population response in MT to individual stimuli, and devised weighted-averaging computations to reconstruct target speed and acceleration from the population response. Target speed could be reconstructed if each neuron's output was weighted according to its preferred speed. Target acceleration could be reconstructed if each neuron's output was weighted according to the product of preferred speed and a measure of the size of its transient response.

Role of the oculomotor vermis in generating pursuit and saccades: effects of microstimulation

We studied the eye movements evoked by applying small amounts of current (2-50 microA) within the oculomotor vermis of two monkeys. We first compared the eye movements evoked by microstimulation applied either during maintained pursuit or during fixation. Smooth, pursuitlike changes in eye velocity caused by the microstimulation were directed toward the ipsilateral side and occurred at short latencies (10-20 ms). The amplitudes of these pursuitlike changes were larger during visually guided pursuit toward the contralateral side than during either fixation or visually guided pursuit toward the ipsilateral side. At these same sites, microstimulation also often produced abrupt, saccadelike changes in eye velocity. In contrast to the smooth changes in eye velocity, these saccadelike effects were more prevalent during fixation and during pursuit toward the ipsilateral side. The amplitude and type of evoked eye movements could also be manipulated at single sites by changing the frequency of microstimulation. Increasing the frequency of microstimulation produced increases in the amplitude of pursuitlike changes, but only up to a certain point. Beyond this point, the value of which depended on the site and whether the monkey was fixating or pursuing, further increases in stimulation frequency produced saccadelike changes of increasing amplitude. To quantify these effects, we introduced a novel method for classifying eye movements as pursuitlike or saccadelike. The results of this analysis showed that the eye movements evoked by microstimulation exhibit a distinct transition point between pursuit and saccadelike effects and that the amplitude of eye movement that corresponds to this transition point depends on the eye movement behavior of the monkey. These results are consistent with accumulating evidence that the oculomotor vermis and its associated deep cerebellar nucleus, the caudal fastigial, are involved in the control of both pursuit and saccadic eye movements. We suggest that the oculomotor vermis might accomplish this role by altering the amplitude of a motor error signal that is common to both saccades and pursuit.

Human motion perception and smooth eye movements show similar directional biases for elongated apertures

Although numerous studies have examined the relationship between smooth-pursuit eye movements and motion perception, it remains unresolved whether a common motion-processing system subserves both perception and pursuit. To address this question, we simultaneously recorded perceptual direction judgments and the concomitant smooth eye-movement response to a plaid stimulus that we have previously shown generates systematic perceptual errors. We measured the perceptual direction biases psychophysically and the smooth eye-movement direction biases using two methods (standard averaging and oculometric analysis). We found that the perceptual and oculomotor biases were nearly identical, suggesting that pursuit and perception share a critical motion processing stage, perhaps in area MT or MST of extrastriate visual cortex.

Intrinsic properties of biological motion detectors prevent the optomotor control system from getting unstable

Compensatory eye, head or body movements are essential to stabilize the gaze or the path of locomotion. Because such compensatory responses usually lag the sensory input by a time delay, the underlying control system is prone to instability, at least if it operates with a high gain in order to compensate disturbances efficiently. In behavioural experiments it could be shown that the optomotor system of the fly does not get unstable even when its overall gain is so high that, on average, imposed disturbances are compensated to a large extent. Fluctuations of the animal’s torque signal do not build up. Rather they are accompanied by only small-amplitude jittery retinal image displacements that rarely slip over more than a few neighbouring photoreceptors. Combined electrophysiological experiments on a pair of neurons in the fly’s optomotor pathway and model simulations of the optomotor control system suggest that this relative stability of the optomotor system is the consequence of the specific velocity dependence of biological movement detectors. The response of the movement detectors first increases with increasing velocity, reaches a maximum and then decreases again. As a consequence, large-amplitude fluctuations in pattern velocity, as are generated when the optomotor system tends to get unstable, are transmitted with a small gain leading to only relatively small torque fluctuations and, thus, small-amplitude image displacements.