Predicting 2D Target Velocity Cannot Help 2D Motion Integration for Smooth Pursuit Initiation

Bayesian approaches to smooth pursuit of random dot kinematograms: effects of varying RDK noise and the predictability of RDK direction

Smooth pursuit eye movements respond on the basis of both immediate and anticipated target motion, where anticipations may be derived from either memory or perceptual cues. To study the combined influence of both immediate sensory motion and anticipation, subjects pursued clear or noisy random dot kinematograms (RDKs) whose mean directions were chosen from Gaussian distributions with SDs = 10° (narrow prior) or 45° (wide prior). Pursuit directions were consistent with Bayesian theory in that transitions over time from dependence on the prior to near total dependence on immediate sensory motion (likelihood) took longer with the noisier RDKs and with the narrower, more reliable, prior. Results were fit to Bayesian models in which parameters representing the variability of the likelihood either were or were not constrained to be the same for both priors. The unconstrained model provided a statistically better fit, with the influence of the prior in the constrained model smaller than predicted from strict reliability-based weighting of prior and likelihood. Factors that may have contributed to this outcome include prior variability different from nominal values, low-level sensorimotor learning with the narrow prior, or departures of pursuit from strict adherence to reliability-based weighting. Although modifications of, or alternatives to, the normative Bayesian model will be required, these results, along with previous studies, suggest that Bayesian approaches are a promising framework to understand how pursuit combines immediate sensory motion, past history, and informative perceptual cues to accurately track the target motion that is most likely to occur in the immediate future.NEW & NOTEWORTHY Smooth pursuit eye movements respond on the basis of anticipated, as well as immediate, target motions. Bayesian models using reliability-based weighting of previous (prior) and immediate target motions (likelihood) accounted for many, but not all, aspects of pursuit of clear and noisy random dot kinematograms with different levels of predictability. Bayesian approaches may solve the long-standing problem of how pursuit combines immediate sensory motion and anticipation of future motion to configure an effective response.

Tracking and perceiving diverse motion signals: Directional biases in human smooth pursuit and perception

Human smooth pursuit eye movements and motion perception behave similarly when observers track and judge the motion of simple objects, such as dots. But moving objects in our natural environment are complex and contain internal motion. We ask how pursuit and perception integrate the motion of objects with motion that is internal to the object. Observers (n = 20) tracked a moving random-dot kinematogram with their eyes and reported the object’s perceived direction. Objects moved horizontally with vertical shifts of 0, ±3, ±6, or ±9° and contained internal dots that were static or moved ±90° up/down. Results show that whereas pursuit direction was consistently biased in the direction of the internal dot motion, perceptual biases differed between observers. Interestingly, the perceptual bias was related to the magnitude of the pursuit bias (r = 0.75): perceptual and pursuit biases were directionally aligned in observers that showed a large pursuit bias, but went in opposite directions in observers with a smaller pursuit bias. Dissociations between perception and pursuit might reflect different functional demands of the two systems. Pursuit integrates all available motion signals in order to maximize the ability to monitor and collect information from the whole scene. Perception needs to recognize and classify visual information, thus segregating the target from its context. Ambiguity in whether internal motion is part of the scene or contributes to object motion might have resulted in individual differences in perception. The perception-pursuit correlation suggests shared early-stage motion processing or perception-pursuit interactions.

Visual selective attention and the control of tracking eye movements: a critical review

People’s eyes are directed at objects of interest with the aim of acquiring visual information. However, processing this information is constrained in capacity, requiring task-driven and salience-driven attentional mechanisms to select few among the many available objects. A wealth of behavioral and neurophysiological evidence has demonstrated that visual selection and the motor selection of saccade targets rely on shared mechanisms. This coupling supports the premotor theory of visual attention put forth more than 30 years ago, postulating visual selection as a necessary stage in motor selection. In this review, we examine to which extent the coupling of visual and motor selection observed with saccades is replicated during ocular tracking. Ocular tracking combines catch-up saccades and smooth pursuit to foveate a moving object. We find evidence that ocular tracking requires visual selection of the speed and direction of the moving target, but the position of the motion signal may not coincide with the position of the pursuit target. Further, visual and motor selection can be spatially decoupled when pursuit is initiated (open-loop pursuit). We propose that a main function of coupled visual and motor selection is to serve the coordination of catch-up saccades and pursuit eye movements. A simple race-to-threshold model is proposed to explain the variable coupling of visual selection during pursuit, catch-up and regular saccades, while generating testable predictions. We discuss pending issues, such as disentangling visual selection from preattentive visual processing and response selection, and the pinpointing of visual selection mechanisms, which have begun to be addressed in the neurophysiological literature.

Expectations about motion direction affect perception and anticipatory smooth pursuit differently

Smooth pursuit eye movements and visual motion perception rely on the integration of current sensory signals with past experience. Experience shapes our expectation of current visual events and can drive eye movement responses made in anticipation of a target, such as anticipatory pursuit. Previous research revealed consistent effects of expectation on anticipatory pursuit-eye movements follow the expected target direction or speed-and contrasting effects on motion perception, but most studies considered either eye movement or perceptual responses. The current study directly compared effects of direction expectation on perception and anticipatory pursuit within the same direction discrimination task to investigate whether both types of responses are affected similarly or differently. Observers (n = 10) viewed high-coherence random-dot kinematograms (RDKs) moving rightward and leftward with a probability of 50%, 70%, or 90% in a given block of trials to build up an expectation of motion direction. They were asked to judge motion direction of interleaved low-coherence RDKs (0%-15%). Perceptual judgements were compared with changes in anticipatory pursuit eye movements as a function of probability. Results show that anticipatory pursuit velocity scaled with probability and followed direction expectation (attraction bias), whereas perceptual judgments were biased opposite to direction expectation (repulsion bias). Control experiments suggest that the repulsion bias in perception was not caused by retinal slip induced by anticipatory pursuit, or by motion adaptation. We conclude that direction expectation can be processed differently for perception and anticipatory pursuit.NEW & NOTEWORTHY We show that expectations about motion direction that are based on long-term trial history affect perception and anticipatory pursuit differently. Whereas anticipatory pursuit direction was coherent with the expected motion direction (attraction bias), perception was biased opposite to the expected direction (repulsion bias). These opposite biases potentially reveal different ways in which perception and action utilize prior information and support the idea of different information processing for perception and pursuit.

Predictive Smooth Pursuit Eye Movements

Smooth pursuit eye movements maintain the line of sight on smoothly moving targets. Although often studied as a response to sensory motion, pursuit anticipates changes in motion trajectories, thus reducing harmful consequences due to sensorimotor processing delays. Evidence for predictive pursuit includes (a) anticipatory smooth eye movements (ASEM) in the direction of expected future target motion that can be evoked by perceptual cues or by memory for recent motion, (b) pursuit during periods of target occlusion, and (c) improved accuracy of pursuit with self-generated or biologically realistic target motions. Predictive pursuit has been linked to neural activity in the frontal cortex and in sensory motion areas. As behavioral and neural evidence for predictive pursuit grows and statistically based models augment or replace linear systems approaches, pursuit is being regarded less as a reaction to immediate sensory motion and more as a predictive response, with retinal motion serving as one of a number of contributing cues.

Illusory motion reveals velocity matching, not foveation, drives smooth pursuit of large objects

When small objects move in a scene, we keep them foveated with smooth pursuit eye movements. Although large objects such as people and animals are common, it is nonetheless unknown how we pursue them since they cannot be foveated. It might be that the brain calculates an object's centroid, and then centers the eyes on it during pursuit as a foveation mechanism might. Alternatively, the brain merely matches the velocity by motion integration. We test these alternatives with an illusory motion stimulus that translates at a speed different from its retinal motion. The stimulus was a Gabor array that translated at a fixed velocity, with component Gabors that drifted with motion consistent or inconsistent with the translation. Velocity matching predicts different pursuit behaviors across drift conditions, while centroid matching predicts no difference. We also tested whether pursuit can segregate and ignore irrelevant local drifts when motion and centroid information are consistent by surrounding the Gabors with solid frames. Finally, observers judged the global translational speed of the Gabors to determine whether smooth pursuit and motion perception share mechanisms. We found that consistent Gabor motion enhanced pursuit gain while inconsistent, opposite motion diminished it, drawing the eyes away from the center of the stimulus and supporting a motion-based pursuit drive. Catch-up saccades tended to counter the position offset, directing the eyes opposite to the deviation caused by the pursuit gain change. Surrounding the Gabors with visible frames canceled both the gain increase and the compensatory saccades. Perceived speed was modulated analogous to pursuit gain. The results suggest that smooth pursuit of large stimuli depends on the magnitude of integrated retinal motion information, not its retinal location, and that the position system might be unnecessary for generating smooth velocity to large pursuit targets.

Orientation-dependent biases in length judgments of isolated stimuli

Vertical line segments tend to be perceived as longer than horizontal ones of the same length, but this may in part be due to configuration effects. To minimize such effects, we used isolated line segments in a two-interval, forced choice paradigm, not limiting ourselves to horizontal and vertical. We fitted psychometric curves using a Bayesian method that assumes that, for a given subject, the lapse rate is the same across all conditions. The closer a line segment's orientation was to vertical, the longer it was perceived to be. Moreover, subjects tended to report the standard line (in the second interval) as longer. The data were well described by a model that contains both an orientation-dependent and an interval-dependent multiplicative bias. Using this model, we estimated that a vertical line was on average perceived as 9.2% ± 2.1% longer than a horizontal line, and a second-interval line was on average perceived as 2.4% ± 0.9% longer than a first-interval line. Moving from a descriptive to an explanatory model, we hypothesized that anisotropy in the polar angle of lines in three dimensions underlies the horizontal-vertical illusion, specifically, that line segments more often have a polar angle of 90° (corresponding to the ground plane) than any other polar angle. This model qualitatively accounts not only for the empirical relationship between projected length and projected orientation that predicts the horizontal-vertical illusion, but also for the empirical distribution of projected orientation in photographs of natural scenes and for paradoxical results reported earlier for slanted surfaces.

Behavioral characterization of prediction and internal models in adolescents with autistic spectrum disorders

Autism has been considered as a deficit in prediction of the upcoming event or of the sensory consequences of our own movements. To test this hypothesis, we recorded eye movements from high-functioning autistic adolescents and from age-matched controls during a blanking paradigm. In this paradigm, adolescents were instructed to follow a moving target with their eyes even during its transient disappearance. Given the absence of visual information during the blanking period, eye movements during this period are solely controlled on the basis of the prediction of the ongoing target motion. Typical markers of predictive eye movements such as the number and accuracy of predictive saccades and the predictive reacceleration before target reappearance were identical in the two populations. In addition, the synergy of predictive saccades and smooth pursuit observed during the blanking periods, which is a marker for the quality of internal models about target/eye motions, was comparable between these two populations. These results suggest that, in our large population of high-functioning autistic adolescent, both predictive abilities and internal models are left intact in Autism, at least for low-level sensorimotor transformations.

Active inference, eye movements and oculomotor delays

This paper considers the problem of sensorimotor delays in the optimal control of (smooth) eye movements under uncertainty. Specifically, we consider delays in the visuo-oculomotor loop and their implications for active inference. Active inference uses a generalisation of Kalman filtering to provide Bayes optimal estimates of hidden states and action in generalised coordinates of motion. Representing hidden states in generalised coordinates provides a simple way of compensating for both sensory and oculomotor delays. The efficacy of this scheme is illustrated using neuronal simulations of pursuit initiation responses, with and without compensation. We then consider an extension of the generative model to simulate smooth pursuit eye movements-in which the visuo-oculomotor system believes both the target and its centre of gaze are attracted to a (hidden) point moving in the visual field. Finally, the generative model is equipped with a hierarchical structure, so that it can recognise and remember unseen (occluded) trajectories and emit anticipatory responses. These simulations speak to a straightforward and neurobiologically plausible solution to the generic problem of integrating information from different sources with different temporal delays and the particular difficulties encountered when a system-like the oculomotor system-tries to control its environment with delayed signals.

Dynamic integration of information about salience and value for smooth pursuit eye movements

Eye movement behavior can be determined by bottom-up factors like visual salience and by top-down factors like expected value. These different types of signals have to be combined for the control of eye movements. In this study we investigated how smooth pursuit eye movements integrate salience and value information. Observers were asked to track a random-dot kinematogram containing two coherent motion directions. To manipulate salience, the coherence or the density of one of the motion signals was varied. To manipulate value, observers won or lost money in a separate experiment if they were tracking one or the other motion direction. Our results show that pursuit direction was initially determined only by salience. 300-400 ms after target motion onset, pursuit steered towards the rewarded direction and the salience effects disappeared. The time course of this effect depended crucially on the difficulty to segment the two signal directions. These results indicate that salience determines early pursuit responses in the same way as saccades with short latencies. Value information is processed slower and dominates pursuit after several 100 ms.

Modelling the dynamics of motion integration with a new luminance-gated diffusion mechanism

The dynamics of motion integration show striking similarities when observed at neuronal, psychophysical, and oculomotor levels. Based on the inter-relation and complementary insights given by those dynamics, our goal was to test how basic mechanisms of dynamical cortical processing can be incorporated in a dynamical model to solve several aspects of 2D motion integration and segmentation. Our model is inspired by the hierarchical processing stages of the primate visual cortex: we describe the interactions between several layers processing local motion and form information through feedforward, feedback, and inhibitive lateral connections. Also, following perceptual studies concerning contour integration and physiological studies of receptive fields, we postulate that motion estimation takes advantage of another low-level cue, which is luminance smoothness along edges or surfaces, in order to gate recurrent motion diffusion. With such a model, we successfully reproduced the temporal dynamics of motion integration on a wide range of simple motion stimuli: line segments, rotating ellipses, plaids, and barber poles. Furthermore, we showed that the proposed computational rule of luminance-gated diffusion of motion information is sufficient to explain a large set of contextual modulations of motion integration and segmentation in more elaborated stimuli such as chopstick illusions, simulated aperture problems, or rotating diamonds. As a whole, in this paper we proposed a new basal luminance-driven motion integration mechanism as an alternative to less parsimonious models, we carefully investigated the dynamics of motion integration, and we established a distinction between simple and complex stimuli according to the kind of information required to solve their ambiguities.

Bayesian modeling of dynamic motion integration

The quality of the representation of an object's motion is limited by the noise in the sensory input as well as by an intrinsic ambiguity due to the spatial limitation of the visual motion analyzers (aperture problem). Perceptual and oculomotor data demonstrate that motion processing of extended objects is initially dominated by the local 1D motion cues, related to the object's edges and orthogonal to them, whereas 2D information, related to terminators (or edge-endings), takes progressively over and leads to the final correct representation of global motion. A Bayesian framework accounting for the sensory noise and general expectancies for object velocities has proven successful in explaining several experimental findings concerning early motion processing [Weiss, Y., Adelson, E., 1998. Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision. MIT Technical report, A.I. Memo 1624]. In particular, these models provide a qualitative account for the initial bias induced by the 1D motion cue. However, a complete functional model, encompassing the dynamical evolution of object motion perception, including the integration of different motion cues, is still lacking. Here we outline several experimental observations concerning human smooth pursuit of moving objects and more particularly the time course of its initiation phase, which reflects the ongoing motion integration process. In addition, we propose a recursive extension of the Bayesian model, motivated and constrained by our oculomotor data, to describe the dynamical integration of 1D and 2D motion information. We compare the model predictions for object motion tracking with human oculomotor recordings.