Dynamics of distributed 1D and 2D motion representations for short-latency ocular following

Weighted power summation and contrast normalization mechanisms account for short-latency eye movements to motion and disparity of sine-wave gratings and broadband visual stimuli in humans

In this paper, we show that the model we proposed earlier to account for the disparity vergence eye movements (disparity vergence responses, or DVRs) in response to horizontal and vertical disparity steps of white noise visual stimuli also provides an excellent description of the short-latency ocular following responses (OFRs) to broadband stimuli in the visual motion domain. In addition, we reanalyzed the data and applied the model to several earlier studies that used sine-wave gratings (single or a combination of two or three gratings) and white noise stimuli. The model provides a very good account of all of these data. The model postulates that the short-latency eye movements-OFRs and DVRs-can be accounted for by the operation of two factors: an excitatory drive, determined by a weighted sum of contributions of stimulus Fourier components, scaled by a global contrast normalization mechanism. The output of the operation of these two factors is then nonlinearly scaled by the total contrast of the stimulus. Despite different roles of disparity (horizontal and vertical) and motion signals in visual scene analyses, the earliest processing stages of these different signals appear to be very similar.

Manipulating the Fourier spectra of stimuli comprising a two-frame kinematogram to study early visual motion-detecting mechanisms: Perception versus short latency ocular-following responses

Two-frame kinematograms have been extensively used to study motion perception in human vision. Measurements of the direction-discrimination performance limits (Dmax) have been the primary subject of such studies, whereas surprisingly little research has asked how the variability in the spatial frequency content of individual frames affects motion processing. Here, we used two-frame one-dimensional vertical pink noise kinematograms, in which images in both frames were bandpass filtered, with the central spatial frequency of the filter manipulated independently for each image. To avoid spatial aliasing, there was no actual leftward-rightward shift of the image: instead, the phases of all Fourier components of the second image were shifted by ±¼ wavelength with respect to those of the first. We recorded ocular-following responses (OFRs) and perceptual direction discrimination in human subjects. OFRs were in the direction of the Fourier components' shift and showed a smooth decline in amplitude, well fit by Gaussian functions, as the difference between the central spatial frequencies of the first and second images increased. In sharp contrast, 100% correct perceptual direction-discrimination performance was observed when the difference between the central spatial frequencies of the first and second images was small, deteriorating rapidly to chance when increased further. Perceptual dependencies moved closer to the OFR ones when subjects were allowed to grade the strength of perceived motion. Response asymmetries common for perceptual judgments and the OFRs suggest that they rely on the same early visual processing mechanisms. The OFR data were quantitatively well described by a model which combined two factors: (1) an excitatory drive determined by a power law sum of stimulus Fourier components' contributions, scaled by (2) a contrast normalization mechanism. Thus, in addition to traditional studies relying on perceptual reports, the OFRs represent a valuable behavioral tool for studying early motion processing on a fine scale.

A dynamical model of visual motion processing for arbitrary stimuli including type II plaids

To explore the operating principle of visual motion processing in the brain underlying perception and eye movements, we model the information processing of velocity estimate of the visual stimulus at the algorithmic level using the dynamical system approach. In this study, we formulate the model as an optimization process of an appropriately defined objective function. The model is applicable to arbitrary visual stimuli. We find that our theoretical predictions qualitatively agree with time evolution of eye movement reported by previous works across various types of stimulus. Our results suggest that the brain implements the present framework as the internal model of motion vision. We anticipate our model to be a promising building block for more profound understanding of visual motion processing as well as for the development of robotics.

A behavioral receptive field for ocular following in monkeys: Spatial summation and its spatial frequency tuning

In human and non-human primates, reflexive tracking eye movements can be initiated at very short latency in response to a rapid shift of the image. Previous studies in humans have shown that only a part of the central visual field is optimal for driving ocular following responses. Herein, we have investigated spatial summation of motion information across a wide range of spatial frequencies and speeds of drifting gratings by recording short-latency ocular following responses in macaque monkeys. We show that optimal stimulus size for driving ocular responses cover a small (<20° diameter), central part of the visual field that shrinks with higher spatial frequency. This signature of linear motion integration remains invariant with speed and temporal frequency. For low and medium spatial frequencies, we found a strong suppressive influence from surround motion, evidenced by a decrease of response amplitude for stimulus sizes larger than optimal. Such suppression disappears with gratings at high frequencies. The contribution of peripheral motion was investigated by presenting grating annuli of increasing eccentricity. We observed an exponential decay of response amplitude with grating eccentricity, the decrease being faster for higher spatial frequencies. Weaker surround suppression can thus be explained by sparser eccentric inputs at high frequencies. A Difference-of-Gaussians model best renders the antagonistic contributions of peripheral and central motions. Its best-fit parameters coincide with several, well-known spatial properties of area MT neuronal populations. These results describe the mechanism by which central motion information is automatically integrated in a context-dependent manner to drive ocular responses.Significance statementOcular following is driven by visual motion at ultra-short latency in both humans and monkeys. Its dynamics reflect the properties of low-level motion integration. Here, we show that a strong center-surround suppression mechanism modulates initial eye velocity. Its spatial properties are dependent upon visual inputs' spatial frequency but are insensitive to either its temporal frequency or speed. These properties are best described with a Difference-of-Gaussian model of spatial integration. The model parameters reflect many spatial characteristics of motion sensitive neuronal populations in monkey area MT. Our results further outline the computational properties of the behavioral receptive field underpinning automatic, context-dependent motion integration.

Speed Estimation for Visual Tracking Emerges Dynamically from Nonlinear Frequency Interactions

Sensing the movement of fast objects within our visual environments is essential for controlling actions. It requires online estimation of motion direction and speed. We probed human speed representation using ocular tracking of stimuli of different statistics. First, we compared ocular responses to single drifting gratings (DGs) with a given set of spatiotemporal frequencies to broadband motion clouds (MCs) of matched mean frequencies. Motion energy distributions of gratings and clouds are point-like, and ellipses oriented along the constant speed axis, respectively. Sampling frequency space, MCs elicited stronger, less variable, and speed-tuned responses. DGs yielded weaker and more frequency-tuned responses. Second, we measured responses to patterns made of two or three components covering a range of orientations within Fourier space. Early tracking initiation of the patterns was best predicted by a linear combination of components before nonlinear interactions emerged to shape later dynamics. Inputs are supralinearly integrated along an iso-velocity line and sublinearly integrated away from it. A dynamical probabilistic model characterizes these interactions as an excitatory pooling along the iso-velocity line and inhibition along the orthogonal "scale" axis. Such crossed patterns of interaction would appropriately integrate or segment moving objects. This study supports the novel idea that speed estimation is better framed as a dynamic channel interaction organized along speed and scale axes.

Responses of neurons in macaque MT to unikinetic plaids

Response properties of MT neurons are often studied with "bikinetic" plaid stimuli, which consist of two superimposed sine wave gratings moving in different directions. Oculomotor studies using "unikinetic plaids" in which only one of the two superimposed gratings moves suggest that the eyes first move reflexively in the direction of the moving grating and only later converge on the perceived direction of the moving pattern. MT has been implicated as the source of visual signals that drives these responses. We wanted to know whether stationary gratings, which have little effect on MT cells when presented alone, would influence MT responses when paired with a moving grating. We recorded extracellularly from neurons in area MT and measured responses to stationary and moving gratings, and to their sums: bikinetic and unikinetic plaids. As expected, stationary gratings presented alone had a very modest influence on the activity of MT neurons. Responses to moving gratings and bikinetic plaids were similar to those previously reported and revealed cells selective for the motion of plaid patterns and of their components (pattern and component cells). When these neurons were probed with unikinetic plaids, pattern cells shifted their direction preferences in a way that revealed the influence of the static grating. Component cell preferences shifted little or not at all. These results support the notion that pattern-selective neurons in area MT integrate component motions that differ widely in speed, and that they do so in a way that is consistent with an intersection-of-constraints model.NEW & NOTEWORTHY Human perceptual and eye movement responses to moving gratings are influenced by adding a second, static grating to create a "unikinetic" plaid. Cells in MT do not respond to static gratings, but those gratings still influence the direction selectivity of some MT cells. The cells influenced by static gratings are those tuned for the motion of global patterns, but not those tuned only for the individual components of moving targets.

Binocular summation for reflexive eye movements

Psychophysical studies and our own subjective experience suggest that, in natural viewing conditions (i.e., at medium to high contrasts), monocularly and binocularly viewed scenes appear very similar, with the exception of the improved depth perception provided by stereopsis. This phenomenon is usually described as a lack of binocular summation. We show here that there is an exception to this rule: Ocular following eye movements induced by the sudden motion of a large stimulus, which we recorded from three human subjects, are much larger when both eyes see the moving stimulus, than when only one eye does. We further discovered that this binocular advantage is a function of the interocular correlation between the two monocular images: It is maximal when they are identical, and reduced when the two eyes are presented with different images. This is possible only if the neurons that underlie ocular following are sensitive to binocular disparity.

A Motion-from-Form Mechanism Contributes to Extracting Pattern Motion from Plaids

Since the discovery of neurons selective for pattern motion direction in primate middle temporal area MT (Albright, 1984; Movshon et al., 1985), the neural computation of this signal has been the subject of intense study. The bulk of this work has explored responses to plaids obtained by summing two drifting sinusoidal gratings. Unfortunately, with these stimuli, many different mechanisms are similarly effective at extracting pattern motion. We devised a new set of stimuli, obtained by summing two random line stimuli with different orientations. This allowed several novel manipulations, including generating plaids that do not contain rigid 2D motion. Importantly, these stimuli do not engage most of the previously proposed mechanisms. We then recorded the ocular following responses that such stimuli induce in human subjects. We found that pattern motion is computed even with stimuli that do not cohere perceptually, including those without rigid motion, and even when the two gratings are presented separately to the two eyes. Moderate temporal and/or spatial separation of the gratings impairs the computation. We show that, of the models proposed so far, only those based on the intersection-of-constraints rule, embedding a motion-from-form mechanism (in which orientation signals are used in the computation of motion direction signals), can account for our results. At least for the eye movements reported here, a motion-from-form mechanism is thus involved in one of the most basic functions of the visual motion system: extracting motion direction from complex scenes.

SIGNIFICANCE STATEMENT

Anatomical considerations led to the proposal that visual function is organized in separate processing streams: one (ventral) devoted to form and one (dorsal) devoted to motion. Several experimental results have challenged this view, arguing in favor of a more integrated view of visual processing. Here we add to this body of work, supporting a role for form information even in a function--extracting pattern motion direction from complex scenes--for which decisive evidence for the involvement of form signals has been lacking.

Dynamic resolution of ambiguity during tri-stable motion perception

Multi-stable perception occurs when an image falling onto the retina has multiple incompatible interpretations. We probed this phenomenon in psychophysical experiments using a moving barber-pole visual stimulus configured as a square to generate three competing perceived directions, horizontal, diagonal and vertical. We characterised patterns in reported switching type and percept duration, classifying switches into three groups related to the direction cues driving such transitions i.e. away from diagonal, towards diagonal and between cardinals. The proportions of each class reported by participants depended on contrast. The two including diagonals dominated at low contrast and those between cardinals increased in proportion as contrast was increased. At low contrasts, the less frequent cardinals persisted for shorter than the dominant diagonals and this was reversed at higher contrasts. This observed asymmetry between the dominance of transition classes appears to be driven by different underlying dynamics between cardinal and the oblique cues and their related transitions. At trial onset we found that transitions away from diagonal dominate, a tendency which later in the trial reverses to dominance by transitions excluding the diagonal, most prominently at higher contrasts. Thus ambiguity is resolved over a contrast dependent temporal integration similar to, but lasting longer than that observed when resolving the aperture problem to estimate direction. When the diagonal direction dominates perception, evidence is found for a noisier competition seen in broader duration distributions than during dominance of cardinal perception. There remain aspects of these identified differences in cardinal and oblique dynamics to be investigated in future.

Bifurcation study of a neural field competition model with an application to perceptual switching in motion integration

Perceptual multistability is a phenomenon in which alternate interpretations of a fixed stimulus are perceived intermittently. Although correlates between activity in specific cortical areas and perception have been found, the complex patterns of activity and the underlying mechanisms that gate multistable perception are little understood. Here, we present a neural field competition model in which competing states are represented in a continuous feature space. Bifurcation analysis is used to describe the different types of complex spatio-temporal dynamics produced by the model in terms of several parameters and for different inputs. The dynamics of the model was then compared to human perception investigated psychophysically during long presentations of an ambiguous, multistable motion pattern known as the barberpole illusion. In order to do this, the model is operated in a parameter range where known physiological response properties are reproduced whilst also working close to bifurcation. The model accounts for characteristic behaviour from the psychophysical experiments in terms of the type of switching observed and changes in the rate of switching with respect to contrast. In this way, the modelling study sheds light on the underlying mechanisms that drive perceptual switching in different contrast regimes. The general approach presented is applicable to a broad range of perceptual competition problems in which spatial interactions play a role.

Temporal evolution of pattern disparity processing in humans

Stereo matching, i.e., the matching by the visual system of corresponding parts of the images seen by the two eyes, is inherently a 2D problem. To gain insights into how this operation is carried out by the visual system, we measured, in human subjects, the reflexive vergence eye movements elicited by the sudden presentation of stereo plaids. We found compelling evidence that the 2D pattern disparity is computed by combining disparities first extracted within orientation selective channels. This neural computation takes 10-15 ms, and is carried out even when subjects perceive not a single plaid but rather two gratings in different depth planes (transparency). However, we found that 1D disparities are not always effectively combined: when spatial frequency and contrast of the gratings are sufficiently different pattern disparity is not computed, a result that cannot be simply attributed to the transparency of such stimuli. Based on our results, we propose that a narrow-band implementation of the IOC (Intersection of Constraints) rule (Fennema and Thompson, 1979; Adelson and Movshon, 1982), preceded by cross-orientation suppression, underlies the extraction of pattern disparity.

More is not always better: adaptive gain control explains dissociation between perception and action

Moving objects generate motion information at different scales, which are processed in the visual system with a bank of spatiotemporal frequency channels. It is not known how the brain pools this information to reconstruct object speed and whether this pooling is generic or adaptive; that is, dependent on the behavioral task. We used rich textured motion stimuli of varying bandwidths to decipher how the human visual motion system computes object speed in different behavioral contexts. We found that, although a simple visuomotor behavior such as short-latency ocular following responses takes advantage of the full distribution of motion signals, perceptual speed discrimination is impaired for stimuli with large bandwidths. Such opposite dependencies can be explained by an adaptive gain control mechanism in which the divisive normalization pool is adjusted to meet the different constraints of perception and action.

Motion-based prediction is sufficient to solve the aperture problem

In low-level sensory systems, it is still unclear how the noisy information collected locally by neurons may give rise to a coherent global percept. This is well demonstrated for the detection of motion in the aperture problem: as luminance of an elongated line is symmetrical along its axis, tangential velocity is ambiguous when measured locally. Here, we develop the hypothesis that motion-based predictive coding is sufficient to infer global motion. Our implementation is based on a context-dependent diffusion of a probabilistic representation of motion. We observe in simulations a progressive solution to the aperture problem similar to physiology and behavior. We demonstrate that this solution is the result of two underlying mechanisms. First, we demonstrate the formation of a tracking behavior favoring temporally coherent features independent of their texture. Second, we observe that incoherent features are explained away, while coherent information diffuses progressively to the global scale. Most previous models included ad hoc mechanisms such as end-stopped cells or a selection layer to track specific luminance-based features as necessary conditions to solve the aperture problem. Here, we have proved that motion-based predictive coding, as it is implemented in this functional model, is sufficient to solve the aperture problem. This solution may give insights into the role of prediction underlying a large class of sensory computations.

Neural mechanisms of tactile motion integration in somatosensory cortex

How are local motion signals integrated to form a global motion percept? We investigate the neural mechanisms of tactile motion integration by presenting tactile gratings and plaids to the fingertips of monkeys, using the tactile analogue of a visual monitor and recording the responses evoked in somatosensory cortical neurons. The perceived directions of the gratings and plaids are measured in parallel psychophysical experiments. We identify a population of somatosensory neurons that exhibit integration properties comparable to those induced by analogous visual stimuli in area MT and find that these neural responses account for the perceived direction of the stimuli across all stimulus conditions tested. The preferred direction of the neurons and the perceived direction of the stimuli can be predicted from the weighted average of the directions of the individual stimulus features, highlighting that the somatosensory system implements a vector average mechanism to compute tactile motion direction that bears striking similarities to its visual counterpart.

Modelling the dynamics of motion integration with a new luminance-gated diffusion mechanism

The dynamics of motion integration show striking similarities when observed at neuronal, psychophysical, and oculomotor levels. Based on the inter-relation and complementary insights given by those dynamics, our goal was to test how basic mechanisms of dynamical cortical processing can be incorporated in a dynamical model to solve several aspects of 2D motion integration and segmentation. Our model is inspired by the hierarchical processing stages of the primate visual cortex: we describe the interactions between several layers processing local motion and form information through feedforward, feedback, and inhibitive lateral connections. Also, following perceptual studies concerning contour integration and physiological studies of receptive fields, we postulate that motion estimation takes advantage of another low-level cue, which is luminance smoothness along edges or surfaces, in order to gate recurrent motion diffusion. With such a model, we successfully reproduced the temporal dynamics of motion integration on a wide range of simple motion stimuli: line segments, rotating ellipses, plaids, and barber poles. Furthermore, we showed that the proposed computational rule of luminance-gated diffusion of motion information is sufficient to explain a large set of contextual modulations of motion integration and segmentation in more elaborated stimuli such as chopstick illusions, simulated aperture problems, or rotating diamonds. As a whole, in this paper we proposed a new basal luminance-driven motion integration mechanism as an alternative to less parsimonious models, we carefully investigated the dynamics of motion integration, and we established a distinction between simple and complex stimuli according to the kind of information required to solve their ambiguities.

Temporal dynamics of 2D motion integration for ocular following in macaque monkeys

Several recent studies have shown that extracting pattern motion direction is a dynamical process where edge motion is first extracted and pattern-related information is encoded with a small time lag by MT neurons. A similar dynamics was found for human reflexive or voluntary tracking. Here, we bring an essential, but still missing, piece of information by documenting macaque ocular following responses to gratings, unikinetic plaids, and barber-poles. We found that ocular tracking was always initiated first in the grating motion direction with ultra-short latencies (approximately 55 ms). A second component was driven only 10-15 ms later, rotating tracking toward pattern motion direction. At the end the open-loop period, tracking direction was aligned with pattern motion direction (plaids) or the average of the line-ending motion directions (barber-poles). We characterized the dependency on contrast of each component. Both timing and direction of ocular following were quantitatively very consistent with the dynamics of neuronal responses reported by others. Overall, we found a remarkable consistency between neuronal dynamics and monkey behavior, advocating for a direct link between the neuronal solution of the aperture problem and primate perception and action.

The effects of prolonged viewing of motion on short-latency ocular following responses

The adaptive effects of prolonged viewing of conditioning motion on ocular following responses (OFRs) elicited by brief test motion of a random-dot pattern were studied in humans. We found that the OFRs were significantly reduced when the directions of the conditioning and test motions were the same. The effect of conditioning motion was still observed when the speeds of the conditioning and test motions did not match. The effect was larger when the conditioning duration was longer, and decayed over time with increased temporal separation between the conditioning and test periods. These results are consistent with the characteristics of motion adaptation on the initial smooth pursuit responses. We also obtained data suggesting that the persistence of the effect depends on visual stimulation in the time between the conditioning and test periods, and that the presence of a stationary visual stimulus facilitates recovery from the motion adaptation.

A neurally plausible model of the dynamics of motion integration in smooth eye pursuit based on recursive Bayesian estimation

In this study, we describe a model of motion integration in smooth eye pursuit based on a recursive Bayesian estimation process, which displays a dynamic behaviour qualitatively similar to the dynamics of the motion integration process observed experimentally, both psychophysically in humans and monkeys, and physiologically in monkeys. By formulating the model as an approximate version of a Kalman filter algorithm, we have been able to show that it can be put into a neurally plausible, distributed recurrent form which coarsely corresponds to the recurrent circuitry of visual cortical areas V1 and MT. The model thus provides further support for the notion that the motion integration process is based on a form of Bayesian estimation, as has been suggested by many psychophysical studies, and moreover suggests that the observed dynamic properties of this process are the result of the recursive nature of the motion estimation.

Modeling spatial integration in the ocular following response using a probabilistic framework

The machinery behind the visual perception of motion and the subsequent sensori-motor transformation, such as in ocular following response (OFR), is confronted to uncertainties which are efficiently resolved in the primate's visual system. We may understand this response as an ideal observer in a probabilistic framework by using Bayesian theory [Weiss, Y., Simoncelli, E.P., Adelson, E.H., 2002. Motion illusions as optimal percepts. Nature Neuroscience, 5(6), 598-604, doi:10.1038/nn858] which we previously proved to be successfully adapted to model the OFR for different levels of noise with full field gratings. More recent experiments of OFR have used disk gratings and bipartite stimuli which are optimized to study the dynamics of center-surround integration. We quantified two main characteristics of the spatial integration of motion: (i) a finite optimal stimulus size for driving OFR, surrounded by an antagonistic modulation and (ii) a direction selective suppressive effect of the surround on the contrast gain control of the central stimuli [Barthélemy, F.V., Vanzetta, I., Masson, G.S., 2006. Behavioral receptive field for ocular following in humans: dynamics of spatial summation and center-surround interactions. Journal of Neurophysiology, (95), 3712-3726, doi:10.1152/jn.00112.2006]. Herein, we extended the ideal observer model to simulate the spatial integration of the different local motion cues within a probabilistic representation. We present analytical results which show that the hypothesis of independence of local measures can describe the spatial integration of the motion signal. Within this framework, we successfully accounted for the contrast gain control mechanisms observed in the behavioral data for center-surround stimuli. However, another inhibitory mechanism had to be added to account for suppressive effects of the surround.