Recurrent network dynamics reconciles visual motion segmentation and integration

A dynamical model of visual motion processing for arbitrary stimuli including type II plaids

To explore the operating principle of visual motion processing in the brain underlying perception and eye movements, we model the information processing of velocity estimate of the visual stimulus at the algorithmic level using the dynamical system approach. In this study, we formulate the model as an optimization process of an appropriately defined objective function. The model is applicable to arbitrary visual stimuli. We find that our theoretical predictions qualitatively agree with time evolution of eye movement reported by previous works across various types of stimulus. Our results suggest that the brain implements the present framework as the internal model of motion vision. We anticipate our model to be a promising building block for more profound understanding of visual motion processing as well as for the development of robotics.

Speed Estimation for Visual Tracking Emerges Dynamically from Nonlinear Frequency Interactions

Sensing the movement of fast objects within our visual environments is essential for controlling actions. It requires online estimation of motion direction and speed. We probed human speed representation using ocular tracking of stimuli of different statistics. First, we compared ocular responses to single drifting gratings (DGs) with a given set of spatiotemporal frequencies to broadband motion clouds (MCs) of matched mean frequencies. Motion energy distributions of gratings and clouds are point-like, and ellipses oriented along the constant speed axis, respectively. Sampling frequency space, MCs elicited stronger, less variable, and speed-tuned responses. DGs yielded weaker and more frequency-tuned responses. Second, we measured responses to patterns made of two or three components covering a range of orientations within Fourier space. Early tracking initiation of the patterns was best predicted by a linear combination of components before nonlinear interactions emerged to shape later dynamics. Inputs are supralinearly integrated along an iso-velocity line and sublinearly integrated away from it. A dynamical probabilistic model characterizes these interactions as an excitatory pooling along the iso-velocity line and inhibition along the orthogonal “scale” axis. Such crossed patterns of interaction would appropriately integrate or segment moving objects. This study supports the novel idea that speed estimation is better framed as a dynamic channel interaction organized along speed and scale axes.

Local motion pooling is continuous, global motion perception is discrete

Perceiving the motion of an object is thought to involve two stages: Local motion energy is measured at each point in space, and these signals are then pooled across space to build coherent global motion. There are several theories of how local-to-global pooling occurs, but they all predict that global motion perception is a continuous process, such that increasing the strength of motion energy should gradually increase the precision of perceived motion directions. We test this prediction against the alternative that global motion perception is discrete: Motion is either perceived with high precision or fails to be perceived altogether. Data from human observers provides clear evidence that, whereas pooling local motion energy is continuous, the segmentation of local signals into coherent global motion patterns is a discrete process. This result adds motion perception to the growing list of processes that exhibit evidence of all-or-none visual awareness. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Dynamic divisive normalization circuits explain and predict change detection in monkey area MT

Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.

Evidence of inverted gravity-driven variation in predictive sensorimotor function

We move our eyes to place the fovea into the part of a viewed scene currently of interest. Recent evidence suggests that each human has signature patterns of eye movements like handwriting which depend on their sensitivity, allocation of attention and experience. Use of implicit knowledge of how earth's gravity influences object motion has been shown to aid dynamic perception. We used a projected ball-tracking task with a plain background offering no context cues to probe the effect of acquired experience about physical laws of gravitation on performance differences of 44 participants under a simulated gravity and an atypical (upward) antigravity condition. Performance measured by the unsigned difference between instantaneous eye and stimulus positions (RMSE) was consistently worse in the antigravity condition. In the vertical RMSE, participants took about 200 ms longer to improve to the best performance for antigravity compared to gravity trials. The antigravity condition produced a divergence of individual performance which was correlated with levels of questionnaire-based quantified traits of schizotypy but not control traits. Grouping participants by high or low traits revealed a negative relationship between schizotypy trait level and both initiation and maintenance of tracking, a result consistent with trait-related impoverished sensory prediction. The findings confirm for the first time that where cues enabling exact estimation of acceleration are unavailable, knowledge of gravity contributes to dynamic prediction improving motion processing. With acceleration expectations violated, we demonstrate that antigravity tracking could act as a multivariate diagnostic window into predictive brain function.

Center-surround velocity-based segmentation: Speed, eccentricity, and timing of visual stimuli interact to determine interocular dominance

We used a novel method to capture the spatial dominance pattern of competing motion fields at rivalry onset. When rivaling velocities were different, the participants reported center-surround segmentation: The slower stimuli often dominated in the center while faster motion persisted along the borders. The size of the central static/slow field scaled with the stimulus size. The central dominance was time-locked to the static stimulus onset but was disrupted if the dynamic stimulus was presented later. We then used the same stimuli as masks in an interocular suppression paradigm. The local suppression strengths were probed with targets at different eccentricities. Consistent with the center-surround segmentation, target speed and location interacted with mask velocities. Specifically, suppression power of the slower masks was nonhomogenous with eccentricity, providing a potential explanation for center-surround velocity-based segmentation. This interaction of speed, eccentricity, and timing has implications for motion processing and interocular suppression. The influence of different masks on which target features get suppressed predicts that some "unconscious effects" are not generalizable across masks and, thus, need to be replicated under various masking conditions.

Motion Perception: From Detection to Interpretation

Visual motion processing can be conceptually divided into two levels. In the lower level, local motion signals are detected by spatiotemporal-frequency-selective sensors and then integrated into a motion vector flow. Although the model based on V1-MT physiology provides a good computational framework for this level of processing, it needs to be updated to fully explain psychophysical findings about motion perception, including complex motion signal interactions in the spatiotemporal-frequency and space domains. In the higher level, the velocity map is interpreted. Although there are many motion interpretation processes, we highlight the recent progress in research on the perception of material (e.g., specular reflection, liquid viscosity) and on animacy perception. We then consider possible linking mechanisms of the two levels and propose intrinsic flow decomposition as the key problem. To provide insights into computational mechanisms of motion perception, in addition to psychophysics and neurosciences, we review machine vision studies seeking to solve similar problems.

Globally Normal Bistable Motion Perception of Anisometropic Amblyopes May Profit From an Unusual Coding Mechanism

Anisometropic amblyopia is a neurodevelopmental disorder of the visual system. There is evidence that the neural deficits spread across visual areas, from the primary cortex up to higher brain areas, including motion coding structures such as MT. Here, we used bistable plaid motion to investigate changes in the underlying mechanisms of motion integration and segmentation and, thus, help us to unravel in more detail deficits in the amblyopic visual motion system. Our results showed that (1) amblyopes globally exhibited normal bistable perception in all viewing conditions compared to the control group and (2) decreased contrast led to a stronger increase in percept switches and decreased percept durations in the control group, while the amblyopic group exhibited no such changes. There were few differences in outcomes dependent upon the use of the weak eye, the strong eye, or both eyes for viewing the stimuli, but this was a general effect present across all subjects, not specific to the amblyopic group. To understand the role of noise and adaptation in such cases of bistable perception, we analyzed predictions from a model and found that contrast does indeed affect percept switches and durations as observed in the control group, in line with the hypothesis that lower stimulus contrast enhances internal noise effects. The combination of experimental and computational results presented here suggests a different motion coding mechanism in the amblyopic visual system, with relatively little effect of stimulus contrast on amblyopes' bistable motion perception.

Criterion-free measurement of motion transparency perception at different speeds

Transparency perception often occurs when objects within the visual scene partially occlude each other or move at the same time, at different velocities across the same spatial region. Although transparent motion perception has been extensively studied, we still do not understand how the distribution of velocities within a visual scene contribute to transparent perception. Here we use a novel psychophysical procedure to characterize the distribution of velocities in a scene that give rise to transparent motion perception. To prevent participants from adopting a subjective decision criterion when discriminating transparent motion, we used an “odd-one-out,” three-alternative forced-choice procedure. Two intervals contained the standard—a random-dot-kinematogram with dot speeds or directions sampled from a uniform distribution. The other interval contained the comparison—speeds or directions sampled from a distribution with the same range as the standard, but with a notch of different widths removed. Our results suggest that transparent motion perception is driven primarily by relatively slow speeds, and does not emerge when only very fast speeds are present within a visual scene. Transparent perception of moving surfaces is modulated by stimulus-based characteristics, such as the separation between the means of the overlapping distributions or the range of speeds presented within an image. Our work illustrates the utility of using objective, forced-choice methods to reveal the mechanisms underlying motion transparency perception.