Motion Perception: From Detection to Interpretation

Transformation of Motion Pattern Selectivity from Retina to Superior Colliculus

The superior colliculus (SC) is a prominent and conserved visual center in all vertebrates. In mice, the most superficial lamina of the SC is enriched with neurons that are selective for the moving direction of visual stimuli. Here, we study how these direction selective neurons respond to complex motion patterns known as plaids, using two-photon calcium imaging in awake male and female mice. The plaid pattern consists of two superimposed sinusoidal gratings moving in different directions, giving an apparent pattern direction that lies between the directions of the two component gratings. Most direction selective neurons in the mouse SC respond robustly to the plaids and show a high selectivity for the moving direction of the plaid pattern but not of its components. Pattern motion selectivity is seen in both excitatory and inhibitory SC neurons and is especially prevalent in response to plaids with large cross angles between the two component gratings. However, retinal inputs to the SC are ambiguous in their selectivity to pattern versus component motion. Modeling suggests that pattern motion selectivity in the SC can arise from a nonlinear transformation of converging retinal inputs. In contrast, the prevalence of pattern motion selective neurons is not seen in the primary visual cortex (V1). These results demonstrate an interesting difference between the SC and V1 in motion processing and reveal the SC as an important site for encoding pattern motion.

Psychophysical measurement of perceived motion flow of naturalistic scenes

The neural and computational mechanisms underlying visual motion perception have been extensively investigated over several decades, but little attempt has been made to measure and analyze, how human observers perceive the map of motion vectors, or optical flow, in complex naturalistic scenes. Here, we developed a psychophysical method to assess human-perceived motion flows using local vector matching and a flash probe. The estimated perceived flow for naturalistic movies agreed with the physically correct flow (ground truth) at many points, but also showed consistent deviations from the ground truth (flow illusions) at other points. Comparisons with the predictions of various computational models, including cutting-edge computer vision algorithms and coordinate transformation models, indicated that some flow illusions are attributable to lower-level factors such as spatiotemporal pooling and signal loss, while others reflect higher-level computations, including vector decomposition. Our study demonstrates a promising data-driven psychophysical paradigm for an advanced understanding of visual motion perception.

Interaction between spatial perception and temporal perception enables preservation of cause-effect relationship: Visual psychophysics and neuronal dynamics

INTRODUCTION

Visual perception of moving objects is integral to our day-to-day life, integrating visual spatial and temporal perception. Most research studies have focused on finding the brain regions activated during motion perception. However, an empirically validated general mathematical model is required to understand the modulation of the motion perception. Here, we develop a mathematical formulation of the modulation of the perception of a moving object due to a change in speed, under the formulation of the invariance of causality.

METHODS

We formulated the perception of a moving object as the coordinate transformation from a retinotopic space onto perceptual space and derived a quantitative relationship between spatiotemporal coordinates. To validate our model, we undertook the analysis of two experiments: (i) the perceived length of the moving arc, and (ii) the perceived time while observing moving stimuli. We performed a magnetic resonance imaging (MRI) tractography investigation of subjects to demarcate the anatomical correlation of the modulation of the perception of moving objects.

RESULTS

Our theoretical model shows that the interaction between visual-spatial and temporal perception, during the perception of moving object is described by coupled linear equations; and experimental observations validate our model. We observed that cerebral area V5 may be an anatomical correlate for this interaction. The physiological basis of interaction is shown by a Lotka-Volterra system delineating interplay between acetylcholine and dopamine neurotransmitters, whose concentrations vary periodically with the orthogonal phase shift between them, occurring at the axodendritic synapse of complex cells at area V5.

CONCLUSION

Under the invariance of causality in the representation of events in retinotopic space and perceptual space, the speed modulates the perception of a moving object. This modulation may be due to variations of the tuning properties of complex cells at area V5 due to the dynamic interaction between acetylcholine and dopamine. Our analysis is the first significant study, to our knowledge, that establishes a mathematical linkage between motion perception and causality invariance.

An interactive motion perception tool for kindergarteners (and vision scientists)

We implement Adelson and Bergen's spatiotemporal energy model with extension to three-dimensional (x-y-t) in an interactive tool. It helps gain an easy understanding of early (first-order) visual motion perception. We demonstrate its usefulness in explaining an assortment of phenomena, including some that are typically not associated with the spatiotemporal energy model.

Going forward: perceptual bias for forward-facing motion in chimpanzees

When a row of objects surrounded by a frame suddenly shifts a certain distance so that part of the row is occluded by the frame, humans perceive ambiguous apparent motion either to the left or the right. However, when the objects have "directionality," humans perceive them as moving forward in the direction in which they are pointing, which is termed forward-facing motion bias. In the present study, five experiments were conducted to address whether, and if so how, physical properties or prior knowledge about the objects affected the perception of their apparent motion in two juvenile chimpanzees (Pan troglodytes). In experiment 1, the chimpanzees did not show a clear forward-facing bias in judging the direction of motion when directed triangles were presented, whereas the human participants did. In contrast, when pictures of the lateral view of chimpanzees with quadrupedal postures were shown, there was a clear bias for going "forward" with regards to the side with the head (experiment 2). We presented pictures of dogs looking back to explore what features caused the forward-facing motion bias (experiment 3). Chimpanzees did not show any bias for these stimuli, suggesting that the direction of the head and body interactively affected the perceptual bias. Experiment 4 tested the role of the head and found that only the lateral view of the heads of chimpanzees or humans caused the bias (experiment 4). Additional tests also showed that the chimpanzees could not solve the task based only on the direction of the stimuli without motion (experiment 5). These results indicate that the perception of motion in the chimpanzees was affected by the biological features of the stimuli, suggesting their prior knowledge of the "body" from a biological (morphological and kinetic) perspective.

Super-resolution microscopy reveals actomyosin dynamics in medioapical arrays

Arrays of actin filaments (F-actin) near the apical surface of epithelial cells (medioapical arrays) contribute to apical constriction and morphogenesis throughout phylogeny. Here, super-resolution approaches (grazing incidence structured illumination, GI-SIM and lattice light sheet, LLSM) microscopy resolve individual, fluorescently labeled F-actin and bipolar myosin filaments that drive amnioserosa cell shape changes during dorsal closure in Drosophila. In expanded cells, F-actin and myosin form loose, apically domed meshworks at the plasma membrane. The arrays condense as cells contract, drawing the domes into the plane of the junctional belts. As condensation continues, individual filaments are no longer uniformly apparent. As cells expand, arrays of actomyosin are again resolved - some F-actin turnover likely occurs, but a large fraction of existing filaments rearrange. In morphologically isotropic cells, actin filaments are randomly oriented and during contraction, are drawn together but remain essentially randomly oriented. In anisotropic cells, largely parallel actin filaments are drawn closer to one another. Our images offer unparalleled resolution of F-actin in embryonic tissue show that medioapical arrays are tightly apposed to the plasma membrane, are continuous with meshworks of lamellar F-actin and thereby constitute modified cell cortex. In concert with other tagged array components, super-resolution imaging of live specimens will offer new understanding of cortical architecture and function. [Media: see text] [Media: see text] [Media: see text] [Media: see text] [Media: see text] [Media: see text] [Media: see text] [Media: see text] [Media: see text] [Media: see text].

Visual discrimination of optical material properties: A large-scale study

Complex visual processing involved in perceiving the object materials can be better elucidated by taking a variety of research approaches. Sharing stimulus and response data is an effective strategy to make the results of different studies directly comparable and can assist researchers with different backgrounds to jump into the field. Here, we constructed a database containing several sets of material images annotated with visual discrimination performance. We created the material images using physically based computer graphics techniques and conducted psychophysical experiments with them in both laboratory and crowdsourcing settings. The observer's task was to discriminate materials on one of six dimensions (gloss contrast, gloss distinctness of image, translucent vs. opaque, metal vs. plastic, metal vs. glass, and glossy vs. painted). The illumination consistency and object geometry were also varied. We used a nonverbal procedure (an oddity task) applicable for diverse use cases, such as cross-cultural, cross-species, clinical, or developmental studies. Results showed that the material discrimination depended on the illuminations and geometries and that the ability to discriminate the spatial consistency of specular highlights in glossiness perception showed larger individual differences than in other tasks. In addition, analysis of visual features showed that the parameters of higher order color texture statistics can partially, but not completely, explain task performance. The results obtained through crowdsourcing were highly correlated with those obtained in the laboratory, suggesting that our database can be used even when the experimental conditions are not strictly controlled in the laboratory. Several projects using our dataset are underway.

Rats spontaneously perceive global motion direction of drifting plaids

Computing global motion direction of extended visual objects is a hallmark of primate high-level vision. Although neurons selective for global motion have also been found in mouse visual cortex, it remains unknown whether rodents can combine multiple motion signals into global, integrated percepts. To address this question, we trained two groups of rats to discriminate either gratings (G group) or plaids (i.e., superpositions of gratings with different orientations; P group) drifting horizontally along opposite directions. After the animals learned the task, we applied a visual priming paradigm, where presentation of the target stimulus was preceded by the brief presentation of either a grating or a plaid. The extent to which rat responses to the targets were biased by such prime stimuli provided a measure of the spontaneous, perceived similarity between primes and targets. We found that gratings and plaids, when used as primes, were equally effective at biasing the perception of plaid direction for the rats of the P group. Conversely, for the G group, only the gratings acted as effective prime stimuli, while the plaids failed to alter the perception of grating direction. To interpret these observations, we simulated a decision neuron reading out the representations of gratings and plaids, as conveyed by populations of either component or pattern cells (i.e., local or global motion detectors). We concluded that the findings for the P group are highly consistent with the existence of a population of pattern cells, playing a functional role similar to that demonstrated in primates. We also explored different scenarios that could explain the failure of the plaid stimuli to elicit a sizable priming magnitude for the G group. These simulations yielded testable predictions about the properties of motion representations in rodent visual cortex at the single-cell and circuitry level, thus paving the way to future neurophysiology experiments.

Adaptation to one perceived motion direction can generate multiple velocity aftereffects

Sensory adaptation is a useful tool to identify the links between perceptual effects and neural mechanisms. Even though motion adaptation is one of the earliest and most documented aftereffects, few studies have investigated the perception of direction and speed of the aftereffect at the same time, that is the perceived velocity. Using a novel experimental paradigm, we simultaneously recorded the perceived direction and speed of leftward or rightward moving random dots before and after adaptation. For the adapting stimulus, we chose a horizontally-oriented broadband grating moving upward behind a circular aperture. Because of the aperture problem, the interpretation of this stimulus is ambiguous, being consistent with multiple velocities, and yet it is systematically perceived as moving at a single direction and speed. Here we ask whether the visual system adapts to the multiple velocities of the adaptor or to just the single perceived velocity. Our results show a strong repulsion aftereffect, away from the adapting velocity (downward and slower), that increases gradually for faster test stimuli as long as these stimuli include some velocities that match some of the ambiguous ones of the adaptor. In summary, the visual system seems to adapt to the multiple velocities of an ambiguous stimulus even though a single velocity is perceived. Our findings can be well described by a computational model that assumes a joint encoding of direction and speed and that includes an extended adaptation component that can represent all the possible velocities of the ambiguous stimulus.

Primate extrastriate cortical area MST: a gateway between sensation and cognition

Primate visual cortex consists of dozens of distinct brain areas, each providing a highly specialized component to the sophisticated task of encoding the incoming sensory information and creating a representation of our visual environment that underlies our perception and action. One such area is the medial superior temporal cortex (MST), a motion-sensitive, direction-selective part of the primate visual cortex. It receives most of its input from the middle temporal (MT) area, but MST cells have larger receptive fields and respond to more complex motion patterns. The finding that MST cells are tuned for optic flow patterns has led to the suggestion that the area plays an important role in the perception of self-motion. This hypothesis has received further support from studies showing that some MST cells also respond selectively to vestibular cues. Furthermore, the area is part of a network that controls the planning and execution of smooth pursuit eye movements and its activity is modulated by cognitive factors, such as attention and working memory. This review of more than 90 studies focuses on providing clarity of the heterogeneous findings on MST in the macaque cortex and its putative homolog in the human cortex. From this analysis of the unique anatomical and functional position in the hierarchy of areas and processing steps in primate visual cortex, MST emerges as a gateway between perception, cognition, and action planning. Given this pivotal role, this area represents an ideal model system for the transition from sensation to cognition.

Visualization by P-flow: gradient- and feature-based optical flow and vector fields extracted from image analysis

We proposed a method for extracting the optical flow suitable for visualization, pseudo-flow (P-flow), from a natural movie [Exp. Brain Res.237, 3321 (2019)EXBRAP0014-481910.1007/s00221-019-05674-0]. The P-flow algorithm comprises two stages: (1) extraction of a local motion vector field from two successive frames and (2) tracking of vectors between two successive frame pairs. In this study, we show that while P-flow takes a feature (vector) tracking approach, it is also classified as a gradient-based approach that satisfies the brightness constancy constraint. We also incorporate interpolation and a corner detector to address the shortcomings associated with the two approaches.

A Model for the Origin of Motion Direction Selectivity in Visual Cortex

Motion perception is a vital part of our sensory repertoire in that it contributes to navigation, awareness of moving objects, and communication. Motion sense in carnivores and primates originates with primary visual cortical neurons selective for motion direction. More than 60 years after the discovery of these neurons, there is still no consensus on the mechanism underlying direction selectivity. This paper describes a model of the cat's visual system in which direction selectivity results from the well-documented orientation selectivity of inhibitory neurons: inhomogeneities in the orientation preference map for inhibitory neurons leads to spatially asymmetric inhibition, and thus to direction selectivity. Stimulation of the model with a drifting grating shows that direction selectivity results from the relative timing of excitatory and inhibitory inputs to a neuron. Using a stationary contrast-reversing grating reveals that the inhibitory input is spatially displaced in the preferred direction relative to the excitatory input, and that this asymmetry leads to the timing difference. More generally, the model yields physiologically realistic estimates of the direction selectivity index, and it reproduces the critical finding with contrast-reversing gratings that response phase advances with grating spatial phase. It is concluded that a model based on intracortical inhibition can account well for the known properties of direction selectivity in carnivores and primates.SIGNIFICANCE STATEMENT Motion perception is vital for navigation, communication, and the awareness of moving objects. Motion sense depends on cortical neurons that are selective for motion direction, and this paper describes a model for the physiological mechanism underlying cortical direction selectivity. The essence of the model is that intracortical inhibition of a direction-selective cell is spatially inhomogeneous and therefore depends on whether a stimulus generates inhibition before or after reaching the cell's receptive field: the response is weaker in the former than in the latter case. If the model is correct, it will contribute to the understanding of motion processing in carnivores and primates.

Measuring and Modeling Visual Appearance

In studying visual perception, we seek to develop models of processing that accurately predict perceptual judgments. Much of this work is focused on judgments of discrimination, and there is a large literature concerning models of visual discrimination. There are, however, non-threshold visual judgments, such as judgments of the magnitude of differences between visual stimuli, that provide a means to bridge the gap between threshold and appearance. We describe two such models of suprathreshold judgments, maximum likelihood difference scaling and maximum likelihood conjoint measurement, and review recent literature that has exploited them.

A Neural Representation of Naturalistic Motion-Guided Behavior in the Zebrafish Brain

All animals must transform ambiguous sensory data into successful behavior. This requires sensory representations that accurately reflect the statistics of natural stimuli and behavior. Multiple studies show that visual motion processing is tuned for accuracy under naturalistic conditions, but the sensorimotor circuits extracting these cues and implementing motion-guided behavior remain unclear. Here we show that the larval zebrafish retina extracts a diversity of naturalistic motion cues, and the retinorecipient pretectum organizes these cues around the elements of behavior. We find that higher-order motion stimuli, gliders, induce optomotor behavior matching expectations from natural scene analyses. We then image activity of retinal ganglion cell terminals and pretectal neurons. The retina exhibits direction-selective responses across glider stimuli, and anatomically clustered pretectal neurons respond with magnitudes matching behavior. Peripheral computations thus reflect natural input statistics, whereas central brain activity precisely codes information needed for behavior. This general principle could organize sensorimotor transformations across animal species.

The physiology of movement

Movement, from foraging to migration, is known to be under the influence of the environment. The translation of environmental cues to individual movement decision making is determined by an individual's internal state and anticipated to balance costs and benefits. General body condition, metabolic and hormonal physiology mechanistically underpin this internal state. These physiological determinants are tightly, and often genetically linked with each other and hence central to a mechanistic understanding of movement. We here synthesise the available evidence of the physiological drivers and signatures of movement and review (1) how physiological state as measured in its most coarse way by body condition correlates with movement decisions during foraging, migration and dispersal, (2) how hormonal changes underlie changes in these movement strategies and (3) how these can be linked to molecular pathways. We reveale that a high body condition facilitates the efficiency of routine foraging, dispersal and migration. Dispersal decision making is, however, in some cases stimulated by a decreased individual condition. Many of the biotic and abiotic stressors that induce movement initiate a physiological cascade in vertebrates through the production of stress hormones. Movement is therefore associated with hormone levels in vertebrates but also insects, often in interaction with factors related to body or social condition. The underlying molecular and physiological mechanisms are currently studied in few model species, and show -in congruence with our insights on the role of body condition- a central role of energy metabolism during glycolysis, and the coupling with timing processes during migration. Molecular insights into the physiological basis of movement remain, however, highly refractory. We finalise this review with a critical reflection on the importance of these physiological feedbacks for a better mechanistic understanding of movement and its effects on ecological dynamics at all levels of biological organization.