Linear responses to stochastic motion signals in area MST

Visual Motion Coherence Responses in Human Visual Cortex

Random dot kinematograms (RDKs) have recently been used to train subjects with cortical scotomas to perform direction of motion discrimination, partially restoring visual motion perception. To study the recovery of visual perception, it is important to understand how visual areas in normal subjects and subjects with cortical scotomas respond to RDK stimuli. Studies in normal subjects have shown that blood oxygen level-dependent (BOLD) responses in human area hV5/MT+ increase monotonically with coherence, in general agreement with electrophysiology studies in primates. However, RDK responses in prior studies were obtained while the subject was performing fixation, not a motion discrimination condition. Furthermore, BOLD responses were gauged against a baseline condition of uniform illumination or static dots, potentially decreasing the specificity of responses for the spatial integration of local motion signals (motion coherence). Here, we revisit this question starting from a baseline RDK condition of no coherence, thereby isolating the component of BOLD response due specifically to the spatial integration of local motion signals. In agreement with prior studies, we found that responses in the area hV5/MT+ of healthy subjects were monotonically increasing when subjects fixated without performing a motion discrimination task. In contrast, when subjects were performing an RDK direction of motion discrimination task, responses in the area hV5/MT+ remained flat, changing minimally, if at all, as a function of motion coherence. A similar pattern of responses was seen in the area hV5/MT+ of subjects with dense cortical scotomas performing direction of motion discrimination for RDKs presented inside the scotoma. Passive RDK presentation within the scotoma elicited no significant hV5/MT+ responses. These observations shed further light on how visual cortex responses behave as a function of motion coherence, helping to prepare the ground for future studies using these methods to study visual system recovery after injury.

Contribution of the slow motion mechanism to global motion revealed by an MAE technique

Two different motion mechanisms have been identified with motion aftereffect (MAE). (1) A slow motion mechanism, accessed by a static MAE, is sensitive to high-spatial and low-temporal frequency; (2) a fast motion mechanism, accessed by a flicker MAE, is sensitive to low-spatial and high-temporal frequency. We examined their respective responses to global motion after adapting to a global motion pattern constructed of multiple compound Gabor patches arranged circularly. Each compound Gabor patch contained two gratings at different spatial frequencies (0.53 and 2.13 cpd) drifting in opposite directions. The participants reported the direction and duration of the MAE for a variety of global motion patterns. We discovered that static MAE durations depended on the global motion patterns, e.g., longer MAE duration to patches arranged to see rotation than to random motion (Exp 1), and increase with global motion strength (patch number in Exp 2). In contrast, flicker MAEs durations are similar across different patterns and adaptation strength. Further, the global integration occurred at the adaptation stage, rather than at the test stage (Exp 3). These results suggest that slow motion mechanism, assessed by static MAE, integrate motion signals over space while fast motion mechanisms do not, at least under the conditions used.

Motion Coherence and Luminance Contrast Interact in Driving Visual Gamma-Band Activity

Synchronized neuronal population activity in the gamma-frequency range (>30 Hz) correlates with the bottom-up drive of various visual features. It has been hypothesized that gamma-band synchronization enhances the gain of neuronal representations, yet evidence remains sparse. We tested a critical prediction of the gain hypothesis, which is that features that drive synchronized gamma-band activity interact super-linearly. To test this prediction, we employed whole-head magnetencephalography in human subjects and investigated if the strength of visual motion (motion coherence) and luminance contrast interact in driving gamma-band activity in visual cortex. We found that gamma-band activity (64-128 Hz) monotonically increased with coherence and contrast, while lower frequency activity (8-32 Hz) decreased with both features. Furthermore, as predicted for a gain mechanism, we found a multiplicative interaction between motion coherence and contrast in their joint drive of gamma-band activity. The lower frequency activity did not show such an interaction. Our findings provide evidence that gamma-band activity acts as a cortical gain mechanism that nonlinearly combines the bottom-up drive of different visual features.

Retinal Stabilization Reveals Limited Influence of Extraretinal Signals on Heading Tuning in the Medial Superior Temporal Area

Heading perception in primates depends heavily on visual optic-flow cues. Yet during self-motion, heading percepts remain stable, even though smooth-pursuit eye movements often distort optic flow. According to theoretical work, self-motion can be represented accurately by compensating for these distortions in two ways: via retinal mechanisms or via extraretinal efference-copy signals, which predict the sensory consequences of movement. Psychophysical evidence strongly supports the efference-copy hypothesis, but physiological evidence remains inconclusive. Neurons that signal the true heading direction during pursuit are found in visual areas of monkey cortex, including the dorsal medial superior temporal area (MSTd). Here we measured heading tuning in MSTd using a novel stimulus paradigm, in which we stabilize the optic-flow stimulus on the retina during pursuit. This approach isolates the effects on neuronal heading preferences of extraretinal signals, which remain active while the retinal stimulus is prevented from changing. Our results from 3 female monkeys demonstrate a significant but small influence of extraretinal signals on the preferred heading directions of MSTd neurons. Under our stimulus conditions, which are rich in retinal cues, we find that retinal mechanisms dominate physiological corrections for pursuit eye movements, suggesting that extraretinal cues, such as predictive efference-copy mechanisms, have a limited role under naturalistic conditions.SIGNIFICANCE STATEMENT Sensory systems discount stimulation caused by an animal's own behavior. For example, eye movements cause irrelevant retinal signals that could interfere with motion perception. The visual system compensates for such self-generated motion, but how this happens is unclear. Two theoretical possibilities are a purely visual calculation or one using an internal signal of eye movements to compensate for their effects. The latter can be isolated by experimentally stabilizing the image on a moving retina, but this approach has never been adopted to study motion physiology. Using this method, we find that extraretinal signals have little influence on activity in visual cortex, whereas visually based corrections for ongoing eye movements have stronger effects and are likely most important under real-world conditions.

Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback

Animals perform near-optimal probabilistic inference in a wide range of psychophysical tasks. Probabilistic inference requires trial-to-trial representation of the uncertainties associated with task variables and subsequent use of this representation. Previous work has implemented such computations using neural networks with hand-crafted and task-dependent operations. We show that generic neural networks trained with a simple error-based learning rule perform near-optimal probabilistic inference in nine common psychophysical tasks. In a probabilistic categorization task, error-based learning in a generic network simultaneously explains a monkey’s learning curve and the evolution of qualitative aspects of its choice behavior. In all tasks, the number of neurons required for a given level of performance grows sublinearly with the input population size, a substantial improvement on previous implementations of probabilistic inference. The trained networks develop a novel sparsity-based probabilistic population code. Our results suggest that probabilistic inference emerges naturally in generic neural networks trained with error-based learning rules.

Behavioural tasks often require probability distributions to be inferred about task specific variables. Here, the authors demonstrate that generic neural networks can be trained using a simple error-based learning rule to perform such probabilistic computations efficiently without any need for task specific operations.

Motion area V5/MT+ response to global motion in the absence of V1 resembles early visual cortex

Motion area V5/MT+ shows a variety of characteristic visual responses, often linked to perception, which are heavily influenced by its rich connectivity with the primary visual cortex (V1). This human motion area also receives a number of inputs from other visual regions, including direct subcortical connections and callosal connections with the contralateral hemisphere. Little is currently known about such alternative inputs to V5/MT+ and how they may drive and influence its activity. Using functional magnetic resonance imaging, the response of human V5/MT+ to increasing the proportion of coherent motion was measured in seven patients with unilateral V1 damage acquired during adulthood, and a group of healthy age-matched controls. When V1 was damaged, the typical V5/MT+ response to increasing coherence was lost. Rather, V5/MT+ in patients showed a negative trend with coherence that was similar to coherence-related activity in V1 of healthy control subjects. This shift to a response-pattern more typical of early visual cortex suggests that in the absence of V1, V5/MT+ activity may be shaped by similar direct subcortical input. This is likely to reflect intact residual pathways rather than a change in connectivity, and has important implications for blindsight function. It also confirms predictions that V1 is critically involved in normal V5/MT+ global motion processing, consistent with a convergent model of V1 input to V5/MT+. Historically, most attempts to model cortical visual responses do not consider the contribution of direct subcortical inputs that may bypass striate cortex, such as input to V5/MT+. We have shown that the signal change driven by these non-striate pathways can be measured, and suggest that models of the intact visual system may benefit from considering their contribution.

Optimal multisensory decision-making in a reaction-time task

Humans and animals can integrate sensory evidence from various sources to make decisions in a statistically near-optimal manner, provided that the stimulus presentation time is fixed across trials. Little is known about whether optimality is preserved when subjects can choose when to make a decision (reaction-time task), nor when sensory inputs have time-varying reliability. Using a reaction-time version of a visual/vestibular heading discrimination task, we show that behavior is clearly sub-optimal when quantified with traditional optimality metrics that ignore reaction times. We created a computational model that accumulates evidence optimally across both cues and time, and trades off accuracy with decision speed. This model quantitatively explains subjects's choices and reaction times, supporting the hypothesis that subjects do, in fact, accumulate evidence optimally over time and across sensory modalities, even when the reaction time is under the subject's control.

Knowing how much you don't know: a neural organization of uncertainty estimates

How we estimate uncertainty is important in decision neuroscience and has wide-ranging implications in basic and clinical neuroscience, from computational models of optimality to ideas on psychopathological disorders including anxiety, depression and schizophrenia. Empirical research in neuroscience, which has been based on divergent theoretical assumptions, has focused on the fundamental question of how uncertainty is encoded in the brain and how it influences behaviour. Here, we integrate several theoretical concepts about uncertainty into a decision-making framework. We conclude that the currently available evidence indicates that distinct neural encoding (including summary statistic-type representations) of uncertainty occurs in distinct neural systems.

Early Alzheimer's disease blocks responses to accelerating self-movement

We assessed the cortical processing of self-movement stimuli in aging and Alzheimer's disease (AD). Our goal was to identify distinguishing effects on neural mechanisms related to driving and navigation. Young (YNC) and older (ONC) normal controls, and early AD patients (EAD) viewed real-world videos and dot motion stimuli simulating self-movement scenes. We recorded visual motion event related potentials (VMERPs) to stimulus motion coherence and speed. Aging delays motion evoked N200s, whereas AD diminishes response amplitudes. Early Alzheimer's disease patients respond to increments in motion coherence, but they are uniquely unresponsive to increments in motion speed that simulate accelerating self-movement. AD-related impairments of self-movement processing may have grave consequences for driving safety and navigational independence.

Neural correlates of reliability-based cue weighting during multisensory integration

Integration of multiple sensory cues is essential for precise and accurate perception and behavioral performance, yet the reliability of sensory signals can vary across modalities and viewing conditions. Human observers typically employ the optimal strategy of weighting each cue in proportion to its reliability, but the neural basis of this computation remains poorly understood. We trained monkeys to perform a heading discrimination task from visual and vestibular cues, varying cue reliability randomly. The monkeys appropriately placed greater weight on the more reliable cue, and population decoding of neural responses in the dorsal medial superior temporal area closely predicted behavioral cue weighting, including modest deviations from optimality. We found that the mathematical combination of visual and vestibular inputs by single neurons is generally consistent with recent theories of optimal probabilistic computation in neural circuits. These results provide direct evidence for a neural mechanism mediating a simple and widespread form of statistical inference.

Driving strategy alters neuronal responses to self-movement: cortical mechanisms of distracted driving

We presented naturalistic combinations of virtual self-movement stimuli while recording neuronal activity in monkey cerebral cortex. Monkeys used a joystick to drive to a straight ahead heading direction guided by either object motion or optic flow. The selected cue dominates neuronal responses, often mimicking responses evoked when that stimulus is presented alone. In some neurons, driving strategy creates selective response additivities. In others, it creates vulnerabilities to the disruptive effects of independently moving objects. Such cue interactions may be related to the disruptive effects of independently moving objects in Alzheimer's disease patients with navigational deficits.

A normalization model of multisensory integration

Responses of neurons that integrate multiple sensory inputs are traditionally characterized in terms of a set of empirical principles. However, a simple computational framework that accounts for these empirical features of multisensory integration has not been established. We propose that divisive normalization, acting at the stage of multisensory integration, can account for many of the empirical principles of multisensory integration shown by single neurons, such as the principle of inverse effectiveness and the spatial principle. This model, which uses a simple functional operation (normalization) for which there is considerable experimental support, also accounts for the recent observation that the mathematical rule by which multisensory neurons combine their inputs changes with cue reliability. The normalization model, which makes a strong testable prediction regarding cross-modal suppression, may therefore provide a simple unifying computational account of the important features of multisensory integration by neurons.

Does the noise matter? Effects of different kinematogram types on smooth pursuit eye movements and perception

We investigated how the human visual system and the pursuit system react to visual motion noise. We presented three different types of random-dot kinematograms at five different coherence levels. For transparent motion, the signal and noise labels on each dot were preserved throughout each trial, and noise dots moved with the same speed as the signal dots but in fixed random directions. For white noise motion, every 20 ms the signal and noise labels were randomly assigned to each dot and noise dots appeared at random positions. For Brownian motion, signal and noise labels were also randomly assigned, but the noise dots moved at the signal speed in a direction that varied randomly from moment to moment. Neither pursuit latency nor early eye acceleration differed among the different types of kinematograms. Late acceleration, pursuit gain, and perceived speed all depended on kinematogram type, with good agreement between pursuit gain and perceived speed. For transparent motion, pursuit gain and perceived speed were independent of coherence level. For white and Brownian motions, pursuit gain and perceived speed increased with coherence but were higher for white than for Brownian motion. This suggests that under our conditions, the pursuit system integrates across all directions of motion but not across all speeds.

Cortical neurons combine visual cues about self-movement

Visual cues about self-movement are derived from the patterns of optic flow and the relative motion of discrete objects. We recorded dorsal medial superior temporal (MSTd) cortical neurons in monkeys that held centered visual fixation while viewing optic flow and object motion stimuli simulating the self-movement cues seen during translation on a circular path. Twenty stimulus configurations presented naturalistic combinations of optic flow with superimposed objects that simulated either earth-fixed landmark objects or independently moving animate objects. Landmarks and animate objects yield the same response interactions with optic flow; mainly additive effects, with a substantial number of sub- and super-additive responses. Sub- and super-additive interactions reflect each neuron's local and global motion sensitivities: Local motion sensitivity is based on the spatial arrangement of directions created by object motion and the surrounding optic flow. Global motion sensitivity is based on the temporal sequence of self-movement headings that define a simulated path through the environment. We conclude that MST neurons' spatio-temporal response properties combine object motion and optic flow cues to represent self-movement in diverse, naturalistic circumstances.

The initial torsional Ocular Following Response (tOFR) in humans: a response to the total motion energy in the stimulus?

We recorded the initial torsional Ocular Following Responses (tOFRs) elicited at short latency by visual images that occupied the frontal plane and rotated about the lines of sight. Using 1-D radial gratings, the local spatio-temporal characteristics of these tOFRs closely resembled those we previously reported for the hOFRs to horizontal motion with 1-D vertical gratings. When the 1-D radial grating was subdivided into a number of concentric annuli, each with the same radial thickness, tOFRs were less than predicted from the sum of the responses to the individual annuli: spatial normalization. However, the normalization was much weaker than that which we previously reported for the hOFRs. Further, when the number, thickness and contrast of these concentric annuli were varied systematically, the latency and magnitude of the tOFRs were well described by single monotonic functions when plotted against the product of the total area of the annuli and the square of their Michelson contrast ("A*C(2)"), consistent with the hypothesis that the onset and magnitude of the initial tOFR are determined by the total motion energy in the stimulus. When our previously published hOFR data were plotted against A*C(2), a single monotonic function sufficed to describe the latency but not the magnitude.

Multisensory integration: psychophysics, neurophysiology, and computation

Fundamental observations and principles derived from traditional physiological studies of multisensory integration have been difficult to reconcile with computational and psychophysical studies that share the foundation of probabilistic (Bayesian) inference. We review recent work on multisensory integration, focusing on experiments that bridge single-cell electrophysiology, psychophysics, and computational principles. These studies show that multisensory (visual-vestibular) neurons can account for near-optimal cue integration during the perception of self-motion. Unlike the nonlinear (superadditive) interactions emphasized in some previous studies, visual-vestibular neurons accomplish near-optimal cue integration through subadditive linear summation of their inputs, consistent with recent computational theories. Important issues remain to be resolved, including the observation that variations in cue reliability appear to change the weights that neurons apply to their different sensory inputs.