Receptive field positions in area MT during slow eye movements

Diverse effects of gaze direction on heading perception in humans

Gaze change can misalign spatial reference frames encoding visual and vestibular signals in cortex, which may affect the heading discrimination. Here, by systematically manipulating the eye-in-head and head-on-body positions to change the gaze direction of subjects, the performance of heading discrimination was tested with visual, vestibular, and combined stimuli in a reaction-time task in which the reaction time is under the control of subjects. We found the gaze change induced substantial biases in perceived heading, increased the threshold of discrimination and reaction time of subjects in all stimulus conditions. For the visual stimulus, the gaze effects were induced by changing the eye-in-world position, and the perceived heading was biased in the opposite direction of gaze. In contrast, the vestibular gaze effects were induced by changing the eye-in-head position, and the perceived heading was biased in the same direction of gaze. Although the bias was reduced when the visual and vestibular stimuli were combined, integration of the 2 signals substantially deviated from predictions of an extended diffusion model that accumulates evidence optimally over time and across sensory modalities. These findings reveal diverse gaze effects on the heading discrimination and emphasize that the transformation of spatial reference frames may underlie the effects.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

A Stable Visual World in Primate Primary Visual Cortex

Humans and other primates rely on eye movements to explore visual scenes and to track moving objects. As a result, the image that is projected onto the retina-and propagated throughout the visual cortical hierarchy-is almost constantly changing and makes little sense without taking into account the momentary direction of gaze. How is this achieved in the visual system? Here, we show that in primary visual cortex (V1), the earliest stage of cortical vision, neural representations carry an embedded "eye tracker" that signals the direction of gaze associated with each image. Using chronically implanted multi-electrode arrays, we recorded the activity of neurons in area V1 of macaque monkeys during tasks requiring fast (exploratory) and slow (pursuit) eye movements. Neurons were stimulated with flickering, full-field luminance noise at all times. As in previous studies, we observed neurons that were sensitive to gaze direction during fixation, despite comparable stimulation of their receptive fields. We trained a decoder to translate neural activity into metric estimates of gaze direction. This decoded signal tracked the eye accurately not only during fixation but also during fast and slow eye movements. After a fast eye movement, the eye-position signal arrived in V1 at approximately the same time at which the new visual information arrived from the retina. Using simulations, we show that this V1 eye-position signal could be used to take into account the sensory consequences of eye movements and map the fleeting positions of objects on the retina onto their stable position in the world.

Perisaccadic visual perception

Primates use frequent, rapid eye movements to sample their visual environment. This is a fruitful strategy to make the best use of the highly sensitive foveal part of the retina, but it requires neural mechanisms to bind the rapidly changing visual input into a single, stable percept. Studies investigating these neural mechanisms have typically assumed that perisaccadic perception in nonhuman primates matches that of humans. We tested this assumption by performing identical experiments in human and nonhuman primates. Our data confirm that perisaccadic visual perception of macaques and humans is qualitatively similar. Specifically, we found a reduction in detectability and mislocalization of targets presented at the time of saccades. We also found substantial differences between human and nonhuman primates. Notably, in nonhuman primates, localization that requires knowledge of eye position was less precise, nonhuman primates detected fewer perisaccadic stimuli, and perisaccadic compression was not towards the saccade target. The qualitative similarities between species support the view that the nonhuman primate is ideally suited to study aspects of brain function—such as those relying on foveal vision—that are uniquely developed in primates. The quantitative differences, however, demonstrate the need for a reassessment of the models purportedly linking neural response changes at the time of saccades with the behavioral phenomena of perisaccadic reduction of detectability and mislocalization.

Recurrent Network Dynamics; a Link between Form and Motion

To discriminate visual features such as corners and contours, the brain must be sensitive to spatial correlations between multiple points in an image. Consistent with this, macaque V2 neurons respond selectively to patterns with well-defined multipoint correlations. Here, we show that a standard feedforward model (a cascade of linear–non-linear filters) does not capture this multipoint selectivity. As an alternative, we developed an artificial neural network model with two hierarchical stages of processing and locally recurrent connectivity. This model faithfully reproduced neurons’ selectivity for multipoint correlations. By probing the model, we gained novel insights into early form processing. First, the diverse selectivity for multipoint correlations and complex response dynamics of the hidden units in the model were surprisingly similar to those observed in V1 and V2. This suggests that both transient and sustained response dynamics may be a vital part of form computations. Second, the model self-organized units with speed and direction selectivity that was correlated with selectivity for multipoint correlations. In other words, the model units that detected multipoint spatial correlations also detected space-time correlations. This leads to the novel hypothesis that higher-order spatial correlations could be computed by the rapid, sequential assessment and comparison of multiple low-order correlations within the receptive field. This computation links spatial and temporal processing and leads to the testable prediction that the analysis of complex form and motion are closely intertwined in early visual cortex.

Evidence and Counterevidence in Motion Perception

Sensory neurons gather evidence in favor of the specific stimuli to which they are tuned, but they could improve their sensitivity by also taking counterevidence into account. The Bours-Lankheet model for motion detection uses counterevidence that relies on a specific combination of the ON and OFF channels in the early visual system. Specifically, the model detects pairs of flashes that occur separated in space and time. If the flashes have the same contrast polarity, they are interpreted as evidence in favor of the corresponding motion. But if they have opposite contrasts, they are interpreted as evidence against it. This mechanism provides an explanation for reverse-phi (the perceived reversal of an apparent motion stimulus due to periodic contrast-inversions) that is a conceptual departure from the standard explanations of the effect. Here, we investigate this counterevidence mechanism by measuring directional tuning curves of neurons in the primary visual and middle temporal cortex areas of awake, behaving macaques using constant-contrast and inverting-contrast moving dot stimuli. Our electrophysiological data support the Bours-Lankheet model and suggest that the counterevidence computation occurs at an early stage of neural processing not captured by the standard models.

Circuits for presaccadic visual remapping

Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF's local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about the reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.

Hunger-Dependent Enhancement of Food Cue Responses in Mouse Postrhinal Cortex and Lateral Amygdala

The needs of the body can direct behavioral and neural processing toward motivationally relevant sensory cues. For example, human imaging studies have consistently found specific cortical areas with biased responses to food-associated visual cues in hungry subjects, but not in sated subjects. To obtain a cellular-level understanding of these hunger-dependent cortical response biases, we performed chronic two-photon calcium imaging in postrhinal association cortex (POR) and primary visual cortex (V1) of behaving mice. As in humans, neurons in mouse POR, but not V1, exhibited biases toward food-associated cues that were abolished by satiety. This emergent bias was mirrored by the innervation pattern of amygdalo-cortical feedback axons. Strikingly, these axons exhibited even stronger food cue biases and sensitivity to hunger state and trial history. These findings highlight a direct pathway by which the lateral amygdala may contribute to state-dependent cortical processing of motivationally relevant sensory cues.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron's spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Testing the assumptions underlying fMRI adaptation using intracortical recordings in area MT

We investigated how neural activity in the middle temporal area of the macaque monkey changes after 3 sec of exposure to a visual stimulus and used this to gain insight into the assumptions underlying the fMRI adaptation method (fMRIa). We studied both changes in tuning curves following weak and strong motion stimuli (adaptation) and the differences between a first and second exposure to the same stimulus (repetition suppression). Typically, tuning curves had smaller amplitudes and narrower tuning widths after strong adaptation; this was true for single neurons, multi-unit activity (MUA), the evoked local field potential (LFP), as well as gamma band activity. Repetition typically led to reduced responses. This reduction was correlated with direction selectivity and not explained by neural fatigue. Our data, however, warn against a simplistic view of the consequences of adaptation. First, a considerable fraction of neurons and sites showed response enhancements after adaptation, especially when probed with a stimulus that moved opposite to the direction of the adapting stimulus. Second, adaptation was stimulus selective only on a time scale of ∼100 msec. Third, aggregate measures of neural activity (MUA, LFPs) had substantially different adaptation effects. Fourth, there were qualitative differences between our findings in MT and earlier findings in IT cortex. We conclude that selective adaptation effects in fMRIa are relatively easy to miss even when they exist (for instance by presenting stimuli for too long, or because neurons that enhance after adaptation cancel out the effect of neurons that suppress). Moreover, we argue that adaptation should be understood in the context of the computations that a neural circuit perform. Using fMRIa as a tool to uncover neural selectivity requires a better understanding of this circuitry and its consequences for adaptation.

Dynamics of visual receptive fields in the macaque frontal eye field

Neuronal receptive fields (RFs) provide the foundation for understanding systems-level sensory processing. In early visual areas, investigators have mapped RFs in detail using stochastic stimuli and sophisticated analytical approaches. Much less is known about RFs in prefrontal cortex. Visual stimuli used for mapping RFs in prefrontal cortex tend to cover a small range of spatial and temporal parameters, making it difficult to understand their role in visual processing. To address these shortcomings, we implemented a generalized linear model to measure the RFs of neurons in the macaque frontal eye field (FEF) in response to sparse, full-field stimuli. Our high-resolution, probabilistic approach tracked the evolution of RFs during passive fixation, and we validated our results against conventional measures. We found that FEF neurons exhibited a surprising level of sensitivity to stimuli presented as briefly as 10 ms or to multiple dots presented simultaneously, suggesting that FEF visual responses are more precise than previously appreciated. FEF RF spatial structures were largely maintained over time and between stimulus conditions. Our results demonstrate that the application of probabilistic RF mapping to FEF and similar association areas is an important tool for clarifying the neuronal mechanisms of cognition.

Dynamics of the functional link between area MT LFPs and motion detection

The evolution of a visually guided perceptual decision results from multiple neural processes, and recent work suggests that signals with different neural origins are reflected in separate frequency bands of the cortical local field potential (LFP). Spike activity and LFPs in the middle temporal area (MT) have a functional link with the perception of motion stimuli (referred to as neural-behavioral correlation). To cast light on the different neural origins that underlie this functional link, we compared the temporal dynamics of the neural-behavioral correlations of MT spikes and LFPs. Wide-band activity was simultaneously recorded from two locations of MT from monkeys performing a threshold, two-stimuli, motion pulse detection task. Shortly after the motion pulse occurred, we found that high-gamma (100-200 Hz) LFPs had a fast, positive correlation with detection performance that was similar to that of the spike response. Beta (10-30 Hz) LFPs were negatively correlated with detection performance, but their dynamics were much slower, peaked late, and did not depend on stimulus configuration or reaction time. A late change in the correlation of all LFPs across the two recording electrodes suggests that a common input arrived at both MT locations prior to the behavioral response. Our results support a framework in which early high-gamma LFPs likely reflected fast, bottom-up, sensory processing that was causally linked to perception of the motion pulse. In comparison, late-arriving beta and high-gamma LFPs likely reflected slower, top-down, sources of neural-behavioral correlation that originated after the perception of the motion pulse.

Motion detection based on recurrent network dynamics

The detection of visual motion requires temporal delays to compare current with earlier visual input. Models of motion detection assume that these delays reside in separate classes of slow and fast thalamic cells, or slow and fast synaptic transmission. We used a data-driven modeling approach to generate a model that instead uses recurrent network dynamics with a single, fixed temporal integration window to implement the velocity computation. This model successfully reproduced the temporal response dynamics of a population of motion sensitive neurons in macaque middle temporal area (MT) and its constituent parts matched many of the properties found in the motion processing pathway (e.g., Gabor-like receptive fields (RFs), simple and complex cells, spatially asymmetric excitation and inhibition). Reverse correlation analysis revealed that a simplified network based on first and second order space-time correlations of the recurrent model behaved much like a feedforward motion energy (ME) model. The feedforward model, however, failed to capture the full speed tuning and direction selectivity properties based on higher than second order space-time correlations typically found in MT. These findings support the idea that recurrent network connectivity can create temporal delays to compute velocity. Moreover, the model explains why the motion detection system often behaves like a feedforward ME network, even though the anatomical evidence strongly suggests that this network should be dominated by recurrent feedback.

Transcranial direct current stimulation over posterior parietal cortex modulates visuospatial localization

Visual localization is based on the complex interplay of bottom-up and top-down processing. Based on previous work, the posterior parietal cortex (PPC) is assumed to play an essential role in this interplay. In this study, we investigated the causal role of the PPC in visual localization. Specifically, our goal was to determine whether modulation of the PPC via transcranial direct current stimulation (tDCS) could induce visual mislocalization similar to that induced by an exogenous attentional cue (Wright, Morris, & Krekelberg, 2011). We placed one stimulation electrode over the right PPC and the other over the left PPC (dual tDCS) and varied the polarity of the stimulation. We found that this manipulation altered visual localization; this supports the causal involvement of the PPC in visual localization. Notably, mislocalization was more rightward when the cathode was placed over the right PPC than when the anode was placed over the right PPC. This mislocalization was found within a few minutes of stimulation onset, it dissipated during stimulation, but then resurfaced after stimulation offset and lasted for another 10-15 min. On the assumption that excitability is reduced beneath the cathode and increased beneath the anode, these findings support the view that each hemisphere biases processing to the contralateral hemifield and that the balance of activation between the hemispheres contributes to position perception (Kinsbourne, 1977; Szczepanski, Konen, & Kastner, 2010).

Transcranial alternating current stimulation attenuates visual motion adaptation

Transcranial alternating current stimulation (tACS) is used in clinical applications and basic neuroscience research. Although its behavioral effects are evident from prior reports, current understanding of the mechanisms that underlie these effects is limited. We used motion perception, a percept with relatively well known properties and underlying neural mechanisms to investigate tACS mechanisms. Healthy human volunteers showed a surprising improvement in motion sensitivity when visual stimuli were paired with 10 Hz tACS. In addition, tACS reduced the motion-after effect, and this reduction was correlated with the improvement in motion sensitivity. Electrical stimulation had no consistent effect when applied before presenting a visual stimulus or during recovery from motion adaptation. Together, these findings suggest that perceptual effects of tACS result from an attenuation of adaptation. Important consequences for the practical use of tACS follow from our work. First, because this mechanism interferes only with adaptation, this suggests that tACS can be targeted at subsets of neurons (by adapting them), even when the applied currents spread widely throughout the brain. Second, by interfering with adaptation, this mechanism provides a means by which electrical stimulation can generate behavioral effects that outlast the stimulation.

Neurons in cortical area MST remap the memory trace of visual motion across saccadic eye movements

Perception of a stable visual world despite eye motion requires integration of visual information across saccadic eye movements. To investigate how the visual system deals with localization of moving visual stimuli across saccades, we observed spatiotemporal changes of receptive fields (RFs) of motion-sensitive neurons across periods of saccades in the middle temporal (MT) and medial superior temporal (MST) areas. We found that the location of the RFs moved with shifts of eye position due to saccades, indicating that motion-sensitive neurons in both areas have retinotopic RFs across saccades. Different characteristic responses emerged when the moving visual stimulus was turned off before the saccades. For MT neurons, virtually no response was observed after the saccade, suggesting that the responses of these neurons simply reflect the reafferent visual information. In contrast, most MST neurons increased their firing rates when a saccade brought the location of the visual stimulus into their RFs, where the visual stimulus itself no longer existed. These findings suggest that the responses of such MST neurons after saccades were evoked by a memory of the stimulus that had preexisted in the postsaccadic RFs ("memory remapping"). A delayed-saccade paradigm further revealed that memory remapping in MST was linked to the saccade itself, rather than to a shift in attention. Thus, the visual motion information across saccades was integrated in spatiotopic coordinates and represented in the activity of MST neurons. This is likely to contribute to the perception of a stable visual world in the presence of eye movements.

Similar adaptation effects in primary visual cortex and area MT of the macaque monkey under matched stimulus conditions

Recent stimulus history, or adaptation, can alter neuronal response properties. Adaptation effects have been characterized in a number of visually responsive structures, from the retina to higher visual cortex. However, it remains unclear whether adaptation effects across stages of the visual system take a similar form in response to a particular sensory event. This is because studies typically probe a single structure or cortical area, using a stimulus ensemble chosen to provide potent drive to the cells of interest. Here we adopt an alternative approach and compare adaptation effects in primary visual cortex (V1) and area MT using identical stimulus ensembles. Previous work has suggested these areas adjust to recent stimulus drive in distinct ways. We show that this is not the case: adaptation effects in V1 and MT can involve weak or strong loss of responsivity and shifts in neuronal preference toward or away from the adapter, depending on stimulus size and adaptation duration. For a particular stimulus size and adaptation duration, however, effects are similar in nature and magnitude in V1 and MT. We also show that adaptation effects in MT of awake animals depend strongly on stimulus size. Our results suggest that the strategies for adjusting to recent stimulus history depend more strongly on adaptation duration and stimulus size than on the cortical area. Moreover, they indicate that different levels of the visual system adapt similarly to recent sensory experience.

Neural mechanisms of speed perception: transparent motion

Visual motion on the macaque retina is processed by direction- and speed-selective neurons in extrastriate middle temporal cortex (MT). There is strong evidence for a link between the activity of these neurons and direction perception. However, there is conflicting evidence for a link between speed selectivity of MT neurons and speed perception. Here we study this relationship by using a strong perceptual illusion in speed perception: when two transparently superimposed dot patterns move in opposite directions, their apparent speed is much larger than the perceived speed of a single pattern moving at that physical speed. Moreover, the sensitivity for speed discrimination is reduced for such bidirectional patterns. We first confirmed these behavioral findings in human subjects and extended them to a monkey subject. Second, we determined speed tuning curves of MT neurons to bidirectional motion and compared these to speed tuning curves for unidirectional motion. Consistent with previous reports, the response to bidirectional motion was often reduced compared with unidirectional motion at the preferred speed. In addition, we found that tuning curves for bidirectional motion were shifted to lower preferred speeds. As a consequence, bidirectional motion of some speeds typically evoked larger responses than unidirectional motion. Third, we showed that these changes in neural responses could explain changes in speed perception with a simple labeled line decoder. These data provide new insight into the encoding of transparent motion patterns and provide support for the hypothesis that MT activity can be decoded for speed perception with a labeled line model.

Motion-dependent representation of space in area MT+

How is visual space represented in cortical area MT+? At a relatively coarse scale, the organization of MT+ is debated; retinotopic, spatiotopic, or mixed representations have all been proposed. However, none of these representations entirely explain the perceptual localization of objects at a fine spatial scale--a scale relevant for tasks like navigating or manipulating objects. For example, perceived positions of objects are strongly modulated by visual motion; stationary flashes appear shifted in the direction of nearby motion. Does spatial coding in MT+ reflect these shifts in perceived position? We performed an fMRI experiment employing this "flash-drag" effect and found that flashes presented near motion produced patterns of activity similar to physically shifted flashes in the absence of motion. This reveals a motion-dependent change in the neural representation of object position in human MT+, a process that could help compensate for perceptual and motor delays in localizing objects in dynamic scenes.

The complex structure of receptive fields in the middle temporal area

Neurons in the middle temporal area (MT) are often viewed as motion detectors that prefer a single direction of motion in a single region of space. This assumption plays an important role in our understanding of visual processing, and models of motion processing in particular. We used extracellular recordings in area MT of awake, behaving monkeys (M. mulatta) to test this assumption with a novel reverse correlation approach. Nearly half of the MT neurons in our sample deviated significantly from the classical view. First, in many cells, direction preference changed with the location of the stimulus within the receptive field. Second, the spatial response profile often had multiple peaks with apparent gaps in between. This shows that visual motion analysis in MT has access to motion detectors that are more complex than commonly thought. This complexity could be a mere byproduct of imperfect development, but can also be understood as the natural consequence of the non-linear, recurrent interactions among laterally connected MT neurons. An important direction for future research is to investigate whether these in homogeneities are advantageous, how they can be incorporated into models of motion detection, and whether they can provide quantitative insight into the underlying effective connectivity.

Dynamics of eye-position signals in the dorsal visual system

BACKGROUND

Many visual areas of the primate brain contain signals related to the current position of the eyes in the orbit. These cortical eye-position signals are thought to underlie the transformation of retinal input-which changes with every eye movement-into a stable representation of visual space. For this coding scheme to work, such signals would need to be updated fast enough to keep up with the eye during normal exploratory behavior. We examined the dynamics of cortical eye-position signals in four dorsal visual areas of the macaque brain: the lateral and ventral intraparietal areas (LIP; VIP), the middle temporal area (MT), and the medial-superior temporal area (MST). We recorded extracellular activity of single neurons while the animal performed sequences of fixations and saccades in darkness.

RESULTS

The data show that eye-position signals are updated predictively, such that the representation shifts in the direction of a saccade prior to (<100 ms) the actual eye movement. Despite this early start, eye-position signals remain inaccurate until shortly after (10-150 ms) the eye movement. By using simulated behavioral experiments, we show that this brief misrepresentation of eye position provides a neural explanation for the psychophysical phenomenon of perisaccadic mislocalization, in which observers misperceive the positions of visual targets flashed around the time of saccadic eye movements.

CONCLUSIONS

Together, these results suggest that eye-position signals in the dorsal visual system are updated rapidly across eye movements and play a direct role in perceptual localization, even when they are erroneous.

A lack of anticipatory remapping of retinotopic receptive fields in the middle temporal area

The middle temporal (MT) area has traditionally been thought to be a retinotopic area. However, recent functional magnetic resonance imaging and psychophysical evidence have suggested that human MT may have some spatiotopic processing. To gain an understanding of the neural mechanisms underlying this process, we recorded neurons from area MT in awake behaving animals performing a simple saccade task in which a spatially stable moving dot stimulus was presented for 500 ms in one of two locations: the presaccadic receptive field or the postsaccadic receptive field. MT neurons responded as if their receptive fields were purely retinotopic. When the stimulus was placed in the presaccadic receptive field, the response was elevated until the saccade took the stimulus out of the receptive field. When the stimulus was placed in the postsaccadic receptive field, the neuron only began its response after the end of the saccade. No evidence of presaccadic or anticipatory remapping was found. We conclude that gain fields are most likely to be responsible for the spatiotopic signal seen in area MT.