Spatiotemporal structure of visual receptive fields in macaque superior colliculus

Decision-related activity and movement selection in primate visual cortex

Fluctuations in the activity of sensory neurons often predict perceptual decisions. This connection can be quantified with a metric called choice probability (CP), and there is a longstanding debate about whether CP reflects a causal influence on decisions or an echo of decision-making activity elsewhere in the brain. Here, we show that CP can reflect a third variable, namely, the movement used to indicate the decision. In a standard visual motion discrimination task, neurons in the middle temporal (MT) area of primate cortex responded more strongly during trials that involved a saccade toward their receptive fields. This variability accounted for much of the CP observed across the neuronal population, and it arose through training. Moreover, pharmacological inactivation of MT biased behavioral responses away from the corresponding visual field locations. These results demonstrate that training on a task with fixed sensorimotor contingencies introduces movement-related activity in sensory brain regions and that this plasticity can shape the neural circuitry of perceptual decision-making.

Express detection of visual objects by primate superior colliculus neurons

Primate superior colliculus (SC) neurons exhibit visual feature tuning properties and are implicated in a subcortical network hypothesized to mediate fast threat and/or conspecific detection. However, the mechanisms through which SC neurons contribute to peripheral object detection, for supporting rapid orienting responses, remain unclear. Here we explored whether, and how quickly, SC neurons detect real-life object stimuli. We presented experimentally-controlled gray-scale images of seven different object categories, and their corresponding luminance- and spectral-matched image controls, within the extrafoveal response fields of SC neurons. We found that all of our functionally-identified SC neuron types preferentially detected real-life objects even in their very first stimulus-evoked visual bursts. Intriguingly, even visually-responsive motor-related neurons exhibited such robust early object detection. We further identified spatial frequency information in visual images as an important, but not exhaustive, source for the earliest (within 100 ms) but not for the late (after 100 ms) component of object detection by SC neurons. Our results demonstrate rapid and robust detection of extrafoveal visual objects by the SC. Besides supporting recent evidence that even SC saccade-related motor bursts can preferentially represent visual objects, these results reveal a plausible mechanism through which rapid orienting responses to extrafoveal visual objects can be mediated.

Visual Functions of the Primate Superior Colliculus

The superior colliculus (SC) is a subcortical brain structure that is relevant for sensation, cognition, and action. In nonhuman primates, a rich history of studies has provided unprecedented detail about this structure's role in controlling orienting behaviors; as a result, the primate SC has become primarily regarded as a motor control structure. However, as in other species, the primate SC is also a highly visual structure: A fraction of its inputs is retinal and complemented by inputs from visual cortical areas, including the primary visual cortex. Motivated by this, recent investigations are revealing the rich visual pattern analysis capabilities of the primate SC, placing this structure in an ideal position to guide orienting movements. The anatomical proximity of the primate SC to both early visual inputs and final motor control apparatuses, as well as its ascending feedback projections to the cortex, affirms an important role for this structure in active perception.

Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression

Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

The functional roles of neural remapping in cortex

Remapping is a property of some cortical and subcortical neurons that update their responses around the time of an eye movement to account for the shift of stimuli on the retina due to the saccade. Physiologically, remapping is traditionally tested by briefly presenting a single stimulus around the time of the saccade and looking at the onset of the response and the locations in space to which the neuron is responsive. Here we suggest that a better way to understand the functional role of remapping is to look at the time at which the neural signal emerges when saccades are made across a stable scene. Based on data obtained using this approach, we suggest that remapping in the lateral intraparietal area is sufficient to play a role in maintaining visual stability across saccades, whereas in the frontal eye field, remapped activity carries information that affects future saccadic choices and, in a separate subset of neurons, is used to maintain a map of locations in the scene that have been previously fixated.

Instantaneous movement-unrelated midbrain activity modifies ongoing eye movements

At any moment in time, new information is sampled from the environment and interacts with ongoing brain state. Often, such interaction takes place within individual circuits that are capable of both mediating the internally ongoing plan as well as representing exogenous sensory events. Here, we investigated how sensory-driven neural activity can be integrated, very often in the same neuron types, into ongoing saccade motor commands. Despite the ballistic nature of saccades, visually induced action potentials in the rhesus macaque superior colliculus (SC), a structure known to drive eye movements, not only occurred intra-saccadically, but they were also associated with highly predictable modifications of ongoing eye movements. Such predictable modifications reflected a simultaneity of movement-related discharge at one SC site and visually induced activity at another. Our results suggest instantaneous readout of the SC during movement generation, irrespective of activity source, and they explain a significant component of kinematic variability of motor outputs.

Perisaccadic remapping: What? How? Why?

About 25 years ago, the discovery of receptive field (RF) remapping in the parietal cortex of nonhuman primates revealed that visual RFs, widely assumed to have a fixed retinotopic organization, can change position before every saccade. Measuring such changes can be deceptively difficult. As a result, studies that followed have generated a fascinating but somewhat confusing picture of the phenomenon. In this review, we describe how observations of RF remapping depend on the spatial and temporal sampling of visual RFs and saccade directions. Further, we summarize some of the theories of how remapping might occur in neural circuitry. Finally, based on neurophysiological and psychophysical observations, we discuss the ways in which remapping information might facilitate computations in downstream brain areas.

Increased preparation time reduces, but does not abolish, action history bias of saccadic eye movements

The characteristics of movements are strongly history-dependent. Marinovic et al. (Marinovic W, Poh E, de Rugy A, Carroll TJ. eLife 6: e26713, 2017) showed that past experience influences the execution of limb movements through a combination of temporally stable processes that are strictly use dependent and dynamically evolving and context-dependent processes that reflect prediction of future actions. Here we tested the basis of history-dependent biases for multiple spatiotemporal features of saccadic eye movements under two preparation time conditions (long and short). Twenty people performed saccades to visual targets. To prompt context-specific expectations of most likely target locations, 1 of 12 potential target locations was specified on ~85% of the trials and each remaining target was presented on ~1% trials. In long preparation trials participants were shown the location of the next target 1 s before its presentation onset, whereas in short preparation trials each target was first specified as the cue to move. Saccade reaction times and direction were biased by recent saccade history but according to distinct spatial tuning profiles. Biases were purely expectation related for saccadic reaction times, which increased linearly as the distance from the repeated target location increased when preparation time was short but were similar to all targets when preparation time was long. By contrast, the directions of saccades were biased toward the repeated target in both preparation time conditions, although to a lesser extent when the target location was precued (long preparation). The results suggest that saccade history affects saccade dynamics via both use- and expectation-dependent mechanisms and that movement history has dissociable effects on reaction time and saccadic direction. NEW & NOTEWORTHY The characteristics of our movements are influenced not only by concurrent sensory inputs but also by how we have moved in the past. For limb movements, history effects involve both use-dependent processes due strictly to movement repetition and processes that reflect prediction of future actions. Here we show that saccade history also affects saccade dynamics via use- and expectation-dependent mechanisms but that movement history has dissociable effects on saccade reaction time and direction.

Coding Schemes in the Archerfish Optic Tectum

Many studies have yielded valuable knowledge on the early visual system but it is biased since the studies have focused on terrestrial mammals alone. Here, to better account for visual systems in different environments and animal classes, we studied the structure of early visual processing in the archerfish which harnesses its extreme visual ability to hunt by shooting water jets at prey hanging on vegetation above the water. Thus, the archerfish provides a unique opportunity to study visual processing in a vertebrate which is an expert vision-guided predator with a very different brain structure than mammals. The receptive field structures in the archerfish (both sexes) optic tectum, the main visual processing region in the fish brain, were measured and linear non-linear cascades were used to analyze their properties. The findings indicate that the spatial receptive field structures lie on a continuum between circular and elliptical shapes. In addition, the cells' functional properties display a richness of response characteristics, since many cells could be captured by more than a single linear filter. Finally, the non-linear response functions that link linear filters and neuronal responses were found to be similar to the non-linear functions of models that describe terrestrial mammalian single cell activity. Overall our results help to better understand the early visual processing system across vertebrates.

Normal Topography and Binocularity of the Superior Colliculus in Strabismus

In subjects with alternating strabismus, either eye can be used to saccade to visual targets. The brain must calculate the correct vector for each saccade, which will depend on the eye chosen to make it. The superior colliculus, a major midbrain center for saccade generation, was examined to determine whether the maps serving each eye were shifted to compensate for strabismus. Alternating exotropia was induced in two male macaques at age 1 month by sectioning the tendons of the medial recti. Once the animals grew to maturity, they were trained to fixate targets with either eye. Receptive fields were mapped in the superior colliculus using a sparse noise stimulus while the monkeys alternated fixation. For some neurons, sparse noise was presented dichoptically to probe for anomalous retinal correspondence. After recordings, microstimulation was applied to compare sensory and motor maps. The data showed that receptive fields were offset in position by the ocular deviation, but otherwise remained aligned. In one animal, the left eye's coordinates were rotated ∼20° clockwise with respect to those of the right eye. This was explained by a corresponding cyclorotation of the ocular fundi, which produced an A-pattern deviation. Microstimulation drove the eyes accurately to the site of receptive fields, as in normal animals. Single-cell recordings uncovered no evidence for anomalous retinal correspondence. Despite strabismus, neurons remained responsive to stimulation of either eye. Misalignment of the eyes early in life does not alter the organization of topographic maps or disrupt binocular convergence in the superior colliculus.SIGNIFICANCE STATEMENT Patients with strabismus are able to make rapid eye movements, known as saccades, toward visual targets almost as gracefully as subjects with normal binocular alignment. They can even exercise the option of using the right eye or the left eye. It is unknown how the brain measures the degree of ocular misalignment and uses it to compute the appropriate saccade for either eye. The obvious place to investigate is the superior colliculus, a midbrain oculomotor center responsible for the generation of saccades. Here, we report the first experiments in the superior colliculus of awake primates with strabismus using a combination of single-cell recordings and microstimulation to explore the organization of its topographic maps.

Dynamics of visual receptive fields in the macaque frontal eye field

Neuronal receptive fields (RFs) provide the foundation for understanding systems-level sensory processing. In early visual areas, investigators have mapped RFs in detail using stochastic stimuli and sophisticated analytical approaches. Much less is known about RFs in prefrontal cortex. Visual stimuli used for mapping RFs in prefrontal cortex tend to cover a small range of spatial and temporal parameters, making it difficult to understand their role in visual processing. To address these shortcomings, we implemented a generalized linear model to measure the RFs of neurons in the macaque frontal eye field (FEF) in response to sparse, full-field stimuli. Our high-resolution, probabilistic approach tracked the evolution of RFs during passive fixation, and we validated our results against conventional measures. We found that FEF neurons exhibited a surprising level of sensitivity to stimuli presented as briefly as 10 ms or to multiple dots presented simultaneously, suggesting that FEF visual responses are more precise than previously appreciated. FEF RF spatial structures were largely maintained over time and between stimulus conditions. Our results demonstrate that the application of probabilistic RF mapping to FEF and similar association areas is an important tool for clarifying the neuronal mechanisms of cognition.

Action and perception are temporally coupled by a common mechanism that leads to a timing misperception

We move our eyes to explore the world, but visual areas determining where to look next (action) are different from those determining what we are seeing (perception). Whether, or how, action and perception are temporally coordinated is not known. The preparation time course of an action (e.g., a saccade) has been widely studied with the gap/overlap paradigm with temporal asynchronies (TA) between peripheral target onset and fixation point offset (gap, synchronous, or overlap). However, whether the subjects perceive the gap or overlap, and when they perceive it, has not been studied. We adapted the gap/overlap paradigm to study the temporal coupling of action and perception. Human subjects made saccades to targets with different TAs with respect to fixation point offset and reported whether they perceived the stimuli as separated by a gap or overlapped in time. Both saccadic and perceptual report reaction times changed in the same way as a function of TA. The TA dependencies of the time change for action and perception were very similar, suggesting a common neural substrate. Unexpectedly, in the perceptual task, subjects misperceived lights overlapping by less than ∼100 ms as separated in time (overlap seen as gap). We present an attention-perception model with a map of prominence in the superior colliculus that modulates the stimulus signal's effectiveness in the action and perception pathways. This common source of modulation determines how competition between stimuli is resolved, causes the TA dependence of action and perception to be the same, and causes the misperception.

Spatio-temporal response properties of local field potentials in the primate superior colliculus

Local field potentials (LFPs) are becoming increasingly popular in neurophysiological studies. However, to date, most of the knowledge about LFPs has been obtained from cortical recordings. Here, we recorded single unit activity (SUA) and LFPs simultaneously from the superior colliculus (SC) of behaving rhesus monkeys. The SC is a midbrain structure that plays a central role in the visual orienting response. Previous studies have characterised the visual and visuomotor response properties of SUA in the superficial layers of the SC and the intermediate layers of the SC, respectively. We found that the signal properties of SUA were well preserved in the LFPs recorded from the SC. The SUA and LFPs had similar spatial and temporal properties, and the response properties of LFPs differed across layers, i.e. purely visual in the superficial layers of the SC but showing significant motor responses in the intermediate layers of the SC. There were also differences between SUA and LFPs. LFPs showed a significant reversal of activity following the phasic visual response, suggesting that the neighboring neurons were suppressed. The results indicate that the LFP can be used as a reliable measure of the SC activity in lieu of SUA, and open up a new way to assess sensorimotor processing within the SC.

A sensorimotor role for traveling waves in primate visual cortex

Traveling waves of neural activity are frequently observed to occur in concert with the presentation of a sensory stimulus or the execution of a movement. Although such waves have been studied for decades, little is known about their function. Here we show that traveling waves in the primate extrastriate visual cortex provide a means of integrating sensory and motor signals. Specifically, we describe a traveling wave of local field potential (LFP) activity in cortical area V4 of macaque monkeys that is triggered by the execution of saccadic eye movements. These waves sweep across the V4 retinotopic map, following a consistent path from the foveal to the peripheral representations of space; their amplitudes correlate with the direction and size of each saccade. Moreover, these waves are associated with a reorganization of the postsaccadic neuronal firing patterns, which follow a similar retinotopic progression, potentially prioritizing the processing of behaviorally relevant stimuli.

Heterogeneity in the spatial receptive field architecture of multisensory neurons of the superior colliculus and its effects on multisensory integration

Multisensory integration has been widely studied in neurons of the mammalian superior colliculus (SC). This has led to the description of various determinants of multisensory integration, including those based on stimulus- and neuron-specific factors. The most widely characterized of these illustrate the importance of the spatial and temporal relationships of the paired stimuli as well as their relative effectiveness in eliciting a response in determining the final integrated output. Although these stimulus-specific factors have generally been considered in isolation (i.e., manipulating stimulus location while holding all other factors constant), they have an intrinsic interdependency that has yet to be fully elucidated. For example, changes in stimulus location will likely also impact both the temporal profile of response and the effectiveness of the stimulus. The importance of better describing this interdependency is further reinforced by the fact that SC neurons have large receptive fields, and that responses at different locations within these receptive fields are far from equivalent. To address these issues, the current study was designed to examine the interdependency between the stimulus factors of space and effectiveness in dictating the multisensory responses of SC neurons. The results show that neuronal responsiveness changes dramatically with changes in stimulus location - highlighting a marked heterogeneity in the spatial receptive fields of SC neurons. More importantly, this receptive field heterogeneity played a major role in the integrative product exhibited by stimulus pairings, such that pairings at weakly responsive locations of the receptive fields resulted in the largest multisensory interactions. Together these results provide greater insight into the interrelationship of the factors underlying multisensory integration in SC neurons, and may have important mechanistic implications for multisensory integration and the role it plays in shaping SC-mediated behaviors.