Color-Change Detection Activity in the Primate Superior Colliculus

Assessing perceptual chromatic equiluminance using a reflexive pupillary response

Equiluminant stimuli help assess the integrity of colour perception and the relationship of colour to other visual features. As a result of individual variation, it is necessary to calibrate experimental visual stimuli to suit each individual's unique equiluminant ratio. Most traditional methods rely on training observers to report their subjective equiluminance point. Such paradigms cannot easily be implemented on pre-verbal or non-verbal observers. Here, we present a novel Pupil Frequency-Tagging Method (PFTM) for detecting a participant's unique equiluminance point without verbal instruction and with minimal training. PFTM analyses reflexive pupil oscillations induced by slow (< 2 Hz) temporal alternations between coloured stimuli. Two equiluminant stimuli will induce a similar pupil dilation response regardless of colour; therefore, an observer's equiluminant point can be identified as the luminance ratio between two colours for which the oscillatory amplitude of the pupil at the tagged frequency is minimal. We compared pupillometry-based equiluminance ratios to those obtained with two established techniques in humans: minimum flicker and minimum motion. In addition, we estimated the equiluminance point in non-human primates, demonstrating that this new technique can be successfully employed in non-verbal subjects.

Express detection of visual objects by primate superior colliculus neurons

Primate superior colliculus (SC) neurons exhibit visual feature tuning properties and are implicated in a subcortical network hypothesized to mediate fast threat and/or conspecific detection. However, the mechanisms through which SC neurons contribute to peripheral object detection, for supporting rapid orienting responses, remain unclear. Here we explored whether, and how quickly, SC neurons detect real-life object stimuli. We presented experimentally-controlled gray-scale images of seven different object categories, and their corresponding luminance- and spectral-matched image controls, within the extrafoveal response fields of SC neurons. We found that all of our functionally-identified SC neuron types preferentially detected real-life objects even in their very first stimulus-evoked visual bursts. Intriguingly, even visually-responsive motor-related neurons exhibited such robust early object detection. We further identified spatial frequency information in visual images as an important, but not exhaustive, source for the earliest (within 100 ms) but not for the late (after 100 ms) component of object detection by SC neurons. Our results demonstrate rapid and robust detection of extrafoveal visual objects by the SC. Besides supporting recent evidence that even SC saccade-related motor bursts can preferentially represent visual objects, these results reveal a plausible mechanism through which rapid orienting responses to extrafoveal visual objects can be mediated.

Visual Functions of the Primate Superior Colliculus

The superior colliculus (SC) is a subcortical brain structure that is relevant for sensation, cognition, and action. In nonhuman primates, a rich history of studies has provided unprecedented detail about this structure's role in controlling orienting behaviors; as a result, the primate SC has become primarily regarded as a motor control structure. However, as in other species, the primate SC is also a highly visual structure: A fraction of its inputs is retinal and complemented by inputs from visual cortical areas, including the primary visual cortex. Motivated by this, recent investigations are revealing the rich visual pattern analysis capabilities of the primate SC, placing this structure in an ideal position to guide orienting movements. The anatomical proximity of the primate SC to both early visual inputs and final motor control apparatuses, as well as its ascending feedback projections to the cortex, affirms an important role for this structure in active perception.

Correlated variability in primate superior colliculus depends on functional class

Correlated variability in neuronal activity (spike count correlations, r_SC) can constrain how information is read out from populations of neurons. Traditionally, r_SC is reported as a single value summarizing a brain area. However, single values, like summary statistics, stand to obscure underlying features of the constituent elements. We predict that in brain areas containing distinct neuronal subpopulations, different subpopulations will exhibit distinct levels of r_SC that are not captured by the population r_SC. We tested this idea in macaque superior colliculus (SC), a structure containing several functional classes (i.e., subpopulations) of neurons. We found that during saccade tasks, different functional classes exhibited differing degrees of r_SC. "Delay class" neurons displayed the highest r_SC, especially during saccades that relied on working memory. Such dependence of r_SC on functional class and cognitive demand underscores the importance of taking functional subpopulations into account when attempting to model or infer population coding principles.

A Midbrain Inspired Recurrent Neural Network Model for Robust Change Detection

We present a biologically inspired recurrent neural network (RNN) that efficiently detects changes in natural images. The model features sparse, topographic connectivity (st-RNN), closely modeled on the circuit architecture of a "midbrain attention network." We deployed the st-RNN in a challenging change blindness task, in which changes must be detected in a discontinuous sequence of images. Compared with a conventional RNN, the st-RNN learned 9x faster and achieved state-of-the-art performance with 15x fewer connections. An analysis of low-dimensional dynamics revealed putative circuit mechanisms, including a critical role for a global inhibitory (GI) motif, for successful change detection. The model reproduced key experimental phenomena, including midbrain neurons' sensitivity to dynamic stimuli, neural signatures of stimulus competition, as well as hallmark behavioral effects of midbrain microstimulation. Finally, the model accurately predicted human gaze fixations in a change blindness experiment, surpassing state-of-the-art saliency-based methods. The st-RNN provides a novel deep learning model for linking neural computations underlying change detection with psychophysical mechanisms.SIGNIFICANCE STATEMENT For adaptive survival, our brains must be able to accurately and rapidly detect changing aspects of our visual world. We present a novel deep learning model, a sparse, topographic recurrent neural network (st-RNN), that mimics the neuroanatomy of an evolutionarily conserved "midbrain attention network." The st-RNN achieved robust change detection in challenging change blindness tasks, outperforming conventional RNN architectures. The model also reproduced hallmark experimental phenomena, both neural and behavioral, reported in seminal midbrain studies. Lastly, the st-RNN outperformed state-of-the-art models at predicting human gaze fixations in a laboratory change blindness experiment. Our deep learning model may provide important clues about key mechanisms by which the brain efficiently detects changes.

Microsaccades as a marker not a cause for attention-related modulation

Recent evidence suggests that microsaccades are causally linked to the attention-related modulation of neurons-specifically, that microsaccades toward the attended location are required for the subsequent changes in firing rate. These findings have raised questions about whether attention-related modulation is due to different states of attention as traditionally assumed or might instead be a secondary effect of microsaccades. Here, in two rhesus macaques, we tested the relationship between microsaccades and attention-related modulation in the superior colliculus (SC), a brain structure crucial for allocating attention. We found that attention-related modulation emerged even in the absence of microsaccades, was already present prior to microsaccades toward the cued stimulus, and persisted through the suppression of activity that accompanied all microsaccades. Nonetheless, consistent with previous findings, we also found significant attention-related modulation when microsaccades were directed toward, rather than away from, the cued location. Thus, despite the clear links between microsaccades and attention, microsaccades are not necessary for attention-related modulation, at least not in the SC. They do, however, provide an additional marker for the state of attention, especially at times when attention is shifting from one location to another.

Neuronal modulation in the mouse superior colliculus during covert visual selective attention

Covert visual attention is accomplished by a cascade of mechanisms distributed across multiple brain regions. Visual cortex is associated with enhanced representations of relevant stimulus features, whereas the contributions of subcortical circuits are less well understood but have been associated with selection of relevant spatial locations and suppression of distracting stimuli. As a step toward understanding these subcortical circuits, here we identified how neuronal activity in the intermediate layers of the superior colliculus (SC) of head-fixed mice is modulated during covert visual attention. We found that spatial cues modulated both firing rate and spike-count correlations. Crucially, the cue-related modulation in firing rate was due to enhancement of activity at the cued spatial location rather than suppression at the uncued location, indicating that SC neurons in our task were modulated by an excitatory or disinhibitory circuit mechanism focused on the relevant location, rather than broad inhibition of irrelevant locations. This modulation improved the neuronal discriminability of visual-change-evoked activity, but only when assessed for neuronal activity between the contralateral and ipsilateral SC. Together, our findings indicate that neurons in the mouse SC can contribute to covert visual selective attention by biasing processing in favor of locations expected to contain task-relevant information.

An inferior-superior colliculus circuit controls auditory cue-directed visual spatial attention

Selective attention modulates neuronal activity in multiple brain regions, but the origins of attention signals remain unclear. We show that, during a visual task requiring spatial attention directed by an auditory cue, an inferior-superior colliculus circuit provides the key attention signal. In mice performing a task based on a visual stimulus in the cued hemifield while ignoring a conflicting stimulus on the uncued side, the visual cortex (V1) and superior colliculus (SC) showed strong attentional modulation, with a shorter latency in the SC. The nucleus of the brachium of the inferior colliculus (nBIC), which provides auditory inputs to the SC, was activated not only at auditory cue onset but also during the delay period before the visual stimulus. The delay activity, but not cue onset activity, was crucial for task performance and attentional modulation in the SC and V1. These results establish a new behavioral paradigm for studying visual attention in mice and identify a midbrain signal controlling auditory cue-directed spatial attention.

Task-Irrelevant Visual Forms Facilitate Covert and Overt Spatial Selection

Covert and overt spatial selection behaviors are guided by both visual saliency maps derived from early visual features as well as priority maps reflecting high-level cognitive factors. However, whether mid-level perceptual processes associated with visual form recognition contribute to covert and overt spatial selection behaviors remains unclear. We hypothesized that if peripheral visual forms contribute to spatial selection behaviors, then they should do so even when the visual forms are task-irrelevant. We tested this hypothesis in male and female human subjects as well as in male macaque monkeys performing a visual detection task. In this task, subjects reported the detection of a suprathreshold target spot presented on top of one of two peripheral images, and they did so with either a speeded manual button press (humans) or a speeded saccadic eye movement response (humans and monkeys). Crucially, the two images, one with a visual form and the other with a partially phase-scrambled visual form, were completely irrelevant to the task. In both manual (covert) and oculomotor (overt) response modalities, and in both humans and monkeys, response times were faster when the target was congruent with a visual form than when it was incongruent. Importantly, incongruent targets were associated with almost all errors, suggesting that forms automatically captured selection behaviors. These findings demonstrate that mid-level perceptual processes associated with visual form recognition contribute to covert and overt spatial selection. This indicates that neural circuits associated with target selection, such as the superior colliculus, may have privileged access to visual form information.SIGNIFICANCE STATEMENT Spatial selection of visual information either with (overt) or without (covert) foveating eye movements is critical to primate behavior. However, it is still not clear whether spatial maps in sensorimotor regions known to guide overt and covert spatial selection are influenced by peripheral visual forms. We probed the ability of humans and monkeys to perform overt and covert target selection in the presence of spatially congruent or incongruent visual forms. Even when completely task-irrelevant, images of visual objects had a dramatic effect on target selection, acting much like spatial cues used in spatial attention tasks. Our results demonstrate that traditional brain circuits for orienting behaviors, such as the superior colliculus, likely have privileged access to visual object representations.

Visual feature tuning of superior colliculus neural reafferent responses after fixational microsaccades

The primate superior colliculus (SC) is causally involved in microsaccade generation. Moreover, visually responsive SC neurons across this structure's topographic map, even at peripheral eccentricities much larger than the tiny microsaccade amplitudes, exhibit significant modulations of evoked response sensitivity when stimuli appear perimicrosaccadically. However, during natural viewing, visual stimuli are normally stably present in the environment and are only shifted on the retina by eye movements. Here we investigated this scenario for the case of microsaccades, asking whether and how SC neurons respond to microsaccade-induced image jitter. We recorded neural activity from two male rhesus macaque monkeys. Within the response field (RF) of a neuron, there was a stable stimulus consisting of a grating of one of three possible spatial frequencies. The grating was stable on the display, but microsaccades periodically jittered the retinotopic RF location over it. We observed clear short-latency visual reafferent responses after microsaccades. These responses were weaker, but earlier (relative to new fixation onset after microsaccade end), than responses to sudden stimulus onsets without microsaccades. The reafferent responses clearly depended on microsaccade amplitude as well as microsaccade direction relative to grating orientation. Our results indicate that one way for microsaccades to influence vision is through modulating how the spatio-temporal landscape of SC visual neural activity represents stable stimuli in the environment. Such representation depends on the specific pattern of temporal luminance modulations expected from the relative relationship between eye movement vector (size and direction) on one hand and spatial visual pattern layout on the other.NEW & NOTEWORTHY Despite being diminutive, microsaccades still jitter retinal images. We investigated how such jitter affects superior colliculus (SC) activity. We found that SC neurons exhibit short-latency visual reafferent bursts after microsaccades. These bursts reflect not only the spatial luminance profiles of visual patterns but also how such profiles are shifted by eye movement size and direction. These results indicate that the SC continuously represents visual patterns, even as they are jittered by the smallest possible saccades.

Superior colliculus modulates cortical coding of somatosensory information

The cortex modulates activity in superior colliculus via a direct projection. What is largely unknown is whether (and if so how) the superior colliculus modulates activity in the cortex. Here, we investigate this issue and show that optogenetic activation of superior colliculus changes the input-output relationship of neurons in somatosensory cortex, enhancing responses to low amplitude whisker deflections. While there is no direct pathway from superior colliculus to somatosensory cortex, we found that activation of superior colliculus drives spiking in the posterior medial (POm) nucleus of the thalamus via a powerful monosynaptic pathway. Furthermore, POm neurons receiving input from superior colliculus provide monosynaptic excitatory input to somatosensory cortex. Silencing POm abolished the capacity of superior colliculus to modulate cortical whisker responses. Our findings indicate that the superior colliculus, which plays a key role in attention, modulates sensory processing in somatosensory cortex via a powerful di-synaptic pathway through the thalamus.

Midbrain activity can explain perceptual decisions during an attention task

We introduce a decision model that interprets the relative levels of moment-by-moment spiking activity from the right and left superior colliculus to distinguish relevant from irrelevant stimulus events. The model explains detection performance in a covert attention task, both in intact animals and when performance is perturbed by causal manipulations. This provides a specific example of how midbrain activity could support perceptual judgments during attention tasks.

Brain regions modulated during covert visual attention in the macaque

Neurophysiological studies of covert visual attention in monkeys have emphasized the modulation of sensory neural responses in the visual cortex. At the same time, electrophysiological correlates of attention have been reported in other cortical and subcortical structures, and recent fMRI studies have identified regions across the brain modulated by attention. Here we used fMRI in two monkeys performing covert attention tasks to reproduce and extend these findings in order to help establish a more complete list of brain structures involved in the control of attention. As expected from previous studies, we found attention-related modulation in frontal, parietal and visual cortical areas as well as the superior colliculus and pulvinar. We also found significant attention-related modulation in cortical regions not traditionally linked to attention - mid-STS areas (anterior FST and parts of IPa, PGa, TPO), as well as the caudate nucleus. A control experiment using a second-order orientation stimulus showed that the observed modulation in a subset of these mid-STS areas did not depend on visual motion. These results identify the mid-STS areas (anterior FST and parts of IPa, PGa, TPO) and caudate nucleus as potentially important brain regions in the control of covert visual attention in monkeys.

Comparing frontal eye field and superior colliculus contributions to covert spatial attention

The causal roles of the frontal eye fields (FEF) and superior colliculus (SC) in spatial selective attention have not been directly compared. Reversible inactivation is an established method for testing causality but comparing results between FEF and SC is complicated by differences in size and morphology of the two brain regions. Here we exploited the fact that inactivation of FEF and SC also changes the metrics of saccadic eye movements, providing an independent benchmark for the strength of the causal manipulation. Using monkeys trained to covertly perform a visual motion-change detection task, we found that inactivation of either FEF or SC could cause deficits in attention task performance. However, SC-induced attention deficits were found with saccade changes half the size needed to get FEF-induced attention deficits. Thus, performance in visual attention tasks is vulnerable to loss of signals from either structure, but suppression of SC activity has a more devastating effect.

Spatial frequency sensitivity in macaque midbrain

Visual brain areas exhibit tuning characteristics well suited for image statistics present in our natural environment. However, visual sensation is an active process, and if there are any brain areas that ought to be particularly in tune with natural scene statistics, it would be sensory-motor areas critical for guiding behavior. Here we found that the rhesus macaque superior colliculus, a structure instrumental for rapid visual exploration with saccades, detects low spatial frequencies, which are the most prevalent in natural scenes, much more rapidly than high spatial frequencies. Importantly, this accelerated detection happens independently of whether a neuron is more or less sensitive to low spatial frequencies to begin with. At the population level, the superior colliculus additionally over-represents low spatial frequencies in neural response sensitivity, even at near-foveal eccentricities. Thus, the superior colliculus possesses both temporal and response gain mechanisms for efficient gaze realignment in low-spatial-frequency-dominated natural environments.

Visual Neurons in the Superior Colliculus Discriminate Many Objects by Their Historical Values

The superior colliculus (SC) is an important structure in the mammalian brain that orients the animal toward distinct visual events. Visually responsive neurons in SC are modulated by visual object features, including size, motion, and color. However, it remains unclear whether SC activity is modulated by non-visual object features, such as the reward value associated with the object. To address this question, three monkeys were trained (>10 days) to saccade to multiple fractal objects, half of which were consistently associated with large rewards while other half were associated with small rewards. This created historically high-valued (‘good’) and low-valued (‘bad’) objects. During the neuronal recordings from the SC, the monkeys maintained fixation at the center while the objects were flashed in the receptive field of the neuron without any reward. We found that approximately half of the visual neurons responded more strongly to the good than bad objects. In some neurons, this value-coding remained intact for a long time (>1 year) after the last object-reward association learning. Notably, the neuronal discrimination of reward values started about 100 ms after the appearance of visual objects and lasted for more than 100 ms. These results provide evidence that SC neurons can discriminate objects by their historical (long-term) values. This object value information may be provided by the basal ganglia, especially the circuit originating from the tail of the caudate nucleus. The information may be used by the neural circuits inside SC for motor (saccade) output or may be sent to the circuits outside SC for future behavior.