Timescales of sensory- and decision-related activity in the middle temporal and medial superior temporal areas

Self-motion perception and sequential decision-making: where are we heading?

To navigate and guide adaptive behaviour in a dynamic environment, animals must accurately estimate their own motion relative to the external world. This is a fundamentally multisensory process involving integration of visual, vestibular and kinesthetic inputs. Ideal observer models, paired with careful neurophysiological investigation, helped to reveal how visual and vestibular signals are combined to support perception of linear self-motion direction, or heading. Recent work has extended these findings by emphasizing the dimension of time, both with regard to stimulus dynamics and the trade-off between speed and accuracy. Both time and certainty-i.e. the degree of confidence in a multisensory decision-are essential to the ecological goals of the system: terminating a decision process is necessary for timely action, and predicting one's accuracy is critical for making multiple decisions in a sequence, as in navigation. Here, we summarize a leading model for multisensory decision-making, then show how the model can be extended to study confidence in heading discrimination. Lastly, we preview ongoing efforts to bridge self-motion perception and navigation per se, including closed-loop virtual reality and active self-motion. The design of unconstrained, ethologically inspired tasks, accompanied by large-scale neural recordings, raise promise for a deeper understanding of spatial perception and decision-making in the behaving animal. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Spike rates of frontal eye field neurons predict reaction times in a spatial attention task

Which neuronal signal(s) predict reaction times when subjects respond to a target at covertly attended locations? Although recent studies showed that spike rates are not predictive, it remains a highly contested question. Therefore, we record single-unit activity from frontal eye field (FEF) neurons while macaques are performing a covert spatial attention task. We find that the attentional modulation of spike rates of FEF neurons is strongly correlated with behavioral reaction times. Moreover, this correlation already emerges 1 s before target dimming, which triggers the behavioral responses. This prediction of reaction times by spike rates is found in neurons showing attention-dependent enhanced and suppressed activity for targets and distractors, respectively, yet in varying degrees across subjects. Thus, spike rates of FEF neurons can predict reaction times persistently and well before the operant behavior during selective attention tasks. Such long prediction windows will be useful for developing spike-based brain-machine interfaces.

A neural correlate of perceptual segmentation in macaque middle temporal cortical area

High-resolution vision requires fine retinal sampling followed by integration to recover object properties. Importantly, accuracy is lost if local samples from different objects are intermixed. Thus, segmentation, grouping of image regions for separate processing, is crucial for perception. Previous work has used bi-stable plaid patterns, which can be perceived as either a single or multiple moving surfaces, to study this process. Here, we report a relationship between activity in a mid-level site in the primate visual pathways and segmentation judgments. Specifically, we find that direction selective middle temporal neurons are sensitive to texturing cues used to bias the perception of bi-stable plaids and exhibit a significant trial-by-trial correlation with subjective perception of a constant stimulus. This correlation is greater in units that signal global motion in patterns with multiple local orientations. Thus, we conclude the middle temporal area contains a signal for segmenting complex scenes into constituent objects and surfaces.

Thalamocortical functional connectivity in patients with insomnia using resting-state fMRI

BACKGROUND

Insomnia is a common disorder that affects a vast number of patients; the hyperarousal theory of insomnia postulates that patients with insomnia are physiologically activated not only at nighttime but also during the daytime. We aimed to investigate the differences in the resting-state functional connectivity (RSFC) of the thalamus with cortical areas between patients with insomnia disorder and healthy controls.

METHODS

All participants completed clinical questionnaires and underwent portable polysomnography and resting-state fMRI.

RESULTS

Patients in the insomnia group (n = 50) showed increased RSFC between the thalamus and right medial superior frontal area, bilateral middle temporal areas, left rectus and right parahippocampal areas compared with controls (n = 42) after controlling for age, sex and education level. Among the pairs that showed increased connectivity, several functional connections were negatively correlated with sleep efficiency, measured by polysomnography.Limitations: We used a small sample size.

CONCLUSION

We consider these results on increased thalamocortical hyperactivity in brain areas related to sensory functions as providing evidence for the hyperarousal theory of insomnia.

Dynamic divisive normalization circuits explain and predict change detection in monkey area MT

Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.

Motion Perception in the Common Marmoset

Visual motion processing is a well-established model system for studying neural population codes in primates. The common marmoset, a small new world primate, offers unparalleled opportunities to probe these population codes in key motion processing areas, such as cortical areas MT and MST, because these areas are accessible for imaging and recording at the cortical surface. However, little is currently known about the perceptual abilities of the marmoset. Here, we introduce a paradigm for studying motion perception in the marmoset and compare their psychophysical performance with human observers. We trained two marmosets to perform a motion estimation task in which they provided an analog report of their perceived direction of motion with an eye movement to a ring that surrounded the motion stimulus. Marmosets and humans exhibited similar trade-offs in speed versus accuracy: errors were larger and reaction times were longer as the strength of the motion signal was reduced. Reverse correlation on the temporal fluctuations in motion direction revealed that both species exhibited short integration windows; however, marmosets had substantially less nondecision time than humans. Our results provide the first quantification of motion perception in the marmoset and demonstrate several advantages to using analog estimation tasks.

Linking Neuronal Direction Selectivity to Perceptual Decisions About Visual Motion

Psychophysical and neurophysiological studies of responses to visual motion have converged on a consistent set of general principles that characterize visual processing of motion information. Both types of approaches have shown that the direction and speed of target motion are among the most important encoded stimulus properties, revealing many parallels between psychophysical and physiological responses to motion. Motivated by these parallels, this review focuses largely on more direct links between the key feature of the neuronal response to motion, direction selectivity, and its utilization in memory-guided perceptual decisions. These links were established during neuronal recordings in monkeys performing direction discriminations, but also by examining perceptual effects of widespread elimination of cortical direction selectivity produced by motion deprivation during development. Other approaches, such as microstimulation and lesions, have documented the importance of direction-selective activity in the areas that are active during memory-guided direction comparisons, area MT and the prefrontal cortex, revealing their likely interactions during behavioral tasks.

Correlated structure of neuronal firing in macaque visual cortex limits information for binocular depth discrimination

Variability in cortical neural activity potentially limits sensory discriminations. Theoretical work shows that information required to discriminate two similar stimuli is limited by the correlation structure of cortical variability. We investigated these information-limiting correlations by recording simultaneously from visual cortical areas primary visual cortex (V1) and extrastriate area V4 in macaque monkeys performing a binocular, stereo depth discrimination task. Within both areas, noise correlations on a rapid temporal scale (20–30 ms) were stronger for neuron pairs with similar selectivity for binocular depth, meaning that these correlations potentially limit information for making the discrimination. Between-area correlations (V1 to V4) were different, being weaker for neuron pairs with similar tuning and having a slower temporal scale (100+ ms). Fluctuations in these information-limiting correlations just prior to the detection event were associated with changes in behavioral accuracy. Although these correlations limit the recovery of information about sensory targets, their impact may be curtailed by integrative processing of signals across multiple brain areas.

NEW & NOTEWORTHY Correlated noise reduces the stimulus information in visual cortical neurons during experimental performance of binocular depth discriminations. The temporal scale of these correlations is important. Rapid (20–30 ms) correlations reduce information within and between areas V1 and V4, whereas slow (>100 ms) correlations between areas do not. Separate cortical areas appear to act together to maintain signal fidelity. Rapid correlations reduce the neuronal signal difference between stimuli and adversely affect perceptual discrimination.

Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Amplitude modulations of cortical sensory responses in pulsatile evidence accumulation

How does the brain internally represent a sequence of sensory information that jointly drives a decision-making behavior? Studies of perceptual decision-making have often assumed that sensory cortices provide noisy but otherwise veridical sensory inputs to downstream processes that accumulate and drive decisions. However, sensory processing in even the earliest sensory cortices can be systematically modified by various external and internal contexts. We recorded from neuronal populations across posterior cortex as mice performed a navigational decision-making task based on accumulating randomly timed pulses of visual evidence. Even in V1, only a small fraction of active neurons had sensory-like responses time-locked to each pulse. Here, we focus on how these 'cue-locked' neurons exhibited a variety of amplitude modulations from sensory to cognitive, notably by choice and accumulated evidence. These task-related modulations affected a large fraction of cue-locked neurons across posterior cortex, suggesting that future models of behavior should account for such influences.

The Correlation of Neuronal Signals with Behavior at Different Levels of Visual Cortex and Their Relative Reliability for Behavioral Decisions

Behavior can be guided by neuronal activity in visual, auditory, or somatosensory cerebral cortex, depending on task requirements. In contrast to this flexible access of cortical signals, several observations suggest that behaviors depend more on neurons in later areas of visual cortex than those in earlier areas, although neurons in earlier areas would provide more reliable signals for many tasks. We recorded from neurons in different levels of visual cortex of 2 male rhesus monkeys while the animals did a visual discrimination task and examined trial-to-trial correlations between neuronal and behavioral responses. These correlations became stronger in primary visual cortex as neuronal signals in that area became more reliable relative to the other areas. The results suggest that the mechanisms that read signals from cortex might access any cortical area depending on the relative value of those signals for the task at hand.SIGNIFICANCE STATEMENT Information is encoded by the action potentials of neurons in various cortical areas in a hierarchical manner such that increasingly complex stimulus features are encoded in successive stages. The brain must extract information from the response of appropriate neurons to drive optimal behavior. A widely held view of this decoding process is that the brain relies on the output of later cortical areas to make decisions, although neurons in earlier areas can provide more reliable signals. We examined correlations between perceptual decisions and the responses of neurons in different levels of monkey visual cortex. The results suggest that the brain may access signals in any cortical area depending on the relative value of those signals for the task at hand.

Attention amplifies neural representations of changes in sensory input at the expense of perceptual accuracy

Attention enhances the neural representations of behaviorally relevant stimuli, typically by a push-pull increase of the neuronal response gain to attended vs. unattended stimuli. This selectively improves perception and consequently behavioral performance. However, to enhance the detectability of stimulus changes, attention might also distort neural representations, compromising accurate stimulus representation. We test this hypothesis by recording neural responses in the visual cortex of rhesus monkeys during a motion direction change detection task. We find that attention indeed amplifies the neural representation of direction changes, beyond a similar effect of adaptation. We further show that humans overestimate such direction changes, providing a perceptual correlate of our neurophysiological observations. Our results demonstrate that attention distorts the neural representations of abrupt sensory changes and consequently perceptual accuracy. This likely represents an evolutionary adaptive mechanism that allows sensory systems to flexibly forgo accurate representation of stimulus features to improve the encoding of stimulus change.

Choice (-history) correlations in sensory cortex: cause or consequence?

One challenge in neuroscience, as in other areas of science, is to make inferences about the underlying causal structure from correlational data. Here, we discuss this challenge in the context of choice correlations in sensory neurons, that is, trial-by-trial correlations, unexplained by the stimulus, between the activity of sensory neurons and an animal's perceptual choice. Do these choice-correlations reflect feedforward, feedback signalling, both, or neither? We highlight recent results of correlational and causal examinations of choice and choice-history signals in sensory, and in part sensorimotor, cortex and address formal statistical frameworks to infer causal interactions from data.

Predicting Perceptual Decisions Using Visual Cortical Population Responses and Choice History

Our understanding of the neural basis of perceptual decision making has been built in part on relating co-fluctuations of single neuron responses to perceptual decisions on a trial-by-trial basis. The strength of this relationship is often compared across neurons or brain areas, recorded in different sessions, animals, or variants of a task. We sought to extend our understanding of perceptual decision making in three ways. First, we measured neuronal activity simultaneously in early [primary visual cortex (V1)] and midlevel (V4) visual cortex while macaque monkeys performed a fine orientation discrimination perceptual task. This allowed a direct comparison of choice signals in these two areas, including their dynamics. Second, we asked how our ability to predict animals' decisions would be improved by considering small simultaneously-recorded neuronal populations rather than individual units. Finally, we asked whether predictions would be improved by taking into account the animals' choice and reward histories, which can strongly influence decision making. We found that responses of individual V4 neurons were weakly predictive of decisions, but only in a brief epoch between stimulus offset and the indication of choice. In V1, few neurons showed significant decision-related activity. Analysis of neuronal population responses revealed robust choice-related information in V4 and substantially weaker signals in V1. Including choice- and reward-history information improved performance further, particularly when the recorded populations contained little decision-related information. Our work shows the power of using neuronal populations and decision history when relating neuronal responses to the perceptual decisions they are thought to underlie.SIGNIFICANCE STATEMENT Decades of research has provided a rich description of how visual information is represented in the visual cortex. Yet how cortical responses relate to visual perception remains poorly understood. Here we relate fluctuations in small neuronal population responses, recorded simultaneously in primary visual cortex (V1) and area V4 of monkeys, to perceptual reports in an orientation discrimination task. Choice-related signals were robust in V4, particularly late in the behavioral trial, but not in V1. Models that include both neuronal responses and choice-history information were able to predict a substantial portion of decisions. Our work shows the power of integrating information across neurons and including decision history in relating neuronal responses to perceptual decisions.

Going with the Flow: The Neural Mechanisms Underlying Illusions of Complex-Flow Motion

Studying the mismatch between perception and reality helps us better understand the constructive nature of the visual brain. The Pinna-Brelstaff motion illusion is a compelling example illustrating how a complex moving pattern can generate an illusory motion perception. When an observer moves toward (expansion) or away (contraction) from the Pinna-Brelstaff figure, the figure appears to rotate. The neural mechanisms underlying the illusory complex-flow motion of rotation, expansion, and contraction remain unknown. We studied this question at both perceptual and neuronal levels in behaving male macaques by using carefully parametrized Pinna-Brelstaff figures that induce the above motion illusions. We first demonstrate that macaques perceive illusory motion in a manner similar to that of human observers. Neurophysiological recordings were subsequently performed in the middle temporal area (MT) and the dorsal portion of the medial superior temporal area (MSTd). We find that subgroups of MSTd neurons encoding a particular global pattern of real complex-flow motion (rotation, expansion, contraction) also represent illusory motion patterns of the same class. They require an extra 15 ms to reliably discriminate the illusion. In contrast, MT neurons encode both real and illusory local motions with similar temporal delays. These findings reveal that illusory complex-flow motion is first represented in MSTd by the same neurons that normally encode real complex-flow motion. However, the extraction of global illusory motion in MSTd from other classes of real complex-flow motion requires extra processing time. Our study illustrates a cascaded integration mechanism from MT to MSTd underlying the transformation from external physical to internal nonveridical flow-motion perception.SIGNIFICANCE STATEMENT The neural basis of the transformation from objective reality to illusory percepts of rotation, expansion, and contraction remains unknown. We demonstrate psychophysically that macaques perceive these illusory complex-flow motions in a manner similar to that of human observers. At the neural level, we show that medial superior temporal (MSTd) neurons represent illusory flow motions as if they were real by globally integrating middle temporal area (MT) local motion signals. Furthermore, while MT neurons reliably encode real and illusory local motions with similar temporal delays, MSTd neurons take a significantly longer time to process the signals associated with illusory percepts. Our work extends previous complex-flow motion studies by providing the first detailed analysis of the neuron-specific mechanisms underlying complex forms of illusory motion integration from MT to MSTd.

Micropools of reliable area MT neurons explain rapid motion detection

Many models of perceptually based decisions postulate that actions are initiated when accumulated sensory signals reach a threshold level of activity. These models have received considerable neurophysiological support from recordings of individual neurons while animals are engaged in motion discrimination tasks. These experiments have found that the activity of neurons in a particular visual area strongly associated with motion processing (MT), when pooled over hundreds of milliseconds, is sufficient to explain behavioral timing and performance. However, this level of pooling may be problematic for urgent perceptual decisions in which rapid detection dictates temporally precise integration. In this paper, we explore the physiological basis of one such task in which macaques detected brief (~70 ms) transients of coherent motion within ~240 ms. We find that a simple linear summation model based on realistic stimulus responses of as few as 40 correlated neurons can predict the reliability and timing of rapid motion detection. The model naturally reproduces a distinctive physiological relationship observed in rapid detection tasks in which the individual neurons with the most reliable stimulus responses are also the most predictive of impending behavioral choices. Remarkably, we observed this relationship across our simulated neuronal populations even when all neurons within the pool were weighted equally with respect to readout. These results demonstrate that small numbers of reliable sensory neurons can dominate perceptual judgments without any explicit reliability based weighting and are sufficient to explain the accuracy, latency, and temporal precision of rapid detection. NEW & NOTEWORTHY Computational and psychophysical models suggest that performance in many perceptual tasks may be based on the preferential sampling of reliable neurons. Recent studies of MT neurons during rapid motion detection, in which only those neurons with the most reliable sensory responses were strongly predictive of the animals' decisions, seemingly support this notion. Here we show that a simple threshold model without explicit reliability biases can explain both the behavioral accuracy and precision of these detections and the distribution of sensory- and choice-related signals across neurons.

The Magnitude, But Not the Sign, of MT Single-Trial Spike-Time Correlations Predicts Motion Detection Performance

Spike-time correlations capture the short timescale covariance between the activity of neurons on a single trial. These correlations can significantly vary in magnitude and sign from trial to trial, and have been proposed to contribute to information encoding in visual cortex. While monkeys performed a motion-pulse detection task, we examined the behavioral impact of both the magnitude and sign of single-trial spike-time correlations between two nonoverlapping pools of middle temporal (MT) neurons. We applied three single-trial measures of spike-time correlation between our multiunit MT spike trains (Pearson's, absolute value of Pearson's, and mutual information), and examined the degree to which they predicted a subject's performance on a trial-by-trial basis. We found that on each trial, positive and negative spike-time correlations were almost equally likely, and, once the correlational sign was accounted for, all three measures were similarly predictive of behavior. Importantly, just before the behaviorally relevant motion pulse occurred, single-trial spike-time correlations were as predictive of the performance of the animal as single-trial firing rates. While firing rates were positively associated with behavioral outcomes, the presence of either strong positive or negative correlations had a detrimental effect on behavior. These correlations occurred on short timescales, and the strongest positive and negative correlations modulated behavioral performance by ∼9%, compared with trials with no correlations. We suggest a model where spike-time correlations are associated with a common noise source for the two MT pools, which in turn decreases the signal-to-noise ratio of the integrated signals that drive motion detection.SIGNIFICANCE STATEMENT Previous work has shown that spike-time correlations occurring on short timescales can affect the encoding of visual inputs. Although spike-time correlations significantly vary in both magnitude and sign across trials, their impact on trial-by-trial behavior is not fully understood. Using neural recordings from area MT (middle temporal) in monkeys performing a motion-detection task using a brief stimulus, we found that both positive and negative spike-time correlations predicted behavioral responses as well as firing rate on a trial-by-trial basis. We propose that strong positive and negative spike-time correlations decreased behavioral performance by reducing the signal-to-noise ratio of integrated MT neural signals.

Methods for identification of spike patterns in massively parallel spike trains

Temporally, precise correlations between simultaneously recorded neurons have been interpreted as signatures of cell assemblies, i.e., groups of neurons that form processing units. Evidence for this hypothesis was found on the level of pairwise correlations in simultaneous recordings of few neurons. Increasing the number of simultaneously recorded neurons increases the chances to detect cell assembly activity due to the larger sample size. Recent technological advances have enabled the recording of 100 or more neurons in parallel. However, these massively parallel spike train data require novel statistical tools to be analyzed for correlations, because they raise considerable combinatorial and multiple testing issues. Recently, various of such methods have started to develop. First approaches were based on population or pairwise measures of synchronization, and later led to methods for the detection of various types of higher-order synchronization and of spatio-temporal patterns. The latest techniques combine data mining with analysis of statistical significance. Here, we give a comparative overview of these methods, of their assumptions and of the types of correlations they can detect.

Long-term sensorimotor adaptation in the ocular following system of primates

The sudden movement of a wide-field image leads to a reflexive eye tracking response referred to as short-latency ocular following. If the image motion occurs soon after a saccade the initial speed of the ocular following is enhanced, a phenomenon known as post-saccadic enhancement. We show in macaque monkeys that repeated exposure to the same stimulus regime over a period of months leads to progressive increases in the initial speeds of ocular following. The improvement in tracking speed occurs for ocular following with and without a prior saccade. As a result of the improvement in ocular following speeds, the influence of post-saccadic enhancement wanes with increasing levels of training. The improvement in ocular following speed following repeated exposure to the same oculomotor task represents a novel form of sensori-motor learning in the context of a reflexive movement.

Task-specific, dimension-based attentional shaping of motion processing in monkey area MT

Nonspatially selective attention is based on the notion that specific features or objects in the visual environment are effectively prioritized in cortical visual processing. Feature-based attention (FBA), in particular, is a well-studied process that dynamically and selectively addresses neurons preferentially processing the attended feature attribute (e.g., leftward motion). In everyday life, however, behavior may require high sensitivity for an entire feature dimension (e.g., motion), but experimental evidence for a feature dimension-specific attentional modulation on a cellular level is lacking. Therefore, we investigated neuronal activity in macaque motion-selective mediotemporal area (MT) in an experimental setting requiring the monkeys to detect either a motion change or a color change. We hypothesized that neural activity in MT is enhanced when the task requires perceptual sensitivity to motion. In line with this, we found that mean firing rates were higher in the motion task and that response variability and latency were lower compared with values in the color task, despite identical visual stimulation. This task-specific, dimension-based modulation of motion processing emerged already in the absence of visual input, was independent of the relation between the attended and stimulating motion direction, and was accompanied by a spatially global reduction of neuronal variability. The results provide single-cell support for the hypothesis of a feature dimension-specific top-down signal emphasizing the processing of an entire feature class.NEW & NOTEWORTHY Cortical processing serving visual perception prioritizes information according to current task requirements. We provide evidence in favor of a dimension-based attentional mechanism addressing all neurons that process visual information in the task-relevant feature domain. Behavioral tasks required monkeys to attend either color or motion, causing modulations of response strength, variability, latency, and baseline activity of motion-selective monkey area MT neurons irrespective of the attended motion direction but specific to the attended feature dimension.

Noisy decision thresholds can account for suboptimal detection of low coherence motion

Noise in sensory signals can vary over both space and time. Moving random dot stimuli are commonly used to quantify how the visual system accounts for spatial noise. In these stimuli, a fixed proportion of "signal" dots move in the same direction and the remaining "noise" dots are randomly replotted. The spatial coherence, or proportion of signal versus noise dots, is fixed across time; however, this means that little is known about how temporally-noisy signals are integrated. Here we use a stimulus with low temporal coherence; the signal direction is only presented on a fraction of frames. Human observers are able to reliably detect and discriminate the direction of a 200 ms motion pulse, even when just 25% of frames within the pulse move in the signal direction. Using psychophysical reverse-correlation analyses, we show that observers are strongly influenced by the number of near-target directions spread throughout the pulse, and that consecutive signal frames have only a small additional influence on perception. Finally, we develop a model inspired by the leaky integration of the responses of direction-selective neurons, which reliably represents motion direction, and which can account for observers' sub-optimal detection of motion pulses by incorporating a noisy decision threshold.

A computational relationship between thalamic sensory neural responses and contrast perception

Uncovering the relationship between sensory neural responses and perceptual decisions remains a fundamental problem in neuroscience. Decades of experimental and modeling work in the sensory cortex have demonstrated that a perceptual decision pool is usually composed of tens to hundreds of neurons, the responses of which are significantly correlated not only with each other, but also with the behavioral choices of an animal. Few studies, however, have measured neural activity in the sensory thalamus of awake, behaving animals. Therefore, it remains unclear how many thalamic neurons are recruited and how the information from these neurons is pooled at subsequent cortical stages to form a perceptual decision. In a previous study we measured neural activity in the macaque lateral geniculate nucleus (LGN) during a two alternative forced choice (2AFC) contrast detection task, and found that single LGN neurons were significantly correlated with the monkeys’ behavioral choices, despite their relatively poor contrast sensitivity and a lack of overall interneuronal correlations. We have now computationally tested a number of specific hypotheses relating these measured LGN neural responses to the contrast detection behavior of the animals. We modeled the perceptual decisions with different numbers of neurons and using a variety of pooling/readout strategies, and found that the most successful model consisted of about 50–200 LGN neurons, with individual neurons weighted differentially according to their signal-to-noise ratios (quantified as d-primes). These results supported the hypothesis that in contrast detection the perceptual decision pool consists of multiple thalamic neurons, and that the response fluctuations in these neurons can influence contrast perception, with the more sensitive thalamic neurons likely to exert a greater influence.

Insights into decision making using choice probability

A long-standing question in systems neuroscience is how the activity of single neurons gives rise to our perceptions and actions. Critical insights into this question occurred in the last part of the 20th century when scientists began linking modulations of neuronal activity directly to perceptual behavior. A significant conceptual advance was the application of signal detection theory to both neuronal activity and behavior, providing a quantitative assessment of the relationship between brain and behavior. One metric that emerged from these efforts was choice probability (CP), which provides information about how well an ideal observer can predict the choice an animal makes from a neuron's discharge rate distribution. In this review, we describe where CP has been studied, locational trends in the values found, and why CP values are typically so low. We discuss its dependence on correlated activity among neurons of a population, assess whether it arises from feedforward or feedback mechanisms, and investigate what CP tells us about how many neurons are required for a decision and how they are pooled to do so.

A Modality-Specific Feedforward Component of Choice-Related Activity in MT

The activity of individual sensory neurons can be predictive of an animal's choices. These decision signals arise from network properties dependent on feedforward and feedback inputs; however, the relative contributions of these inputs are poorly understood. We determined the role of feedforward pathways to decision signals in MT by recording neuronal activity while monkeys performed motion and depth tasks. During each session, we reversibly inactivated V2 and V3, which provide feedforward input to MT that conveys more information about depth than motion. We thus monitored the choice-related activity of the same neuron both before and during V2/V3 inactivation. During inactivation, MT neurons became less predictive of decisions for the depth task but not the motion task, indicating that a feedforward pathway that gives rise to tuning preferences also contributes to decision signals. We show that our data are consistent with V2/V3 input conferring structured noise correlations onto the MT population.

The functional asymmetry of ON and OFF channels in the perception of contrast

To fully understand the relationship between perception and single neural responses, one should take into consideration the early stages of sensory processing. Few studies, however, have directly examined the neural underpinning of visual perception in the lateral geniculate nucleus (LGN), only one synapse away from the retina. In this study we recorded from LGN parvocellular (P) ON-center and OFF-center neurons while monkeys either passively viewed or actively detected a full range of contrasts. We found that OFF neurons were more sensitive in detecting negative contrasts than ON neurons were in detecting positive contrasts. Also, OFF neurons had higher spontaneous activities, higher peak response amplitudes, and were more sustained than ON neurons in their contrast responses. Puzzlingly, OFF neurons failed to show any significant correlations with the monkeys' perceptual choices, despite their greater contrast sensitivities. If, however, choice probabilities were calculated from interspike intervals instead of spike counts (thus taking into account the higher firing rates of OFF neurons), OFF neurons but not ON neurons were significantly correlated with behavioral choices. Taken together, these results demonstrate in awake, behaving animals that: 1) the ON and OFF pathways do not simply mirror each other in their functionality but instead carry qualitatively different types of information, and 2) the responses of ON and OFF neurons can be correlated with perceptual choices even in the absence of physical stimuli and interneuronal correlations.

Effects of Microstimulation in the Anterior Intraparietal Area during Three-Dimensional Shape Categorization

The anterior intraparietal area (AIP) of rhesus monkeys is part of the dorsal visual stream and contains neurons whose visual response properties are commensurate with a role in three-dimensional (3D) shape perception. Neuronal responses in AIP signal the depth structure of disparity-defined 3D shapes, reflect the choices of monkeys while they categorize 3D shapes, and mirror the behavioral variability across different stimulus conditions during 3D-shape categorization. However, direct evidence for a role of AIP in 3D-shape perception has been lacking. We trained rhesus monkeys to categorize disparity-defined 3D shapes and examined AIP's contribution to 3D-shape categorization by microstimulating in clusters of 3D-shape selective AIP neurons during task performance. We find that microstimulation effects on choices (monkey M1) and reaction times (monkey M1 and M2) depend on the 3D-shape preference of the stimulated site. Moreover, electrical stimulation of the same cells, during either the 3D-shape-categorization task or a saccade task, could affect behavior differently. Interestingly, in one monkey we observed a strong correlation between the strength of choice-related AIP activity (choice probabilities) and the influence of microstimulation on 3D-shape-categorization behavior (choices and reaction time). These findings propose AIP as part of the network responsible for 3D-shape perception. The results also show that the anterior intraparietal cortex contains cells with different tuning properties, i.e. 3D-shape- or saccade-related, that can be dynamically read out depending on the requirements of the task at hand.

Perceptual decision related activity in the lateral geniculate nucleus

Fundamental to neuroscience is the understanding of how the language of neurons relates to behavior. In the lateral geniculate nucleus (LGN), cells show distinct properties such as selectivity for particular wavelengths, increments or decrements in contrast, or preference for fine detail versus rapid motion. No studies, however, have measured how LGN cells respond when an animal is challenged to make a perceptual decision using information within the receptive fields of those LGN cells. In this study we measured neural activity in the macaque LGN during a two-alternative, forced-choice (2AFC) contrast detection task or during a passive fixation task and found that a small proportion (13.5%) of single LGN parvocellular (P) and magnocellular (M) neurons matched the psychophysical performance of the monkey. The majority of LGN neurons measured in both tasks were not as sensitive as the monkey. The covariation between neural response and behavior (quantified as choice probability) was significantly above chance during active detection, even when there was no external stimulus. Interneuronal correlations and task-related gain modulations were negligible under the same condition. A bottom-up pooling model that used sensory neural responses to compute perceptual choices in the absence of interneuronal correlations could fully explain these results at the level of the LGN, supporting the hypothesis that the perceptual decision pool consists of multiple sensory neurons and that response fluctuations in these neurons can influence perception.

Effects of attention and distractor contrast on the responses of middle temporal area neurons to transient motion direction changes

The ability of primates to detect transient changes in a visual scene can be influenced by the allocation of attention, as well as by the presence of distractors. We investigated the neural substrates of these effects by recording the responses of neurons in the middle temporal area (MT) of two monkeys while they detected a transient motion direction change in a moving target. We found that positioning a distractor near the target impaired the change-detection performance of the animals. This impairment monotonically decreased as the distractor's contrast decreased. A neural correlate of this effect was a decrease in the ability of MT neurons to signal the direction change (detection sensitivity or DS) when a distractor was near the target, both located inside the neuron's receptive field. Moreover, decreasing distractor contrast increased neuronal DS. On the other hand, directing attention away from the target decreased neuronal DS. At the level of individual neurons, we found a negative correlation between the degree of response normalization and the DS. Finally, the intensity of a neuron's response to the change was predictive of the animal's reaction time, suggesting that the activity of our recorded neurons was linked to the animal's detection performance. Our results suggest that the ability of an MT neuron to signal a transient direction change is regulated by the degree of inhibitory drive into the cell. The presence of distractors, their contrast and the allocation of attention influence such inhibitory drive, therefore modulating the ability of the neurons to signal transient changes in stimulus features and consequently behavioral performance.

On the number of neurons and time scale of integration underlying the formation of percepts in the brain

All of our perceptual experiences arise from the activity of neural populations. Here we study the formation of such percepts under the assumption that they emerge from a linear readout, i.e., a weighted sum of the neurons’ firing rates. We show that this assumption constrains the trial-to-trial covariance structure of neural activities and animal behavior. The predicted covariance structure depends on the readout parameters, and in particular on the temporal integration window w and typical number of neurons K used in the formation of the percept. Using these predictions, we show how to infer the readout parameters from joint measurements of a subject’s behavior and neural activities. We consider three such scenarios: (1) recordings from the complete neural population, (2) recordings of neuronal sub-ensembles whose size exceeds K, and (3) recordings of neuronal sub-ensembles that are smaller than K. Using theoretical arguments and artificially generated data, we show that the first two scenarios allow us to recover the typical spatial and temporal scales of the readout. In the third scenario, we show that the readout parameters can only be recovered by making additional assumptions about the structure of the full population activity. Our work provides the first thorough interpretation of (feed-forward) percept formation from a population of sensory neurons. We discuss applications to experimental recordings in classic sensory decision-making tasks, which will hopefully provide new insights into the nature of perceptual integration.

This article deals with the interpretation of neural activities during perceptual decision-making tasks, where animals must assess the value of a sensory stimulus and take a decision on the basis of their percept. A “standard model” for these tasks has progressively emerged, whence the animal’s percept and subsequent choice on each trial are obtained from a linear integration of the activity of sensory neurons. However, up to date, there has been no principled method to estimate the parameters of this model: mainly, the typical number of neurons K from the population involved in conveying the percept, and the typical time scale w during which these neurons’ activities are integrated. In this article, we propose a novel method to estimate these quantities from experimental data, and thus assess the validity of the standard model of percept formation. In the process, we clarify the predictions of the standard model regarding two classic experimental measures in these tasks: sensitivity, which is the animal’s ability to distinguish nearby stimulus values, and choice signals, which assess the amount of correlation between the activity of single neurons and the animal’s ultimate choice on each trial.

Choice-related Activity in the Anterior Intraparietal Area during 3-D Structure Categorization

The anterior intraparietal area (AIP) of macaques contains neurons that signal the depth structure of disparity-defined 3-D shapes. Previous studies have suggested that AIP's depth information is used for sensorimotor transformations related to the efficient grasping of 3-D objects. We trained monkeys to categorize disparity-defined 3-D shapes and examined whether neuronal activity in AIP may also underlie pure perceptual categorization behavior. We first show that neurons with a similar 3-D shape preference cluster in AIP. We then demonstrate that the monkeys' 3-D shape discrimination performance depends on the position in depth of the stimulus and that this performance difference is reflected in the activity of AIP neurons. We further reveal correlations between the neuronal activity in AIP and the subject's subsequent choices and RTs during 3-D shape categorization. Our findings propose AIP as an important processing stage for 3-D shape perception.

Feature attention for binocular disparity in primate area MT depends on tuning strength

Attending to a stimulus modulates the responses of sensory neurons that represent features of that stimulus, a phenomenon named "feature attention." For example, attending to a stimulus containing upward motion enhances the responses of upward-preferring direction-selective neurons in the middle temporal area (MT) and suppresses the responses of downward-preferring neurons, even when the attended stimulus is outside of the spatial receptive fields of the recorded neurons (Treue S, Martinez-Trujillo JC. Nature 399: 575-579, 1999). This modulation renders the representation of sensory information across a neuronal population more selective for the features present in the attended stimulus (Martinez-Trujillo JC, Treue S. Curr Biol 14: 744-751, 2004). We hypothesized that if feature attention modulates neurons according to their tuning preferences, it should also be sensitive to their tuning strength, which is the magnitude of the difference in responses to preferred and null stimuli. We measured how the effects of feature attention on MT neurons in rhesus monkeys (Macaca mulatta) depended on the relationship between features-in our case, direction of motion and binocular disparity-of the attended stimulus and a neuron's tuning for those features. We found that, as for direction, attention to stimuli containing binocular disparity cues modulated the responses of MT neurons and that the magnitude of the modulation depended on both a neuron's tuning preferences and its tuning strength. Our results suggest that modulation by feature attention may depend not just on which features a neuron represents but also on how well the neuron represents those features.

Transient activity in monkey area MT represents speed changes and is correlated with human behavioral performance

Neurons in the middle temporal area (MT) respond to motion onsets and speed changes with a transient-sustained firing pattern. The latency of the transient response has recently been shown to correlate with reaction time in a speed change detection task, but it is not known how the sign, the amplitude, and the latency of this response depend on the sign and the magnitude of a speed change, and whether these transients can be decoded to explain speed change detection behavior. To investigate this issue, we measured the neuronal representation of a wide range of positive and negative speed changes in area MT of fixating macaques and obtained three major findings. First, speed change transients not only reflect a neuron's absolute speed tuning but are shaped by an additional gain that scales the tuned response according to the magnitude of a relative speed change. Second, by means of a threshold model positive and negative population transients of a moderate number of MT neurons explain detection of both positive and negative speed changes, respectively, at a level comparable to human detection rates under identical visual stimulation. Third, like reaction times in a psychophysical model of velocity detection, speed change response latencies follow a power-law function of the absolute difference of a speed change. Both this neuronal representation and its close correlation with behavioral measures of speed change detection suggest that neuronal transients in area MT facilitate the detection of rapid changes in visual input.

ROC-based estimates of neural-behavioral covariations using matched filters

Correlations between responses in visual cortex and perceptual performance help draw a functional link between neural activity and visually guided behavior. These correlations are commonly derived with ROC-based neural-behavioral covariances (referred to as choice or detect probability) using boxcar analysis windows. Although boxcar windows capture the covariation between neural activity and behavior during steady-state stimulus presentations, they are not optimized to capture these correlations during short time-varying visual inputs. In this study, we implemented a matched-filter technique, combined with cross-validation, to improve the estimation of ROC-based neural-behavioral covariance under short and dynamic stimulus conditions. We show that this approach maximizes the area under the ROC curve and converges to the true neural-behavioral covariance using a Poisson spiking model. We also demonstrate that the matched filter, combined with cross-validation, reveals the dynamics of the neural-behavioral covariations of individual MT neurons during the detection of a brief motion stimulus.

Identifying and quantifying multisensory integration: a tutorial review

We process information from the world through multiple senses, and the brain must decide what information belongs together and what information should be segregated. One challenge in studying such multisensory integration is how to quantify the multisensory interactions, a challenge that is amplified by the host of methods that are now used to measure neural, behavioral, and perceptual responses. Many of the measures that have been developed to quantify multisensory integration (and which have been derived from single unit analyses), have been applied to these different measures without much consideration for the nature of the process being studied. Here, we provide a review focused on the means with which experimenters quantify multisensory processes and integration across a range of commonly used experimental methodologies. We emphasize the most commonly employed measures, including single- and multiunit responses, local field potentials, functional magnetic resonance imaging, and electroencephalography, along with behavioral measures of detection, accuracy, and response times. In each section, we will discuss the different metrics commonly used to quantify multisensory interactions, including the rationale for their use, their advantages, and the drawbacks and caveats associated with them. Also discussed are possible alternatives to the most commonly used metrics.

Adaptation to speed in macaque middle temporal and medial superior temporal areas

The response of a sensory neuron to an unchanging stimulus typically adapts, showing decreases in response gain that are accompanied by changes in the shape of tuning curves. It remains unclear whether these changes arise purely due to spike rate adaptation within single neurons or whether they are dependent on network interactions between neurons. Further, it is unclear how the timescales of neural and perceptual adaptation are related. To examine this issue, we compared speed tuning of middle temporal (MT) and medial superior temporal neurons in macaque visual cortex after adaptation to two different reference speeds. For 75% of speed-tuned units, adaptation caused significant changes in tuning that could be explained equally well as lateral shifts, vertical gain changes, or both. These tuning changes occurred rapidly, as both neuronal firing rate and Fano factor showed no evidence of changing beyond the first 500 ms after motion onset, and the magnitude of tuning curve changes showed no difference between trials with adaptation durations shorter or longer than 1 s. Importantly, the magnitude of tuning shifts was correlated with the transient-sustained index, which measures a well characterized form of rapid response adaptation in MT, and is likely associated with changes at the level of neuronal networks. Tuning curves changed in a manner that increased neuronal sensitivity around the adapting speed, consistent with improvements in human and macaque psychophysical performance that we observed over the first several hundred ms of adaptation.

Decision making by urgency gating: theory and experimental support

It is often suggested that decisions are made when accumulated sensory information reaches a fixed accuracy criterion. This is supported by many studies showing a gradual build up of neural activity to a threshold. However, the proposal that this build up is caused by sensory accumulation is challenged by findings that decisions are based on information from a time window much shorter than the build-up process. Here, we propose that in natural conditions where the environment can suddenly change, the policy that maximizes reward rate is to estimate evidence by accumulating only novel information and then compare the result to a decreasing accuracy criterion. We suggest that the brain approximates this policy by multiplying an estimate of sensory evidence with a motor-related urgency signal and that the latter is primarily responsible for neural activity build up. We support this hypothesis using human behavioral data from a modified random-dot motion task in which motion coherence changes during each trial.

Population-wide distributions of neural activity during perceptual decision-making

Cortical activity involves large populations of neurons, even when it is limited to functionally coherent areas. Electrophysiological recordings, on the other hand, involve comparatively small neural ensembles, even when modern-day techniques are used. Here we review results which have started to fill the gap between these two scales of inquiry, by shedding light on the statistical distributions of activity in large populations of cells. We put our main focus on data recorded in awake animals that perform simple decision-making tasks and consider statistical distributions of activity throughout cortex, across sensory, associative, and motor areas. We transversally review the complexity of these distributions, from distributions of firing rates and metrics of spike-train structure, through distributions of tuning to stimuli or actions and of choice signals, and finally the dynamical evolution of neural population activity and the distributions of (pairwise) neural interactions. This approach reveals shared patterns of statistical organization across cortex, including: (i) long-tailed distributions of activity, where quasi-silence seems to be the rule for a majority of neurons; that are barely distinguishable between spontaneous and active states; (ii) distributions of tuning parameters for sensory (and motor) variables, which show an extensive extrapolation and fragmentation of their representations in the periphery; and (iii) population-wide dynamics that reveal rotations of internal representations over time, whose traces can be found both in stimulus-driven and internally generated activity. We discuss how these insights are leading us away from the notion of discrete classes of cells, and are acting as powerful constraints on theories and models of cortical organization and population coding.

Potential confounds in estimating trial-to-trial correlations between neuronal response and behavior using choice probabilities

Correlations between trial-to-trial fluctuations in the responses of individual sensory neurons and perceptual reports, commonly quantified with choice probability (CP), have been widely used as an important tool for assessing the contributions of neurons to behavior. These correlations are usually weak and often require a large number of trials for a reliable estimate. Therefore, working with measures such as CP warrants care in data analysis as well as rigorous controls during data collection. Here we identify potential confounds that can arise in data analysis and lead to biased estimates of CP, and suggest methods to avoid the bias. In particular, we show that the common practice of combining neuronal responses across different stimulus conditions with z-score normalization can result in an underestimation of CP when the ratio of the numbers of trials for the two behavioral response categories differs across the stimulus conditions. We also discuss the effects of using variable time intervals for quantifying neuronal response on CP measurements. Finally, we demonstrate that serious artifacts can arise in reaction time tasks that use varying measurement intervals if the mean neuronal response and mean behavioral performance vary over time within trials. To emphasize the importance of addressing these concerns in neurophysiological data, we present a set of data collected from V1 cells in macaque monkeys while the animals performed a detection task.

Decision-related activity in sensory neurons: correlations among neurons and with behavior

Neurons in early sensory cortex show weak but systematic correlations with perceptual decisions when trained animals perform at psychophysical threshold. These correlations are observed across repeated presentations of identical stimuli and cannot be explained by variation in external factors. The relationship between the activity of individual sensory neurons and the animal's behavioral choice means that even neurons in early sensory cortex carry information about an upcoming decision. This relationship, termed choice probability, may reflect the effect of fluctuations in neuronal firing rate on the animal's decision, but it can also reflect modulation of sensory responses by cognitive factors, or network properties such as variability that is shared among populations of neurons. Here, we review recent work clarifying the relationship among fluctuations in the responses of individual neurons, correlated variability, and behavior in a variety of tasks and cortical areas. We also discuss the possibility that choice probability may in part reflect the influence of cognitive factors on sensory neurons and explore the situations in which choice probability can be used to make inferences about the role of particular sensory neurons in the decision-making process.

The functional link between area MT neural fluctuations and detection of a brief motion stimulus

Fluctuations of neural firing rates in visual cortex are known to be correlated with variations in perceptual performance. It is important to know whether these fluctuations are functionally linked to perception in a causal manner or instead reflect non-causal processes that arise after the perceptual decision is made. We recorded from middle temporal (MT) neurons from monkey subjects while they detected the random occurrence of a brief 50 ms motion pulse that occurred in either of two (or simultaneously in both) random dot patches located in the same hemisphere. The receptive field parameters of the motion pulse were matched to that preferred by each MT neuron under study. This task contained uncertainty in both space and time because, on any given trial, the subjects did not know which patch would contain the motion pulse or when the motion pulse would occur. Covariations between MT activity and behavior began just before the motion pulse onset and peaked at the maximum neural response. These neural-behavioral covariations were strongest when only one patch contained the motion pulse and were still weakly present when a patch did not contain a motion pulse. A feedforward temporal integration model with two independent detector channels captured both the detection performance and evolution of the neural-behavior covariations over time and stimulus condition. The results suggest that, when detecting a brief visual stimulus, there is a causal relationship between fluctuations in neural activity and variations in behavior across trials.

Ensemble encoding of nociceptive stimulus intensity in the rat medial and lateral pain systems

BACKGROUND

The ability to encode noxious stimulus intensity is essential for the neural processing of pain perception. It is well accepted that the intensity information is transmitted within both sensory and affective pathways. However, it remains unclear what the encoding patterns are in the thalamocortical brain regions, and whether the dual pain systems share similar responsibility in intensity coding.

RESULTS

Multichannel single-unit recordings were used to investigate the activity of individual neurons and neuronal ensembles in the rat brain following the application of noxious laser stimuli of increasing intensity to the hindpaw. Four brain regions were monitored, including two within the lateral sensory pain pathway, namely, the ventral posterior lateral thalamic nuclei and the primary somatosensory cortex, and two in the medial pathway, namely, the medial dorsal thalamic nuclei and the anterior cingulate cortex. Neuron number, firing rate, and ensemble spike count codings were examined in this study. Our results showed that the noxious laser stimulation evoked double-peak responses in all recorded brain regions. Significant correlations were found between the laser intensity and the number of responsive neurons, the firing rates, as well as the mass spike counts (MSCs). MSC coding was generally more efficient than the other two methods. Moreover, the coding capacities of neurons in the two pathways were comparable.

CONCLUSION

This study demonstrated the collective contribution of medial and lateral pathway neurons to the noxious intensity coding. Additionally, we provide evidence that ensemble spike count may be the most reliable method for coding pain intensity in the brain.

Measuring and interpreting neuronal correlations

Mounting evidence suggests that understanding how the brain encodes information and performs computations will require studying the correlations between neurons. The recent advent of recording techniques such as multielectrode arrays and two-photon imaging has made it easier to measure correlations, opening the door for detailed exploration of their properties and contributions to cortical processing. However, studies have reported discrepant findings, providing a confusing picture. Here we briefly review these studies and conduct simulations to explore the influence of several experimental and physiological factors on correlation measurements. Differences in response strength, the time window over which spikes are counted, spike sorting conventions and internal states can all markedly affect measured correlations and systematically bias estimates. Given these complicating factors, we offer guidelines for interpreting correlation data and a discussion of how best to evaluate the effect of correlations on cortical processing.