Choice-related activity and correlated noise in subcortical vestibular neurons

Neural populations within macaque early vestibular pathways are adapted to encode natural self-motion

How the activities of large neural populations are integrated in the brain to ensure accurate perception and behavior remains a central problem in systems neuroscience. Here, we investigated population coding of naturalistic self-motion by neurons within early vestibular pathways in rhesus macaques (Macacca mulatta). While vestibular neurons displayed similar dynamic tuning to self-motion, inspection of their spike trains revealed significant heterogeneity. Further analysis revealed that, during natural but not artificial stimulation, heterogeneity resulted primarily from variability across neurons as opposed to trial-to-trial variability. Interestingly, vestibular neurons displayed different correlation structures during naturalistic and artificial self-motion. Specifically, while correlations due to the stimulus (i.e., signal correlations) did not differ, correlations between the trial-to-trial variabilities of neural responses (i.e., noise correlations) were instead significantly positive during naturalistic but not artificial stimulation. Using computational modeling, we show that positive noise correlations during naturalistic stimulation benefits information transmission by heterogeneous vestibular neural populations. Taken together, our results provide evidence that neurons within early vestibular pathways are adapted to the statistics of natural self-motion stimuli at the population level. We suggest that similar adaptations will be found in other systems and species.

Brain-wide neural activity underlying memory-guided movement

Behavior relies on activity in structured neural circuits that are distributed across the brain, but most experiments probe neurons in a single area at a time. Using multiple Neuropixels probes, we recorded from multi-regional loops connected to the anterior lateral motor cortex (ALM), a circuit node mediating memory-guided directional licking. Neurons encoding sensory stimuli, choices, and actions were distributed across the brain. However, choice coding was concentrated in the ALM and subcortical areas receiving input from the ALM in an ALM-dependent manner. Diverse orofacial movements were encoded in the hindbrain; midbrain; and, to a lesser extent, forebrain. Choice signals were first detected in the ALM and the midbrain, followed by the thalamus and other brain areas. At movement initiation, choice-selective activity collapsed across the brain, followed by new activity patterns driving specific actions. Our experiments provide the foundation for neural circuit models of decision-making and movement initiation.

Self-motion perception and sequential decision-making: where are we heading?

To navigate and guide adaptive behaviour in a dynamic environment, animals must accurately estimate their own motion relative to the external world. This is a fundamentally multisensory process involving integration of visual, vestibular and kinesthetic inputs. Ideal observer models, paired with careful neurophysiological investigation, helped to reveal how visual and vestibular signals are combined to support perception of linear self-motion direction, or heading. Recent work has extended these findings by emphasizing the dimension of time, both with regard to stimulus dynamics and the trade-off between speed and accuracy. Both time and certainty-i.e. the degree of confidence in a multisensory decision-are essential to the ecological goals of the system: terminating a decision process is necessary for timely action, and predicting one's accuracy is critical for making multiple decisions in a sequence, as in navigation. Here, we summarize a leading model for multisensory decision-making, then show how the model can be extended to study confidence in heading discrimination. Lastly, we preview ongoing efforts to bridge self-motion perception and navigation per se, including closed-loop virtual reality and active self-motion. The design of unconstrained, ethologically inspired tasks, accompanied by large-scale neural recordings, raise promise for a deeper understanding of spatial perception and decision-making in the behaving animal. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Temporal and spatial properties of vestibular signals for perception of self-motion

It is well recognized that the vestibular system is involved in numerous important cognitive functions, including self-motion perception, spatial orientation, locomotion, and vector-based navigation, in addition to basic reflexes, such as oculomotor or body postural control. Consistent with this rationale, vestibular signals exist broadly in the brain, including several regions of the cerebral cortex, potentially allowing tight coordination with other sensory systems to improve the accuracy and precision of perception or action during self-motion. Recent neurophysiological studies in animal models based on single-cell resolution indicate that vestibular signals exhibit complex spatiotemporal dynamics, producing challenges in identifying their exact functions and how they are integrated with other modality signals. For example, vestibular and optic flow could provide congruent and incongruent signals regarding spatial tuning functions, reference frames, and temporal dynamics. Comprehensive studies, including behavioral tasks, neural recording across sensory and sensory-motor association areas, and causal link manipulations, have provided some insights into the neural mechanisms underlying multisensory self-motion perception.

Descending pathways increase sensory neural response heterogeneity to facilitate decoding and behavior

The functional role of heterogeneous spiking responses of otherwise similarly tuned neurons to stimulation, which has been observed ubiquitously, remains unclear to date. Here, we demonstrate that such response heterogeneity serves a beneficial function that is used by downstream brain areas to generate behavioral responses that follows the detailed timecourse of the stimulus. Multi-unit recordings from sensory pyramidal cells within the electrosensory system of Apteronotus leptorhynchus were performed and revealed highly heterogeneous responses that were similar for all cell types. By comparing the coding properties of a given neural population before and after inactivation of descending pathways, we found that heterogeneities were beneficial as decoding was then more robust to the addition of noise. Taken together, our results not only reveal that descending pathways actively promote response heterogeneity within a given cell type, but also uncover a beneficial function for such heterogeneity that is used by the brain to generate behavior.

Noise and vestibular perception of passive self-motion

Noise defined as random disturbances is ubiquitous in both the external environment and the nervous system. Depending on the context, noise can degrade or improve information processing and performance. In all cases, it contributes to neural systems dynamics. We review some effects of various sources of noise on the neural processing of self-motion signals at different stages of the vestibular pathways and the resulting perceptual responses. Hair cells in the inner ear reduce the impact of noise by means of mechanical and neural filtering. Hair cells synapse on regular and irregular afferents. Variability of discharge (noise) is low in regular afferents and high in irregular units. The high variability of irregular units provides information about the envelope of naturalistic head motion stimuli. A subset of neurons in the vestibular nuclei and thalamus are optimally tuned to noisy motion stimuli that reproduce the statistics of naturalistic head movements. In the thalamus, variability of neural discharge increases with increasing motion amplitude but saturates at high amplitudes, accounting for behavioral violation of Weber's law. In general, the precision of individual vestibular neurons in encoding head motion is worse than the perceptual precision measured behaviorally. However, the global precision predicted by neural population codes matches the high behavioral precision. The latter is estimated by means of psychometric functions for detection or discrimination of whole-body displacements. Vestibular motion thresholds (inverse of precision) reflect the contribution of intrinsic and extrinsic noise to perception. Vestibular motion thresholds tend to deteriorate progressively after the age of 40 years, possibly due to oxidative stress resulting from high discharge rates and metabolic loads of vestibular afferents. In the elderly, vestibular thresholds correlate with postural stability: the higher the threshold, the greater is the postural imbalance and risk of falling. Experimental application of optimal levels of either galvanic noise or whole-body oscillations can ameliorate vestibular function with a mechanism reminiscent of stochastic resonance. Assessment of vestibular thresholds is diagnostic in several types of vestibulopathies, and vestibular stimulation might be useful in vestibular rehabilitation.

Hierarchical differences in the encoding of sound and choice in the subcortical auditory system

Detection of sounds is a fundamental function of the auditory system. Although studies of auditory cortex have gained substantial insight into detection performance using behaving animals, previous subcortical studies have mostly taken place under anesthesia, in passively listening animals, or have not measured performance at threshold. These limitations preclude direct comparisons between neuronal responses and behavior. To address this, we simultaneously measured auditory detection performance and single-unit activity in the inferior colliculus (IC) and cochlear nucleus (CN) in macaques. The spontaneous activity and response variability of CN neurons were higher than those observed for IC neurons. Signal detection theoretic methods revealed that the magnitude of responses of IC neurons provided more reliable estimates of psychometric threshold and slope compared with the responses of single CN neurons. However, pooling small populations of CN neurons provided reliable estimates of psychometric threshold and slope, suggesting sufficient information in CN population activity. Trial-by-trial correlations between spike count and behavioral response emerged 50-75 ms after sound onset for most IC neurons, but for few neurons in the CN. These results highlight hierarchical differences between neurometric-psychometric correlations in CN and IC and have important implications for how subcortical information could be decoded.NEW & NOTEWORTHY The cerebral cortex is widely recognized to play a role in sensory processing and decision-making. Accounts of the neural basis of auditory perception and its dysfunction are based on this idea. However, significantly less attention has been paid to midbrain and brainstem structures in this regard. Here, we find that subcortical auditory neurons represent stimulus information sufficient for detection and predict behavioral choice on a trial-by-trial basis.

Parallel processing, hierarchical transformations, and sensorimotor associations along the 'where' pathway

Visually guided behaviors require the brain to transform ambiguous retinal images into object-level spatial representations and implement sensorimotor transformations. These processes are supported by the dorsal ‘where’ pathway. However, the specific functional contributions of areas along this pathway remain elusive due in part to methodological differences across studies. We previously showed that macaque caudal intraparietal (CIP) area neurons possess robust 3D visual representations, carry choice- and saccade-related activity, and exhibit experience-dependent sensorimotor associations (Chang et al., 2020b). Here, we used a common experimental design to reveal parallel processing, hierarchical transformations, and the formation of sensorimotor associations along the ‘where’ pathway by extending the investigation to V3A, a major feedforward input to CIP. Higher-level 3D representations and choice-related activity were more prevalent in CIP than V3A. Both areas contained saccade-related activity that predicted the direction/timing of eye movements. Intriguingly, the time course of saccade-related activity in CIP aligned with the temporally integrated V3A output. Sensorimotor associations between 3D orientation and saccade direction preferences were stronger in CIP than V3A, and moderated by choice signals in both areas. Together, the results explicate parallel representations, hierarchical transformations, and functional associations of visual and saccade-related signals at a key juncture in the ‘where’ pathway.

Bridging neuronal correlations and dimensionality reduction

Two commonly used approaches to study interactions among neurons are spike count correlation, which describes pairs of neurons, and dimensionality reduction, applied to a population of neurons. Although both approaches have been used to study trial-to-trial neuronal variability correlated among neurons, they are often used in isolation and have not been directly related. We first established concrete mathematical and empirical relationships between pairwise correlation and metrics of population-wide covariability based on dimensionality reduction. Applying these insights to macaque V4 population recordings, we found that the previously reported decrease in mean pairwise correlation associated with attention stemmed from three distinct changes in population-wide covariability. Overall, our work builds the intuition and formalism to bridge between pairwise correlation and population-wide covariability and presents a cautionary tale about the inferences one can make about population activity by using a single statistic, whether it be mean pairwise correlation or dimensionality.

How multisensory neurons solve causal inference

We perceive our environment through multiple independent sources of sensory input. The brain is tasked with deciding whether multiple signals are produced by the same or different events (i.e., solve the problem of causal inference). Here, we train a neural network to solve causal inference by either combining or separating visual and vestibular inputs in order to estimate self- and scene motion. We find that the network recapitulates key neurophysiological (i.e., congruent and opposite neurons) and behavioral (e.g., reliability-based cue weighting) properties of biological systems. We show how congruent and opposite neurons support motion estimation and how the balance in activity between these subpopulations determines whether to combine or separate multisensory signals.

Sitting in a static railway carriage can produce illusory self-motion if the train on an adjoining track moves off. While our visual system registers motion, vestibular signals indicate that we are stationary. The brain is faced with a difficult challenge: is there a single cause of sensations (I am moving) or two causes (I am static, another train is moving)? If a single cause, integrating signals produces a more precise estimate of self-motion, but if not, one cue should be ignored. In many cases, this process of causal inference works without error, but how does the brain achieve it? Electrophysiological recordings show that the macaque medial superior temporal area contains many neurons that encode combinations of vestibular and visual motion cues. Some respond best to vestibular and visual motion in the same direction (“congruent” neurons), while others prefer opposing directions (“opposite” neurons). Congruent neurons could underlie cue integration, but the function of opposite neurons remains a puzzle. Here, we seek to explain this computational arrangement by training a neural network model to solve causal inference for motion estimation. Like biological systems, the model develops congruent and opposite units and recapitulates known behavioral and neurophysiological observations. We show that all units (both congruent and opposite) contribute to motion estimation. Importantly, however, it is the balance between their activity that distinguishes whether visual and vestibular cues should be integrated or separated. This explains the computational purpose of puzzling neural representations and shows how a relatively simple feedforward network can solve causal inference.

Sensory coding and the causal impact of mouse cortex in a visual decision

Correlates of sensory stimuli and motor actions are found in multiple cortical areas, but such correlates do not indicate whether these areas are causally relevant to task performance. We trained mice to discriminate visual contrast and report their decision by steering a wheel. Widefield calcium imaging and Neuropixels recordings in cortex revealed stimulus-related activity in visual (VIS) and frontal (MOs) areas, and widespread movement-related activity across the whole dorsal cortex. Optogenetic inactivation biased choices only when targeted at VIS and MOs,proportionally to each site's encoding of the visual stimulus, and at times corresponding to peak stimulus decoding. A neurometric model based on summing and subtracting activity in VIS and MOs successfully described behavioral performance and predicted the effect of optogenetic inactivation. Thus, sensory signals localized in visual and frontal cortex play a causal role in task performance, while widespread dorsal cortical signals correlating with movement reflect processes that do not play a causal role.

Synergistic population coding of natural communication stimuli by hindbrain electrosensory neurons

Understanding how neural populations encode natural stimuli with complex spatiotemporal structure to give rise to perception remains a central problem in neuroscience. Here we investigated population coding of natural communication stimuli by hindbrain neurons within the electrosensory system of weakly electric fish Apteronotus leptorhynchus. Overall, we found that simultaneously recorded neural activities were correlated: signal but not noise correlations were variable depending on the stimulus waveform as well as the distance between neurons. Combining the neural activities using an equal-weight sum gave rise to discrimination performance between different stimulus waveforms that was limited by redundancy introduced by noise correlations. However, using an evolutionary algorithm to assign different weights to individual neurons before combining their activities (i.e., a weighted sum) gave rise to increased discrimination performance by revealing synergistic interactions between neural activities. Our results thus demonstrate that correlations between the neural activities of hindbrain electrosensory neurons can enhance information about the structure of natural communication stimuli that allow for reliable discrimination between different waveforms by downstream brain areas.

Population Coding of Natural Electrosensory Stimuli by Midbrain Neurons

Natural stimuli display spatiotemporal characteristics that typically vary over orders of magnitude, and their encoding by sensory neurons remains poorly understood. We investigated population coding of highly heterogeneous natural electrocommunication stimuli in Apteronotus leptorhynchus of either sex. Neuronal activities were positively correlated with one another in the absence of stimulation, and correlation magnitude decayed with increasing distance between recording sites. Under stimulation, we found that correlations between trial-averaged neuronal responses (i.e., signal correlations) were positive and higher in magnitude for neurons located close to another, but that correlations between the trial-to-trial variability (i.e., noise correlations) were independent of physical distance. Overall, signal and noise correlations were independent of stimulus waveform as well as of one another. To investigate how neuronal populations encoded natural electrocommunication stimuli, we considered a nonlinear decoder for which the activities were combined. Decoding performance was best for a timescale of 6 ms, indicating that midbrain neurons transmit information via precise spike timing. A simple summation of neuronal activities (equally weighted sum) revealed that noise correlations limited decoding performance by introducing redundancy. Using an evolution algorithm to optimize performance when considering instead unequally weighted sums of neuronal activities revealed much greater performance values, indicating that midbrain neuron populations transmit information that reliably enable discrimination between different stimulus waveforms. Interestingly, we found that different weight combinations gave rise to similar discriminability, suggesting robustness. Our results have important implications for understanding how natural stimuli are integrated by downstream brain areas to give rise to behavioral responses.SIGNIFICANCE STATEMENT We show that midbrain electrosensory neurons display correlations between their activities and that these can significantly impact performance of decoders. While noise correlations limited discrimination performance by introducing redundancy, considering unequally weighted sums of neuronal activities gave rise to much improved performance and mitigated the deleterious effects of noise correlations. Further analysis revealed that increased discriminability was achieved by making trial-averaged responses more separable, as well as by reducing trial-to-trial variability by eliminating noise correlations. We further found that multiple combinations of weights could give rise to similar discrimination performances, which suggests that such combinatorial codes could be achieved in the brain. We conclude that the activities of midbrain neuronal populations can be used to reliably discriminate between highly heterogeneous stimulus waveforms.

Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Neural activity during a simple reaching task in macaques is counter to gating and rebound in basal ganglia-thalamic communication

Task-related activity in the ventral thalamus, a major target of basal ganglia output, is often assumed to be permitted or triggered by changes in basal ganglia activity through gating- or rebound-like mechanisms. To test those hypotheses, we sampled single-unit activity from connected basal ganglia output and thalamic nuclei (globus pallidus-internus [GPi] and ventrolateral anterior nucleus [VLa]) in monkeys performing a reaching task. Rate increases were the most common peri-movement change in both nuclei. Moreover, peri-movement changes generally began earlier in VLa than in GPi. Simultaneously recorded GPi-VLa pairs rarely showed short-time-scale spike-to-spike correlations or slow across-trials covariations, and both were equally positive and negative. Finally, spontaneous GPi bursts and pauses were both followed by small, slow reductions in VLa rate. These results appear incompatible with standard gating and rebound models. Still, gating or rebound may be possible in other physiological situations: simulations show how GPi-VLa communication can scale with GPi synchrony and GPi-to-VLa convergence, illuminating how synchrony of basal ganglia output during motor learning or in pathological conditions may render this pathway effective. Thus, in the healthy state, basal ganglia-thalamic communication during learned movement is more subtle than expected, with changes in firing rates possibly being dominated by a common external source.

The Neural Basis for Response Latency in a Sensory-Motor Behavior

We seek a neural circuit explanation for sensory-motor reaction times. In the smooth eye movement region of the frontal eye fields (FEFSEM), the latencies of pairs of neurons show trial-by-trial correlations that cause trial-by-trial correlations in neural and behavioral latency. These correlations can account for two-third of the observed variation in behavioral latency. The amplitude of preparatory activity also could contribute, but the responses of many FEFSEM neurons fail to support predictions of the traditional "ramp-to-threshold" model. As a correlate of neural processing that determines reaction time, the local field potential in FEFSEM includes a brief wave in the 5-15-Hz frequency range that precedes pursuit initiation and whose phase is correlated with the latency of pursuit in individual trials. We suggest that the latency of the incoming visual motion signals combines with the state of preparatory activity to determine the latency of the transient response that controls eye movement.

IMPACT STATEMENT

The motor cortex for smooth pursuit eye movements contributes to sensory-motor reaction time through the amplitude of preparatory activity and the latency of transient, visually driven responses.

Cerebellar Neurodynamics Predict Decision Timing and Outcome on the Single-Trial Level

Goal-directed behavior requires the interaction of multiple brain regions. How these regions and their interactions with brain-wide activity drive action selection is less understood. We have investigated this question by combining whole-brain volumetric calcium imaging using light-field microscopy and an operant-conditioning task in larval zebrafish. We find global, recurring dynamics of brain states to exhibit pre-motor bifurcations toward mutually exclusive decision outcomes. These dynamics arise from a distributed network displaying trial-by-trial functional connectivity changes, especially between cerebellum and habenula, which correlate with decision outcome. Within this network the cerebellum shows particularly strong and predictive pre-motor activity (>10 s before movement initiation), mainly within the granule cells. Turn directions are determined by the difference neuroactivity between the ipsilateral and contralateral hemispheres, while the rate of bi-hemispheric population ramping quantitatively predicts decision time on the trial-by-trial level. Our results highlight a cognitive role of the cerebellum and its importance in motor planning.

Functional links between sensory representations, choice activity, and sensorimotor associations in parietal cortex

Three-dimensional (3D) representations of the environment are often critical for selecting actions that achieve desired goals. The success of these goal-directed actions relies on 3D sensorimotor transformations that are experience-dependent. Here we investigated the relationships between the robustness of 3D visual representations, choice-related activity, and motor-related activity in parietal cortex. Macaque monkeys performed an eight-alternative 3D orientation discrimination task and a visually guided saccade task while we recorded from the caudal intraparietal area using laminar probes. We found that neurons with more robust 3D visual representations preferentially carried choice-related activity. Following the onset of choice-related activity, the robustness of the 3D representations further increased for those neurons. We additionally found that 3D orientation and saccade direction preferences aligned, particularly for neurons with choice-related activity, reflecting an experience-dependent sensorimotor association. These findings reveal previously unrecognized links between the fidelity of ecologically relevant object representations, choice-related activity, and motor-related activity.

Interneuronal correlations at longer time scales predict decision signals for bistable structure-from-motion perception

Perceptual decisions are thought to depend on the activation of task-relevant neurons, whose activity is often correlated in time. Here, we examined how the temporal structure of shared variability in neuronal firing relates to perceptual choices. We recorded stimulus-selective neurons from visual area V5/MT while two monkeys (Macaca mulatta) made perceptual decisions about the rotation direction of structure-from-motion cylinders. Interneuronal correlations for a perceptually ambiguous cylinder stimulus were significantly higher than those for unambiguous cylinders or for random 2D motion during passive viewing. Much of the difference arose from correlations at relatively long timescales (hundreds of milliseconds). Choice-related neural activity (quantified as choice probability; CP) for ambiguous cylinders was positively correlated with interneuronal correlations and was specifically associated with their long timescale component. Furthermore, the slope of the long timescale - but not the instantaneous - component of the correlation predicted higher CPs towards the end of the trial i.e. close to the decision. Our results suggest that the perceptual stability of structure-from-motion cylinders may be controlled by enhanced interneuronal correlations on longer timescales. We propose this as a potential signature of top-down influences onto V5/MT processing that shape and stabilize the appearance of 3D-motion percepts.

Sensory-to-Category Transformation via Dynamic Reorganization of Ensemble Structures in Mouse Auditory Cortex

The ability to group physical stimuli into behaviorally relevant categories is fundamental to perception and cognition. Despite a large body of work on stimulus categorization at the behavioral and cognitive levels, little is known about the underlying mechanisms at the neuronal level. Here, combining mouse auditory psychophysical behavior and in vivo two-photon imaging from the auditory cortex, we investigate how sensory-to-category transformation is implemented by cortical neurons during a stimulus categorization task. Distinct from responses during passive listening, many neurons exhibited emergent selectivity to stimuli near the category boundary during task performance, reshaping local tuning maps; other neurons became more selective to category membership of stimuli. At the population level, local cortical ensembles robustly encode category information and predict trial-by-trial decisions during task performance. Our data uncover a task-dependent dynamic reorganization of cortical response patterns serving as a neural mechanism for sensory-to-category transformation during perceptual decision-making.

Predicting Perceptual Decisions Using Visual Cortical Population Responses and Choice History

Our understanding of the neural basis of perceptual decision making has been built in part on relating co-fluctuations of single neuron responses to perceptual decisions on a trial-by-trial basis. The strength of this relationship is often compared across neurons or brain areas, recorded in different sessions, animals, or variants of a task. We sought to extend our understanding of perceptual decision making in three ways. First, we measured neuronal activity simultaneously in early [primary visual cortex (V1)] and midlevel (V4) visual cortex while macaque monkeys performed a fine orientation discrimination perceptual task. This allowed a direct comparison of choice signals in these two areas, including their dynamics. Second, we asked how our ability to predict animals' decisions would be improved by considering small simultaneously-recorded neuronal populations rather than individual units. Finally, we asked whether predictions would be improved by taking into account the animals' choice and reward histories, which can strongly influence decision making. We found that responses of individual V4 neurons were weakly predictive of decisions, but only in a brief epoch between stimulus offset and the indication of choice. In V1, few neurons showed significant decision-related activity. Analysis of neuronal population responses revealed robust choice-related information in V4 and substantially weaker signals in V1. Including choice- and reward-history information improved performance further, particularly when the recorded populations contained little decision-related information. Our work shows the power of using neuronal populations and decision history when relating neuronal responses to the perceptual decisions they are thought to underlie.SIGNIFICANCE STATEMENT Decades of research has provided a rich description of how visual information is represented in the visual cortex. Yet how cortical responses relate to visual perception remains poorly understood. Here we relate fluctuations in small neuronal population responses, recorded simultaneously in primary visual cortex (V1) and area V4 of monkeys, to perceptual reports in an orientation discrimination task. Choice-related signals were robust in V4, particularly late in the behavioral trial, but not in V1. Models that include both neuronal responses and choice-history information were able to predict a substantial portion of decisions. Our work shows the power of integrating information across neurons and including decision history in relating neuronal responses to perceptual decisions.

Population Coding and Correlated Variability in Electrosensory Pathways

The fact that perception and behavior depend on the simultaneous and coordinated activity of neural populations is well established. Understanding encoding through neuronal population activity is however complicated by the statistical dependencies between the activities of neurons, which can be present in terms of both their mean (signal correlations) and their response variability (noise correlations). Here, we review the state of knowledge regarding population coding and the influence of correlated variability in the electrosensory pathways of the weakly electric fish Apteronotus leptorhynchus. We summarize known population coding strategies at the peripheral level, which are largely unaffected by noise correlations. We then move on to the hindbrain, where existing data from the electrosensory lateral line lobe (ELL) shows the presence of noise correlations. We summarize the current knowledge regarding the mechanistic origins of noise correlations and known mechanisms of stimulus dependent correlation shaping in ELL. We finish by considering future directions for understanding population coding in the electrosensory pathways of weakly electric fish, highlighting the benefits of this model system for understanding the origins and impact of noise correlations on population coding.

Inferring decoding strategies for multiple correlated neural populations

Studies of neuron-behaviour correlation and causal manipulation have long been used separately to understand the neural basis of perception. Yet these approaches sometimes lead to drastically conflicting conclusions about the functional role of brain areas. Theories that focus only on choice-related neuronal activity cannot reconcile those findings without additional experiments involving large-scale recordings to measure interneuronal correlations. By expanding current theories of neural coding and incorporating results from inactivation experiments, we demonstrate here that it is possible to infer decoding weights of different brain areas at a coarse scale without precise knowledge of the correlation structure. We apply this technique to neural data collected from two different cortical areas in macaque monkeys trained to perform a heading discrimination task. We identify two opposing decoding schemes, each consistent with data depending on the nature of correlated noise. Our theory makes specific testable predictions to distinguish these scenarios experimentally without requiring measurement of the underlying noise correlations.

The neocortex is structurally organized into distinct brain areas. The role of specific brain areas in sensory perception is typically studied using two kinds of laboratory experiments: those that measure correlations between neural activity and reported percepts, and those that inactivate a brain region and measure the resulting changes in percepts. The two types of experiments have generally been interpreted in isolation, in part because no theory has been able combine their outcomes. Here, we describe a mathematical framework that synthesizes both kinds of results, giving us a new way to assess how different brain areas contribute to perception. When we apply our framework to experiments on behaving monkeys, we discover two models that can explain the perplexing finding that one brain area can predict an animal’s reported percepts, even though the percepts are not affected when that brain area is inactivated. The two models ascribe dramatically different efficiencies to brain computation. We show that these two models could be distinguished by a proposed experiment that measures correlations while inactivating different brain areas.

Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation

It is controversial whether covariation between neuronal activity and perceptual choice (i.e., choice correlation) reflects the functional readout of sensory signals. Here, we combined choice-correlation measures and electrical microstimulation on a site-to-site basis in the medial superior temporal area (MST), middle temporal area (MT), and ventral intraparietal area (VIP) when macaques discriminated between motion directions in both fine and coarse tasks. Microstimulation generated comparable effects between tasks but heterogeneous effects across and within brain regions. Within the MST and MT, microstimulation significantly biased an animal's choice toward the sensory preference instead of choice-related signals of the stimulated units. This was particularly evident for sites with conflict preference of sensory and choice-related signals. In the VIP, microstimulation failed to produce significant effects in either task despite strong choice correlations presented in this area. Our results suggest that sensory readout may not be inferred from choice-related signals during perceptual decision-making tasks.

Descending pathways generate perception of and neural responses to weak sensory input

Natural sensory stimuli frequently consist of a fast time-varying waveform whose amplitude or contrast varies more slowly. While changes in contrast carry behaviorally relevant information necessary for sensory perception, their processing by the brain remains poorly understood to this day. Here, we investigated the mechanisms that enable neural responses to and perception of low-contrast stimuli in the electrosensory system of the weakly electric fish Apteronotus leptorhynchus. We found that fish reliably detected such stimuli via robust behavioral responses. Recordings from peripheral electrosensory neurons revealed stimulus-induced changes in firing activity (i.e., phase locking) but not in their overall firing rate. However, central electrosensory neurons receiving input from the periphery responded robustly via both phase locking and increases in firing rate. Pharmacological inactivation of feedback input onto central electrosensory neurons eliminated increases in firing rate but did not affect phase locking for central electrosensory neurons in response to low-contrast stimuli. As feedback inactivation eliminated behavioral responses to these stimuli as well, our results show that it is changes in central electrosensory neuron firing rate that are relevant for behavior, rather than phase locking. Finally, recordings from neurons projecting directly via feedback to central electrosensory neurons revealed that they provide the necessary input to cause increases in firing rate. Our results thus provide the first experimental evidence that feedback generates both neural and behavioral responses to low-contrast stimuli that are commonly found in the natural environment.

Feedback input from more central to more peripheral brain areas is found ubiquitously in the central nervous system of vertebrates. In this study, we used a combination of electrophysiological, behavioral, and pharmacological approaches to reveal a novel function for feedback pathways in generating neural and behavioral responses to weak sensory input in the weakly electric fish. We first determined that weak sensory input gives rise to responses that are phase locked in both peripheral sensory neurons and in the central neurons that are their downstream targets. However, central neurons also responded to weak sensory inputs that were not relayed via a feedforward input from the periphery, because complete inactivation of the feedback pathway abolished increases in firing rate but not the phase locking in response to weak sensory input. Because such inactivation also abolished the behavioral responses, our results show that the increases in firing rate in central neurons, and not the phase locking, are decoded downstream to give rise to perception. Finally, we discovered that the neurons providing feedback input were also activated by weak sensory input, thereby offering further evidence that feedback is necessary to elicit increases in firing rate that are needed for perception.

A cerebellar adaptation to uncertain inputs

Noise and variability are inherent and unavoidable features of neural processing. Despite this physiological challenge, brain systems function well, suggesting the existence of adaptations that cope with noise. We report a novel adaptation that the cerebellum implements to maintain correct responses in the face of ambiguous inputs. We found that under these conditions, the cerebellum used a probabilistic binary choice: Although the probability of behavioral response gradually increased or decreased depending on the degree of similarity between current and trained inputs, the size of response remained constant. That way the cerebellum kept responses adaptive to trained input corrupted by noise while minimizing false responses to novel stimuli. Recordings and analysis of Purkinje cells activity showed that the binary choice is made in the cerebellar cortex. Results from large-scale simulation suggest that internal feedback from cerebellar nucleus back to cerebellar cortex plays a critical role in implementation of binary choice.

Orbitofrontal Cortex: A Neural Circuit for Economic Decisions

Economic choice behavior entails the computation and comparison of subjective values. A central contribution of neuroeconomics has been to show that subjective values are represented explicitly at the neuronal level. With this result at hand, the field has increasingly focused on the difficult question of where in the brain and how exactly subjective values are compared to make a decision. Here, we review a broad range of experimental and theoretical results suggesting that good-based decisions are generated in a neural circuit within the orbitofrontal cortex (OFC). The main lines of evidence supporting this proposal include the fact that goal-directed behavior is specifically disrupted by OFC lesions, the fact that different groups of neurons in this area encode the input and the output of the decision process, the fact that activity fluctuations in each of these cell groups correlate with choice variability, and the fact that these groups of neurons are computationally sufficient to generate decisions. Results from other brain regions are consistent with the idea that good-based decisions take place in OFC and indicate that value signals inform a variety of mental functions. We also contrast the present proposal with other leading models for the neural mechanisms of economic decisions. Finally, we indicate open questions and suggest possible directions for future research.

Decoupled choice-driven and stimulus-related activity in parietal neurons may be misrepresented by choice probabilities

Trial-by-trial correlations between neural responses and choices (choice probabilities) are often interpreted to reflect a causal contribution of neurons to task performance. However, choice probabilities may arise from top-down, rather than bottom-up, signals. We isolated distinct sensory and decision contributions to single-unit activity recorded from the dorsal medial superior temporal (MSTd) and ventral intraparietal (VIP) areas of monkeys during perception of self-motion. Superficially, neurons in both areas show similar tuning curves during task performance. However, tuning in MSTd neurons primarily reflects sensory inputs, whereas choice-related signals dominate tuning in VIP neurons. Importantly, the choice-related activity of VIP neurons is not predictable from their stimulus tuning, and these factors are often confounded in choice probability measurements. This finding was confirmed in a subset of neurons for which stimulus tuning was measured during passive fixation. Our findings reveal decoupled stimulus and choice signals in the VIP area, and challenge our understanding of choice signals in the brain.Choice-related signals in neuronal activity may reflect bottom-up sensory processes, top-down decision-related influences, or a combination of the two. Here the authors report that choice-related activity in VIP neurons is not predictable from their stimulus tuning, and that dominant choice signals can bias the standard metric of choice preference (choice probability).

Contribution of spiking activity in the primary auditory cortex to detection in noise

A fundamental problem in hearing is detecting a "target" stimulus (e.g., a friend's voice) that is presented with a noisy background (e.g., the din of a crowded restaurant). Despite its importance to hearing, a relationship between spiking activity and behavioral performance during such a "detection-in-noise" task has yet to be fully elucidated. In this study, we recorded spiking activity in primary auditory cortex (A1) while rhesus monkeys detected a target stimulus that was presented with a noise background. Although some neurons were modulated, the response of the typical A1 neuron was not modulated by the stimulus- and task-related parameters of our task. In contrast, we found more robust representations of these parameters in population-level activity: small populations of neurons matched the monkeys' behavioral sensitivity. Overall, these findings are consistent with the hypothesis that the sensory evidence, which is needed to solve such detection-in-noise tasks, is represented in population-level A1 activity and may be available to be read out by downstream neurons that are involved in mediating this task.NEW & NOTEWORTHY This study examines the contribution of A1 to detecting a sound that is presented with a noisy background. We found that population-level A1 activity, but not single neurons, could provide the evidence needed to make this perceptual decision.

Decision-Related Activity in Macaque V2 for Fine Disparity Discrimination Is Not Compatible with Optimal Linear Readout

Fine judgments of stereoscopic depth rely mainly on relative judgments of depth (relative binocular disparity) between objects, rather than judgments of the distance to where the eyes are fixating (absolute disparity). In macaques, visual area V2 is the earliest site in the visual processing hierarchy for which neurons selective for relative disparity have been observed (Thomas et al., 2002). Here, we found that, in macaques trained to perform a fine disparity discrimination task, disparity-selective neurons in V2 were highly selective for the task, and their activity correlated with the animals' perceptual decisions (unexplained by the stimulus). This may partially explain similar correlations reported in downstream areas. Although compatible with a perceptual role of these neurons for the task, the interpretation of such decision-related activity is complicated by the effects of interneuronal "noise" correlations between sensory neurons. Recent work has developed simple predictions to differentiate decoding schemes (Pitkow et al., 2015) without needing measures of noise correlations, and found that data from early sensory areas were compatible with optimal linear readout of populations with information-limiting correlations. In contrast, our data here deviated significantly from these predictions. We additionally tested this prediction for previously reported results of decision-related activity in V2 for a related task, coarse disparity discrimination (Nienborg and Cumming, 2006), thought to rely on absolute disparity. Although these data followed the predicted pattern, they violated the prediction quantitatively. This suggests that optimal linear decoding of sensory signals is not generally a good predictor of behavior in simple perceptual tasks.

SIGNIFICANCE STATEMENT

Activity in sensory neurons that correlates with an animal's decision is widely believed to provide insights into how the brain uses information from sensory neurons. Recent theoretical work developed simple predictions to differentiate decoding schemes, and found support for optimal linear readout of early sensory populations with information-limiting correlations. Here, we observed decision-related activity for neurons in visual area V2 of macaques performing fine disparity discrimination, as yet the earliest site for this task. These findings, and previously reported results from V2 in a different task, deviated from the predictions for optimal linear readout of a population with information-limiting correlations. Our results suggest that optimal linear decoding of early sensory information is not a general decoding strategy used by the brain.

Distinct Correlation Structure Supporting a Rate-Code for Sound Localization in the Owl's Auditory Forebrain

While a topographic map of auditory space exists in the vertebrate midbrain, it is absent in the forebrain. Yet, both brain regions are implicated in sound localization. The heterogeneous spatial tuning of adjacent sites in the forebrain compared to the midbrain reflects different underlying circuitries, which is expected to affect the correlation structure, i.e., signal (similarity of tuning) and noise (trial-by-trial variability) correlations. Recent studies have drawn attention to the impact of response correlations on the information readout from a neural population. We thus analyzed the correlation structure in midbrain and forebrain regions of the barn owl’s auditory system. Tetrodes were used to record in the midbrain and two forebrain regions, Field L and the downstream auditory arcopallium (AAr), in anesthetized owls. Nearby neurons in the midbrain showed high signal and noise correlations (R_NCs), consistent with shared inputs. As previously reported, Field L was arranged in random clusters of similarly tuned neurons. Interestingly, AAr neurons displayed homogeneous monotonic azimuth tuning, while response variability of nearby neurons was significantly less correlated than the midbrain. Using a decoding approach, we demonstrate that low R_NC in AAr restricts the potentially detrimental effect it can have on information, assuming a rate code proposed for mammalian sound localization. This study harnesses the power of correlation structure analysis to investigate the coding of auditory space. Our findings demonstrate distinct correlation structures in the auditory midbrain and forebrain, which would be beneficial for a rate-code framework for sound localization in the nontopographic forebrain representation of auditory space.

Inference in the Brain: Statistics Flowing in Redundant Population Codes

It is widely believed that the brain performs approximate probabilistic inference to estimate causal variables in the world from ambiguous sensory data. To understand these computations, we need to analyze how information is represented and transformed by the actions of nonlinear recurrent neural networks. We propose that these probabilistic computations function by a message-passing algorithm operating at the level of redundant neural populations. To explain this framework, we review its underlying concepts, including graphical models, sufficient statistics, and message-passing, and then describe how these concepts could be implemented by recurrently connected probabilistic population codes. The relevant information flow in these networks will be most interpretable at the population level, particularly for redundant neural codes. We therefore outline a general approach to identify the essential features of a neural message-passing algorithm. Finally, we argue that to reveal the most important aspects of these neural computations, we must study large-scale activity patterns during moderately complex, naturalistic behaviors.

Transformation of spatiotemporal dynamics in the macaque vestibular system from otolith afferents to cortex

Sensory signals undergo substantial recoding when neural activity is relayed from sensors through pre-thalamic and thalamic nuclei to cortex. To explore how temporal dynamics and directional tuning are sculpted in hierarchical vestibular circuits, we compared responses of macaque otolith afferents with neurons in the vestibular and cerebellar nuclei, as well as five cortical areas, to identical three-dimensional translational motion. We demonstrate a remarkable spatio-temporal transformation: otolith afferents carry spatially aligned cosine-tuned translational acceleration and jerk signals. In contrast, brainstem and cerebellar neurons exhibit non-linear, mixed selectivity for translational velocity, acceleration, jerk and position. Furthermore, these components often show dissimilar spatial tuning. Moderate further transformation of translation signals occurs in the cortex, such that similar spatio-temporal properties are found in multiple cortical areas. These results suggest that the first synapse represents a key processing element in vestibular pathways, robustly shaping how self-motion is represented in central vestibular circuits and cortical areas.

Self-motion evokes precise spike timing in the primate vestibular system

The accurate representation of self-motion requires the efficient processing of sensory input by the vestibular system. Conventional wisdom is that vestibular information is exclusively transmitted through changes in firing rate, yet under this assumption vestibular neurons display relatively poor detection and information transmission. Here, we carry out an analysis of the system's coding capabilities by recording neuronal responses to repeated presentations of naturalistic stimuli. We find that afferents with greater intrinsic variability reliably discriminate between different stimulus waveforms through differential patterns of precise (∼6 ms) spike timing, while those with minimal intrinsic variability do not. A simple mathematical model provides an explanation for this result. Postsynaptic central neurons also demonstrate precise spike timing, suggesting that higher brain areas also represent self-motion using temporally precise firing. These findings demonstrate that two distinct sensory channels represent vestibular information: one using rate coding and the other that takes advantage of precise spike timing.

Early vestibular pathways are thought to code sensory inputs regarding self-motion via changes in firing rate. Here, the authors record from both regular and irregular afferents in macaques, and find both irregular afferents and central neurons also represent self-motion via temporally precise spike timing.

Short Time-Scale Sensory Coding in S1 during Discrimination of Whisker Vibrotactile Sequences

Rodent whisker input consists of dense microvibration sequences that are often temporally integrated for perceptual discrimination. Whether primary somatosensory cortex (S1) participates in temporal integration is unknown. We trained rats to discriminate whisker impulse sequences that varied in single-impulse kinematics (5–20-ms time scale) and mean speed (150-ms time scale). Rats appeared to use the integrated feature, mean speed, to guide discrimination in this task, consistent with similar prior studies. Despite this, 52% of S1 units, including 73% of units in L4 and L2/3, encoded sequences at fast time scales (≤20 ms, mostly 5–10 ms), accurately reflecting single impulse kinematics. 17% of units, mostly in L5, showed weaker impulse responses and a slow firing rate increase during sequences. However, these units did not effectively integrate whisker impulses, but instead combined weak impulse responses with a distinct, slow signal correlated to behavioral choice. A neural decoder could identify sequences from fast unit spike trains and behavioral choice from slow units. Thus, S1 encoded fast time scale whisker input without substantial temporal integration across whisker impulses.

Recordings in whisker somatosensory cortex of rats during discrimination of rapid whisker deflection sequences show that whisker input is encoded at very short time scales (less than 20 ms).

Sensory input is rich in temporal patterns, but how the brain processes this temporal information is not well understood. This process is important in the whisker tactile system of rodents, in which active whisking on objects generates dense streams of stick-slip and contact events. Rats can discriminate vibrotactile sequences applied to the whiskers, and prior studies show that this often involves behavioral integration over time to calculate mean whisker-speed. How the brain represents and integrates vibrotactile input is not known. We recorded neural activity in primary somatosensory cortex as rats discriminated rapid vibrotactile sequences. We found that neurons in the primary somatosensory cortex encoded whisker sensory information at very fast time scales (<20 ms), without evidence for substantial temporal integration. A subset of neurons encoded relatively little stimulus information but strongly encoded the rat’s behavioral choice on each trial. Thus, primary sensory cortex represents immediate sensory input, suggesting that temporal integration occurs in downstream brain areas.

Evidence for a Causal Contribution of Macaque Vestibular, But Not Intraparietal, Cortex to Heading Perception

Multisensory convergence of visual and vestibular signals has been observed within a network of cortical areas involved in representing heading. Vestibular-dominant heading tuning has been found in the macaque parietoinsular vestibular cortex (PIVC) and the adjacent visual posterior sylvian (VPS) area, whereas relatively balanced visual/vestibular tuning was encountered in the ventral intraparietal (VIP) area and visual-dominant tuning was found in the dorsal medial superior temporal (MSTd) area. Although the respective functional roles of these areas remain unclear, perceptual deficits in heading discrimination following reversible chemical inactivation of area MSTd area suggested that areas with vestibular-dominant heading tuning also contribute to behavior. To explore the roles of other areas in heading perception, muscimol injections were used to reversibly inactivate either the PIVC or the VIP area bilaterally in macaques. Inactivation of the anterior PIVC increased psychophysical thresholds when heading judgments were based on either optic flow or vestibular cues, although effects were stronger for vestibular stimuli. All behavioral deficits recovered within 36 h. Visual deficits were larger following inactivation of the posterior portion of the PIVC, likely because these injections encroached upon the VPS area, which contains neurons with optic flow tuning (unlike the PIVC). In contrast, VIP inactivation led to no behavioral deficits, despite the fact that VIP neurons show much stronger choice-related activity than MSTd neurons. These results suggest that the VIP area either provides a parallel and partially redundant pathway for this task, or does not participate in heading discrimination. In contrast, the PIVC/VPS area, along with the MSTd area, make causal contributions to heading perception based on either vestibular or visual signals.

SIGNIFICANCE STATEMENT

Multisensory vestibular and visual signals are found in multiple cortical areas, but their causal contribution to self-motion perception has been previously tested only in the dorsal medial superior temporal (MSTd) area. In these experiments, we show that inactivation of the parietoinsular vestibular cortex (PIVC) also results in causal deficits during heading discrimination for both visual and vestibular cues. In contrast, ventral intraparietal (VIP) area inactivation led to no behavioral deficits, despite the fact that VIP neurons show much stronger choice-related activity than MSTd or PIVC neurons. These results demonstrate that choice-related activity does not always imply a causal role in sensory perception.

Dissociated functional significance of decision-related activity in the primate dorsal stream

During decision making, neurons in multiple brain regions exhibit responses that are correlated with decisions. However, it remains uncertain whether or not various forms of decision-related activity are causally related to decision making. Here we address this question by recording and reversibly inactivating the lateral intraparietal (LIP) and middle temporal (MT) areas of rhesus macaques performing a motion direction discrimination task. Neurons in area LIP exhibited firing rate patterns that directly resembled the evidence accumulation process posited to govern decision making, with strong correlations between their response fluctuations and the animal's choices. Neurons in area MT, in contrast, exhibited weak correlations between their response fluctuations and choices, and had firing rate patterns consistent with their sensory role in motion encoding. The behavioural impact of pharmacological inactivation of each area was inversely related to their degree of decision-related activity: while inactivation of neurons in MT profoundly impaired psychophysical performance, inactivation in LIP had no measurable impact on decision-making performance, despite having silenced the very clusters that exhibited strong decision-related activity. Although LIP inactivation did not impair psychophysical behaviour, it did influence spatial selection and oculomotor metrics in a free-choice control task. The absence of an effect on perceptual decision making was stable over trials and sessions and was robust to changes in stimulus type and task geometry, arguing against several forms of compensation. Thus, decision-related signals in LIP do not appear to be critical for computing perceptual decisions, and may instead reflect secondary processes. Our findings highlight a dissociation between decision correlation and causation, showing that strong neuron-decision correlations do not necessarily offer direct access to the neural computations underlying decisions.

A neural basis for the spatial suppression of visual motion perception

In theory, sensory perception should be more accurate when more neurons contribute to the representation of a stimulus. However, psychophysical experiments that use larger stimuli to activate larger pools of neurons sometimes report impoverished perceptual performance. To determine the neural mechanisms underlying these paradoxical findings, we trained monkeys to discriminate the direction of motion of visual stimuli that varied in size across trials, while simultaneously recording from populations of motion-sensitive neurons in cortical area MT. We used the resulting data to constrain a computational model that explained the behavioral data as an interaction of three main mechanisms: noise correlations, which prevented stimulus information from growing with stimulus size; neural surround suppression, which decreased sensitivity for large stimuli; and a read-out strategy that emphasized neurons with receptive fields near the stimulus center. These results suggest that paradoxical percepts reflect tradeoffs between sensitivity and noise in neuronal populations.

DOI:http://dx.doi.org/10.7554/eLife.16167.001

People usually find it easier to see things when they are big and bright, but there are occasionally exceptions. One example concerns moving objects: when they are small, we can identify their direction of motion easily, but this becomes much more difficult for larger objects. This decreased perceptual sensitivity appears to be linked to other mental processes. For example, studies have suggested that people with high IQs have more difficulty perceiving the motion of large objects, whereas people with various psychiatric disorders, such as schizophrenia, are better able to see such movement. Although several theories have been proposed, there is currently no good explanation for these findings.

Liu et al. set out to determine why the part of the brain that is responsible for vision (the visual cortex) fails to register the direction of large moving objects and how this failure might relate to mental function in general. To do this, Liu et al. trained monkeys to report which direction different sized stimuli were moving on a screen. The electrical activity of nerve cells in the part of the visual cortex that deals with movement was recorded while the monkeys performed this task. The results of the experiments revealed that, on average, these cells responded strongly to large moving stimuli, even though the monkeys had trouble seeing their motion. However, nerve cells are “noisy” – they respond a bit differently every time they are presented with the same stimulus – and this noise was stronger for larger stimuli.

By studying the mathematical relationship between the noise and what the animals perceived, Liu et al. found that the visual cortex attempts to suppress the noise and in the process often shuts off the responses to large stimuli entirely. This suppression is likely to cause the movement of large stimuli to be poorly perceived.

If suppressing this kind of noise is really responsible for failures in perceiving motion, then this mechanism could also explain the connection between motion perception and other mental processes. Liu et al. are currently testing this idea.

DOI:http://dx.doi.org/10.7554/eLife.16167.002

Feedforward and feedback sources of choice probability in neural population responses

How the processing of signals carried by sensory neurons supports perceptual decisions is a long-standing question in neuroscience. The ability to record neuronal activity in awake animals while they perform psychophysical tasks near threshold has been a key advance in studying these questions. Trial-to-trial correlations between the activity of sensory neurons and the decisions reported by animals ('choice probabilities'), even when measured across repeated presentations of an identical stimulus provide insights into this problem. But understanding the sources of such co-variability between sensory neurons and behavior has proven more difficult than it initially appeared. Below, we discuss our current understanding of what gives rise to these correlations.

Origins of choice-related activity in mouse somatosensory cortex

During perceptual decisions about faint or ambiguous sensory stimuli, even identical stimuli can produce different choices. Spike trains from sensory cortex neurons can predict trial-to-trial variability in choice. Choice-related spiking is widely studied to link cortical activity to perception, but its origins remain unclear. Using imaging and electrophysiology, we found that mouse primary somatosensory cortex neurons showed robust choice-related activity during a tactile detection task. Spike trains from primary mechanoreceptive neurons did not predict choices about identical stimuli. Spike trains from thalamic relay neurons showed highly transient, weak choice-related activity. Intracellular recordings in cortex revealed a prolonged choice-related depolarization in most neurons that was not accounted for by feedforward thalamic input. Top-down axons projecting from secondary to primary somatosensory cortex signaled choice. An intracellular measure of stimulus sensitivity determined which neurons converted choice-related depolarization into spiking. Our results reveal how choice-related spiking emerges across neural circuits and within single neurons.

Neural Coding of Perceived Odor Intensity

Stimulus intensity is a fundamental perceptual feature in all sensory systems. In olfaction, perceived odor intensity depends on at least two variables: odor concentration; and duration of the odor exposure or adaptation. To examine how neural activity at early stages of the olfactory system represents features relevant to intensity perception, we studied the responses of mitral/tufted cells (MTCs) while manipulating odor concentration and exposure duration. Temporal profiles of MTC responses to odors changed both as a function of concentration and with adaptation. However, despite the complexity of these responses, adaptation and concentration dependencies behaved similarly. These similarities were visualized by principal component analysis of average population responses and were quantified by discriminant analysis in a trial-by-trial manner. The qualitative functional dependencies of neuronal responses paralleled psychophysics results in humans. We suggest that temporal patterns of MTC responses in the olfactory bulb contribute to an internal perceptual variable: odor intensity.

A computational relationship between thalamic sensory neural responses and contrast perception

Uncovering the relationship between sensory neural responses and perceptual decisions remains a fundamental problem in neuroscience. Decades of experimental and modeling work in the sensory cortex have demonstrated that a perceptual decision pool is usually composed of tens to hundreds of neurons, the responses of which are significantly correlated not only with each other, but also with the behavioral choices of an animal. Few studies, however, have measured neural activity in the sensory thalamus of awake, behaving animals. Therefore, it remains unclear how many thalamic neurons are recruited and how the information from these neurons is pooled at subsequent cortical stages to form a perceptual decision. In a previous study we measured neural activity in the macaque lateral geniculate nucleus (LGN) during a two alternative forced choice (2AFC) contrast detection task, and found that single LGN neurons were significantly correlated with the monkeys’ behavioral choices, despite their relatively poor contrast sensitivity and a lack of overall interneuronal correlations. We have now computationally tested a number of specific hypotheses relating these measured LGN neural responses to the contrast detection behavior of the animals. We modeled the perceptual decisions with different numbers of neurons and using a variety of pooling/readout strategies, and found that the most successful model consisted of about 50–200 LGN neurons, with individual neurons weighted differentially according to their signal-to-noise ratios (quantified as d-primes). These results supported the hypothesis that in contrast detection the perceptual decision pool consists of multiple thalamic neurons, and that the response fluctuations in these neurons can influence contrast perception, with the more sensitive thalamic neurons likely to exert a greater influence.

Insights into decision making using choice probability

A long-standing question in systems neuroscience is how the activity of single neurons gives rise to our perceptions and actions. Critical insights into this question occurred in the last part of the 20th century when scientists began linking modulations of neuronal activity directly to perceptual behavior. A significant conceptual advance was the application of signal detection theory to both neuronal activity and behavior, providing a quantitative assessment of the relationship between brain and behavior. One metric that emerged from these efforts was choice probability (CP), which provides information about how well an ideal observer can predict the choice an animal makes from a neuron's discharge rate distribution. In this review, we describe where CP has been studied, locational trends in the values found, and why CP values are typically so low. We discuss its dependence on correlated activity among neurons of a population, assess whether it arises from feedforward or feedback mechanisms, and investigate what CP tells us about how many neurons are required for a decision and how they are pooled to do so.

The functional asymmetry of ON and OFF channels in the perception of contrast

To fully understand the relationship between perception and single neural responses, one should take into consideration the early stages of sensory processing. Few studies, however, have directly examined the neural underpinning of visual perception in the lateral geniculate nucleus (LGN), only one synapse away from the retina. In this study we recorded from LGN parvocellular (P) ON-center and OFF-center neurons while monkeys either passively viewed or actively detected a full range of contrasts. We found that OFF neurons were more sensitive in detecting negative contrasts than ON neurons were in detecting positive contrasts. Also, OFF neurons had higher spontaneous activities, higher peak response amplitudes, and were more sustained than ON neurons in their contrast responses. Puzzlingly, OFF neurons failed to show any significant correlations with the monkeys' perceptual choices, despite their greater contrast sensitivities. If, however, choice probabilities were calculated from interspike intervals instead of spike counts (thus taking into account the higher firing rates of OFF neurons), OFF neurons but not ON neurons were significantly correlated with behavioral choices. Taken together, these results demonstrate in awake, behaving animals that: 1) the ON and OFF pathways do not simply mirror each other in their functionality but instead carry qualitatively different types of information, and 2) the responses of ON and OFF neurons can be correlated with perceptual choices even in the absence of physical stimuli and interneuronal correlations.

How Can Single Sensory Neurons Predict Behavior?

Single sensory neurons can be surprisingly predictive of behavior in discrimination tasks. We propose this is possible because sensory information extracted from neural populations is severely restricted, either by near-optimal decoding of a population with information-limiting correlations or by suboptimal decoding that is blind to correlations. These have different consequences for choice correlations, the correlations between neural responses and behavioral choices. In the vestibular and cerebellar nuclei and the dorsal medial superior temporal area, we found that choice correlations during heading discrimination are consistent with near-optimal decoding of neuronal responses corrupted by information-limiting correlations. In the ventral intraparietal area, the choice correlations are also consistent with the presence of information-limiting correlations, but this area does not appear to influence behavior, although the choice correlations are particularly large. These findings demonstrate how choice correlations can be used to assess the efficiency of the downstream readout and detect the presence of information-limiting correlations.

Neuronal variability in orbitofrontal cortex during economic decisions

Neuroeconomic models assume that economic decisions are based on the activity of offer value cells in the orbitofrontal cortex (OFC), but testing this assertion has proven difficult. In principle, the decision made on a given trial should correlate with the stochastic fluctuations of these cells. However, this correlation, measured as a choice probability (CP), is small. Importantly, a neuron's CP reflects not only its individual contribution to the decision (termed readout weight), but also the intensity and the structure of correlated variability across the neuronal population (termed noise correlation). A precise mathematical relation between CPs, noise correlations, and readout weights was recently derived by Haefner and colleagues (Haefner RM, Gerwinn S, Macke JH, Bethge M. Nat Neurosci 16: 235-242, 2013) for a linear decision model. In this framework, concurrent measurements of noise correlations and CPs can provide quantitative information on how a population of cells contributes to a decision. Here we examined neuronal variability in the OFC of rhesus monkeys during economic decisions. Noise correlations had similar structure but considerably lower strength compared with those typically measured in sensory areas during perceptual decisions. In contrast, variability in the activity of individual cells was high and comparable to that recorded in other cortical regions. Simulation analyses based on Haefner's equation showed that noise correlations measured in the OFC combined with a plausible readout of offer value cells reproduced the experimental measures of CPs. In other words, the results obtained for noise correlations and those obtained for CPs taken together support the hypothesis that economic decisions are primarily based on the activity of offer value cells.

Perceptual decision related activity in the lateral geniculate nucleus

Fundamental to neuroscience is the understanding of how the language of neurons relates to behavior. In the lateral geniculate nucleus (LGN), cells show distinct properties such as selectivity for particular wavelengths, increments or decrements in contrast, or preference for fine detail versus rapid motion. No studies, however, have measured how LGN cells respond when an animal is challenged to make a perceptual decision using information within the receptive fields of those LGN cells. In this study we measured neural activity in the macaque LGN during a two-alternative, forced-choice (2AFC) contrast detection task or during a passive fixation task and found that a small proportion (13.5%) of single LGN parvocellular (P) and magnocellular (M) neurons matched the psychophysical performance of the monkey. The majority of LGN neurons measured in both tasks were not as sensitive as the monkey. The covariation between neural response and behavior (quantified as choice probability) was significantly above chance during active detection, even when there was no external stimulus. Interneuronal correlations and task-related gain modulations were negligible under the same condition. A bottom-up pooling model that used sensory neural responses to compute perceptual choices in the absence of interneuronal correlations could fully explain these results at the level of the LGN, supporting the hypothesis that the perceptual decision pool consists of multiple sensory neurons and that response fluctuations in these neurons can influence perception.

Neuronal thresholds and choice-related activity of otolith afferent fibers during heading perception

How activity of sensory neurons leads to perceptual decisions remains a challenge to understand. Correlations between choices and single neuron firing rates have been found early in vestibular processing, in the brainstem and cerebellum. To investigate the origins of choice-related activity, we have recorded from otolith afferent fibers while animals performed a fine heading discrimination task. We find that afferent fibers have similar discrimination thresholds as central cells, and the most sensitive fibers have thresholds that are only twofold or threefold greater than perceptual thresholds. Unlike brainstem and cerebellar nuclei neurons, spike counts from afferent fibers do not exhibit trial-by-trial correlations with perceptual decisions. This finding may reflect the fact that otolith afferent responses are poorly suited for driving heading perception because they fail to discriminate self-motion from changes in orientation relative to gravity. Alternatively, if choice probabilities reflect top-down inference signals, they are not relayed to the vestibular periphery.

A functional link between MT neurons and depth perception based on motion parallax

As an observer translates, objects lying at different distances from the observer have differential image motion on the retina (motion parallax). It is well established psychophysically that humans perceive depth rather precisely from motion parallax and that extraretinal signals may be used to correctly perceive the sign of depth (near vs far) when binocular and pictorial depth cues are absent or weak. However, the neural basis for this capacity remains poorly understood. We have shown previously that neurons in the macaque middle temporal (MT) area combine retinal image motion with smooth eye movement command signals to signal depth sign from motion parallax. However, those studies were performed in animals that were required simply to track a visual target, thus precluding direct comparisons between neural activity and behavior. Here, we examine the activity of MT neurons in rhesus monkeys that were trained to discriminate depth sign based on motion parallax, in the absence of binocular disparity and pictorial depth cues. We find that the most sensitive MT neurons approach behavioral sensitivity, whereas the average neuron is twofold to threefold less sensitive than the animal. We also find that MT responses are predictive of perceptual decisions (independent of the visual stimulus), consistent with a role for MT in providing sensory signals for this behavior. Our findings suggest that, in addition to its established roles in processing stereoscopic depth, area MT is well suited to contribute to perception of depth based on motion parallax.