Decision-related activity in sensory neurons: correlations among neurons and with behavior

Origins of choice-related activity in mouse somatosensory cortex

During perceptual decisions about faint or ambiguous sensory stimuli, even identical stimuli can produce different choices. Spike trains from sensory cortex neurons can predict trial-to-trial variability in choice. Choice-related spiking is widely studied to link cortical activity to perception, but its origins remain unclear. Using imaging and electrophysiology, we found that mouse primary somatosensory cortex neurons showed robust choice-related activity during a tactile detection task. Spike trains from primary mechanoreceptive neurons did not predict choices about identical stimuli. Spike trains from thalamic relay neurons showed highly transient, weak choice-related activity. Intracellular recordings in cortex revealed a prolonged choice-related depolarization in most neurons that was not accounted for by feedforward thalamic input. Top-down axons projecting from secondary to primary somatosensory cortex signaled choice. An intracellular measure of stimulus sensitivity determined which neurons converted choice-related depolarization into spiking. Our results reveal how choice-related spiking emerges across neural circuits and within single neurons.

The importance of decision onset

The neural mechanisms of decision making are thought to require the integration of evidence over time until a response threshold is reached. Much work suggests that response threshold can be adjusted via top-down control as a function of speed or accuracy requirements. In contrast, the time of integration onset has received less attention and is believed to be determined mostly by afferent or preprocessing delays. However, a number of influential studies over the past decade challenge this assumption and begin to paint a multifaceted view of the phenomenology of decision onset. This review highlights the challenges involved in initiating the integration of evidence at the optimal time and the potential benefits of adjusting integration onset to task demands. The review outlines behavioral and electrophysiolgical studies suggesting that the onset of the integration process may depend on properties of the stimulus, the task, attention, and response strategy. Most importantly, the aggregate findings in the literature suggest that integration onset may be amenable to top-down regulation, and may be adjusted much like response threshold to exert cognitive control and strategically optimize the decision process to fit immediate behavioral requirements.

A computational relationship between thalamic sensory neural responses and contrast perception

Uncovering the relationship between sensory neural responses and perceptual decisions remains a fundamental problem in neuroscience. Decades of experimental and modeling work in the sensory cortex have demonstrated that a perceptual decision pool is usually composed of tens to hundreds of neurons, the responses of which are significantly correlated not only with each other, but also with the behavioral choices of an animal. Few studies, however, have measured neural activity in the sensory thalamus of awake, behaving animals. Therefore, it remains unclear how many thalamic neurons are recruited and how the information from these neurons is pooled at subsequent cortical stages to form a perceptual decision. In a previous study we measured neural activity in the macaque lateral geniculate nucleus (LGN) during a two alternative forced choice (2AFC) contrast detection task, and found that single LGN neurons were significantly correlated with the monkeys’ behavioral choices, despite their relatively poor contrast sensitivity and a lack of overall interneuronal correlations. We have now computationally tested a number of specific hypotheses relating these measured LGN neural responses to the contrast detection behavior of the animals. We modeled the perceptual decisions with different numbers of neurons and using a variety of pooling/readout strategies, and found that the most successful model consisted of about 50–200 LGN neurons, with individual neurons weighted differentially according to their signal-to-noise ratios (quantified as d-primes). These results supported the hypothesis that in contrast detection the perceptual decision pool consists of multiple thalamic neurons, and that the response fluctuations in these neurons can influence contrast perception, with the more sensitive thalamic neurons likely to exert a greater influence.

A Hardwired Circuit Supplemented with Endocannabinoids Encodes Behavioral Choice in Zebrafish

Animals constantly make behavioral choices to facilitate moving efficiently through their environment. When faced with a threat, animals make decisions in the midst of other ongoing behaviors through a context-dependent integration of sensory stimuli. In vertebrates, the mechanisms underlying behavioral selection are poorly understood. Here, we show that ongoing swimming in zebrafish is suppressed by escape. The selection of escape over swimming is mediated by switching between two distinct motoneuron pools. A hardwired circuit mediates this switch by acting as a clutch-like mechanism to disengage the swimming motoneuron pool and engage the escape motoneuron pool. Threshold for escape initiation is lowered and swimming suppression is prolonged by endocannabinoid neuromodulation. Thus, our results reveal a novel cellular mechanism involving a hardwired circuit supplemented with endocannabinoids acting as a clutch-like mechanism to engage/disengage distinct motor pools to ensure behavioral selection and a smooth execution of motor action sequences in a vertebrate system.

Insights into decision making using choice probability

A long-standing question in systems neuroscience is how the activity of single neurons gives rise to our perceptions and actions. Critical insights into this question occurred in the last part of the 20th century when scientists began linking modulations of neuronal activity directly to perceptual behavior. A significant conceptual advance was the application of signal detection theory to both neuronal activity and behavior, providing a quantitative assessment of the relationship between brain and behavior. One metric that emerged from these efforts was choice probability (CP), which provides information about how well an ideal observer can predict the choice an animal makes from a neuron's discharge rate distribution. In this review, we describe where CP has been studied, locational trends in the values found, and why CP values are typically so low. We discuss its dependence on correlated activity among neurons of a population, assess whether it arises from feedforward or feedback mechanisms, and investigate what CP tells us about how many neurons are required for a decision and how they are pooled to do so.

A Modality-Specific Feedforward Component of Choice-Related Activity in MT

The activity of individual sensory neurons can be predictive of an animal's choices. These decision signals arise from network properties dependent on feedforward and feedback inputs; however, the relative contributions of these inputs are poorly understood. We determined the role of feedforward pathways to decision signals in MT by recording neuronal activity while monkeys performed motion and depth tasks. During each session, we reversibly inactivated V2 and V3, which provide feedforward input to MT that conveys more information about depth than motion. We thus monitored the choice-related activity of the same neuron both before and during V2/V3 inactivation. During inactivation, MT neurons became less predictive of decisions for the depth task but not the motion task, indicating that a feedforward pathway that gives rise to tuning preferences also contributes to decision signals. We show that our data are consistent with V2/V3 input conferring structured noise correlations onto the MT population.

The functional asymmetry of ON and OFF channels in the perception of contrast

To fully understand the relationship between perception and single neural responses, one should take into consideration the early stages of sensory processing. Few studies, however, have directly examined the neural underpinning of visual perception in the lateral geniculate nucleus (LGN), only one synapse away from the retina. In this study we recorded from LGN parvocellular (P) ON-center and OFF-center neurons while monkeys either passively viewed or actively detected a full range of contrasts. We found that OFF neurons were more sensitive in detecting negative contrasts than ON neurons were in detecting positive contrasts. Also, OFF neurons had higher spontaneous activities, higher peak response amplitudes, and were more sustained than ON neurons in their contrast responses. Puzzlingly, OFF neurons failed to show any significant correlations with the monkeys' perceptual choices, despite their greater contrast sensitivities. If, however, choice probabilities were calculated from interspike intervals instead of spike counts (thus taking into account the higher firing rates of OFF neurons), OFF neurons but not ON neurons were significantly correlated with behavioral choices. Taken together, these results demonstrate in awake, behaving animals that: 1) the ON and OFF pathways do not simply mirror each other in their functionality but instead carry qualitatively different types of information, and 2) the responses of ON and OFF neurons can be correlated with perceptual choices even in the absence of physical stimuli and interneuronal correlations.

Neuronal variability in orbitofrontal cortex during economic decisions

Neuroeconomic models assume that economic decisions are based on the activity of offer value cells in the orbitofrontal cortex (OFC), but testing this assertion has proven difficult. In principle, the decision made on a given trial should correlate with the stochastic fluctuations of these cells. However, this correlation, measured as a choice probability (CP), is small. Importantly, a neuron's CP reflects not only its individual contribution to the decision (termed readout weight), but also the intensity and the structure of correlated variability across the neuronal population (termed noise correlation). A precise mathematical relation between CPs, noise correlations, and readout weights was recently derived by Haefner and colleagues (Haefner RM, Gerwinn S, Macke JH, Bethge M. Nat Neurosci 16: 235-242, 2013) for a linear decision model. In this framework, concurrent measurements of noise correlations and CPs can provide quantitative information on how a population of cells contributes to a decision. Here we examined neuronal variability in the OFC of rhesus monkeys during economic decisions. Noise correlations had similar structure but considerably lower strength compared with those typically measured in sensory areas during perceptual decisions. In contrast, variability in the activity of individual cells was high and comparable to that recorded in other cortical regions. Simulation analyses based on Haefner's equation showed that noise correlations measured in the OFC combined with a plausible readout of offer value cells reproduced the experimental measures of CPs. In other words, the results obtained for noise correlations and those obtained for CPs taken together support the hypothesis that economic decisions are primarily based on the activity of offer value cells.

Perceptual decision related activity in the lateral geniculate nucleus

Fundamental to neuroscience is the understanding of how the language of neurons relates to behavior. In the lateral geniculate nucleus (LGN), cells show distinct properties such as selectivity for particular wavelengths, increments or decrements in contrast, or preference for fine detail versus rapid motion. No studies, however, have measured how LGN cells respond when an animal is challenged to make a perceptual decision using information within the receptive fields of those LGN cells. In this study we measured neural activity in the macaque LGN during a two-alternative, forced-choice (2AFC) contrast detection task or during a passive fixation task and found that a small proportion (13.5%) of single LGN parvocellular (P) and magnocellular (M) neurons matched the psychophysical performance of the monkey. The majority of LGN neurons measured in both tasks were not as sensitive as the monkey. The covariation between neural response and behavior (quantified as choice probability) was significantly above chance during active detection, even when there was no external stimulus. Interneuronal correlations and task-related gain modulations were negligible under the same condition. A bottom-up pooling model that used sensory neural responses to compute perceptual choices in the absence of interneuronal correlations could fully explain these results at the level of the LGN, supporting the hypothesis that the perceptual decision pool consists of multiple sensory neurons and that response fluctuations in these neurons can influence perception.

Neuronal thresholds and choice-related activity of otolith afferent fibers during heading perception

How activity of sensory neurons leads to perceptual decisions remains a challenge to understand. Correlations between choices and single neuron firing rates have been found early in vestibular processing, in the brainstem and cerebellum. To investigate the origins of choice-related activity, we have recorded from otolith afferent fibers while animals performed a fine heading discrimination task. We find that afferent fibers have similar discrimination thresholds as central cells, and the most sensitive fibers have thresholds that are only twofold or threefold greater than perceptual thresholds. Unlike brainstem and cerebellar nuclei neurons, spike counts from afferent fibers do not exhibit trial-by-trial correlations with perceptual decisions. This finding may reflect the fact that otolith afferent responses are poorly suited for driving heading perception because they fail to discriminate self-motion from changes in orientation relative to gravity. Alternatively, if choice probabilities reflect top-down inference signals, they are not relayed to the vestibular periphery.

Ambiguity and nonidentifiability in the statistical analysis of neural codes

Many experimental studies of neural coding rely on a statistical interpretation of the theoretical notion of the rate at which a neuron fires spikes. For example, neuroscientists often ask, "Does a population of neurons exhibit more synchronous spiking than one would expect from the covariability of their instantaneous firing rates?" For another example, "How much of a neuron's observed spiking variability is caused by the variability of its instantaneous firing rate, and how much is caused by spike timing variability?" However, a neuron's theoretical firing rate is not necessarily well-defined. Consequently, neuroscientific questions involving the theoretical firing rate do not have a meaning in isolation but can only be interpreted in light of additional statistical modeling choices. Ignoring this ambiguity can lead to inconsistent reasoning or wayward conclusions. We illustrate these issues with examples drawn from the neural-coding literature.

Cell types, circuits, and receptive fields in the mouse visual cortex

Over the past decade, the mouse has emerged as an important model system for studying cortical function, owing to the advent of powerful tools that can record and manipulate neural activity in intact neural circuits. This advance has been particularly prominent in the visual cortex, where studies in the mouse have begun to bridge the gap between cortical structure and function, allowing investigators to determine the circuits that underlie specific visual computations. This review describes the advances in our understanding of the mouse visual cortex, including neural coding, the role of different cell types, and links between vision and behavior, and discusses how recent findings and new approaches can guide future studies.

Choice-correlated activity fluctuations underlie learning of neuronal category representation

The ability to categorize stimuli into discrete behaviourally relevant groups is an essential cognitive function. To elucidate the neural mechanisms underlying categorization, we constructed a cortical circuit model that is capable of learning a motion categorization task through reward-dependent plasticity. Here we show that stable category representations develop in neurons intermediate to sensory and decision layers if they exhibit choice-correlated activity fluctuations (choice probability). In the model, choice probability and task-specific interneuronal correlations emerge from plasticity of top-down projections from decision neurons. Specific model predictions are confirmed by analysis of single-neuron activity from the monkey parietal cortex, which reveals a mixture of directional and categorical tuning, and a positive correlation between category selectivity and choice probability. Beyond demonstrating a circuit mechanism for categorization, the present work suggests a key role of plastic top-down feedback in simultaneously shaping both neural tuning and correlated neural variability.

The ability to categorize stimuli into discrete behaviourally relevant groups is an essential cognitive function. Here, the authors demonstrate a critical role for choice-correlated activity fluctuations in the emergence of stable cortical category representations.

Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT

Neuronal variability in sensory cortex predicts perceptual decisions. This relationship, termed choice probability (CP), can arise from sensory variability biasing behaviour and from top-down signals reflecting behaviour. To investigate the interaction of these mechanisms during the decision-making process, we use a hierarchical network model composed of reciprocally connected sensory and integration circuits. Consistent with monkey behaviour in a fixed-duration motion discrimination task, the model integrates sensory evidence transiently, giving rise to a decaying bottom-up CP component. However, the dynamics of the hierarchical loop recruits a concurrently rising top-down component, resulting in sustained CP. We compute the CP time-course of neurons in the medial temporal area (MT) and find an early transient component and a separate late contribution reflecting decision build-up. The stability of individual CPs and the dynamics of noise correlations further support this decomposition. Our model provides a unified understanding of the circuit dynamics linking neural and behavioural variability.

Effective connectivity of depth-structure-selective patches in the lateral bank of the macaque intraparietal sulcus

Extrastriate cortical areas are frequently composed of subpopulations of neurons encoding specific features or stimuli, such as color, disparity, or faces, and patches of neurons encoding similar stimulus properties are typically embedded in interconnected networks, such as the attention or face-processing network. The goal of the current study was to examine the effective connectivity of subsectors of neurons in the same cortical area with highly similar neuronal response properties. We first recorded single- and multi-unit activity to identify two neuronal patches in the anterior part of the macaque intraparietal sulcus (IPS) showing the same depth structure selectivity and then employed electrical microstimulation during functional magnetic resonance imaging in these patches to determine the effective connectivity of these patches. The two IPS subsectors we identified-with the same neuronal response properties and in some cases separated by only 3 mm-were effectively connected to remarkably distinct cortical networks in both dorsal and ventral stream in three macaques. Conversely, the differences in effective connectivity could account for the known visual-to-motor gradient within the anterior IPS. These results clarify the role of the anterior IPS as a pivotal brain region where dorsal and ventral visual stream interact during object analysis. Thus, in addition to the anatomical connectivity of cortical areas and the properties of individual neurons in these areas, the effective connectivity provides novel key insights into the widespread functional networks that support behavior.

Choice-related Activity in the Anterior Intraparietal Area during 3-D Structure Categorization

The anterior intraparietal area (AIP) of macaques contains neurons that signal the depth structure of disparity-defined 3-D shapes. Previous studies have suggested that AIP's depth information is used for sensorimotor transformations related to the efficient grasping of 3-D objects. We trained monkeys to categorize disparity-defined 3-D shapes and examined whether neuronal activity in AIP may also underlie pure perceptual categorization behavior. We first show that neurons with a similar 3-D shape preference cluster in AIP. We then demonstrate that the monkeys' 3-D shape discrimination performance depends on the position in depth of the stimulus and that this performance difference is reflected in the activity of AIP neurons. We further reveal correlations between the neuronal activity in AIP and the subject's subsequent choices and RTs during 3-D shape categorization. Our findings propose AIP as an important processing stage for 3-D shape perception.

Role of medio-dorsal frontal and posterior parietal neurons during auditory detection performance in rats

To further characterize the role of frontal and parietal cortices in rat cognition, we recorded action potentials simultaneously from multiple sites in the medio-dorsal frontal cortex and posterior parietal cortex of rats while they performed a two-choice auditory detection task. We quantified neural correlates of task performance, including response movements, perception of a target tone, and the differentiation between stimuli with distinct features (different pitches or durations). A minority of units--15% in frontal cortex, 23% in parietal cortex--significantly distinguished hit trials (successful detections, response movement to the right) from correct rejection trials (correct leftward response to the absence of the target tone). Estimating the contribution of movement-related activity to these responses suggested that more than half of these units were likely signaling correct perception of the auditory target, rather than merely movement direction. In addition, we found a smaller and mostly not overlapping population of units that differentiated stimuli based on task-irrelevant details. The detection-related spiking responses we observed suggest that correlates of perception in the rat are sparsely represented among neurons in the rat's frontal-parietal network, without being concentrated preferentially in frontal or parietal areas.

Rapid shape detection signals in area V4

Vision in foveate animals is an active process that requires rapid and constant decision-making. For example, when a new object appears in the visual field, we can quickly decide to inspect it by directing our eyes to the object's location. We studied the contribution of primate area V4 to these types of rapid foveation decisions. Animals performed a reaction time task that required them to report when any shape appeared within a peripherally-located noisy stimulus by making a saccade to the stimulus location. We found that about half of the randomly sampled V4 neurons not only rapidly and precisely represented the appearance of this shape, but they were also predictive of the animal's saccades. A neuron's ability to predict the animal's saccades was not related to the specificity with which the cell represented a single type of shape but rather to its ability to signal whether any shape was present. This relationship between sensory sensitivity and behavioral predictiveness was not due to global effects such as alertness, as it was equally likely to be observed for cells with increases and decreases in firing rate. Careful analysis of the timescales of reliability in these neurons implies that they reflect both feedforward and feedback shape detecting processes. In approximately 7% of our recorded sample, individual neurons were able to predict both the delay and precision of the animal's shape detection performance. This suggests that a subset of V4 neurons may have been directly and causally contributing to task performance and that area V4 likely plays a critical role in guiding rapid, form-based foveation decisions.

Visual impairment by surrounding noise is due to interactions among stimuli in the higher-order visual cortex

Observers have difficulty identifying a target in their peripheral vision in the presence of surrounding stimuli. Although hypotheses addressing this phenomenon have been proposed, such as the integration of stimuli and surround suppression in the higher-order visual cortex, no direct comparisons of the psychophysical and neuronal sensitivities have been performed. Here we measured the performance of monkeys with a variant of the direction discrimination task using a center/surround bipartite random-dot stimulus while simultaneously recording from isolated neurons from the middle temporal visual area (MT). The psychophysical threshold increased with the addition of a task-irrelevant noise annulus that surrounded the task-relevant motion stimuli. The neuronal threshold of MT neurons also increased at a spatial scale similar to the psychophysical threshold. This suggests that the impaired ability in our task resulted from impairment in the MT area. Importantly, reduced neuronal performance was due to both a reduced response to preferred motion and an enhanced response to nonpreferred motion. These observations suggest that impairment caused by surrounding noise results from interactions between stimuli and noise and not from a reduction in the response of visual neurons.

Neural correlates of auditory streaming in an objective behavioral task

Segregating streams of sounds from sources in complex acoustic scenes is crucial for perception in real world situations. We analyzed an objective psychophysical measure of stream segregation obtained while simultaneously recording forebrain neurons in the European starlings to investigate neural correlates of segregating a stream of A tones from a stream of B tones presented at one-half the rate. The objective measure, sensitivity for time shift detection of the B tone, was higher when the A and B tones were of the same frequency (one stream) compared with when there was a 6- or 12-semitone difference between them (two streams). The sensitivity for representing time shifts in spiking patterns was correlated with the behavioral sensitivity. The spiking patterns reflected the stimulus characteristics but not the behavioral response, indicating that the birds' primary cortical field represents the segregated streams, but not the decision process.

Contribution of correlated noise and selective decoding to choice probability measurements in extrastriate visual cortex

Trial by trial covariations between neural activity and perceptual decisions (quantified by choice Probability, CP) have been used to probe the contribution of sensory neurons to perceptual decisions. CPs are thought to be determined by both selective decoding of neural activity and by the structure of correlated noise among neurons, but the respective roles of these factors in creating CPs have been controversial. We used biologically-constrained simulations to explore this issue, taking advantage of a peculiar pattern of CPs exhibited by multisensory neurons in area MSTd that represent self-motion. Although models that relied on correlated noise or selective decoding could both account for the peculiar pattern of CPs, predictions of the selective decoding model were substantially more consistent with various features of the neural and behavioral data. While correlated noise is essential to observe CPs, our findings suggest that selective decoding of neuronal signals also plays important roles.

DOI:http://dx.doi.org/10.7554/eLife.02670.001

Even the simplest tasks require the brain to process vast amounts of information. To take a step forward, for example, the brain must process information about the orientation of the animal's body and what the animal is seeing, hearing and feeling in order to determine whether any obstacles stand in the way. The brain must integrate all this information to make decisions about how to proceed. And once a decision is made, the brain must send signals via the nervous system to the muscles to physically move the foot forward.

Specialized brain cells called sensory neurons help to process this sensory information. For example, visual neurons process information about what the animal sees, while auditory neurons process information about what it hears. Other sensory neurons—called multisensory neurons—can process information coming from more than one of an animal's senses.

For more than two decades, researchers have known that the firing of an individual sensory neuron can be linked to the decision that an animal makes about the meaning of the sensory information it has received. The ability to predict whether an animal will make a given decision based on the firing of individual sensory neurons is often referred to as a ‘choice probability’. Measurements of single neurons have often been used to try to work out how the brain decodes the sensory information that is needed to carry out a specific task. However, it remains unclear whether choice probabilities really reflect how sensory information is decoded in the brain, or whether these measurements are just reflecting coordinated patterns of background ‘noise’ among the neurons as the decisions are being made.

Gu et al. set out to help resolve this debate by examining choice probabilities in the multisensory neurons in one area of the brain. A series of experiments was conducted to see how these neurons process information, both from the eyes and the part of the inner ear that helps control balance, to work out the direction in which an animal was moving. By performing computer simulations of the activity of groups of neurons, Gu et al. found that choice probability measurements are better explained by the models whereby these measurements did reflect the strategy that is used to decode the sensory information. Models based solely on patterns of correlated noise did not explain the data as well, though Gu et al. suggest that this noise is likely to also contribute to the observed effects.

Following on from the work of Gu et al., a major challenge will be to see if it is possible to infer how the brain extracts the relevant information from the different sensory neurons. This may require recordings from large groups of neurons, but it might help us to decipher how patterns of activity in the brain lead to decisions about the world around us.

DOI:http://dx.doi.org/10.7554/eLife.02670.002

Decision-related activity in sensory neurons may depend on the columnar architecture of cerebral cortex

Many studies have reported correlations between the activity of sensory neurons and animals' judgments in discrimination tasks. Here, we suggest that such neuron-behavior correlations may require a cortical map for the task relevant features. This would explain why studies using discrimination tasks based on disparity in area V1 have not found these correlations: V1 contains no map for disparity. This scheme predicts that activity of V1 neurons correlates with decisions in an orientation-discrimination task. To test this prediction, we trained two macaque monkeys in a coarse orientation discrimination task using band-pass-filtered dynamic noise. The two orientations were always 90° apart and task difficulty was controlled by varying the orientation bandwidth of the filter. While the trained animals performed this task, we recorded from orientation-selective V1 neurons (n = 82, n = 31 for Monkey 1, n = 51 for Monkey 2). For both monkeys, we observed significant correlation (quantified as "choice probabilities") of the V1 activity with the monkeys' perceptual judgments (mean choice probability 0.54, p = 10(-5)). In one of these animals, we had previously measured choice probabilities in a disparity discrimination task in V1, which had been at chance (0.49, not significantly different from 0.5). The choice probabilities in this monkey for the orientation discrimination task were significantly larger than those for the disparity discrimination task (p = 0.032). These results are predicted by our suggestion that choice probabilities are only observed for cortical sensory neurons that are organized in maps for the task-relevant feature.

Ongoing behavior predicts perceptual report of interval duration

The ability to estimate the passage of time is essential for adaptive behavior in complex environments. Yet, it is not known how the brain encodes time over the durations necessary to explain animal behavior. Under temporally structured reinforcement schedules, animals tend to develop temporally structured behavior, and interval timing has been suggested to be accomplished by learning sequences of behavioral states. If this is true, trial to trial fluctuations in behavioral sequences should be predictive of fluctuations in time estimation. We trained rodents in an duration categorization task while continuously monitoring their behavior with a high speed camera. Animals developed highly reproducible behavioral sequences during the interval being timed. Moreover, those sequences were often predictive of perceptual report from early in the trial, providing support to the idea that animals may use learned behavioral patterns to estimate the duration of time intervals. To better resolve the issue, we propose that continuous and simultaneous behavioral and neural monitoring will enable identification of neural activity related to time perception that is not explained by ongoing behavior.

Neuronal origins of choice variability in economic decisions

To investigate the mechanisms through which economic decisions are formed, I examined the activity of neurons in the orbitofrontal cortex while monkeys chose between different juice types. Different classes of cells encoded the value of individual offers (offer value), the value of the chosen option (chosen value), or the identity of the chosen juice (chosen juice). Choice variability was partly explained by the tendency to repeat choices (choice hysteresis). Surprisingly, near-indifference decisions did not reflect fluctuations in the activity of offer value cells. In contrast, near-indifference decisions correlated with fluctuations in the preoffer activity of chosen juice cells. After the offer, the activity of chosen juice cells reflected the decision difficulty but did not resemble a race-to-threshold. Finally, chosen value cells presented an "activity overshooting" closely related to the decision difficulty and possibly due to fluctuations in the relative value of the juices. This overshooting was independent of choice hysteresis.

The evolution of a disparity decision in human visual cortex

We used fMRI-informed EEG source-imaging in humans to characterize the dynamics of cortical responses during a disparity-discrimination task. After the onset of a disparity-defined target, decision-related activity was found within an extended cortical network that included several occipital regions of interest (ROIs): V4, V3A, hMT+ and the Lateral Occipital Complex (LOC). By using a response-locked analysis, we were able to determine the timing relationships in this network of ROIs relative to the subject's behavioral response. Choice-related activity appeared first in the V4 ROI almost 200 ms before the button press and then subsequently in the V3A ROI. Modeling of the responses in the V4 ROI suggests that this area provides an early contribution to disparity discrimination. Choice-related responses were also found after the button-press in ROIs V4, V3A, LOC and hMT+. Outside the visual cortex, choice-related activity was found in the frontal and temporal poles before the button-press. By combining the spatial resolution of fMRI-informed EEG source imaging with the ability to sort out neural activity occurring before, during and after the behavioral manifestation of the decision, our study is the first to assign distinct functional roles to the extra-striate ROIs involved in perceptual decisions based on disparity, the primary cue for depth.

Emerging principles of population coding: in search for the neural code

Population coding theory aims to provide quantitative tests for hypotheses concerning the neural code. Over the last two decades theory has focused on analyzing the ways in which various parameters that characterize neuronal responses to external stimuli affect the information content of these responses. This article reviews and provides an intuitive explanation for the major effects of noise correlations and neuronal heterogeneity, and discusses their implications for our ability to investigate the neural code. It is argued that to test neural code hypotheses further, additional constraints are required, including relating trial-to-trial variation in neuronal population responses to behavioral decisions and specifying how information is decoded by downstream networks.

Animation of natural scene by virtual eye-movements evokes high precision and low noise in V1 neurons

Synaptic noise is thought to be a limiting factor for computational efficiency in the brain. In visual cortex (V1), ongoing activity is present in vivo, and spiking responses to simple stimuli are highly unreliable across trials. Stimulus statistics used to plot receptive fields, however, are quite different from those experienced during natural visuomotor exploration. We recorded V1 neurons intracellularly in the anaesthetized and paralyzed cat and compared their spiking and synaptic responses to full field natural images animated by simulated eye-movements to those evoked by simpler (grating) or higher dimensionality statistics (dense noise). In most cells, natural scene animation was the only condition where high temporal precision (in the 10–20 ms range) was maintained during sparse and reliable activity. At the subthreshold level, irregular but highly reproducible membrane potential dynamics were observed, even during long (several 100 ms) “spike-less” periods. We showed that both the spatial structure of natural scenes and the temporal dynamics of eye-movements increase the signal-to-noise ratio by a non-linear amplification of the signal combined with a reduction of the subthreshold contextual noise. These data support the view that the sparsening and the time precision of the neural code in V1 may depend primarily on three factors: (1) broadband input spectrum: the bandwidth must be rich enough for recruiting optimally the diversity of spatial and time constants during recurrent processing; (2) tight temporal interplay of excitation and inhibition: conductance measurements demonstrate that natural scene statistics narrow selectively the duration of the spiking opportunity window during which the balance between excitation and inhibition changes transiently and reversibly; (3) signal energy in the lower frequency band: a minimal level of power is needed below 10 Hz to reach consistently the spiking threshold, a situation rarely reached with visual dense noise.

Variance in population firing rate as a measure of slow time-scale correlation

Correlated variability in the spiking responses of pairs of neurons, also known as spike count correlation, is a key indicator of functional connectivity and a critical factor in population coding. Underscoring the importance of correlation as a measure for cognitive neuroscience research is the observation that spike count correlations are not fixed, but are rather modulated by perceptual and cognitive context. Yet while this context fluctuates from moment to moment, correlation must be calculated over multiple trials. This property undermines its utility as a dependent measure for investigations of cognitive processes which fluctuate on a trial-to-trial basis, such as selective attention. A measure of functional connectivity that can be assayed on a moment-to-moment basis is needed to investigate the single-trial dynamics of populations of spiking neurons. Here, we introduce the measure of population variance in normalized firing rate for this goal. We show using mathematical analysis, computer simulations and in vivo data how population variance in normalized firing rate is inversely related to the latent correlation in the population, and how this measure can be used to reliably classify trials from different typical correlation conditions, even when firing rate is held constant. We discuss the potential advantages for using population variance in normalized firing rate as a dependent measure for both basic and applied neuroscience research.

The what, where and how of auditory-object perception

The fundamental perceptual unit in hearing is the 'auditory object'. Similar to visual objects, auditory objects are the computational result of the auditory system's capacity to detect, extract, segregate and group spectrotemporal regularities in the acoustic environment; the multitude of acoustic stimuli around us together form the auditory scene. However, unlike the visual scene, resolving the component objects within the auditory scene crucially depends on their temporal structure. Neural correlates of auditory objects are found throughout the auditory system. However, neural responses do not become correlated with a listener's perceptual reports until the level of the cortex. The roles of different neural structures and the contribution of different cognitive states to the perception of auditory objects are not yet fully understood.

An optimal decision population code that accounts for correlated variability unambiguously predicts a subject's choice

Decisions emerge from the concerted activity of neuronal populations distributed across brain circuits. However, the analytical tools best suited to decode decision signals from neuronal populations remain unknown. Here we show that knowledge of correlated variability between pairs of cortical neurons allows perfect decoding of decisions from population firing rates. We recorded pairs of neurons from secondary somatosensory (S2) and premotor (PM) cortices while monkeys reported the presence or absence of a tactile stimulus. We found that while populations of S2 and sensory-like PM neurons are only partially correlated with behavior, those PM neurons active during a delay period preceding the motor report predict unequivocally the animal's decision report. Thus, a population rate code that optimally reveals a subject's perceptual decisions can be implemented just by knowing the correlations of PM neurons representing decision variables.

Reduced choice-related activity and correlated noise accompany perceptual deficits following unilateral vestibular lesion

Signals from the bilateral vestibular labyrinths work in tandem to generate robust estimates of our motion and orientation in the world. The relative contributions of each labyrinth to behavior, as well as how the brain recovers after unilateral peripheral damage, have been characterized for motor reflexes, but never for perceptual functions. Here we measure perceptual deficits in a heading discrimination task following surgical ablation of the neurosensory epithelium in one labyrinth. We found large increases in heading discrimination thresholds and large perceptual biases at 1 wk postlesion. Repeated testing thereafter improved heading perception, but vestibular discrimination thresholds remained elevated 3 mo postlesion. Electrophysiological recordings from the contralateral vestibular and cerebellar nuclei revealed elevated neuronal discrimination thresholds, elevated neurometric-to-psychometric threshold ratios, and reduced trial-by-trial correlations with perceptual decisions ["choice probabilities" (CPs)]. The relationship between CP and neuronal threshold was shallower, but not significantly altered, suggesting that smaller CPs in lesioned animals could be largely attributable to greater neuronal thresholds. Simultaneous recordings from pairs of neurons revealed that correlated noise among neurons was also reduced following the lesion. Simulations of a simple pooling model, which takes into account the observed changes in tuning slope and correlated noise, qualitatively accounts for the elevated psychophysical thresholds and neurometric-to-psychometric ratios, as well as the decreased CPs. Thus, cross-labyrinthine interactions appear to play important roles in enhancing neuronal and perceptual sensitivity, strengthening interneuronal correlations, and facilitating correlations between neural activity and perceptual decisions.

A causal role for V5/MT neurons coding motion-disparity conjunctions in resolving perceptual ambiguity

Judgments about the perceptual appearance of visual objects require the combination of multiple parameters, like location, direction, color, speed, and depth. Our understanding of perceptual judgments has been greatly informed by studies of ambiguous figures, which take on different appearances depending upon the brain state of the observer. Here we probe the neural mechanisms hypothesized as responsible for judging the apparent direction of rotation of ambiguous structure from motion (SFM) stimuli. Resolving the rotation direction of SFM cylinders requires the conjoint decoding of direction of motion and binocular depth signals [1, 2]. Within cortical visual area V5/MT of two macaque monkeys, we applied electrical stimulation at sites with consistent multiunit tuning to combinations of binocular depth and direction of motion, while the monkey made perceptual decisions about the rotation of SFM stimuli. For both ambiguous and unambiguous SFM figures, rotation judgments shifted as if we had added a specific conjunction of disparity and motion signals to the stimulus elements. This is the first causal demonstration that the activity of neurons in V5/MT contributes directly to the perception of SFM stimuli and by implication to decoding the specific conjunction of disparity and motion, the two different visual cues whose combination drives the perceptual judgment.

•

Targeted search for brain sites encoding conjunctions of stereo and motion

•

Electrical microstimulation alters perceptual choice and resolves visual ambiguity

•

Causal evidence for direct role of V5/MT in cue combination

•

Stimulation at cortical sites with conjoint encoding is particularly effective

Common rules guide comparisons of speed and direction of motion in the dorsolateral prefrontal cortex

When a monkey needs to decide whether motion direction of one stimulus is the same or different as that of another held in working memory, neurons in dorsolateral prefrontal cortex (DLPFC) faithfully represent the motion directions being evaluated and contribute to their comparison. Here, we examined whether DLPFC neurons are more generally involved in other types of sensory comparisons. Such involvement would support the existence of generalized sensory comparison mechanisms within DLPFC, shedding light on top-down influences this region is likely to provide to the upstream sensory neurons during comparison tasks. We recorded activity of individual neurons in the DLPFC while monkeys performed a memory-guided decision task in which the important dimension was the speed of two sequentially presented moving random-dot stimuli. We found that many neurons, both narrow-spiking putative local interneurons and broad-spiking putative pyramidal output cells, were speed-selective, with tuning reminiscent of that observed in the motion processing middle temporal (MT) cortical area. Throughout the delay, broad-spiking neurons were more active, showing anticipatory rate modulation and transient periods of speed selectivity. During the comparison stimulus, responses of both cell types were modulated by the speed of the first stimulus, and their activity was highly predictive of the animals' behavioral report. These results are similar to those found for comparisons of motion direction, suggesting the existence of generalized neural mechanisms in the DLPFC subserving the comparison of sensory signals.

Auditory cortex represents both pitch judgments and the corresponding acoustic cues

The neural processing of sensory stimuli involves a transformation of physical stimulus parameters into perceptual features, and elucidating where and how this transformation occurs is one of the ultimate aims of sensory neurophysiology. Recent studies have shown that the firing of neurons in early sensory cortex can be modulated by multisensory interactions [1-5], motor behavior [1, 3, 6, 7], and reward feedback [1, 8, 9], but it remains unclear whether neural activity is more closely tied to perception, as indicated by behavioral choice, or to the physical properties of the stimulus. We investigated which of these properties are predominantly represented in auditory cortex by recording local field potentials (LFPs) and multiunit spiking activity in ferrets while they discriminated the pitch of artificial vowels. We found that auditory cortical activity is informative both about the fundamental frequency (F0) of a target sound and also about the pitch that the animals appear to perceive given their behavioral responses. Surprisingly, although the stimulus F0 was well represented at the onset of the target sound, neural activity throughout auditory cortex frequently predicted the reported pitch better than the target F0.

Functional specializations of the ventral intraparietal area for multisensory heading discrimination

The ventral intraparietal area (VIP) of the macaque brain is a multimodal cortical region with directionally selective responses to visual and vestibular stimuli. To explore how these signals contribute to self-motion perception, neural activity in VIP was monitored while macaques performed a fine heading discrimination task based on vestibular, visual, or multisensory cues. For neurons with congruent visual and vestibular heading tuning, discrimination thresholds improved during multisensory stimulation, suggesting that VIP (like the medial superior temporal area; MSTd) may contribute to the improved perceptual discrimination seen during cue combination. Unlike MSTd, however, few VIP neurons showed opposite visual/vestibular tuning over the range of headings relevant to behavior, and those few cells showed reduced sensitivity under cue combination. Our data suggest that the heading tuning of some VIP neurons may be locally remodeled to increase the proportion of cells with congruent tuning over the behaviorally relevant stimulus range. VIP neurons also showed much stronger trial-by-trial correlations with perceptual decisions (choice probabilities; CPs) than MSTd neurons. While this may suggest that VIP neurons are more strongly linked to heading perception, we also find that correlated noise is much stronger among pairs of VIP neurons, with noise correlations averaging 0.14 in VIP as compared with 0.04 in MSTd. Thus, the large CPs in VIP could be a consequence of strong interneuronal correlations. Together, our findings suggest that VIP neurons show specializations that may make them well equipped to play a role in multisensory integration for heading perception.

Inferring decoding strategies from choice probabilities in the presence of correlated variability

The activity of cortical neurons in sensory areas covaries with perceptual decisions, a relationship that is often quantified by choice probabilities. Although choice probabilities have been measured extensively, their interpretation has remained fraught with difficulty. We derive the mathematical relationship between choice probabilities, read-out weights and correlated variability in the standard neural decision-making model. Our solution allowed us to prove and generalize earlier observations on the basis of numerical simulations and to derive new predictions. Notably, our results indicate how the read-out weight profile, or decoding strategy, can be inferred from experimentally measurable quantities. Furthermore, we developed a test to decide whether the decoding weights of individual neurons are optimal for the task, even without knowing the underlying correlations. We confirmed the practicality of our approach using simulated data from a realistic population model. Thus, our findings provide a theoretical foundation for a growing body of experimental results on choice probabilities and correlations.