Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron's causality?

Mapping model units to visual neurons reveals population code for social behaviour

The rich variety of behaviours observed in animals arises through the interplay between sensory processing and motor control. To understand these sensorimotor transformations, it is useful to build models that predict not only neural responses to sensory input1-5 but also how each neuron causally contributes to behaviour6,7. Here we demonstrate a novel modelling approach to identify a one-to-one mapping between internal units in a deep neural network and real neurons by predicting the behavioural changes that arise from systematic perturbations of more than a dozen neuronal cell types. A key ingredient that we introduce is 'knockout training', which involves perturbing the network during training to match the perturbations of the real neurons during behavioural experiments. We apply this approach to model the sensorimotor transformations of Drosophila melanogaster males during a complex, visually guided social behaviour8-11. The visual projection neurons at the interface between the optic lobe and central brain form a set of discrete channels12, and prior work indicates that each channel encodes a specific visual feature to drive a particular behaviour13,14. Our model reaches a different conclusion: combinations of visual projection neurons, including those involved in non-social behaviours, drive male interactions with the female, forming a rich population code for behaviour. Overall, our framework consolidates behavioural effects elicited from various neural perturbations into a single, unified model, providing a map from stimulus to neuronal cell type to behaviour, and enabling future incorporation of wiring diagrams of the brain15 into the model.

Temporal and spatial properties of vestibular signals for perception of self-motion

It is well recognized that the vestibular system is involved in numerous important cognitive functions, including self-motion perception, spatial orientation, locomotion, and vector-based navigation, in addition to basic reflexes, such as oculomotor or body postural control. Consistent with this rationale, vestibular signals exist broadly in the brain, including several regions of the cerebral cortex, potentially allowing tight coordination with other sensory systems to improve the accuracy and precision of perception or action during self-motion. Recent neurophysiological studies in animal models based on single-cell resolution indicate that vestibular signals exhibit complex spatiotemporal dynamics, producing challenges in identifying their exact functions and how they are integrated with other modality signals. For example, vestibular and optic flow could provide congruent and incongruent signals regarding spatial tuning functions, reference frames, and temporal dynamics. Comprehensive studies, including behavioral tasks, neural recording across sensory and sensory-motor association areas, and causal link manipulations, have provided some insights into the neural mechanisms underlying multisensory self-motion perception.

Contributions of early and mid-level visual cortex to high-level object categorization

The complexity of visual features for which neurons are tuned increases from early to late stages of the ventral visual stream. Thus, the standard hypothesis is that high-level functions like object categorization are primarily mediated by higher visual areas because they require more complex image formats that are not evident in early visual processing stages. However, human observers can categorize images as objects or animals or as big or small even when the images preserve only some low- and mid-level features but are rendered unidentifiable ('texforms', Long et al., 2018). This observation suggests that even the early visual cortex, in which neurons respond to simple stimulus features, may already encode signals about these more abstract high-level categorical distinctions. We tested this hypothesis by recording from populations of neurons in early and mid-level visual cortical areas while rhesus monkeys viewed texforms and their unaltered source stimuli (simultaneous recordings from areas V1 and V4 in one animal and separate recordings from V1 and V4 in two others). Using recordings from a few dozen neurons, we could decode the real-world size and animacy of both unaltered images and texforms. Furthermore, this neural decoding accuracy across stimuli was related to the ability of human observers to categorize texforms by real-world size and animacy. Our results demonstrate that neuronal populations early in the visual hierarchy contain signals useful for higher-level object perception and suggest that the responses of early visual areas to simple stimulus features display preliminary untangling of higher-level distinctions.

From calcium imaging to graph topology

Systems neuroscience is facing an ever-growing mountain of data. Recent advances in protein engineering and microscopy have together led to a paradigm shift in neuroscience; using fluorescence, we can now image the activity of every neuron through the whole brain of behaving animals. Even in larger organisms, the number of neurons that we can record simultaneously is increasing exponentially with time. This increase in the dimensionality of the data is being met with an explosion of computational and mathematical methods, each using disparate terminology, distinct approaches, and diverse mathematical concepts. Here we collect, organize, and explain multiple data analysis techniques that have been, or could be, applied to whole-brain imaging, using larval zebrafish as an example model. We begin with methods such as linear regression that are designed to detect relations between two variables. Next, we progress through network science and applied topological methods, which focus on the patterns of relations among many variables. Finally, we highlight the potential of generative models that could provide testable hypotheses on wiring rules and network progression through time, or disease progression. While we use examples of imaging from larval zebrafish, these approaches are suitable for any population-scale neural network modeling, and indeed, to applications beyond systems neuroscience. Computational approaches from network science and applied topology are not limited to larval zebrafish, or even to systems neuroscience, and we therefore conclude with a discussion of how such methods can be applied to diverse problems across the biological sciences.

Behaviorally relevant decision coding in primary somatosensory cortex neurons

Primary sensory cortex is thought to process incoming sensory information, while decision variables important for driving behavior are assumed to arise downstream in the processing hierarchy. Here, we used population two-photon calcium imaging and targeted two-photon optogenetic stimulation of neurons in layer 2/3 of mouse primary somatosensory cortex (S1) during a texture discrimination task to test for the presence of decision signals and probe their behavioral relevance. Small but distinct populations of neurons carried information about the stimulus irrespective of the behavioral outcome (stimulus neurons), or about the choice irrespective of the presented stimulus (decision neurons). Decision neurons show categorical coding that develops during learning, and lack a conclusive decision signal in Miss trials. All-optical photostimulation of decision neurons during behavior improves behavioral performance, establishing a causal role in driving behavior. The fact that stimulus and decision neurons are intermingled challenges the idea of S1 as a purely sensory area, and causal perturbation suggests a direct involvement of S1 decision neurons in the decision-making process.

Task-induced neural covariability as a signature of approximate Bayesian learning and inference

Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called “differential correlations” as the observer’s internal model learns the stimulus distribution, and the observer’s behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject’s internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.

Perceptual decision-making has classically been studied in the context of feedforward encoding/ decoding models. Here, we derive predictions for the responses of sensory neurons under the assumption that the brain performs hierarchical Bayesian inference, including feedback signals that communicate task-specific prior expectations. Interestingly, those predictions stand in contrast to some of the conclusions drawn in the classic framework. In particular, we find that Bayesian learning predicts the increase of a type of correlated variability called “differential correlations” over the course of learning. Differential correlations limit information, and hence are seen as harmful in feedforward models. Since our results are also specific to the statistics of a given task, and since they hold under a wide class of theories about how Bayesian probabilities may be represented by neural responses, they constitute a strong test of the Bayesian Brain hypothesis. Our results can explain the task-dependence of correlated variability in prior studies and suggest a reason why these kinds of correlations are surprisingly common in empirical data. Interpreted in a probabilistic framework, correlated variability provides a window into an observer’s task-related beliefs.

Attention improves information flow between neuronal populations without changing the communication subspace

Visual attention allows observers to change the influence of different parts of a visual scene on their behavior, suggesting that information can be flexibly shared between visual cortex and neurons involved in decision making. We investigated the neural substrate of flexible information routing by analyzing the activity of populations of visual neurons in the medial temporal area (MT) and oculo-motor neurons in the superior colliculus (SC) while rhesus monkeys switched spatial attention. We demonstrated that attention increases the efficacy of visuomotor communication: trial-to-trial variability in SC population activity could be better predicted by the activity of the MT population (and vice versa) when attention was directed toward their joint receptive fields. Surprisingly, this improvement in prediction was not explained by changes in the dimensionality of the shared subspace or in the magnitude of local or shared pairwise noise correlations. These results lay a foundation for future theoretical and experimental studies into how visual attention can flexibly change information flow between sensory and decision neurons.

Sensory coding and the causal impact of mouse cortex in a visual decision

Correlates of sensory stimuli and motor actions are found in multiple cortical areas, but such correlates do not indicate whether these areas are causally relevant to task performance. We trained mice to discriminate visual contrast and report their decision by steering a wheel. Widefield calcium imaging and Neuropixels recordings in cortex revealed stimulus-related activity in visual (VIS) and frontal (MOs) areas, and widespread movement-related activity across the whole dorsal cortex. Optogenetic inactivation biased choices only when targeted at VIS and MOs,proportionally to each site's encoding of the visual stimulus, and at times corresponding to peak stimulus decoding. A neurometric model based on summing and subtracting activity in VIS and MOs successfully described behavioral performance and predicted the effect of optogenetic inactivation. Thus, sensory signals localized in visual and frontal cortex play a causal role in task performance, while widespread dorsal cortical signals correlating with movement reflect processes that do not play a causal role.

Correlated structure of neuronal firing in macaque visual cortex limits information for binocular depth discrimination

Variability in cortical neural activity potentially limits sensory discriminations. Theoretical work shows that information required to discriminate two similar stimuli is limited by the correlation structure of cortical variability. We investigated these information-limiting correlations by recording simultaneously from visual cortical areas primary visual cortex (V1) and extrastriate area V4 in macaque monkeys performing a binocular, stereo depth discrimination task. Within both areas, noise correlations on a rapid temporal scale (20–30 ms) were stronger for neuron pairs with similar selectivity for binocular depth, meaning that these correlations potentially limit information for making the discrimination. Between-area correlations (V1 to V4) were different, being weaker for neuron pairs with similar tuning and having a slower temporal scale (100+ ms). Fluctuations in these information-limiting correlations just prior to the detection event were associated with changes in behavioral accuracy. Although these correlations limit the recovery of information about sensory targets, their impact may be curtailed by integrative processing of signals across multiple brain areas.

NEW & NOTEWORTHY Correlated noise reduces the stimulus information in visual cortical neurons during experimental performance of binocular depth discriminations. The temporal scale of these correlations is important. Rapid (20–30 ms) correlations reduce information within and between areas V1 and V4, whereas slow (>100 ms) correlations between areas do not. Separate cortical areas appear to act together to maintain signal fidelity. Rapid correlations reduce the neuronal signal difference between stimuli and adversely affect perceptual discrimination.

Stimulus-dependent relationships between behavioral choice and sensory neural responses

Understanding perceptual decision-making requires linking sensory neural responses to behavioral choices. In two-choice tasks, activity-choice covariations are commonly quantified with a single measure of choice probability (CP), without characterizing their changes across stimulus levels. We provide theoretical conditions for stimulus dependencies of activity-choice covariations. Assuming a general decision-threshold model, which comprises both feedforward and feedback processing and allows for a stimulus-modulated neural population covariance, we analytically predict a very general and previously unreported stimulus dependence of CPs. We develop new tools, including refined analyses of CPs and generalized linear models with stimulus-choice interactions, which accurately assess the stimulus- or choice-driven signals of each neuron, characterizing stimulus-dependent patterns of choice-related signals. With these tools, we analyze CPs of macaque MT neurons during a motion discrimination task. Our analysis provides preliminary empirical evidence for the promise of studying stimulus dependencies of choice-related signals, encouraging further assessment in wider data sets.

Estimating Fisher discriminant error in a linear integrator model of neural population activity

Decoding approaches provide a useful means of estimating the information contained in neuronal circuits. In this work, we analyze the expected classification error of a decoder based on Fisher linear discriminant analysis. We provide expressions that relate decoding error to the specific parameters of a population model that performs linear integration of sensory input. Results show conditions that lead to beneficial and detrimental effects of noise correlation on decoding. Further, the proposed framework sheds light on the contribution of neuronal noise, highlighting cases where, counter-intuitively, increased noise may lead to improved decoding performance. Finally, we examined the impact of dynamical parameters, including neuronal leak and integration time constant, on decoding. Overall, this work presents a fruitful approach to the study of decoding using a comprehensive theoretical framework that merges dynamical parameters with estimates of readout error.

Frequency-separated principal component analysis of cortical population activity

A hallmark of neocortical activity is the presence of low-dimensional fluctuations in firing rate that are coordinated across neurons. However, the impact of these fluctuations on sensory processing remains unclear. Here, we examined fluctuations in populations of orientation-selective neurons from anesthetized macaque primary visual cortex (V1) during stimulus viewing as well as spontaneous activity. We introduce a novel approach termed frequency-separated principal component analysis (FS-PCA) to characterize these fluctuations. This method unveiled a distribution of components with a broad range of frequencies whose eigenvalues and variance followed an approximate power law. During stimulus viewing, subpopulations of V1 neurons correlated either positively or negatively with low-dimensional fluctuations. These two subpopulations displayed distinct activation properties and noise correlations in response to sensory input. Together, results suggest that slow, low-dimensional fluctuations in V1 population activity shape the response of individual neurons to oriented stimuli and may impact the transmission of sensory information to downstream regions of the primary visual system.NEW & NOTEWORTHY A method termed frequency-separated principal component analysis (FS-PCA) is introduced for analyzing populations of simultaneously recorded neurons. This framework extends standard principal component analysis by extracting components of activity delimited to specific frequency bands. FS-PCA revealed that circuits of the primary visual cortex generate a broad range of components dominated by low-frequency activity. Furthermore, low-dimensional fluctuations in population activity modulated the response of individual neurons to sensory input.

Temporal information loss in the macaque early visual system

Stimuli that modulate neuronal activity are not always detectable, indicating a loss of information between the modulated neurons and perception. To identify where in the macaque visual system information about periodic light modulations is lost, signal-to-noise ratios were compared across simulated cone photoreceptors, lateral geniculate nucleus (LGN) neurons, and perceptual judgements. Stimuli were drifting, threshold-contrast Gabor patterns on a photopic background. The sensitivity of LGN neurons, extrapolated to populations, was similar to the monkeys' at low temporal frequencies. At high temporal frequencies, LGN sensitivity exceeded the monkeys' and approached the upper bound set by cone photocurrents. These results confirm a loss of high-frequency information downstream of the LGN. However, this loss accounted for only about 5% of the total. Phototransduction accounted for essentially all of the rest. Together, these results show that low temporal frequency information is lost primarily between the cones and the LGN, whereas high-frequency information is lost primarily within the cones, with a small additional loss downstream of the LGN.

Choice (-history) correlations in sensory cortex: cause or consequence?

One challenge in neuroscience, as in other areas of science, is to make inferences about the underlying causal structure from correlational data. Here, we discuss this challenge in the context of choice correlations in sensory neurons, that is, trial-by-trial correlations, unexplained by the stimulus, between the activity of sensory neurons and an animal's perceptual choice. Do these choice-correlations reflect feedforward, feedback signalling, both, or neither? We highlight recent results of correlational and causal examinations of choice and choice-history signals in sensory, and in part sensorimotor, cortex and address formal statistical frameworks to infer causal interactions from data.

The Segmentation of Proto-Objects in the Monkey Primary Visual Cortex

During visual perception, the brain enhances the representations of image regions that belong to figures and suppresses those that belong to the background. Natural images contain many regions that initially appear to be part of a figure when analyzed locally (proto-objects) but are actually part of the background if the whole image is considered. These proto-grounds must be correctly assigned to the background to allow correct shape identification and guide behavior. To understand how the brain resolves this conflict between local and global processing, we recorded neuronal activity from the primary visual cortex (V1) of macaque monkeys while they discriminated between n/u shapes that have a central proto-ground region. We studied the fine-grained spatiotemporal profile of neural activity evoked by the n/u shape and found that neural representation of the object proceeded from a coarse-to-fine resolution. Approximately 100 ms after the stimulus onset, the representation of the proto-ground region was enhanced together with the rest of the n/u surface, but after ∼115 ms, the proto-ground was suppressed back to the level of the background. Suppression of the proto-ground was only present in animals that had been trained to perform the shape-discrimination task, and it predicted the choice of the animal on a trial-by-trial basis. Attention enhanced figure-ground modulation, but it had no effect on the strength of proto-ground suppression. The results indicate that the accuracy of scene segmentation is sharpened by a suppressive process that resolves local ambiguities by assigning proto-grounds to the background.

Laws of concatenated perception: Vision goes for novelty, decisions for perseverance

Every instant of perception depends on a cascade of brain processes calibrated to the history of sensory and decisional events. In the present work, we show that human visual perception is constantly shaped by two contrasting forces exerted by sensory adaptation and past decisions. In a series of experiments, we used multilevel modeling and cross-validation approaches to investigate the impact of previous stimuli and decisions on behavioral reports during adjustment and forced-choice tasks. Our results revealed that each perceptual report is permeated by opposite biases from a hierarchy of serially dependent processes: Low-level adaptation repels perception away from previous stimuli, whereas decisional traces attract perceptual reports toward the recent past. In this hierarchy of serial dependence, "continuity fields" arise from the inertia of decisional templates and not from low-level sensory processes. This finding is consistent with a Two-process model of serial dependence in which the persistence of readout weights in a decision unit compensates for sensory adaptation, leading to attractive biases in sequential perception. We propose a unified account of serial dependence in which functionally distinct mechanisms, operating at different stages, promote the differentiation and integration of visual information over time.

Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation

It is controversial whether covariation between neuronal activity and perceptual choice (i.e., choice correlation) reflects the functional readout of sensory signals. Here, we combined choice-correlation measures and electrical microstimulation on a site-to-site basis in the medial superior temporal area (MST), middle temporal area (MT), and ventral intraparietal area (VIP) when macaques discriminated between motion directions in both fine and coarse tasks. Microstimulation generated comparable effects between tasks but heterogeneous effects across and within brain regions. Within the MST and MT, microstimulation significantly biased an animal's choice toward the sensory preference instead of choice-related signals of the stimulated units. This was particularly evident for sites with conflict preference of sensory and choice-related signals. In the VIP, microstimulation failed to produce significant effects in either task despite strong choice correlations presented in this area. Our results suggest that sensory readout may not be inferred from choice-related signals during perceptual decision-making tasks.

Resveratrol Prevents Cellular and Behavioral Sensory Alterations in the Animal Model of Autism Induced by Valproic Acid

Autism spectrum disorder (ASD) is characterized by impairments in both social communication and interaction and repetitive or stereotyped behaviors. Although its etiology remains unknown, genetic and environmental risk factors have been associated with this disorder, including the exposure to valproic acid (VPA) during pregnancy. Resveratrol (RSV) is an anti-inflammatory and antioxidant molecule known to prevent social impairments in the VPA animal model of autism. This study aimed to analyze the effects of prenatal exposure to VPA, as well as possible preventive effects of RSV, on sensory behavior, the localization of GABAergic parvalbumin (PV⁺) neurons in sensory brain regions and the expression of proteins of excitatory and inhibitory synapses. Pregnant rats were treated daily with RSV (3.6 mg/kg) from E6.5 to E18.5 and injected with VPA (600 mg/kg) in the E12.5. Male pups were analyzed in Nest Seeking (NS) behavior and in whisker nuisance task (WNT). At P30, the tissues were removed and analyzed by immunofluorescence and western blotting. Our data showed for the first time an altered localization of PV⁺-neurons in primary sensory cortex and amygdala. We also showed a reduced level of gephyrin in the primary somatosensory area (PSSA) of VPA animals. The treatment with RSV prevented all the aforementioned alterations triggered by VPA. Our data shed light on the relevance of sensory component in ASD and highlights the interplay between RSV and VPA animal model as an important tool to investigate the pathophysiology of ASD.

Optimal Readout of Correlated Neural Activity in a Decision-Making Circuit

The neural correlates of decision making have been extensively studied with tasks involving a choice between two alternatives that is guided by visual cues. While a large body of work argues for a role of the lateral intraparietal (LIP) region of cortex in these tasks, this role may be confounded by the interaction between LIP and other regions, including medial temporal (MT) cortex. Here, we describe a simplified linear model of decision making that is adapted to two tasks: a motion discrimination and a categorization task. We show that the distinct contribution of MT and LIP may indeed be confounded in these tasks. In particular, we argue that the motion discrimination task relies on a straightforward visuomotor mapping, which leads to redundant information between MT and LIP. The categorization task requires a more complex mapping between visual information and decision behavior, and therefore does not lead to redundancy between MT and LIP. Going further, the model predicts that noise correlations within LIP should be greater in the categorization compared to the motion discrimination task due to the presence of shared inputs from MT. The impact of these correlations on task performance is examined by analytically deriving error estimates of an optimal linear readout for shared and unique inputs. Taken together, results clarify the contribution of MT and LIP to decision making and help characterize the role of noise correlations in these regions.

Feedback determines the structure of correlated variability in primary visual cortex

The variable responses of sensory neurons tend to be weakly correlated (spike-count correlation, r_sc). This is widely thought to reflect noise in shared afferents, in which case r_sc can limit the reliability of sensory coding. However, it could also be due to feedback from higher-order brain regions. Currently, the relative contributions of these sources are unknown. We addressed this by recording from populations of V1 neurons in macaques performing different discrimination tasks involving the same visual input. We found that the structure of r_sc (the way r_sc varied with neuronal stimulus preference) changed systematically with task instruction. Therefore, even at the earliest stage in the cortical visual hierarchy, r_sc structure during task performance primarily reflects feedback dynamics. Consequently, previous proposals for how r_sc constrains sensory processing need not apply. Furthermore, we show that correlations between the activity of single neurons and choice depend on feedback engaged by the task.

Perceptual Decisions in the Presence of Relevant and Irrelevant Sensory Evidence

Perceptual decisions in the presence of decision-irrelevant sensory information require a selection of decision-relevant sensory evidence. To characterize the mechanism that is responsible for separating decision-relevant from irrelevant sensory information we asked human subjects to make judgments about one of two simultaneously present motion components in a random dot stimulus. Subjects were able to ignore the decision-irrelevant component to a large degree, but their decisions were still influenced by the irrelevant sensory information. Computational modeling revealed that this influence was not simply the consequence of subjects forgetting at times which stimulus component they had been instructed to base their decision on. Instead, residual irrelevant information always seems to be leaking through, and the decision process is captured by a net sensory evidence signal being accumulated to a decision threshold. This net sensory evidence is a linear combination of decision-relevant and irrelevant sensory information. The selection process is therefore well-described by a strong linear gain modulation, which, in our experiment, resulted in the relevant sensory evidence having at least 10 times more impact on the decision than the irrelevant evidence.

Organization of orientation-specific whisker deflection responses in layer 2/3 of mouse somatosensory cortex

The rodent whisker-barrel system is characterized by its patterned somatotopic mapping between the sensory periphery and multiple regions of the brain. While somatotopy in the whisker system is established, we know far less about how preferences for stimulus orientation or other features are organized. Mouse somatosensation is an increasingly popular model for circuit-based dissection of perceptual decision making and learning, yet our understanding of how stimulus feature representations are organized in the cortex is incomplete. Here, we used in vivo two-photon calcium imaging to monitor activity of populations of layer (L) 2/3 neurons in the mouse primary somatosensory cortex during deflections of a single whisker in two orthogonal orientations (azimuthal or elevational). We split the population response to whisker deflections into an orientation-specific component and a non-specific component that reflected overall excitability in response to deflection of a single whisker. Orientation-specific responses were organized in a locally heterogeneous and spatially distributed manner. Correlations in the stimulus-independent trial-to-trial variability of pairs of neurons were higher among neurons that preferred the same orientation. These correlations depended on similarity in both orientation-specific and non-specific components of responses to single-whisker deflections. Our results shed light on L2/3 organization in mouse somatosensory cortex, and lay a foundation for dissecting circuit mechanisms of perceptual learning and decision-making during orientation discrimination tasks.

Decision-Related Activity in Macaque V2 for Fine Disparity Discrimination Is Not Compatible with Optimal Linear Readout

Fine judgments of stereoscopic depth rely mainly on relative judgments of depth (relative binocular disparity) between objects, rather than judgments of the distance to where the eyes are fixating (absolute disparity). In macaques, visual area V2 is the earliest site in the visual processing hierarchy for which neurons selective for relative disparity have been observed (Thomas et al., 2002). Here, we found that, in macaques trained to perform a fine disparity discrimination task, disparity-selective neurons in V2 were highly selective for the task, and their activity correlated with the animals' perceptual decisions (unexplained by the stimulus). This may partially explain similar correlations reported in downstream areas. Although compatible with a perceptual role of these neurons for the task, the interpretation of such decision-related activity is complicated by the effects of interneuronal "noise" correlations between sensory neurons. Recent work has developed simple predictions to differentiate decoding schemes (Pitkow et al., 2015) without needing measures of noise correlations, and found that data from early sensory areas were compatible with optimal linear readout of populations with information-limiting correlations. In contrast, our data here deviated significantly from these predictions. We additionally tested this prediction for previously reported results of decision-related activity in V2 for a related task, coarse disparity discrimination (Nienborg and Cumming, 2006), thought to rely on absolute disparity. Although these data followed the predicted pattern, they violated the prediction quantitatively. This suggests that optimal linear decoding of sensory signals is not generally a good predictor of behavior in simple perceptual tasks.

SIGNIFICANCE STATEMENT

Activity in sensory neurons that correlates with an animal's decision is widely believed to provide insights into how the brain uses information from sensory neurons. Recent theoretical work developed simple predictions to differentiate decoding schemes, and found support for optimal linear readout of early sensory populations with information-limiting correlations. Here, we observed decision-related activity for neurons in visual area V2 of macaques performing fine disparity discrimination, as yet the earliest site for this task. These findings, and previously reported results from V2 in a different task, deviated from the predictions for optimal linear readout of a population with information-limiting correlations. Our results suggest that optimal linear decoding of early sensory information is not a general decoding strategy used by the brain.

Attention Increases Spike Count Correlations between Visual Cortical Areas

Visual attention, which improves perception of attended locations or objects, has long been known to affect many aspects of the responses of neuronal populations in visual cortex. There are two nonmutually exclusive hypotheses concerning the neuronal mechanisms that underlie these perceptual improvements. The first hypothesis, that attention improves the information encoded by a population of neurons in a particular cortical area, has considerable physiological support. The second hypothesis is that attention improves perception by selectively communicating relevant visual information. This idea has been tested primarily by measuring interactions between neurons on very short timescales, which are mathematically nearly independent of neuronal interactions on longer timescales. We tested the hypothesis that attention changes the way visual information is communicated between cortical areas on longer timescales by recording simultaneously from neurons in primary visual cortex (V1) and the middle temporal area (MT) in rhesus monkeys. We used two independent and complementary approaches. Our correlative experiment showed that attention increases the trial-to-trial response variability that is shared between the two areas. In our causal experiment, we electrically microstimulated V1 and found that attention increased the effect of stimulation on MT responses. Together, our results suggest that attention affects both the way visual stimuli are encoded within a cortical area and the extent to which visual information is communicated between areas on behaviorally relevant timescales.

SIGNIFICANCE STATEMENT

Visual attention dramatically improves the perception of attended stimuli. Attention has long been thought to act by selecting relevant visual information for further processing. It has been hypothesized that this selection is accomplished by increasing communication between neurons that encode attended information in different cortical areas. We recorded simultaneously from neurons in primary visual cortex and the middle temporal area while rhesus monkeys performed an attention task. We found that attention increased shared variability between neurons in the two areas and that attention increased the effect of microstimulation in V1 on the firing rates of MT neurons. Our results provide support for the hypothesis that attention increases communication between neurons in different brain areas on behaviorally relevant timescales.

Feature-Selective Attention Adaptively Shifts Noise Correlations in Primary Auditory Cortex

Sensory environments often contain an overwhelming amount of information, with both relevant and irrelevant information competing for neural resources. Feature attention mediates this competition by selecting the sensory features needed to form a coherent percept. How attention affects the activity of populations of neurons to support this process is poorly understood because population coding is typically studied through simulations in which one sensory feature is encoded without competition. Therefore, to study the effects of feature attention on population-based neural coding, investigations must be extended to include stimuli with both relevant and irrelevant features. We measured noise correlations (r_noise) within small neural populations in primary auditory cortex while rhesus macaques performed a novel feature-selective attention task. We found that the effect of feature-selective attention on r_noise depended not only on the population tuning to the attended feature, but also on the tuning to the distractor feature. To attempt to explain how these observed effects might support enhanced perceptual performance, we propose an extension of a simple and influential model in which shifts in r_noise can simultaneously enhance the representation of the attended feature while suppressing the distractor. These findings present a novel mechanism by which attention modulates neural populations to support sensory processing in cluttered environments.SIGNIFICANCE STATEMENT Although feature-selective attention constitutes one of the building blocks of listening in natural environments, its neural bases remain obscure. To address this, we developed a novel auditory feature-selective attention task and measured noise correlations (r_noise) in rhesus macaque A1 during task performance. Unlike previous studies showing that the effect of attention on r_noise depends on population tuning to the attended feature, we show that the effect of attention depends on the tuning to the distractor feature as well. We suggest that these effects represent an efficient process by which sensory cortex simultaneously enhances relevant information and suppresses irrelevant information.

Neuronal pattern separation of motion-relevant input in LIP activity

In various regions of the brain, neurons discriminate sensory stimuli by decreasing the similarity between ambiguous input patterns. Here, we examine whether this process of pattern separation may drive the rapid discrimination of visual motion stimuli in the lateral intraparietal area (LIP). Starting with a simple mean-rate population model that captures neuronal activity in LIP, we show that overlapping input patterns can be reformatted dynamically to give rise to separated patterns of neuronal activity. The population model predicts that a key ingredient of pattern separation is the presence of heterogeneity in the response of individual units. Furthermore, the model proposes that pattern separation relies on heterogeneity in the temporal dynamics of neural activity and not merely in the mean firing rates of individual neurons over time. We confirm these predictions in recordings of macaque LIP neurons and show that the accuracy of pattern separation is a strong predictor of behavioral performance. Overall, results propose that LIP relies on neuronal pattern separation to facilitate decision-relevant discrimination of sensory stimuli.NEW & NOTEWORTHY A new hypothesis is proposed on the role of the lateral intraparietal (LIP) region of cortex during rapid decision making. This hypothesis suggests that LIP alters the representation of ambiguous inputs to reduce their overlap, thus improving sensory discrimination. A combination of computational modeling, theoretical analysis, and electrophysiological data shows that the pattern separation hypothesis links neural activity to behavior and offers novel predictions on the role of LIP during sensory discrimination.

Feedforward and feedback sources of choice probability in neural population responses

How the processing of signals carried by sensory neurons supports perceptual decisions is a long-standing question in neuroscience. The ability to record neuronal activity in awake animals while they perform psychophysical tasks near threshold has been a key advance in studying these questions. Trial-to-trial correlations between the activity of sensory neurons and the decisions reported by animals ('choice probabilities'), even when measured across repeated presentations of an identical stimulus provide insights into this problem. But understanding the sources of such co-variability between sensory neurons and behavior has proven more difficult than it initially appeared. Below, we discuss our current understanding of what gives rise to these correlations.

Near-optimal Integration of Magnitude in the Human Parietal Cortex

Humans are often observed to make optimal sensorimotor decisions but to be poor judges of situations involving explicit estimation of magnitudes or numerical quantities. For example, when drawing conclusions from data, humans tend to neglect the size of the sample from which it was collected. Here, we asked whether this sample size neglect is a general property of human decisions and investigated its neural implementation. Participants viewed eight discrete visual arrays (samples) depicting variable numbers of blue and pink balls. They then judged whether the samples were being drawn from an urn in which blue or pink predominated. A participant who neglects the sample size will integrate the ratio of balls on each array, giving equal weight to each sample. However, we found that human behavior resembled that of an optimal observer, giving more credence to larger sample sizes. Recording scalp EEG signals while participants performed the task allowed us to assess the decision information that was computed during integration. We found that neural signals over the posterior cortex after each sample correlated first with the sample size and then with the difference in the number of balls in either category. Moreover, lateralized beta-band activity over motor cortex was predicted by the cumulative difference in number of balls in each category. Together, these findings suggest that humans achieve statistically near-optimal decisions by adding up the difference in evidence on each sample, and imply that sample size neglect may not be a general feature of human decision-making.

Anxiety dissociates the adaptive functions of sensory and motor response enhancements to social threats

Efficient detection and reaction to negative signals in the environment is essential for survival. In social situations, these signals are often ambiguous and can imply different levels of threat for the observer, thereby making their recognition susceptible to contextual cues – such as gaze direction when judging facial displays of emotion. However, the mechanisms underlying such contextual effects remain poorly understood. By computational modeling of human behavior and electrical brain activity, we demonstrate that gaze direction enhances the perceptual sensitivity to threat-signaling emotions – anger paired with direct gaze, and fear paired with averted gaze. This effect arises simultaneously in ventral face-selective and dorsal motor cortices at 200 ms following face presentation, dissociates across individuals as a function of anxiety, and does not reflect increased attention to threat-signaling emotions. These findings reveal that threat tunes neural processing in fast, selective, yet attention-independent fashion in sensory and motor systems, for different adaptive purposes.

DOI:http://dx.doi.org/10.7554/eLife.10274.001

Facial expressions can communicate important social signals, and understanding these signals can be essential for surviving threatening situations. Past studies have identified changes to brain activity and behavior in response to particular social threats, but it is not clear how the brain processes information from the facial expressions of others to identify these threats. Here, El Zein, Wyart and Grèzes aimed to identify how signals of threat are represented in the human brain.

The experiment used a technique called electroencephalography to record brain activity in healthy human volunteers as they examined angry and fearful facial expressions. El Zein, Wyart and Grèzes found that emotions that signaled a threat to the observer are better represented in particular regions of the brain – including those that control action – within a fraction of a second after the facial expression was shown to the volunteer. Moreover, the response of the brain regions that control action was greater in volunteers with higher levels of anxiety, which highlights the role of anxiety in reacting rapidly to social threats in the environment.

El Zein, Wyart and Grèzes’ findings show that social threats can alter brain activity very rapidly, and in a more selective manner than previously believed. A future challenge is to find out whether other aspects in threatening environments can stimulate similar increases in brain activity.

DOI:http://dx.doi.org/10.7554/eLife.10274.002

Origins of choice-related activity in mouse somatosensory cortex

During perceptual decisions about faint or ambiguous sensory stimuli, even identical stimuli can produce different choices. Spike trains from sensory cortex neurons can predict trial-to-trial variability in choice. Choice-related spiking is widely studied to link cortical activity to perception, but its origins remain unclear. Using imaging and electrophysiology, we found that mouse primary somatosensory cortex neurons showed robust choice-related activity during a tactile detection task. Spike trains from primary mechanoreceptive neurons did not predict choices about identical stimuli. Spike trains from thalamic relay neurons showed highly transient, weak choice-related activity. Intracellular recordings in cortex revealed a prolonged choice-related depolarization in most neurons that was not accounted for by feedforward thalamic input. Top-down axons projecting from secondary to primary somatosensory cortex signaled choice. An intracellular measure of stimulus sensitivity determined which neurons converted choice-related depolarization into spiking. Our results reveal how choice-related spiking emerges across neural circuits and within single neurons.

A computational relationship between thalamic sensory neural responses and contrast perception

Uncovering the relationship between sensory neural responses and perceptual decisions remains a fundamental problem in neuroscience. Decades of experimental and modeling work in the sensory cortex have demonstrated that a perceptual decision pool is usually composed of tens to hundreds of neurons, the responses of which are significantly correlated not only with each other, but also with the behavioral choices of an animal. Few studies, however, have measured neural activity in the sensory thalamus of awake, behaving animals. Therefore, it remains unclear how many thalamic neurons are recruited and how the information from these neurons is pooled at subsequent cortical stages to form a perceptual decision. In a previous study we measured neural activity in the macaque lateral geniculate nucleus (LGN) during a two alternative forced choice (2AFC) contrast detection task, and found that single LGN neurons were significantly correlated with the monkeys’ behavioral choices, despite their relatively poor contrast sensitivity and a lack of overall interneuronal correlations. We have now computationally tested a number of specific hypotheses relating these measured LGN neural responses to the contrast detection behavior of the animals. We modeled the perceptual decisions with different numbers of neurons and using a variety of pooling/readout strategies, and found that the most successful model consisted of about 50–200 LGN neurons, with individual neurons weighted differentially according to their signal-to-noise ratios (quantified as d-primes). These results supported the hypothesis that in contrast detection the perceptual decision pool consists of multiple thalamic neurons, and that the response fluctuations in these neurons can influence contrast perception, with the more sensitive thalamic neurons likely to exert a greater influence.

A Modality-Specific Feedforward Component of Choice-Related Activity in MT

The activity of individual sensory neurons can be predictive of an animal's choices. These decision signals arise from network properties dependent on feedforward and feedback inputs; however, the relative contributions of these inputs are poorly understood. We determined the role of feedforward pathways to decision signals in MT by recording neuronal activity while monkeys performed motion and depth tasks. During each session, we reversibly inactivated V2 and V3, which provide feedforward input to MT that conveys more information about depth than motion. We thus monitored the choice-related activity of the same neuron both before and during V2/V3 inactivation. During inactivation, MT neurons became less predictive of decisions for the depth task but not the motion task, indicating that a feedforward pathway that gives rise to tuning preferences also contributes to decision signals. We show that our data are consistent with V2/V3 input conferring structured noise correlations onto the MT population.

How Can Single Sensory Neurons Predict Behavior?

Single sensory neurons can be surprisingly predictive of behavior in discrimination tasks. We propose this is possible because sensory information extracted from neural populations is severely restricted, either by near-optimal decoding of a population with information-limiting correlations or by suboptimal decoding that is blind to correlations. These have different consequences for choice correlations, the correlations between neural responses and behavioral choices. In the vestibular and cerebellar nuclei and the dorsal medial superior temporal area, we found that choice correlations during heading discrimination are consistent with near-optimal decoding of neuronal responses corrupted by information-limiting correlations. In the ventral intraparietal area, the choice correlations are also consistent with the presence of information-limiting correlations, but this area does not appear to influence behavior, although the choice correlations are particularly large. These findings demonstrate how choice correlations can be used to assess the efficiency of the downstream readout and detect the presence of information-limiting correlations.

Perceptual decision related activity in the lateral geniculate nucleus

Fundamental to neuroscience is the understanding of how the language of neurons relates to behavior. In the lateral geniculate nucleus (LGN), cells show distinct properties such as selectivity for particular wavelengths, increments or decrements in contrast, or preference for fine detail versus rapid motion. No studies, however, have measured how LGN cells respond when an animal is challenged to make a perceptual decision using information within the receptive fields of those LGN cells. In this study we measured neural activity in the macaque LGN during a two-alternative, forced-choice (2AFC) contrast detection task or during a passive fixation task and found that a small proportion (13.5%) of single LGN parvocellular (P) and magnocellular (M) neurons matched the psychophysical performance of the monkey. The majority of LGN neurons measured in both tasks were not as sensitive as the monkey. The covariation between neural response and behavior (quantified as choice probability) was significantly above chance during active detection, even when there was no external stimulus. Interneuronal correlations and task-related gain modulations were negligible under the same condition. A bottom-up pooling model that used sensory neural responses to compute perceptual choices in the absence of interneuronal correlations could fully explain these results at the level of the LGN, supporting the hypothesis that the perceptual decision pool consists of multiple sensory neurons and that response fluctuations in these neurons can influence perception.

Mutual information against correlations in binary communication channels

Background

Explaining how the brain processing is so fast remains an open problem (van Hemmen JL, Sejnowski T., 2004). Thus, the analysis of neural transmission (Shannon CE, Weaver W., 1963) processes basically focuses on searching for effective encoding and decoding schemes. According to the Shannon fundamental theorem, mutual information plays a crucial role in characterizing the efficiency of communication channels. It is well known that this efficiency is determined by the channel capacity that is already the maximal mutual information between input and output signals. On the other hand, intuitively speaking, when input and output signals are more correlated, the transmission should be more efficient. A natural question arises about the relation between mutual information and correlation. We analyze the relation between these quantities using the binary representation of signals, which is the most common approach taken in studying neuronal processes of the brain.

Results

We present binary communication channels for which mutual information and correlation coefficients behave differently both quantitatively and qualitatively. Despite this difference in behavior, we show that the noncorrelation of binary signals implies their independence, in contrast to the case for general types of signals.

Conclusions

Our research shows that the mutual information cannot be replaced by sheer correlations. Our results indicate that neuronal encoding has more complicated nature which cannot be captured by straightforward correlations between input and output signals once the mutual information takes into account the structure and patterns of the signals.

Neuronal thresholds and choice-related activity of otolith afferent fibers during heading perception

How activity of sensory neurons leads to perceptual decisions remains a challenge to understand. Correlations between choices and single neuron firing rates have been found early in vestibular processing, in the brainstem and cerebellum. To investigate the origins of choice-related activity, we have recorded from otolith afferent fibers while animals performed a fine heading discrimination task. We find that afferent fibers have similar discrimination thresholds as central cells, and the most sensitive fibers have thresholds that are only twofold or threefold greater than perceptual thresholds. Unlike brainstem and cerebellar nuclei neurons, spike counts from afferent fibers do not exhibit trial-by-trial correlations with perceptual decisions. This finding may reflect the fact that otolith afferent responses are poorly suited for driving heading perception because they fail to discriminate self-motion from changes in orientation relative to gravity. Alternatively, if choice probabilities reflect top-down inference signals, they are not relayed to the vestibular periphery.

On the number of neurons and time scale of integration underlying the formation of percepts in the brain

All of our perceptual experiences arise from the activity of neural populations. Here we study the formation of such percepts under the assumption that they emerge from a linear readout, i.e., a weighted sum of the neurons’ firing rates. We show that this assumption constrains the trial-to-trial covariance structure of neural activities and animal behavior. The predicted covariance structure depends on the readout parameters, and in particular on the temporal integration window w and typical number of neurons K used in the formation of the percept. Using these predictions, we show how to infer the readout parameters from joint measurements of a subject’s behavior and neural activities. We consider three such scenarios: (1) recordings from the complete neural population, (2) recordings of neuronal sub-ensembles whose size exceeds K, and (3) recordings of neuronal sub-ensembles that are smaller than K. Using theoretical arguments and artificially generated data, we show that the first two scenarios allow us to recover the typical spatial and temporal scales of the readout. In the third scenario, we show that the readout parameters can only be recovered by making additional assumptions about the structure of the full population activity. Our work provides the first thorough interpretation of (feed-forward) percept formation from a population of sensory neurons. We discuss applications to experimental recordings in classic sensory decision-making tasks, which will hopefully provide new insights into the nature of perceptual integration.

This article deals with the interpretation of neural activities during perceptual decision-making tasks, where animals must assess the value of a sensory stimulus and take a decision on the basis of their percept. A “standard model” for these tasks has progressively emerged, whence the animal’s percept and subsequent choice on each trial are obtained from a linear integration of the activity of sensory neurons. However, up to date, there has been no principled method to estimate the parameters of this model: mainly, the typical number of neurons K from the population involved in conveying the percept, and the typical time scale w during which these neurons’ activities are integrated. In this article, we propose a novel method to estimate these quantities from experimental data, and thus assess the validity of the standard model of percept formation. In the process, we clarify the predictions of the standard model regarding two classic experimental measures in these tasks: sensitivity, which is the animal’s ability to distinguish nearby stimulus values, and choice signals, which assess the amount of correlation between the activity of single neurons and the animal’s ultimate choice on each trial.

Feature attention for binocular disparity in primate area MT depends on tuning strength

Attending to a stimulus modulates the responses of sensory neurons that represent features of that stimulus, a phenomenon named "feature attention." For example, attending to a stimulus containing upward motion enhances the responses of upward-preferring direction-selective neurons in the middle temporal area (MT) and suppresses the responses of downward-preferring neurons, even when the attended stimulus is outside of the spatial receptive fields of the recorded neurons (Treue S, Martinez-Trujillo JC. Nature 399: 575-579, 1999). This modulation renders the representation of sensory information across a neuronal population more selective for the features present in the attended stimulus (Martinez-Trujillo JC, Treue S. Curr Biol 14: 744-751, 2004). We hypothesized that if feature attention modulates neurons according to their tuning preferences, it should also be sensitive to their tuning strength, which is the magnitude of the difference in responses to preferred and null stimuli. We measured how the effects of feature attention on MT neurons in rhesus monkeys (Macaca mulatta) depended on the relationship between features-in our case, direction of motion and binocular disparity-of the attended stimulus and a neuron's tuning for those features. We found that, as for direction, attention to stimuli containing binocular disparity cues modulated the responses of MT neurons and that the magnitude of the modulation depended on both a neuron's tuning preferences and its tuning strength. Our results suggest that modulation by feature attention may depend not just on which features a neuron represents but also on how well the neuron represents those features.

Processing of sub- and supra-second intervals in the primate brain results from the calibration of neuronal oscillators via sensory, motor, and feedback processes

The processing of time intervals in the sub- to supra-second range by the brain is critical for the interaction of primates with their surroundings in activities, such as foraging and hunting. For an accurate processing of time intervals by the brain, representation of physical time within neuronal circuits is necessary. I propose that time dimension of the physical surrounding is represented in the brain by different types of neuronal oscillators, generating spikes or spike bursts at regular intervals. The proposed oscillators include the pacemaker neurons, tonic inputs, and synchronized excitation and inhibition of inter-connected neurons. Oscillators, which are built inside various circuits of brain, help to form modular clocks, processing time intervals or other temporal characteristics specific to functions of a circuit. Relative or absolute duration is represented within neuronal oscillators by "neural temporal unit," defined as the interval between regularly occurring spikes or spike bursts. Oscillator output is processed to produce changes in activities of neurons, named frequency modulator neuron, wired within a separate module, represented by the rate of change in frequency, and frequency of activities, proposed to encode time intervals. Inbuilt oscillators are calibrated by (a) feedback processes, (b) input of time intervals resulting from rhythmic external sensory stimulation, and (c) synchronous effects of feedback processes and evoked sensory activity. A single active clock is proposed per circuit, which is calibrated by one or more mechanisms. Multiple calibration mechanisms, inbuilt oscillators, and the presence of modular connections prevent a complete loss of interval timing functions of the brain.

Contribution of correlated noise and selective decoding to choice probability measurements in extrastriate visual cortex

Trial by trial covariations between neural activity and perceptual decisions (quantified by choice Probability, CP) have been used to probe the contribution of sensory neurons to perceptual decisions. CPs are thought to be determined by both selective decoding of neural activity and by the structure of correlated noise among neurons, but the respective roles of these factors in creating CPs have been controversial. We used biologically-constrained simulations to explore this issue, taking advantage of a peculiar pattern of CPs exhibited by multisensory neurons in area MSTd that represent self-motion. Although models that relied on correlated noise or selective decoding could both account for the peculiar pattern of CPs, predictions of the selective decoding model were substantially more consistent with various features of the neural and behavioral data. While correlated noise is essential to observe CPs, our findings suggest that selective decoding of neuronal signals also plays important roles.

DOI:http://dx.doi.org/10.7554/eLife.02670.001

Even the simplest tasks require the brain to process vast amounts of information. To take a step forward, for example, the brain must process information about the orientation of the animal's body and what the animal is seeing, hearing and feeling in order to determine whether any obstacles stand in the way. The brain must integrate all this information to make decisions about how to proceed. And once a decision is made, the brain must send signals via the nervous system to the muscles to physically move the foot forward.

Specialized brain cells called sensory neurons help to process this sensory information. For example, visual neurons process information about what the animal sees, while auditory neurons process information about what it hears. Other sensory neurons—called multisensory neurons—can process information coming from more than one of an animal's senses.

For more than two decades, researchers have known that the firing of an individual sensory neuron can be linked to the decision that an animal makes about the meaning of the sensory information it has received. The ability to predict whether an animal will make a given decision based on the firing of individual sensory neurons is often referred to as a ‘choice probability’. Measurements of single neurons have often been used to try to work out how the brain decodes the sensory information that is needed to carry out a specific task. However, it remains unclear whether choice probabilities really reflect how sensory information is decoded in the brain, or whether these measurements are just reflecting coordinated patterns of background ‘noise’ among the neurons as the decisions are being made.

Gu et al. set out to help resolve this debate by examining choice probabilities in the multisensory neurons in one area of the brain. A series of experiments was conducted to see how these neurons process information, both from the eyes and the part of the inner ear that helps control balance, to work out the direction in which an animal was moving. By performing computer simulations of the activity of groups of neurons, Gu et al. found that choice probability measurements are better explained by the models whereby these measurements did reflect the strategy that is used to decode the sensory information. Models based solely on patterns of correlated noise did not explain the data as well, though Gu et al. suggest that this noise is likely to also contribute to the observed effects.

Following on from the work of Gu et al., a major challenge will be to see if it is possible to infer how the brain extracts the relevant information from the different sensory neurons. This may require recordings from large groups of neurons, but it might help us to decipher how patterns of activity in the brain lead to decisions about the world around us.

DOI:http://dx.doi.org/10.7554/eLife.02670.002

Population coding in mouse visual cortex: response reliability and dissociability of stimulus tuning and noise correlation

The primary visual cortex is an excellent model system for investigating how neuronal populations encode information, because of well-documented relationships between stimulus characteristics and neuronal activation patterns. We used two-photon calcium imaging data to relate the performance of different methods for studying population coding (population vectors, template matching, and Bayesian decoding algorithms) to their underlying assumptions. We show that the variability of neuronal responses may hamper the decoding of population activity, and that a normalization to correct for this variability may be of critical importance for correct decoding of population activity. Second, by comparing noise correlations and stimulus tuning we find that these properties have dissociated anatomical correlates, even though noise correlations have been previously hypothesized to reflect common synaptic input. We hypothesize that noise correlations arise from large non-specific increases in spiking activity acting on many weak synapses simultaneously, while neuronal stimulus response properties are dependent on more reliable connections. Finally, this paper provides practical guidelines for further research on population coding and shows that population coding cannot be approximated by a simple summation of inputs, but is heavily influenced by factors such as input reliability and noise correlation structure.

Decision-related activity in sensory neurons may depend on the columnar architecture of cerebral cortex

Many studies have reported correlations between the activity of sensory neurons and animals' judgments in discrimination tasks. Here, we suggest that such neuron-behavior correlations may require a cortical map for the task relevant features. This would explain why studies using discrimination tasks based on disparity in area V1 have not found these correlations: V1 contains no map for disparity. This scheme predicts that activity of V1 neurons correlates with decisions in an orientation-discrimination task. To test this prediction, we trained two macaque monkeys in a coarse orientation discrimination task using band-pass-filtered dynamic noise. The two orientations were always 90° apart and task difficulty was controlled by varying the orientation bandwidth of the filter. While the trained animals performed this task, we recorded from orientation-selective V1 neurons (n = 82, n = 31 for Monkey 1, n = 51 for Monkey 2). For both monkeys, we observed significant correlation (quantified as "choice probabilities") of the V1 activity with the monkeys' perceptual judgments (mean choice probability 0.54, p = 10(-5)). In one of these animals, we had previously measured choice probabilities in a disparity discrimination task in V1, which had been at chance (0.49, not significantly different from 0.5). The choice probabilities in this monkey for the orientation discrimination task were significantly larger than those for the disparity discrimination task (p = 0.032). These results are predicted by our suggestion that choice probabilities are only observed for cortical sensory neurons that are organized in maps for the task-relevant feature.

Gamma synchrony predicts neuron-neuron correlations and correlations with motor behavior in extrastriate visual area MT

Correlated variability of neuronal responses is an important factor in estimating sensory parameters from a population response. Large correlations among neurons reduce the effective size of a neural population and increase the variation of the estimates. They also allow the activity of one neuron to be informative about impending perceptual decisions or motor actions on single trials. In extrastriate visual area MT of the rhesus macaque, for example, some but not all neurons show nonzero "choice probabilities" for perceptual decisions or non-zero "MT-pursuit" correlations between the trial-by-trial variations in neural activity and smooth pursuit eye movements. To understand the functional implications of zero versus nonzero correlations between neural responses and impending perceptions or actions, we took advantage of prior observations that specific frequencies of local field potentials reflect the correlated activity of neurons. We found that the strength of the spike-field coherence of a neuron in the gamma-band frequency range is related to the size of its MT-pursuit correlations for eye direction, as well as to the size of the neuron-neuron correlations. Spike-field coherence predicts MT-pursuit correlations better for direction than for speed, perhaps because the topographic organization of direction preference in MT is more amenable to creating meaningful local field potentials. We suggest that the relationship between spiking and local-field potentials is stronger for neurons that have larger correlations with their neighbors; larger neuron-neuron correlations create stronger MT-pursuit correlations. Neurons that lack strong correlations with their neighbors also have weaker correlations with pursuit behavior, but still could drive pursuit strongly.

An optimal decision population code that accounts for correlated variability unambiguously predicts a subject's choice

Decisions emerge from the concerted activity of neuronal populations distributed across brain circuits. However, the analytical tools best suited to decode decision signals from neuronal populations remain unknown. Here we show that knowledge of correlated variability between pairs of cortical neurons allows perfect decoding of decisions from population firing rates. We recorded pairs of neurons from secondary somatosensory (S2) and premotor (PM) cortices while monkeys reported the presence or absence of a tactile stimulus. We found that while populations of S2 and sensory-like PM neurons are only partially correlated with behavior, those PM neurons active during a delay period preceding the motor report predict unequivocally the animal's decision report. Thus, a population rate code that optimally reveals a subject's perceptual decisions can be implemented just by knowing the correlations of PM neurons representing decision variables.

Inattention blindness to motion in middle temporal area

Subjects naturally form and use expectations to solve familiar tasks, but the accuracy of these expectations and the neuronal mechanisms by which these expectations enhance behavior are unclear. We trained animals (Macaca mulatta) in a challenging perceptual task in which the likelihood of a very brief pulse of motion was consistently modulated over time and space. Pulse likelihood had dramatic effects on behavior: unexpected pulses were nearly invisible to the animals. To examine the neuronal basis of such inattention blindness, we recorded from single neurons in the middle temporal (MT) area, an area related to motion perception. Fluctuations in how reliably MT neurons both signaled stimulus events and predicted behavioral choices were highly correlated with changes in performance over the course of individual trials. A simple neuronal pooling model reveals that the dramatic behavioral effects of attention in this task can be completely explained by changes in the reliability of a small number of MT neurons.