Emerging principles of population coding: in search for the neural code

Adaptation of the inferior temporal neurons and efficient visual processing

Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.

A quantitative model of ensemble perception as summed activation in feature space

Ensemble perception is a process by which we summarize complex scenes. Despite the importance of ensemble perception to everyday cognition, there are few computational models that provide a formal account of this process. Here we develop and test a model in which ensemble representations reflect the global sum of activation signals across all individual items. We leverage this set of minimal assumptions to formally connect a model of memory for individual items to ensembles. We compare our ensemble model against a set of alternative models in five experiments. Our approach uses performance on a visual memory task for individual items to generate zero-free-parameter predictions of interindividual and intraindividual differences in performance on an ensemble continuous-report task. Our top-down modelling approach formally unifies models of memory for individual items and ensembles and opens a venue for building and comparing models of distinct memory processes and representations.

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

Receptive field sizes and neuronal encoding bandwidth are constrained by axonal conduction delays

Studies on population coding implicitly assume that spikes from the presynaptic cells arrive simultaneously at the integrating neuron. In natural neuronal populations, this is usually not the case-neuronal signaling takes time and populations cover a certain space. The spread of spike arrival times depends on population size, cell density and axonal conduction velocity. Here we analyze the consequences of population size and axonal conduction delays on the stimulus encoding performance in the electrosensory system of the electric fish Apteronotus leptorhynchus. We experimentally locate p-type electroreceptor afferents along the rostro-caudal body axis and relate locations to neurophysiological response properties. In an information-theoretical approach we analyze the coding performance in homogeneous and heterogeneous populations. As expected, the amount of information increases with population size and, on average, heterogeneous populations encode better than the average same-size homogeneous population, if conduction delays are compensated for. The spread of neuronal conduction delays within a receptive field strongly degrades encoding of high-frequency stimulus components. Receptive field sizes typically found in the electrosensory lateral line lobe of A. leptorhynchus appear to be a good compromise between the spread of conduction delays and encoding performance. The limitations imposed by finite axonal conduction velocity are relevant for any converging network as is shown by model populations of LIF neurons. The bandwidth of natural stimuli and the maximum meaningful population sizes are constrained by conduction delays and may thus impact the optimal design of nervous systems.

Intracortical brain-computer interfaces in primates: a review and outlook

Brain-computer interfaces (BCI) translate brain signals into artificial output to restore or replace natural central nervous system (CNS) functions. Multiple processes, including sensorimotor integration, decision-making, motor planning, execution, and updating, are involved in any movement. For example, a BCI may be better able to restore naturalistic motor behaviors if it uses signals from multiple brain areas and decodes natural behaviors' cognitive and motor aspects. This review provides an overview of the preliminary information necessary to plan a BCI project focusing on intracortical implants in primates. Since the brain structure and areas of non-human primates (NHP) are similar to humans, exploring the result of NHP studies will eventually benefit human BCI studies. The different types of BCI systems based on the target cortical area, types of signals, and decoding methods will be discussed. In addition, various successful state-of-the-art cases will be reviewed in more detail, focusing on the general algorithm followed in the real-time system. Finally, an outlook for improving the current BCI research studies will be debated.

Sparse spike trains and the limitation of rate codes underlying rapid behaviours

Animals live in dynamic worlds where they use sensorimotor circuits to rapidly process information and drive behaviours. For example, dragonflies are aerial predators that react to movements of prey within tens of milliseconds. These pursuits are likely controlled by identified neurons in the dragonfly, which have well-characterized physiological responses to moving targets. Predominantly, neural activity in these circuits is interpreted in context of a rate code, where information is conveyed by changes in the number of spikes over a time period. However, such a description of neuronal activity is difficult to achieve in real-world, real-time scenarios. Here, we contrast a neuroscientists' post-hoc view of spiking activity with the information available to the animal in real-time. We describe how performance of a rate code is readily overestimated and outline a rate code's significant limitations in driving rapid behaviours.

Mapping thalamic innervation to individual L2/3 pyramidal neurons and modeling their 'readout' of visual input

The thalamus is the main gateway for sensory information from the periphery to the mammalian cerebral cortex. A major conundrum has been the discrepancy between the thalamus's central role as the primary feedforward projection system into the neocortex and the sparseness of thalamocortical synapses. Here we use new methods, combining genetic tools and scalable tissue expansion microscopy for whole-cell synaptic mapping, revealing the number, density and size of thalamic versus cortical excitatory synapses onto individual layer 2/3 (L2/3) pyramidal cells (PCs) of the mouse primary visual cortex. We find that thalamic inputs are not only sparse, but remarkably heterogeneous in number and density across individual dendrites and neurons. Most surprising, despite their sparseness, thalamic synapses onto L2/3 PCs are smaller than their cortical counterparts. Incorporating these findings into fine-scale, anatomically faithful biophysical models of L2/3 PCs reveals how individual neurons with sparse and weak thalamocortical synapses, embedded in small heterogeneous neuronal ensembles, may reliably 'read out' visually driven thalamic input.

Unraveling Functional Diversity of Cortical Synaptic Architecture Through the Lens of Population Coding

The synaptic inputs to single cortical neurons exhibit substantial diversity in their sensory-driven activity. What this diversity reflects is unclear, and appears counter-productive in generating selective somatic responses to specific stimuli. One possibility is that this diversity reflects the propagation of information from one neural population to another. To test this possibility, we bridge population coding theory with measurements of synaptic inputs recorded in vivo with two-photon calcium imaging. We construct a probabilistic decoder to estimate the stimulus orientation from the responses of a realistic, hypothetical input population of neurons to compare with synaptic inputs onto individual neurons of ferret primary visual cortex (V1) recorded with two-photon calcium imaging in vivo. We find that optimal decoding requires diverse input weights and provides a straightforward mapping from the decoder weights to excitatory synapses. Analytically derived weights for biologically realistic input populations closely matched the functional heterogeneity of dendritic spines imaged in vivo with two-photon calcium imaging. Our results indicate that synaptic diversity is a necessary component of information transmission and reframes studies of connectivity through the lens of probabilistic population codes. These results suggest that the mapping from synaptic inputs to somatic selectivity may not be directly interpretable without considering input covariance and highlights the importance of population codes in pursuit of the cortical connectome.

Editing reality in the brain

Recent information technologies such as virtual reality (VR) and augmented reality (AR) allow the creation of simulated sensory worlds with which we can interact. Using programming language, digital details can be overlaid onto displays of our environment, confounding what is real and what has been artificially engineered. Natural language, particularly the use of direct verbal suggestion (DVS) in everyday and hypnotic contexts, can also manipulate the meaning and significance of objects and events in ourselves and others. In this review, we focus on how socially rewarding language can construct and influence reality. Language is symbolic, automatic and flexible and can be used to augment bodily sensations e.g. feelings of heaviness in a limb or suggest a colour that is not there. We introduce the term 'suggested reality' (SR) to refer to the important role that language, specifically DVS, plays in constructing, maintaining and manipulating our shared reality. We also propose the term edited reality to encompass the wider influence of information technology and linguistic techniques that results in altered subjective experience and review its use in clinical settings, while acknowledging its limitations. We develop a cognitive model indicating how the brain's central executive structures use our personal and linguistic-based narrative in subjective awareness, arguing for a central role for language in DVS. A better understanding of the characteristics of VR, AR and SR and their applications in everyday life, research and clinical settings can help us to better understand our own reality and how it can be edited.

Scaling of sensory information in large neural populations shows signatures of information-limiting correlations

How is information distributed across large neuronal populations within a given brain area? Information may be distributed roughly evenly across neuronal populations, so that total information scales linearly with the number of recorded neurons. Alternatively, the neural code might be highly redundant, meaning that total information saturates. Here we investigate how sensory information about the direction of a moving visual stimulus is distributed across hundreds of simultaneously recorded neurons in mouse primary visual cortex. We show that information scales sublinearly due to correlated noise in these populations. We compartmentalized noise correlations into information-limiting and nonlimiting components, then extrapolate to predict how information grows with even larger neural populations. We predict that tens of thousands of neurons encode 95% of the information about visual stimulus direction, much less than the number of neurons in primary visual cortex. These findings suggest that the brain uses a widely distributed, but nonetheless redundant code that supports recovering most sensory information from smaller subpopulations.

Inferring neural information flow from spiking data

The brain can be regarded as an information processing system in which neurons store and propagate information about external stimuli and internal processes. Therefore, estimating interactions between neural activity at the cellular scale has significant implications in understanding how neuronal circuits encode and communicate information across brain areas to generate behavior. While the number of simultaneously recorded neurons is growing exponentially, current methods relying only on pairwise statistical dependencies still suffer from a number of conceptual and technical challenges that preclude experimental breakthroughs describing neural information flows. In this review, we examine the evolution of the field over the years, starting from descriptive statistics to model-based and model-free approaches. Then, we discuss in detail the Granger Causality framework, which includes many popular state-of-the-art methods and we highlight some of its limitations from a conceptual and practical estimation perspective. Finally, we discuss directions for future research, including the development of theoretical information flow models and the use of dimensionality reduction techniques to extract relevant interactions from large-scale recording datasets.

Global Motion Processing by Populations of Direction-Selective Retinal Ganglion Cells

Simple stimuli have been critical to understanding neural population codes in sensory systems. Yet it remains necessary to determine the extent to which this understanding generalizes to more complex conditions. To examine this problem, we measured how populations of direction-selective ganglion cells (DSGCs) from the retinas of male and female mice respond to a global motion stimulus with its direction and speed changing dynamically. We then examined the encoding and decoding of motion direction in both individual and populations of DSGCs. Individual cells integrated global motion over ∼200 ms, and responses were tuned to direction. However, responses were sparse and broadly tuned, which severely limited decoding performance from small DSGC populations. In contrast, larger populations compensated for response sparsity, enabling decoding with high temporal precision (<100 ms). At these timescales, correlated spiking was minimal and had little impact on decoding performance, unlike results obtained using simpler local motion stimuli decoded over longer timescales. We use these data to define different DSGC population decoding regimes that use or mitigate correlated spiking to achieve high-spatial versus high-temporal resolution.SIGNIFICANCE STATEMENT ON-OFF direction-selective ganglion cells (ooDSGCs) in the mammalian retina are typically thought to signal local motion to the brain. However, several recent studies suggest they may signal global motion. Here we analyze the fidelity of encoding and decoding global motion in a natural scene across large populations of ooDSGCs. We show that large populations of DSGCs are capable of signaling rapid changes in global motion.

Linking neural responses to behavior with information-preserving population vectors

All systems for processing signals, both artificial and within animals, must obey fundamental statistical laws for how information can be processed. We discuss here recent results using information theory that provide a blueprint for building circuits where signals can be read-out without information loss. Many properties that are necessary to build information-preserving circuits are actually observed in real neurons, at least approximately. One such property is the use of logistic nonlinearity for relating inputs to neural response probability. Such nonlinearities are common in neural and intracellular networks. With this nonlinearity type, there is a linear combination of neural responses that is guaranteed to preserve Shannon information contained in the response of a neural population, no matter how many neurons it contains. This read-out measure is related to a classic quantity known as the population vector that has been quite successful in relating neural responses to animal behavior in a wide variety of cases. Nevertheless, the population vector did not withstand the scrutiny of detailed information-theoretical analyses that showed that it discards substantial amounts of information contained in the responses of a neural population. We discuss recent theoretical results showing how to modify the population vector expression to make it 'information-preserving', and what is necessary in terms of neural circuit organization to allow for lossless information transfer. Implementing these strategies within artificial systems is likely to increase their efficiency, especially for brain-machine interfaces.

Representational untangling by the firing rate nonlinearity in V1 simple cells

An important computational goal of the visual system is ‘representational untangling’ (RU): representing increasingly complex features of visual scenes in an easily decodable format. RU is typically assumed to be achieved in high-level visual cortices via several stages of cortical processing. Here we show, using a canonical population coding model, that RU of low-level orientation information is already performed at the first cortical stage of visual processing, but not before that, by a fundamental cellular-level property: the thresholded firing rate nonlinearity of simple cells in the primary visual cortex (V1). We identified specific, experimentally measurable parameters that determined the optimal firing threshold for RU and found that the thresholds of V1 simple cells extracted from in vivo recordings in awake behaving mice were near optimal. These results suggest that information re-formatting, rather than maximisation, may already be a relevant computational goal for the early visual system.

Quantifying encoding redundancy induced by rate correlations in Poisson neurons

Temporal correlations in neuronal spike trains are known to introduce redundancy to stimulus encoding. However, exact methods to describe how these correlations impact neural information transmission quantitatively are lacking. Here, we provide a general measure for the information carried by correlated rate modulations only, neglecting other spike correlations, and use it to investigate the effect of rate correlations on encoding redundancy. We derive it analytically by calculating the mutual information between a time-correlated, rate modulating signal and the resulting spikes of Poisson neurons. Whereas this information is determined by spike autocorrelations only, the redundancy in information encoding due to rate correlations depends on both the distribution and the autocorrelation of the rate histogram. We further demonstrate that at very small signal strengths the information carried by rate correlated spikes becomes identical to that of independent spikes, in effect measuring the signal modulation depth. In contrast, a vanishing signal correlation time maximizes information but does not generally yield the information of independent spikes. Overall, our study sheds light on the role of signal-induced temporal correlations for neural coding, by providing insight into how signal features shape redundancy and by establishing mathematical links between existing methods.

Correlated states in balanced neuronal networks

Understanding the magnitude and structure of interneuronal correlations and their relationship to synaptic connectivity structure is an important and difficult problem in computational neuroscience. Early studies show that neuronal network models with excitatory-inhibitory balance naturally create very weak spike train correlations, defining the "asynchronous state." Later work showed that, under some connectivity structures, balanced networks can produce larger correlations between some neuron pairs, even when the average correlation is very small. All of these previous studies assume that the local network receives feedforward synaptic input from a population of uncorrelated spike trains. We show that when spike trains providing feedforward input are correlated, the downstream recurrent network produces much larger correlations. We provide an in-depth analysis of the resulting "correlated state" in balanced networks and show that, unlike the asynchronous state, it produces a tight excitatory-inhibitory balance consistent with in vivo cortical recordings.

Weighting neurons by selectivity produces near-optimal population codes

Perception is produced by "reading out" the representation of a sensory stimulus contained in the activity of a population of neurons. To examine experimentally how populations code information, a common approach is to decode a linearly weighted sum of the neurons' spike counts. This approach is popular because of the biological plausibility of weighted, nonlinear integration. For neurons recorded in vivo, weights are highly variable when derived through optimization methods, but it is unclear how the variability affects decoding performance in practice. To address this, we recorded from neurons in the middle temporal area (MT) of anesthetized marmosets (Callithrix jacchus) viewing stimuli comprising a sheet of dots that moved coherently in 1 of 12 different directions. We found that high peak response and direction selectivity both predicted that a neuron would be weighted more highly in an optimized decoding model. Although learned weights differed markedly from weights chosen according to a priori rules based on a neuron's tuning profile, decoding performance was only marginally better for the learned weights. In the models with a priori rules, selectivity is the best predictor of weighting, and defining weights according to a neuron's preferred direction and selectivity improves decoding performance to very near the maximum level possible, as defined by the learned weights. NEW & NOTEWORTHY We examined which aspects of a neuron's tuning account for its contribution to sensory coding. Strongly direction-selective neurons are weighted most highly by optimal decoders trained to discriminate motion direction. Models with predefined decoding weights demonstrate that this weighting scheme causally improved direction representation by a neuronal population. Optimizing decoders (using a generalized linear model or Fisher's linear discriminant) led to only marginally better performance than decoders based purely on a neuron's preferred direction and selectivity.

Assessing the Relevance of Specific Response Features in the Neural Code

The study of the neural code aims at deciphering how the nervous system maps external stimuli into neural activity—the encoding phase—and subsequently transforms such activity into adequate responses to the original stimuli—the decoding phase. Several information-theoretical methods have been proposed to assess the relevance of individual response features, as for example, the spike count of a given neuron, or the amount of correlation in the activity of two cells. These methods work under the premise that the relevance of a feature is reflected in the information loss that is induced by eliminating the feature from the response. The alternative methods differ in the procedure by which the tested feature is removed, and the algorithm with which the lost information is calculated. Here we compare these methods, and show that more often than not, each method assigns a different relevance to the tested feature. We demonstrate that the differences are both quantitative and qualitative, and connect them with the method employed to remove the tested feature, as well as the procedure to calculate the lost information. By studying a collection of carefully designed examples, and working on analytic derivations, we identify the conditions under which the relevance of features diagnosed by different methods can be ranked, or sometimes even equated. The condition for equality involves both the amount and the type of information contributed by the tested feature. We conclude that the quest for relevant response features is more delicate than previously thought, and may yield to multiple answers depending on methodological subtleties.

Unconscious Biases in Neural Populations Coding Multiple Stimuli

Throughout the nervous system, information is commonly coded in activity distributed over populations of neurons. In idealized situations where a single, continuous stimulus is encoded in a homogeneous population code, the value of the encoded stimulus can be read out without bias. However, in many situations, multiple stimuli are simultaneously present; for example, multiple motion patterns might overlap. Here we find that when multiple stimuli that overlap in their neural representation are simultaneously encoded in the population, biases in the read-out emerge. Although the bias disappears in the absence of noise, the bias is remarkably persistent at low noise levels. The bias can be reduced by competitive encoding schemes or by employing complex decoders. To study the origin of the bias, we develop a novel general framework based on gaussian processes that allows an accurate calculation of the estimate distributions of maximum likelihood decoders, and reveals that the distribution of estimates is bimodal for overlapping stimuli. The results have implications for neural coding and behavioral experiments on, for instance, overlapping motion patterns.

Sequential Nonlinear Filtering of Local Motion Cues by Global Motion Circuits

Many animals guide their movements using optic flow, the displacement of stationary objects across the retina caused by self-motion. How do animals selectively synthesize a global motion pattern from its local motion components? To what extent does this feature selectivity rely on circuit mechanisms versus dendritic processing? Here we used in vivo calcium imaging to identify pre- and postsynaptic mechanisms for processing local motion signals in global motion detection circuits in Drosophila. Lobula plate tangential cells (LPTCs) detect global motion by pooling input from local motion detectors, T4/T5 neurons. We show that T4/T5 neurons suppress responses to adjacent local motion signals whereas LPTC dendrites selectively amplify spatiotemporal sequences of local motion signals consistent with preferred global patterns. We propose that sequential nonlinear suppression and amplification operations allow optic flow circuitry to simultaneously prevent saturating responses to local signals while creating selectivity for global motion patterns critical to behavior.

Effects of V1 surround modulation tuning on visual saliency and the tilt illusion

Neurons in the primary visual cortex respond to oriented stimuli placed in the center of their receptive field, yet their response is modulated by stimuli outside the receptive field (the surround). Classically, this surround modulation is assumed to be strongest if the orientation of the surround stimulus aligns with the neuron's preferred orientation, irrespective of the actual center stimulus. This neuron-dependent surround modulation has been used to explain a wide range of psychophysical phenomena, such as biased tilt perception and saliency of stimuli with contrasting orientation. However, several neurophysiological studies have shown that for most neurons surround modulation is instead center dependent: it is strongest if the surround orientation aligns with the center stimulus. As the impact of such center-dependent modulation on the population level is unknown, we examine this using computational models. We find that with neuron-dependent modulation the biases in orientation coding, commonly used to explain the tilt illusion, are larger than psychophysically reported, but disappear with center-dependent modulation. Therefore we suggest that a mixture of the two modulation types is necessary to quantitatively explain the psychophysically observed biases. Next, we find that under center-dependent modulation average population responses are more sensitive to orientation differences between stimuli, which in theory could improve saliency detection. However, this effect depends on the specific saliency model. Overall, our results thus show that center-dependent modulation reduces coding bias, while possibly increasing the sensitivity to salient features. NEW & NOTEWORTHY Neural responses in the primary visual cortex are modulated by stimuli surrounding the receptive field. Most earlier studies assume this modulation depends on the neuron's tuning properties, but experiments have shown that instead it depends mostly on the stimulus characteristics. We show that this simple change leads to neural coding that is less biased and under some conditions more sensitive to salient features.

Relating the Structure of Noise Correlations in Macaque Primary Visual Cortex to Decoder Performance

Noise correlations in neuronal responses can have a strong influence on the information available in large populations. In addition, the structure of noise correlations may have a great impact on the utility of different algorithms to extract this information that may depend on the specific algorithm, and hence may affect our understanding of population codes in the brain. Thus, a better understanding of the structure of noise correlations and their interplay with different readout algorithms is required. Here we use eigendecomposition to investigate the structure of noise correlations in populations of about 50–100 simultaneously recorded neurons in the primary visual cortex of anesthetized monkeys, and we relate this structure to the performance of two common decoders: the population vector and the optimal linear estimator. Our analysis reveals a non-trivial correlation structure, in which the eigenvalue spectrum is composed of several distinct large eigenvalues that represent different shared modes of fluctuation extending over most of the population, and a semi-continuous tail. The largest eigenvalue represents a uniform collective mode of fluctuation. The second and third eigenvalues typically show either a clear functional (i.e., dependent on the preferred orientation of the neurons) or spatial structure (i.e., dependent on the physical position of the neurons). We find that the number of shared modes increases with the population size, being roughly 10% of that size. Furthermore, we find that the noise in each of these collective modes grows linearly with the population. This linear growth of correlated noise power can have limiting effects on the utility of averaging neuronal responses across large populations, depending on the readout. Specifically, the collective modes of fluctuation limit the accuracy of the population vector but not of the optimal linear estimator.

Neuronal Correlations in MT and MST Impair Population Decoding of Opposite Directions of Random Dot Motion

The study of neuronal responses to random-dot motion patterns has provided some of the most valuable insights into how the activity of neurons is related to perception. In the opposite directions of motion paradigm, the motion signal strength is decreased by manipulating the coherence of random dot patterns to examine how well the activity of single neurons represents the direction of motion. To extend this paradigm to populations of neurons, studies have used modelling based on data from pairs of neurons, but several important questions require further investigation with larger neuronal datasets. We recorded neuronal populations in the middle temporal (MT) and medial superior temporal (MST) areas of anaesthetized marmosets with electrode arrays, while varying the coherence of random dot patterns in two opposite directions of motion (left and right). Using the spike rates of simultaneously recorded neurons, we decoded the direction of motion at each level of coherence with linear classifiers. We found that the presence of correlations had a detrimental effect to decoding performance, but that learning the correlation structure produced better decoding performance compared to decoders that ignored the correlation structure. We also found that reducing motion coherence increased neuronal correlations, but decoders did not need to be optimized for each coherence level. Finally, we showed that decoder weights depend of left-right selectivity at 100% coherence, rather than the preferred direction. These results have implications for understanding how the information encoded by populations of neurons is affected by correlations in spiking activity.

Robust information propagation through noisy neural circuits

Sensory neurons give highly variable responses to stimulation, which can limit the amount of stimulus information available to downstream circuits. Much work has investigated the factors that affect the amount of information encoded in these population responses, leading to insights about the role of covariability among neurons, tuning curve shape, etc. However, the informativeness of neural responses is not the only relevant feature of population codes; of potentially equal importance is how robustly that information propagates to downstream structures. For instance, to quantify the retina’s performance, one must consider not only the informativeness of the optic nerve responses, but also the amount of information that survives the spike-generating nonlinearity and noise corruption in the next stage of processing, the lateral geniculate nucleus. Our study identifies the set of covariance structures for the upstream cells that optimize the ability of information to propagate through noisy, nonlinear circuits. Within this optimal family are covariances with “differential correlations”, which are known to reduce the information encoded in neural population activities. Thus, covariance structures that maximize information in neural population codes, and those that maximize the ability of this information to propagate, can be very different. Moreover, redundancy is neither necessary nor sufficient to make population codes robust against corruption by noise: redundant codes can be very fragile, and synergistic codes can—in some cases—optimize robustness against noise.

Information about the outside world, which originates in sensory neurons, propagates through multiple stages of processing before reaching the neural structures that control behavior. While much work in neuroscience has investigated the factors that affect the amount of information contained in peripheral sensory areas, very little work has asked how much of that information makes it through subsequent processing stages. That’s the focus of this paper, and it’s an important issue because information that fails to propagate cannot be used to affect decision-making. We find a tradeoff between information content and information transmission: neural codes which contain a large amount of information can transmit that information poorly to subsequent processing stages. Thus, the problem of robust information propagation—which has largely been overlooked in previous research—may be critical for determining how our sensory organs communicate with our brains. We identify the conditions under which information propagates well—or poorly—through multiple stages of neural processing.

Cracking the Neural Code for Sensory Perception by Combining Statistics, Intervention, and Behavior

The two basic processes underlying perceptual decisions-how neural responses encode stimuli, and how they inform behavioral choices-have mainly been studied separately. Thus, although many spatiotemporal features of neural population activity, or "neural codes," have been shown to carry sensory information, it is often unknown whether the brain uses these features for perception. To address this issue, we propose a new framework centered on redefining the neural code as the neural features that carry sensory information used by the animal to drive appropriate behavior; that is, the features that have an intersection between sensory and choice information. We show how this framework leads to a new statistical analysis of neural activity recorded during behavior that can identify such neural codes, and we discuss how to combine intersection-based analysis of neural recordings with intervention on neural activity to determine definitively whether specific neural activity features are involved in a task.

The spatial structure of correlated neuronal variability

Shared neural variability is ubiquitous in cortical populations. While this variability is presumed to arise from overlapping synaptic input, its precise relationship to local circuit architecture remains unclear. We combine computational models and in vivo recordings to study the relationship between the spatial structure of connectivity and correlated variability in neural circuits. Extending the theory of networks with balanced excitation and inhibition, we find that spatially localized lateral projections promote weakly correlated spiking, but broader lateral projections produce a distinctive spatial correlation structure: nearby neuron pairs are positively correlated, pairs at intermediate distances are negatively correlated and distant pairs are weakly correlated. This non-monotonic dependence of correlation on distance is revealed in a new analysis of recordings from superficial layers of macaque primary visual cortex. Our findings show that incorporating distance-dependent connectivity improves the extent to which balanced network theory can explain correlated neural variability.

A Readout Mechanism for Latency Codes

Response latency has been suggested as a possible source of information in the central nervous system when fast decisions are required. The accuracy of latency codes was studied in the past using a simplified readout algorithm termed the temporal-winner-take-all (tWTA). The tWTA is a competitive readout algorithm in which populations of neurons with a similar decision preference compete, and the algorithm selects according to the preference of the population that reaches the decision threshold first. It has been shown that this algorithm can account for accurate decisions among a small number of alternatives during short biologically relevant time periods. However, one of the major points of criticism of latency codes has been that it is unclear how can such a readout be implemented by the central nervous system. Here we show that the solution to this long standing puzzle may be rather simple. We suggest a mechanism that is based on reciprocal inhibition architecture, similar to that of the conventional winner-take-all, and show that under a wide range of parameters this mechanism is sufficient to implement the tWTA algorithm. This is done by first analyzing a rate toy model, and demonstrating its ability to discriminate short latency differences between its inputs. We then study the sensitivity of this mechanism to fine-tuning of its initial conditions, and show that it is robust to wide range of noise levels in the initial conditions. These results are then generalized to a Hodgkin-Huxley type of neuron model, using numerical simulations. Latency codes have been criticized for requiring a reliable stimulus-onset detection mechanism as a reference for measuring latency. Here we show that this frequent assumption does not hold, and that, an additional onset estimator is not needed to trigger this simple tWTA mechanism.

Effects of receptor correlations on molecular information transmission

Cells measure concentrations of external ligands by capturing ligand molecules with cell surface receptors. The numbers of molecules captured by different receptors co-vary because they depend on the same extrinsic ligand fluctuations. However, these numbers also counter-vary due to the intrinsic stochasticity of chemical processes because a single molecule randomly captured by a receptor cannot be captured by another. Such structure of receptor correlations is generally believed to lead to an increase in information about the external signal compared to the case of independent receptors. We analyze a solvable model of two molecular receptors and show that, contrary to this widespread expectation, the correlations have a small and negative effect on the information about the ligand concentration. Further, we show that measurements that average over multiple receptors are almost as informative as those that track the states of every individual one.

Direction-Selective Circuits Shape Noise to Ensure a Precise Population Code

Neural responses are noisy, and circuit structure can correlate this noise across neurons. Theoretical studies show that noise correlations can have diverse effects on population coding, but these studies rarely explore stimulus dependence of noise correlations. Here, we show that noise correlations in responses of ON-OFF direction-selective retinal ganglion cells are strongly stimulus dependent, and we uncover the circuit mechanisms producing this stimulus dependence. A population model based on these mechanistic studies shows that stimulus-dependent noise correlations improve the encoding of motion direction 2-fold compared to independent noise. This work demonstrates a mechanism by which a neural circuit effectively shapes its signal and noise in concert, minimizing corruption of signal by noise. Finally, we generalize our findings beyond direction coding in the retina and show that stimulus-dependent correlations will generally enhance information coding in populations of diversely tuned neurons.

Complementary Functional Organization of Neuronal Activity Patterns in the Perirhinal, Lateral Entorhinal, and Medial Entorhinal Cortices

It is commonly conceived that the cortical areas of the hippocampal region are functionally divided into the perirhinal cortex (PRC) and the lateral entorhinal cortex (LEC), which selectively process object information; and the medial entorhinal cortex (MEC), which selectively processes spatial information. Contrary to this notion, in rats performing a task that demands both object and spatial information processing, single neurons in PRC, LEC, and MEC, including those in both superficial and deep cortical areas and in grid, border, and head direction cells of MEC, have a highly similar range of selectivity to object and spatial dimensions of the task. By contrast, representational similarity analysis of population activity reveals a key distinction in the organization of information in these areas, such that PRC and LEC populations prioritize object over location information, whereas MEC populations prioritize location over object information. These findings bring to the hippocampal system a growing emphasis on population analyses as a powerful tool for characterizing neural representations supporting cognition and memory.

SIGNIFICANCE STATEMENT

Contrary to the common view that brain regions in the "what" and "where" streams distinctly process object and spatial cues, respectively, we found that both streams encode both object and spatial information but distinctly organize memories for objects and space. Specifically, perirhinal cortex and lateral entorhinal cortex represent objects and, within the object-specific representations, the locations where they occur. Conversely, medial entorhinal cortex represents relevant locations and, within those spatial representations, the objects that occupy them. Furthermore, these findings reach beyond simple notions of perirhinal cortex and lateral entorhinal cortex neurons as object detectors and MEC neurons as position detectors, and point to a more complex organization of memory representations within the medial temporal lobe system.

Structure, Function, and Propagation of Information across Living Two, Four, and Eight Node Degree Topologies

In this study, we created four network topologies composed of living cortical neurons and compared resultant structural-functional dynamics including the nature and quality of information transmission. Each living network was composed of living cortical neurons and were created using microstamping of adhesion promoting molecules and each was "designed" with different levels of convergence embedded within each structure. Networks were cultured over a grid of electrodes that permitted detailed measurements of neural activity at each node in the network. Of the topologies we tested, the "Random" networks in which neurons connect based on their own intrinsic properties transmitted information embedded within their spike trains with higher fidelity relative to any other topology we tested. Within our patterned topologies in which we explicitly manipulated structure, the effect of convergence on fidelity was dependent on both topology and time-scale (rate vs. temporal coding). A more detailed examination using tools from network analysis revealed that these changes in fidelity were also associated with a number of other structural properties including a node's degree, degree-degree correlations, path length, and clustering coefficients. Whereas information transmission was apparent among nodes with few connections, the greatest transmission fidelity was achieved among the few nodes possessing the highest number of connections (high degree nodes or putative hubs). These results provide a unique view into the relationship between structure and its affect on transmission fidelity, at least within these small neural populations with defined network topology. They also highlight the potential role of tools such as microstamp printing and microelectrode array recordings to construct and record from arbitrary network topologies to provide a new direction in which to advance the study of structure-function relationships.

Dynamics of robust pattern separability in the hippocampal dentate gyrus

The dentate gyrus (DG) is thought to perform pattern separation on inputs received from the entorhinal cortex, such that the DG forms distinct representations of different input patterns. Neuronal responses, however, are known to be variable, and that variability has the potential to confuse the representations of different inputs, thereby hindering the pattern separation function. This variability can be especially problematic for tissues such as the DG, in which the responses can persist for tens of seconds following stimulation: the long response duration allows for variability from many different sources to accumulate. To understand how the DG can robustly encode different input patterns, we investigated a recently developed in vitro hippocampal DG preparation that generates persistent responses to transient electrical stimulation. For 10-20 s after stimulation, the responses are indicative of the pattern of stimulation that was applied, even though the responses exhibit significant trial-to-trial variability. Analyzing the dynamical trajectories of the evoked responses, we found that, following stimulation, the neural responses follow distinct paths through the space of possible neural activations, with a different path associated with each stimulation pattern. The neural responses' trial-to-trial variability shifts the responses along these paths rather than between them, maintaining the separability of the input patterns. Manipulations that redistributed the variability more isotropically over the space of possible neural activations impeded the pattern separation function. Consequently, we conclude that the confinement of neuronal variability to these one-dimensional paths mitigates the impacts of variability on pattern encoding and, thus, may be an important aspect of the DG's ability to robustly encode input patterns.

Network Anisotropy Trumps Noise for Efficient Object Coding in Macaque Inferior Temporal Cortex

How neuronal ensembles compute information is actively studied in early visual cortex. Much less is known about how local ensembles function in inferior temporal (IT) cortex, the last stage of the ventral visual pathway that supports visual recognition. Previous reports suggested that nearby neurons carry information mostly independently, supporting efficient processing (Barlow, 1961). However, others postulate that noise covariation effects may depend on network anisotropy/homogeneity and on how the covariation relates to representation. Do slow trial-by-trial noise covariations increase or decrease IT's object coding capability, how does encoding capability relate to correlational structure (i.e., the spatial pattern of signal and noise redundancy/homogeneity across neurons), and does knowledge of correlational structure matter for decoding? We recorded simultaneously from ∼80 spiking neurons in ∼1 mm(3) of macaque IT under light neurolept anesthesia. Noise correlations were stronger for neurons with correlated tuning, and noise covariations reduced object encoding capability, including generalization across object pose and illumination. Knowledge of noise covariations did not lead to better decoding performance. However, knowledge of anisotropy/homogeneity improved encoding and decoding efficiency by reducing the number of neurons needed to reach a given performance level. Such correlated neurons were found mostly in supragranular and infragranular layers, supporting theories that link recurrent circuitry to manifold representation. These results suggest that redundancy benefits manifold learning of complex high-dimensional information and that subsets of neurons may be more immune to noise covariation than others.

SIGNIFICANCE STATEMENT

How noise affects neuronal population coding is poorly understood. By sampling densely from local populations supporting visual object recognition, we show that recurrent circuitry supports useful representations and that subsets of neurons may be more immune to noise covariation than others.

Visual coding with a population of direction-selective neurons

The brain decodes the visual scene from the action potentials of ∼20 retinal ganglion cell types. Among the retinal ganglion cells, direction-selective ganglion cells (DSGCs) encode motion direction. Several studies have focused on the encoding or decoding of motion direction by recording multiunit activity, mainly in the visual cortex. In this study, we simultaneously recorded from all four types of ON-OFF DSGCs of the rabbit retina using a microelectronics-based high-density microelectrode array (HDMEA) and decoded their concerted activity using probabilistic and linear decoders. Furthermore, we investigated how the modification of stimulus parameters (velocity, size, angle of moving object) and the use of different tuning curve fits influenced decoding precision. Finally, we simulated ON-OFF DSGC activity, based on real data, in order to understand how tuning curve widths and the angular distribution of the cells' preferred directions influence decoding performance. We found that probabilistic decoding strategies outperformed, on average, linear methods and that decoding precision was robust to changes in stimulus parameters such as velocity. The removal of noise correlations among cells, by random shuffling trials, caused a drop in decoding precision. Moreover, we found that tuning curves are broad in order to minimize large errors at the expense of a higher average error, and that the retinal direction-selective system would not substantially benefit, on average, from having more than four types of ON-OFF DSGCs or from a perfect alignment of the cells' preferred directions.

Triplet correlations among similarly tuned cells impact population coding

Which statistical features of spiking activity matter for how stimuli are encoded in neural populations? A vast body of work has explored how firing rates in individual cells and correlations in the spikes of cell pairs impact coding. Recent experiments have shown evidence for the existence of higher-order spiking correlations, which describe simultaneous firing in triplets and larger ensembles of cells; however, little is known about their impact on encoded stimulus information. Here, we take a first step toward closing this gap. We vary triplet correlations in small (approximately 10 cell) neural populations while keeping single cell and pairwise statistics fixed at typically reported values. This connection with empirically observed lower-order statistics is important, as it places strong constraints on the level of triplet correlations that can occur. For each value of triplet correlations, we estimate the performance of the neural population on a two-stimulus discrimination task. We find that the allowed changes in the level of triplet correlations can significantly enhance coding, in particular if triplet correlations differ for the two stimuli. In this scenario, triplet correlations must be included in order to accurately quantify the functionality of neural populations. When both stimuli elicit similar triplet correlations, however, pairwise models provide relatively accurate descriptions of coding accuracy. We explain our findings geometrically via the skew that triplet correlations induce in population-wide distributions of neural responses. Finally, we calculate how many samples are necessary to accurately measure spiking correlations of this type, providing an estimate of the necessary recording times in future experiments.

Long-range intralaminar noise correlations in the barrel cortex

Identifying the properties of correlations in the firing of neocortical neurons is central to our understanding of cortical information processing. It has been generally assumed, by virtue of the columnar organization of the neocortex, that the firing of neurons residing in a certain vertical domain is highly correlated. On the other hand, firing correlations between neurons steeply decline with horizontal distance. Technical difficulties in sampling neurons with sufficient spatial information have precluded the critical evaluation of these notions. We used 128-channel "silicon probes" to examine the spike-count noise correlations during spontaneous activity between multiple neurons with identified laminar position and over large horizontal distances in the anesthetized rat barrel cortex. Eigen decomposition of correlation coefficient matrices revealed that the laminar position of a neuron is a significant determinant of these correlations, such that the fluctuations of layer 5B/6 neurons are in opposite direction to those of layers 5A and 4. Moreover, we found that within each experiment, the distribution of horizontal, intralaminar spike-count correlation coefficients, up to a distance of ∼1.5 mm, is practically identical to the distribution of vertical correlations. Taken together, these data reveal that the neuron's laminar position crucially affects its role in cortical processing. Moreover, our analyses reveal that this laminar effect extends over several functional columns. We propose that within the cortex the influence of the horizontal elements exists in a dynamic balance with the influence of the vertical domain and this balance is modulated with brain states to shape the network's behavior.

Neural population coding: combining insights from microscopic and mass signals

Behavior relies on the distributed and coordinated activity of neural populations. Population activity can be measured using multi-neuron recordings and neuroimaging. Neural recordings reveal how the heterogeneity, sparseness, timing, and correlation of population activity shape information processing in local networks, whereas neuroimaging shows how long-range coupling and brain states impact on local activity and perception. To obtain an integrated perspective on neural information processing we need to combine knowledge from both levels of investigation. We review recent progress of how neural recordings, neuroimaging, and computational approaches begin to elucidate how interactions between local neural population activity and large-scale dynamics shape the structure and coding capacity of local information representations, make them state-dependent, and control distributed populations that collectively shape behavior.

Correlated activity supports efficient cortical processing

Visual recognition is a computational challenge that is thought to occur via efficient coding. An important concept is sparseness, a measure of coding efficiency. The prevailing view is that sparseness supports efficiency by minimizing redundancy and correlations in spiking populations. Yet, we recently reported that "choristers", neurons that behave more similarly (have correlated stimulus preferences and spontaneous coincident spiking), carry more generalizable object information than uncorrelated neurons ("soloists") in macaque inferior temporal (IT) cortex. The rarity of choristers (as low as 6% of IT neurons) indicates that they were likely missed in previous studies. Here, we report that correlation strength is distinct from sparseness (choristers are not simply broadly tuned neurons), that choristers are located in non-granular output layers, and that correlated activity predicts human visual search efficiency. These counterintuitive results suggest that a redundant correlational structure supports efficient processing and behavior.

Nonlinear dendritic integration of electrical and chemical synaptic inputs drives fine-scale correlations

Throughout the CNS, gap junction-mediated electrical signals synchronize neural activity on millisecond timescales via cooperative interactions with chemical synapses. However, gap junction-mediated synchrony has rarely been studied in the context of varying spatiotemporal patterns of electrical and chemical synaptic activity. Thus, the mechanism underlying fine-scale synchrony and its relationship to neural coding remain unclear. We examined spike synchrony in pairs of genetically identified, electrically coupled ganglion cells in mouse retina. We found that coincident electrical and chemical synaptic inputs, but not electrical inputs alone, elicited synchronized dendritic spikes in subregions of coupled dendritic trees. The resulting nonlinear integration produced fine-scale synchrony in the cells' spike output, specifically for light stimuli driving input to the regions of dendritic overlap. In addition, the strength of synchrony varied inversely with spike rate. Together, these features may allow synchronized activity to encode information about the spatial distribution of light that is ambiguous on the basis of spike rate alone.

Computational neuroscience: beyond the local circuit

Computational neuroscience has focused largely on the dynamics and function of local circuits of neuronal populations dedicated to a common task, such as processing a common sensory input, storing its features in working memory, choosing between a set of options dictated by controlled experimental settings or generating the appropriate actions. Most of current circuit models suggest mechanisms for computations that can be captured by networks of simplified neurons connected via simple synaptic weights. In this article I review the progress of this approach and its limitations. It is argued that new experimental techniques will yield data that might challenge the present paradigms in that they will (1) demonstrate the computational importance of microscopic structural and physiological complexity and specificity; (2) highlight the importance of models of large brain structures engaged in a variety of tasks; and (3) reveal the necessity of coupling the neuronal networks to chemical and environmental variables.