Input nonlinearities can shape beyond-pairwise correlations and improve information transmission by neural populations

Uncovering hidden network architecture from spiking activities using an exact statistical input-output relation of neurons

Identifying network architecture from observed neural activities is crucial in neuroscience studies. A key requirement is knowledge of the statistical input-output relation of single neurons in vivo. By utilizing an exact analytical solution of the spike-timing for leaky integrate-and-fire neurons under noisy inputs balanced near the threshold, we construct a framework that links synaptic type, strength, and spiking nonlinearity with the statistics of neuronal population activity. The framework explains structured pairwise and higher-order interactions of neurons receiving common inputs under different architectures. We compared the theoretical predictions with the activity of monkey and mouse V1 neurons and found that excitatory inputs given to pairs explained the observed sparse activity characterized by strong negative triple-wise interactions, thereby ruling out the alternative explanation by shared inhibition. Moreover, we showed that the strong interactions are a signature of excitatory rather than inhibitory inputs whenever the spontaneous rate is low. We present a guide map of neural interactions that help researchers to specify the hidden neuronal motifs underlying observed interactions found in empirical data.

The structures and functions of correlations in neural population codes

The collective activity of a population of neurons, beyond the properties of individual cells, is crucial for many brain functions. A fundamental question is how activity correlations between neurons affect how neural populations process information. Over the past 30 years, major progress has been made on how the levels and structures of correlations shape the encoding of information in population codes. Correlations influence population coding through the organization of pairwise-activity correlations with respect to the similarity of tuning of individual neurons, by their stimulus modulation and by the presence of higher-order correlations. Recent work has shown that correlations also profoundly shape other important functions performed by neural populations, including generating codes across multiple timescales and facilitating information transmission to, and readout by, downstream brain areas to guide behaviour. Here, we review this recent work and discuss how the structures of correlations can have opposite effects on the different functions of neural populations, thus creating trade-offs and constraints for the structure-function relationships of population codes. Further, we present ideas on how to combine large-scale simultaneous recordings of neural populations, computational models, analyses of behaviour, optogenetics and anatomy to unravel how the structures of correlations might be optimized to serve multiple functions.

The Geometry of Information Coding in Correlated Neural Populations

Neurons in the brain represent information in their collective activity. The fidelity of this neural population code depends on whether and how variability in the response of one neuron is shared with other neurons. Two decades of studies have investigated the influence of these noise correlations on the properties of neural coding. We provide an overview of the theoretical developments on the topic. Using simple, qualitative, and general arguments, we discuss, categorize, and relate the various published results. We emphasize the relevance of the fine structure of noise correlation, and we present a new approach to the issue. Throughout this review, we emphasize a geometrical picture of how noise correlations impact the neural code.

Autonomous emergence of connectivity assemblies via spike triplet interactions

Non-random connectivity can emerge without structured external input driven by activity-dependent mechanisms of synaptic plasticity based on precise spiking patterns. Here we analyze the emergence of global structures in recurrent networks based on a triplet model of spike timing dependent plasticity (STDP), which depends on the interactions of three precisely-timed spikes, and can describe plasticity experiments with varying spike frequency better than the classical pair-based STDP rule. We derive synaptic changes arising from correlations up to third-order and describe them as the sum of structural motifs, which determine how any spike in the network influences a given synaptic connection through possible connectivity paths. This motif expansion framework reveals novel structural motifs under the triplet STDP rule, which support the formation of bidirectional connections and ultimately the spontaneous emergence of global network structure in the form of self-connected groups of neurons, or assemblies. We propose that under triplet STDP assembly structure can emerge without the need for externally patterned inputs or assuming a symmetric pair-based STDP rule common in previous studies. The emergence of non-random network structure under triplet STDP occurs through internally-generated higher-order correlations, which are ubiquitous in natural stimuli and neuronal spiking activity, and important for coding. We further demonstrate how neuromodulatory mechanisms that modulate the shape of the triplet STDP rule or the synaptic transmission function differentially promote structural motifs underlying the emergence of assemblies, and quantify the differences using graph theoretic measures.

Emergent non-random connectivity structures in different brain regions are tightly related to specific patterns of neural activity and support diverse brain functions. For instance, self-connected groups of neurons, known as assemblies, have been proposed to represent functional units in brain circuits and can emerge even without patterned external instruction. Here we investigate the emergence of non-random connectivity in recurrent networks using a particular plasticity rule, triplet STDP, which relies on the interaction of spike triplets and can capture higher-order statistical dependencies in neural activity. We derive the evolution of the synaptic strengths in the network and explore the conditions for the self-organization of connectivity into assemblies. We demonstrate key differences of the triplet STDP rule compared to the classical pair-based rule in terms of how assemblies are formed, including the realistic asymmetric shape and influence of novel connectivity motifs on network plasticity driven by higher-order correlations. Assembly formation depends on the specific shape of the STDP window and synaptic transmission function, pointing towards an important role of neuromodulatory signals on formation of intrinsically generated assemblies.

Ensemble inhibition and excitation in the human cortex: An Ising-model analysis with uncertainties

The pairwise maximum entropy model, also known as the Ising model, has been widely used to analyze the collective activity of neurons. However, controversy persists in the literature about seemingly inconsistent findings, whose significance is unclear due to lack of reliable error estimates. We therefore develop a method for accurately estimating parameter uncertainty based on random walks in parameter space using adaptive Markov-chain Monte Carlo after the convergence of the main optimization algorithm. We apply our method to the activity patterns of excitatory and inhibitory neurons recorded with multielectrode arrays in the human temporal cortex during the wake-sleep cycle. Our analysis shows that the Ising model captures neuronal collective behavior much better than the independent model during wakefulness, light sleep, and deep sleep when both excitatory (E) and inhibitory (I) neurons are modeled; ignoring the inhibitory effects of I neurons dramatically overestimates synchrony among E neurons. Furthermore, information-theoretic measures reveal that the Ising model explains about 80-95% of the correlations, depending on sleep state and neuron type. Thermodynamic measures show signatures of criticality, although we take this with a grain of salt as it may be merely a reflection of long-range neural correlations.

A Moment-Based Maximum Entropy Model for Fitting Higher-Order Interactions in Neural Data

Correlations in neural activity have been demonstrated to have profound consequences for sensory encoding. To understand how neural populations represent stimulus information, it is therefore necessary to model how pairwise and higher-order spiking correlations between neurons contribute to the collective structure of population-wide spiking patterns. Maximum entropy models are an increasingly popular method for capturing collective neural activity by including successively higher-order interaction terms. However, incorporating higher-order interactions in these models is difficult in practice due to two factors. First, the number of parameters exponentially increases as higher orders are added. Second, because triplet (and higher) spiking events occur infrequently, estimates of higher-order statistics may be contaminated by sampling noise. To address this, we extend previous work on the Reliable Interaction class of models to develop a normalized variant that adaptively identifies the specific pairwise and higher-order moments that can be estimated from a given dataset for a specified confidence level. The resulting “Reliable Moment” model is able to capture cortical-like distributions of population spiking patterns. Finally, we show that, compared with the Reliable Interaction model, the Reliable Moment model infers fewer strong spurious higher-order interactions and is better able to predict the frequencies of previously unobserved spiking patterns.

Robust information propagation through noisy neural circuits

Sensory neurons give highly variable responses to stimulation, which can limit the amount of stimulus information available to downstream circuits. Much work has investigated the factors that affect the amount of information encoded in these population responses, leading to insights about the role of covariability among neurons, tuning curve shape, etc. However, the informativeness of neural responses is not the only relevant feature of population codes; of potentially equal importance is how robustly that information propagates to downstream structures. For instance, to quantify the retina’s performance, one must consider not only the informativeness of the optic nerve responses, but also the amount of information that survives the spike-generating nonlinearity and noise corruption in the next stage of processing, the lateral geniculate nucleus. Our study identifies the set of covariance structures for the upstream cells that optimize the ability of information to propagate through noisy, nonlinear circuits. Within this optimal family are covariances with “differential correlations”, which are known to reduce the information encoded in neural population activities. Thus, covariance structures that maximize information in neural population codes, and those that maximize the ability of this information to propagate, can be very different. Moreover, redundancy is neither necessary nor sufficient to make population codes robust against corruption by noise: redundant codes can be very fragile, and synergistic codes can—in some cases—optimize robustness against noise.

Information about the outside world, which originates in sensory neurons, propagates through multiple stages of processing before reaching the neural structures that control behavior. While much work in neuroscience has investigated the factors that affect the amount of information contained in peripheral sensory areas, very little work has asked how much of that information makes it through subsequent processing stages. That’s the focus of this paper, and it’s an important issue because information that fails to propagate cannot be used to affect decision-making. We find a tradeoff between information content and information transmission: neural codes which contain a large amount of information can transmit that information poorly to subsequent processing stages. Thus, the problem of robust information propagation—which has largely been overlooked in previous research—may be critical for determining how our sensory organs communicate with our brains. We identify the conditions under which information propagates well—or poorly—through multiple stages of neural processing.

Accuracy of rate coding: When shorter time window and higher spontaneous activity help

It is widely accepted that neuronal firing rates contain a significant amount of information about the stimulus intensity. Nevertheless, theoretical studies on the coding accuracy inferred from the exact spike counting distributions are rare. We present an analysis based on the number of observed spikes assuming the stochastic perfect integrate-and-fire model with a change point, representing the stimulus onset, for which we calculate the corresponding Fisher information to investigate the accuracy of rate coding. We analyze the effect of changing the duration of the time window and the influence of several parameters of the model, in particular the level of the presynaptic spontaneous activity and the level of random fluctuation of the membrane potential, which can be interpreted as noise of the system. The results show that the Fisher information is nonmonotonic with respect to the length of the observation period. This counterintuitive result is caused by the discrete nature of the count of spikes. We observe also that the signal can be enhanced by noise, since the Fisher information is nonmonotonic with respect to the level of spontaneous activity and, in some cases, also with respect to the level of fluctuation of the membrane potential.

Higher-Order Synaptic Interactions Coordinate Dynamics in Recurrent Networks

Linking synaptic connectivity to dynamics is key to understanding information processing in neocortex. Circuit dynamics emerge from complex interactions of interconnected neurons, necessitating that links between connectivity and dynamics be evaluated at the network level. Here we map propagating activity in large neuronal ensembles from mouse neocortex and compare it to a recurrent network model, where connectivity can be precisely measured and manipulated. We find that a dynamical feature dominates statistical descriptions of propagating activity for both neocortex and the model: convergent clusters comprised of fan-in triangle motifs, where two input neurons are themselves connected. Fan-in triangles coordinate the timing of presynaptic inputs during ongoing activity to effectively generate postsynaptic spiking. As a result, paradoxically, fan-in triangles dominate the statistics of spike propagation even in randomly connected recurrent networks. Interplay between higher-order synaptic connectivity and the integrative properties of neurons constrains the structure of network dynamics and shapes the routing of information in neocortex.

Triplet correlations among similarly tuned cells impact population coding

Which statistical features of spiking activity matter for how stimuli are encoded in neural populations? A vast body of work has explored how firing rates in individual cells and correlations in the spikes of cell pairs impact coding. Recent experiments have shown evidence for the existence of higher-order spiking correlations, which describe simultaneous firing in triplets and larger ensembles of cells; however, little is known about their impact on encoded stimulus information. Here, we take a first step toward closing this gap. We vary triplet correlations in small (approximately 10 cell) neural populations while keeping single cell and pairwise statistics fixed at typically reported values. This connection with empirically observed lower-order statistics is important, as it places strong constraints on the level of triplet correlations that can occur. For each value of triplet correlations, we estimate the performance of the neural population on a two-stimulus discrimination task. We find that the allowed changes in the level of triplet correlations can significantly enhance coding, in particular if triplet correlations differ for the two stimuli. In this scenario, triplet correlations must be included in order to accurately quantify the functionality of neural populations. When both stimuli elicit similar triplet correlations, however, pairwise models provide relatively accurate descriptions of coding accuracy. We explain our findings geometrically via the skew that triplet correlations induce in population-wide distributions of neural responses. Finally, we calculate how many samples are necessary to accurately measure spiking correlations of this type, providing an estimate of the necessary recording times in future experiments.

The sign rule and beyond: boundary effects, flexibility, and noise correlations in neural population codes

Over repeat presentations of the same stimulus, sensory neurons show variable responses. This “noise” is typically correlated between pairs of cells, and a question with rich history in neuroscience is how these noise correlations impact the population's ability to encode the stimulus. Here, we consider a very general setting for population coding, investigating how information varies as a function of noise correlations, with all other aspects of the problem – neural tuning curves, etc. – held fixed. This work yields unifying insights into the role of noise correlations. These are summarized in the form of theorems, and illustrated with numerical examples involving neurons with diverse tuning curves. Our main contributions are as follows. (1) We generalize previous results to prove a sign rule (SR) — if noise correlations between pairs of neurons have opposite signs vs. their signal correlations, then coding performance will improve compared to the independent case. This holds for three different metrics of coding performance, and for arbitrary tuning curves and levels of heterogeneity. This generality is true for our other results as well. (2) As also pointed out in the literature, the SR does not provide a necessary condition for good coding. We show that a diverse set of correlation structures can improve coding. Many of these violate the SR, as do experimentally observed correlations. There is structure to this diversity: we prove that the optimal correlation structures must lie on boundaries of the possible set of noise correlations. (3) We provide a novel set of necessary and sufficient conditions, under which the coding performance (in the presence of noise) will be as good as it would be if there were no noise present at all.

Sensory systems communicate information to the brain — and brain areas communicate between themselves — via the electrical activities of their respective neurons. These activities are “noisy”: repeat presentations of the same stimulus do not yield to identical responses every time. Furthermore, the neurons' responses are not independent: the variability in their responses is typically correlated from cell to cell. How does this change the impact of the noise — for better or for worse? Our goal here is to classify (broadly) the sorts of noise correlations that are either good or bad for enabling populations of neurons to transmit information. This is helpful as there are many possibilities for the noise correlations, and the set of possibilities becomes large for even modestly sized neural populations. We prove mathematically that, for larger populations, there are many highly diverse ways that favorable correlations can occur. These often differ from the noise correlation structures that are typically identified as beneficial for information transmission – those that follow the so-called “sign rule.” Our results help in interpreting some recent data that seems puzzling from the perspective of this rule.