Approximate Inference for Time-Varying Interactions and Macroscopic Dynamics of Neural Populations

Adaptive modeling and inference of higher-order coordination in neuronal assemblies: A dynamic greedy estimation approach

Central in the study of population codes, coordinated ensemble spiking activity is widely observable in neural recordings with hypothesized roles in robust stimulus representation, interareal communication, and learning and memory formation. Model-free measures of synchrony characterize coherent pairwise activity but not higher-order interactions, a limitation transcended by statistical models of ensemble spiking activity. However, existing model-based analyses often impose assumptions about the relevance of higher-order interactions and require repeated trials to characterize dynamics in the correlational structure of ensemble activity. To address these shortcomings, we propose an adaptive greedy filtering algorithm based on a discretized mark point-process model of ensemble spiking and a corresponding statistical inference framework to identify significant higher-order coordination. In the course of developing a precise statistical test, we show that confidence intervals can be constructed for greedily estimated parameters. We demonstrate the utility of our proposed methods on simulated neuronal assemblies. Applied to multi-electrode recordings from human and rat cortical assemblies, our proposed methods provide new insights into the dynamics underlying localized population activity during transitions between brain states.

Hippocampal and medial prefrontal ensemble spiking represents episodes and rules in similar task spaces

Episodic memory requires the hippocampus and prefrontal cortex to guide decisions by representing events in spatial, temporal, and personal contexts. Both brain regions have been described by cognitive theories that represent events in context as locations in maps or memory spaces. We query whether ensemble spiking in these regions described spatial structures as rats performed memory tasks. From each ensemble, we construct a state-space with each point defined by the coordinated spiking of single and pairs of units in 125-ms bins and investigate how state-space locations discriminate task features. Trajectories through state-spaces correspond with behavioral episodes framed by spatial, temporal, and internal contexts. Both hippocampal and prefrontal ensembles distinguish maze locations, task intervals, and goals by distances between state-space locations, consistent with cognitive mapping and relational memory space theories of episodic memory. Prefrontal modulation of hippocampal activity may guide choices by directing memory representations toward appropriate state-space goal locations.

Adaptive modeling and inference of higher-order coordination in neuronal assemblies: a dynamic greedy estimation approach

Central in the study of population codes, coordinated ensemble spiking activity is widely observable in neural recordings with hypothesized roles in robust stimulus representation, interareal communication, and learning and memory formation. Model-free measures of synchrony characterize coherent pairwise activity but not higher-order interactions, a limitation transcended by statistical models of ensemble spiking activity. However, existing model-based analyses often impose assumptions about the relevance of higher-order interactions and require repeated trials to characterize dynamics in the correlational structure of ensemble activity. To address these shortcomings, we propose an adaptive greedy filtering algorithm based on a discretized mark point-process model of ensemble spiking and a corresponding statistical inference framework to identify significant higher-order coordination. In the course of developing a precise statistical test, we show that confidence intervals can be constructed for greedily estimated parameters. We demonstrate the utility of our proposed methods on simulated neuronal assemblies. Applied to multi-electrode recordings from human and rat cortical assemblies, our proposed methods provide new insights into the dynamics underlying localized population activity during transitions between brain states.

Nonequilibrium thermodynamics of the asymmetric Sherrington-Kirkpatrick model

Most natural systems operate far from equilibrium, displaying time-asymmetric, irreversible dynamics characterized by a positive entropy production while exchanging energy and matter with the environment. Although stochastic thermodynamics underpins the irreversible dynamics of small systems, the nonequilibrium thermodynamics of larger, more complex systems remains unexplored. Here, we investigate the asymmetric Sherrington-Kirkpatrick model with synchronous and asynchronous updates as a prototypical example of large-scale nonequilibrium processes. Using a path integral method, we calculate a generating functional over trajectories, obtaining exact solutions of the order parameters, path entropy, and steady-state entropy production of infinitely large networks. Entropy production peaks at critical order-disorder phase transitions, but is significantly larger for quasi-deterministic disordered dynamics. Consequently, entropy production can increase under distinct scenarios, requiring multiple thermodynamic quantities to describe the system accurately. These results contribute to developing an exact analytical theory of the nonequilibrium thermodynamics of large-scale physical and biological systems and their phase transitions.

Uncovering hidden network architecture from spiking activities using an exact statistical input-output relation of neurons

Identifying network architecture from observed neural activities is crucial in neuroscience studies. A key requirement is knowledge of the statistical input-output relation of single neurons in vivo. By utilizing an exact analytical solution of the spike-timing for leaky integrate-and-fire neurons under noisy inputs balanced near the threshold, we construct a framework that links synaptic type, strength, and spiking nonlinearity with the statistics of neuronal population activity. The framework explains structured pairwise and higher-order interactions of neurons receiving common inputs under different architectures. We compared the theoretical predictions with the activity of monkey and mouse V1 neurons and found that excitatory inputs given to pairs explained the observed sparse activity characterized by strong negative triple-wise interactions, thereby ruling out the alternative explanation by shared inhibition. Moreover, we showed that the strong interactions are a signature of excitatory rather than inhibitory inputs whenever the spontaneous rate is low. We present a guide map of neural interactions that help researchers to specify the hidden neuronal motifs underlying observed interactions found in empirical data.

An in silico model for determining the influence of neuronal co-activity on rodent spatial behavior

BACKGROUND

Neuropsychological and neurophysiological analyses focus on understanding how neuronal activity and co-activity predict behavior. Experimental techniques allow for modulation of neuronal activity, but do not control neuronal ensemble spatiotemporal firing patterns, and there are few, if any, sophisticated in silico techniques which accurately reconstruct physiological neural spike trains and behavior using unit co-activity as an input parameter.

NEW METHOD

Our approach to simulation of neuronal spike trains is based on using state space modeling to estimate a weighted graph of interaction strengths between pairs of neurons along with separate estimations of spiking threshold voltage and neuronal membrane leakage. These parameters allow us to tune a biophysical model which is then employed to accurately reconstruct spike trains from freely behaving animals and then use these spike trains to estimate an animal's spatial behavior. The reconstructed spatial behavior allows us to confirm the same information is present in both the recorded and simulated spike trains.

RESULTS

Our method reconstructs spike trains (98 ± 0.0013% like original spike trains, mean ± SEM) and animal position (9.468 ± 0.240 cm, mean ± SEM) with high fidelity.

COMPARISON WITH EXISTING METHOD(S)

To our knowledge, this is the first method that uses empirically derived network connectivity to constrain biophysical parameters and predict spatial behavior. Together, these methods allow in silico quantification of the contribution of specific unit activity and co-activity to animal spatial behavior.

CONCLUSIONS

Our novel approach provides a flexible, robust in silico technique for determining the contribution of specific neuronal activity and co-activity to spatial behavior.

Optimal Population Coding for Dynamic Input by Nonequilibrium Networks

The efficient coding hypothesis states that neural response should maximize its information about the external input. Theoretical studies focus on optimal response in single neuron and population code in networks with weak pairwise interactions. However, more biological settings with asymmetric connectivity and the encoding for dynamical stimuli have not been well-characterized. Here, we study the collective response in a kinetic Ising model that encodes the dynamic input. We apply gradient-based method and mean-field approximation to reconstruct networks given the neural code that encodes dynamic input patterns. We measure network asymmetry, decoding performance, and entropy production from networks that generate optimal population code. We analyze how stimulus correlation, time scale, and reliability of the network affect optimal encoding networks. Specifically, we find network dynamics altered by statistics of the dynamic input, identify stimulus encoding strategies, and show optimal effective temperature in the asymmetric networks. We further discuss how this approach connects to the Bayesian framework and continuous recurrent neural networks. Together, these results bridge concepts of nonequilibrium physics with the analyses of dynamics and coding in networks.

A unifying framework for mean-field theories of asymmetric kinetic Ising systems

Kinetic Ising models are powerful tools for studying the non-equilibrium dynamics of complex systems. As their behavior is not tractable for large networks, many mean-field methods have been proposed for their analysis, each based on unique assumptions about the system’s temporal evolution. This disparity of approaches makes it challenging to systematically advance mean-field methods beyond previous contributions. Here, we propose a unifying framework for mean-field theories of asymmetric kinetic Ising systems from an information geometry perspective. The framework is built on Plefka expansions of a system around a simplified model obtained by an orthogonal projection to a sub-manifold of tractable probability distributions. This view not only unifies previous methods but also allows us to develop novel methods that, in contrast with traditional approaches, preserve the system’s correlations. We show that these new methods can outperform previous ones in predicting and assessing network properties near maximally fluctuating regimes.

Many mean-field theories are proposed for studying the non-equilibrium dynamics of complex systems, each based on specific assumptions about the system’s temporal evolution. Here, Aguilera et al. propose a unified framework for mean-field theories of asymmetric kinetic Ising systems to study non-equilibrium dynamics.

Estimation of neuron parameters from imperfect observations

The estimation of parameters controlling the electrical properties of biological neurons is essential to determine their complement of ion channels and to understand the function of biological circuits. By synchronizing conductance models to time series observations of the membrane voltage, one may construct models capable of predicting neuronal dynamics. However, identifying the actual set of parameters of biological ion channels remains a formidable theoretical challenge. Here, we present a regularization method that improves convergence towards this optimal solution when data are noisy and the model is unknown. Our method relies on the existence of an offset in parameter space arising from the interplay between model nonlinearity and experimental error. By tuning this offset, we induce saddle-node bifurcations from sub-optimal to optimal solutions. This regularization method increases the probability of finding the optimal set of parameters from 67% to 94.3%. We also reduce parameter correlations by implementing adaptive sampling and stimulation protocols compatible with parameter identifiability requirements. Our results show that the optimal model parameters may be inferred from imperfect observations provided the conditions of observability and identifiability are fulfilled.

Inverse Ising problem in continuous time: A latent variable approach

We consider the inverse Ising problem: the inference of network couplings from observed spin trajectories for a model with continuous time Glauber dynamics. By introducing two sets of auxiliary latent random variables we render the likelihood into a form which allows for simple iterative inference algorithms with analytical updates. The variables are (1) Poisson variables to linearize an exponential term which is typical for point process likelihoods and (2) Pólya-Gamma variables, which make the likelihood quadratic in the coupling parameters. Using the augmented likelihood, we derive an expectation-maximization (EM) algorithm to obtain the maximum likelihood estimate of network parameters. Using a third set of latent variables we extend the EM algorithm to sparse couplings via L1 regularization. Finally, we develop an efficient approximate Bayesian inference algorithm using a variational approach. We demonstrate the performance of our algorithms on data simulated from an Ising model. For data which are simulated from a more biologically plausible network with spiking neurons, we show that the Ising model captures well the low order statistics of the data and how the Ising couplings are related to the underlying synaptic structure of the simulated network.

Probabilistic models for neural populations that naturally capture global coupling and criticality

Advances in multi-unit recordings pave the way for statistical modeling of activity patterns in large neural populations. Recent studies have shown that the summed activity of all neurons strongly shapes the population response. A separate recent finding has been that neural populations also exhibit criticality, an anomalously large dynamic range for the probabilities of different population activity patterns. Motivated by these two observations, we introduce a class of probabilistic models which takes into account the prior knowledge that the neural population could be globally coupled and close to critical. These models consist of an energy function which parametrizes interactions between small groups of neurons, and an arbitrary positive, strictly increasing, and twice differentiable function which maps the energy of a population pattern to its probability. We show that: 1) augmenting a pairwise Ising model with a nonlinearity yields an accurate description of the activity of retinal ganglion cells which outperforms previous models based on the summed activity of neurons; 2) prior knowledge that the population is critical translates to prior expectations about the shape of the nonlinearity; 3) the nonlinearity admits an interpretation in terms of a continuous latent variable globally coupling the system whose distribution we can infer from data. Our method is independent of the underlying system’s state space; hence, it can be applied to other systems such as natural scenes or amino acid sequences of proteins which are also known to exhibit criticality.

Populations of sensory neurons represent information about the outside environment in a collective fashion. A salient property of this distributed neural code is criticality. Yet most models used to date to analyze recordings from large neural populations do not take this observation explicitly into account. Here we aim to bridge this gap by designing probabilistic models whose structure reflects the expectation that the population is close to critical. We show that such principled approach improves previously considered models, and we demonstrate a connection between our models and the presence of continuous latent variables which is a recently proposed mechanism underlying criticality in many natural systems.