Emergent spike patterns in neuronal populations

A large-scale neuronal network modelling study: Stimulus size modulates gamma oscillations in the primary visual cortex by long-range connections

Stimulus size modulation of neuronal firing activity is a fundamental property of the primary visual cortex. Numerous biological experiments have shown that stimulus size modulation is affected by multiple factors at different spatiotemporal scales, but the exact pathways and mechanisms remain incompletely understood. In this paper, we establish a large-scale neuronal network model of primary visual cortex with layer 2/3 to study how gamma oscillation properties are modulated by stimulus size and especially how long-range connections affect the modulation as realistic neuronal properties and spatial distributions of synaptic connections are considered. It is shown that long-range horizontal synaptic connections are sufficient to produce dimensional modulation of firing rates and gamma oscillations. In particular, with increasing grating stimulus size, the firing rate increases and then decreases, the peak frequency of gamma oscillations decreases and the spectral power increases. These are consistent with biological experimental observations. Furthermore, we explain in detail how the number and spatial distribution of long-range connections affect the size modulation of gamma oscillations by using the analysis of neuronal firing activity and synaptic current fluctuations. Our results provide a mechanism explanation for size modulation of gamma oscillations in the primary visual cortex and reveal the important and unique role played by long-range connections, which contributes to a deeper understanding of the cognitive function of gamma oscillations in visual cortex.

Learning spiking neuronal networks with artificial neural networks: neural oscillations

First-principles-based modelings have been extremely successful in providing crucial insights and predictions for complex biological functions and phenomena. However, they can be hard to build and expensive to simulate for complex living systems. On the other hand, modern data-driven methods thrive at modeling many types of high-dimensional and noisy data. Still, the training and interpretation of these data-driven models remain challenging. Here, we combine the two types of methods to model stochastic neuronal network oscillations. Specifically, we develop a class of artificial neural networks to provide faithful surrogates to the high-dimensional, nonlinear oscillatory dynamics produced by a spiking neuronal network model. Furthermore, when the training data set is enlarged within a range of parameter choices, the artificial neural networks become generalizable to these parameters, covering cases in distinctly different dynamical regimes. In all, our work opens a new avenue for modeling complex neuronal network dynamics with artificial neural networks.

Multi-band oscillations emerge from a simple spiking network

In the brain, coherent neuronal activities often appear simultaneously in multiple frequency bands, e.g., as combinations of alpha (8-12 Hz), beta (12.5-30 Hz), and gamma (30-120 Hz) oscillations, among others. These rhythms are believed to underlie information processing and cognitive functions and have been subjected to intense experimental and theoretical scrutiny. Computational modeling has provided a framework for the emergence of network-level oscillatory behavior from the interaction of spiking neurons. However, due to the strong nonlinear interactions between highly recurrent spiking populations, the interplay between cortical rhythms in multiple frequency bands has rarely been theoretically investigated. Many studies invoke multiple physiological timescales (e.g., various ion channels or multiple types of inhibitory neurons) or oscillatory inputs to produce rhythms in multi-bands. Here, we demonstrate the emergence of multi-band oscillations in a simple network consisting of one excitatory and one inhibitory neuronal population driven by constant input. First, we construct a data-driven, Poincaré section theory for robust numerical observations of single-frequency oscillations bifurcating into multiple bands. Then, we develop model reductions of the stochastic, nonlinear, high-dimensional neuronal network to capture the appearance of multi-band dynamics and the underlying bifurcations theoretically. Furthermore, when viewed within the reduced state space, our analysis reveals conserved geometrical features of the bifurcations on low-dimensional dynamical manifolds. These results suggest a simple geometric mechanism behind the emergence of multi-band oscillations without appealing to oscillatory inputs or multiple synaptic or neuronal timescales. Thus, our work points to unexplored regimes of stochastic competition between excitation and inhibition behind the generation of dynamic, patterned neuronal activities.

The Use of Reduced Models to Generate Irregular, Broad-Band Signals That Resemble Brain Rhythms

The brain produces rhythms in a variety of frequency bands. Some are likely by-products of neuronal processes; others are thought to be top-down. Produced entirely naturally, these rhythms have clearly recognizable beats, but they are very far from periodic in the sense of mathematics. The signals are broad-band, episodic, wandering in amplitude and frequency; the rhythm comes and goes, degrading and regenerating. Gamma rhythms, in particular, have been studied by many authors in computational neuroscience, using reduced models as well as networks of hundreds to thousands of integrate-and-fire neurons. All of these models captured successfully the oscillatory nature of gamma rhythms, but the irregular character of gamma in reduced models has not been investigated thoroughly. In this article, we tackle the mathematical question of whether signals with the properties of brain rhythms can be generated from low dimensional dynamical systems. We found that while adding white noise to single periodic cycles can to some degree simulate gamma dynamics, such models tend to be limited in their ability to capture the range of behaviors observed. Using an ODE with two variables inspired by the FitzHugh-Nagumo and Leslie-Gower models, with stochastically varying coefficients designed to control independently amplitude, frequency, and degree of degeneracy, we were able to replicate the qualitative characteristics of natural brain rhythms. To demonstrate model versatility, we simulate the power spectral densities of gamma rhythms in various brain states recorded in experiments.

Multilevel monte carlo for cortical circuit models

Multilevel Monte Carlo (MLMC) methods aim to speed up computation of statistics from dynamical simulations. MLMC is easy to implement and is sometimes very effective, but its efficacy may depend on the underlying dynamics. We apply MLMC to networks of spiking neurons and assess its effectiveness on prototypical models of cortical circuitry under different conditions. We find that MLMC can be very efficient for computing reliable features, i.e., features of network dynamics that are reproducible upon repeated presentation of the same external forcing. In contrast, MLMC is less effective for complex, internally generated activity. Qualitative explanations are given using concepts from random dynamical systems theory.

A data-informed mean-field approach to mapping of cortical parameter landscapes

Constraining the many biological parameters that govern cortical dynamics is computationally and conceptually difficult because of the curse of dimensionality. This paper addresses these challenges by proposing (1) a novel data-informed mean-field (MF) approach to efficiently map the parameter space of network models; and (2) an organizing principle for studying parameter space that enables the extraction biologically meaningful relations from this high-dimensional data. We illustrate these ideas using a large-scale network model of the Macaque primary visual cortex. Of the 10-20 model parameters, we identify 7 that are especially poorly constrained, and use the MF algorithm in (1) to discover the firing rate contours in this 7D parameter cube. Defining a "biologically plausible" region to consist of parameters that exhibit spontaneous Excitatory and Inhibitory firing rates compatible with experimental values, we find that this region is a slightly thickened codimension-1 submanifold. An implication of this finding is that while plausible regimes depend sensitively on parameters, they are also robust and flexible provided one compensates appropriately when parameters are varied. Our organizing principle for conceptualizing parameter dependence is to focus on certain 2D parameter planes that govern lateral inhibition: Intersecting these planes with the biologically plausible region leads to very simple geometric structures which, when suitably scaled, have a universal character independent of where the intersections are taken. In addition to elucidating the geometry of the plausible region, this invariance suggests useful approximate scaling relations. Our study offers, for the first time, a complete characterization of the set of all biologically plausible parameters for a detailed cortical model, which has been out of reach due to the high dimensionality of parameter space.

Model Reduction Captures Stochastic Gamma Oscillations on Low-Dimensional Manifolds

Gamma frequency oscillations (25–140 Hz), observed in the neural activities within many brain regions, have long been regarded as a physiological basis underlying many brain functions, such as memory and attention. Among numerous theoretical and computational modeling studies, gamma oscillations have been found in biologically realistic spiking network models of the primary visual cortex. However, due to its high dimensionality and strong non-linearity, it is generally difficult to perform detailed theoretical analysis of the emergent gamma dynamics. Here we propose a suite of Markovian model reduction methods with varying levels of complexity and apply it to spiking network models exhibiting heterogeneous dynamical regimes, ranging from nearly homogeneous firing to strong synchrony in the gamma band. The reduced models not only successfully reproduce gamma oscillations in the full model, but also exhibit the same dynamical features as we vary parameters. Most remarkably, the invariant measure of the coarse-grained Markov process reveals a two-dimensional surface in state space upon which the gamma dynamics mainly resides. Our results suggest that the statistical features of gamma oscillations strongly depend on the subthreshold neuronal distributions. Because of the generality of the Markovian assumptions, our dimensional reduction methods offer a powerful toolbox for theoretical examinations of other complex cortical spatio-temporal behaviors observed in both neurophysiological experiments and numerical simulations.

A strategy for mapping biophysical to abstract neuronal network models applied to primary visual cortex

A fundamental challenge for the theoretical study of neuronal networks is to make the link between complex biophysical models based directly on experimental data, to progressively simpler mathematical models that allow the derivation of general operating principles. We present a strategy that successively maps a relatively detailed biophysical population model, comprising conductance-based Hodgkin-Huxley type neuron models with connectivity rules derived from anatomical data, to various representations with fewer parameters, finishing with a firing rate network model that permits analysis. We apply this methodology to primary visual cortex of higher mammals, focusing on the functional property of stimulus orientation selectivity of receptive fields of individual neurons. The mapping produces compact expressions for the parameters of the abstract model that clearly identify the impact of specific electrophysiological and anatomical parameters on the analytical results, in particular as manifested by specific functional signatures of visual cortex, including input-output sharpening, conductance invariance, virtual rotation and the tilt after effect. Importantly, qualitative differences between model behaviours point out consequences of various simplifications. The strategy may be applied to other neuronal systems with appropriate modifications.

A hierarchy of theoretical approaches to study a neuronal network depends on a tradeoff between biological fidelity and mathematical tractibility. Biophysically-detailed models consider cellular mechanisms and anatomically defined synaptic circuits, but are often too complex to reveal insights into fundamental principles. In contrast, increasingly abstract reduced models facilitate analytical insights. To better ground the latter to the underlying biology, we describe a systematic procedure to move across the model hierarchy that allows understanding how changes in biological parameters—physiological, pathophysiological, or because of new data—impact the behaviour of the network. We apply this approach to mammalian primary visual cortex, and examine how the different models in the hierarchy reproduce functional signatures of this area, in particular the tuning of neurons to the orientation of a visual stimulus. Our work provides a navigation of the complex parameter space of neural network models faithful to biology, as well as highlighting how simplifications made for mathematical convenience can fundamentally change their behaviour.

Malleability of gamma rhythms enhances population-level correlations

An important problem in systems neuroscience is to understand how information is communicated among brain regions, and it has been proposed that communication is mediated by neuronal oscillations, such as rhythms in the gamma band. We sought to investigate this idea by using a network model with two components, a source (sending) and a target (receiving) component, both built to resemble local populations in the cerebral cortex. To measure the effectiveness of communication, we used population-level correlations in spike times between the source and target. We found that after correcting for a response time that is independent of initial conditions, spike-time correlations between the source and target are significant, due in large measure to the alignment of firing events in their gamma rhythms. But, we also found that regular oscillations cannot produce the results observed in our model simulations of cortical neurons. Surprisingly, it is the irregularity of gamma rhythms, the absence of internal clocks, together with the malleability of these rhythms and their tendency to align with external pulses - features that are known to be present in gamma rhythms in the real cortex - that produced the results observed. These findings and the mechanistic explanations we offered are our primary results. Our secondary result is a mathematical relationship between correlations and the sizes of the samples used for their calculation. As improving technology enables recording simultaneously from increasing numbers of neurons, this relationship could be useful for interpreting results from experimental recordings.

Effective behavior of cooperative and nonidentical molecular motors

Analytical formulas for effective drift, diffusivity, run times, and run lengths are derived for an intracellular transport system consisting of a cargo attached to two cooperative but not identical molecular motors (for example, kinesin-1 and kinesin-2) which can each attach and detach from a microtubule. The dynamics of the motor and cargo in each phase are governed by stochastic differential equations, and the switching rates depend on the spatial configuration of the motor and cargo. This system is analyzed in a limit where the detached motors have faster dynamics than the cargo, which in turn has faster dynamics than the attached motors. The attachment and detachment rates are also taken to be slow relative to the spatial dynamics. Through an application of iterated stochastic averaging to this system, and the use of renewal-reward theory to stitch together the progress within each switching phase, we obtain explicit analytical expressions for the effective drift, diffusivity, and processivity of the motor-cargo system. Our approach accounts in particular for jumps in motor-cargo position that occur during attachment and detachment events, as the cargo tracking variable makes a rapid adjustment due to the averaged fast scales. The asymptotic formulas are in generally good agreement with direct stochastic simulations of the detailed model based on experimental parameters for various pairings of kinesin-1 and kinesin-2 under assisting, hindering, or no load.

DNN-assisted statistical analysis of a model of local cortical circuits

In neuroscience, computational modeling is an effective way to gain insight into cortical mechanisms, yet the construction and analysis of large-scale network models—not to mention the extraction of underlying principles—are themselves challenging tasks, due to the absence of suitable analytical tools and the prohibitive costs of systematic numerical exploration of high-dimensional parameter spaces. In this paper, we propose a data-driven approach assisted by deep neural networks (DNN). The idea is to first discover certain input-output relations, and then to leverage this information and the superior computation speeds of the well-trained DNN to guide parameter searches and to deduce theoretical understanding. To illustrate this novel approach, we used as a test case a medium-size network of integrate-and-fire neurons intended to model local cortical circuits. With the help of an accurate yet extremely efficient DNN surrogate, we revealed the statistics of model responses, providing a detailed picture of model behavior. The information obtained is both general and of a fundamental nature, with direct application to neuroscience. Our results suggest that the methodology proposed can be scaled up to larger and more complex biological networks when used in conjunction with other techniques of biological modeling.

Entropy, mutual information, and systematic measures of structured spiking neural networks

The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that are based on mutual information, for a class of structured spiking neuronal networks. In order to analyze and compute these information-theoretic measures for large networks, we coarse-grained the data by ignoring the order of spikes that fall into the same small time bin. The resultant coarse-grained entropy mainly captures the information contained in the rhythm produced by a local population of the network. We first show that these information theoretical measures are well-defined and computable by proving stochastic stability and the law of large numbers. Then we use three neuronal network examples, from simple to complex, to investigate these information-theoretic measures. Several analytical and computational results about properties of these information-theoretic measures are given.

Extreme value theory of evolving phenomena in complex dynamical systems: Firing cascades in a model of a neural network

We extend the scope of the dynamical theory of extreme values to include phenomena that do not happen instantaneously but evolve over a finite, albeit unknown at the onset, time interval. We consider complex dynamical systems composed of many individual subsystems linked by a network of interactions. As a specific example of the general theory, a model of a neural network, previously introduced by other authors to describe the electrical activity of the cerebral cortex, is analyzed in detail. On the basis of this analysis, we propose a novel definition of a neuronal cascade, a physiological phenomenon of primary importance. We derive extreme value laws for the statistics of these cascades, both from the point of view of exceedances (that satisfy critical scaling theory in a certain regime) and of block maxima.

Stochastic neural field model: multiple firing events and correlations

This paper studies a nonlinear dynamical phenomenon called the multiple firing event (MFE) in a spatially heterogeneous stochastic neural field model, which is extended from that in our previous paper (Li et al. in J Math Biol 78:83-115, 2018). MFEs are a partially synchronized spiking barrages that are believed to be responsible for the Gamma oscillation. Rigorous results about the stochastic stability and the law of large numbers are proved, which further imply the well-definedness and computability of many quantities related to MFEs. Then we devote to study spatial and temporal properties of MFEs. Our key finding is that MFEs are spatially correlated but the spatial correlation decays quickly. Detailed mathematical justifications are made based on our qualitative models that aim to demonstrate the mechanism of MFEs.

Rhythm and Synchrony in a Cortical Network Model

We studied mechanisms for cortical gamma-band activity in the cerebral cortex and identified neurobiological factors that affect such activity. This was done by analyzing the behavior of a previously developed, data-driven, large-scale network model that simulated many visual functions of monkey V1 cortex (Chariker et al., 2016). Gamma activity was an emergent property of the model. The model's gamma activity, like that of the real cortex, was (1) episodic, (2) variable in frequency and phase, and (3) graded in power with stimulus variables like orientation. The spike firing of the model's neuronal population was only partially synchronous during multiple firing events (MFEs) that occurred at gamma rates. Detailed analysis of the model's MFEs showed that gamma-band activity was multidimensional in its sources. Most spikes were evoked by excitatory inputs. A large fraction of these inputs came from recurrent excitation within the local circuit, but feedforward and feedback excitation also contributed, either through direct pulsing or by raising the overall baseline. Inhibition was responsible for ending MFEs, but disinhibition led directly to only a small minority of the synchronized spikes. As a potential explanation for the wide range of gamma characteristics observed in different parts of cortex, we found that the relative rise times of AMPA and GABA synaptic conductances have a strong effect on the degree of synchrony in gamma.SIGNIFICANCE STATEMENT Canonical computations used throughout the cerebral cortex are performed in primary visual cortex (V1). Providing theoretical mechanisms for these computations will advance understanding of computation throughout cortex. We studied one dynamical feature, gamma-band rhythms, in a large-scale, data-driven, computational model of monkey V1. Our most significant conclusion is that the sources of gamma band activity are multidimensional. A second major finding is that the relative rise times of excitatory and inhibitory synaptic potentials have strong effects on spike synchrony and peak gamma band power. Insight gained from studying our V1 model can shed light on the functions of other cortical regions.

How well do reduced models capture the dynamics in models of interacting neurons?

This paper introduces a class of stochastic models of interacting neurons with emergent dynamics similar to those seen in local cortical populations. Rigorous results on existence and uniqueness of nonequilibrium steady states are proved. These network models are then compared to very simple reduced models driven by the same mean excitatory and inhibitory currents. Discrepancies in firing rates between network and reduced models are investigated and explained by correlations in spiking, or partial synchronization, working in concert with "nonlinearities" in the time evolution of membrane potentials. The use of simple random walks and their first passage times to simulate fluctuations in neuronal membrane potentials and interspike times is also considered.

Orientation Selectivity from Very Sparse LGN Inputs in a Comprehensive Model of Macaque V1 Cortex

A new computational model of the primary visual cortex (V1) of the macaque monkey was constructed to reconcile the visual functions of V1 with anatomical data on its LGN input, the extreme sparseness of which presented serious challenges to theoretically sound explanations of cortical function. We demonstrate that, even with such sparse input, it is possible to produce robust orientation selectivity, as well as continuity in the orientation map. We went beyond that to find plausible dynamic regimes of our new model that emulate simultaneously experimental data for a wide range of V1 phenomena, beginning with orientation selectivity but also including diversity in neuronal responses, bimodal distributions of the modulation ratio (the simple/complex classification), and dynamic signatures, such as gamma-band oscillations. Intracortical interactions play a major role in all aspects of the visual functions of the model.

SIGNIFICANCE STATEMENT

We present the first realistic model that has captured the sparseness of magnocellular LGN inputs to the macaque primary visual cortex and successfully derived orientation selectivity from them. Three implications are (1) even in input layers to the visual cortex, the system is less feedforward and more dominated by intracortical signals than previously thought, (2) interactions among cortical neurons in local populations produce dynamics not explained by single neurons, and (3) such dynamics are important for function. Our model also shows that a comprehensive picture is necessary to explain function, because different visual properties are related. This study points to the need for paradigm shifts in neuroscience modeling: greater emphasis on population dynamics and, where possible, a move toward data-driven, comprehensive models.