Firing rate distributions in spiking networks with heterogeneous connectivity

Moment neural network and an efficient numerical method for modeling irregular spiking activity

Continuous rate-based neural networks have been widely applied to modeling the dynamics of cortical circuits. However, cortical neurons in the brain exhibit irregular spiking activity with complex correlation structures that cannot be captured by mean firing rate alone. To close this gap, we consider a framework for modeling irregular spiking activity, called the moment neural network, which naturally generalizes rate models to second-order moments and can accurately capture the firing statistics of spiking neural networks. We propose an efficient numerical method that allows for rapid evaluation of moment mappings for neuronal activations without solving the underlying Fokker-Planck equation. This allows simulation of coupled interactions of mean firing rate and firing variability of large-scale neural circuits while retaining the advantage of analytical tractability of continuous rate models. We demonstrate how the moment neural network can explain a range of phenomena including diverse Fano factor in networks with quenched disorder and the emergence of irregular oscillatory dynamics in excitation-inhibition networks with delay.

Top-down modulation in canonical cortical circuits with short-term plasticity

Cortical dynamics and computations are strongly influenced by diverse GABAergic interneurons, including those expressing parvalbumin (PV), somatostatin (SST), and vasoactive intestinal peptide (VIP). Together with excitatory (E) neurons, they form a canonical microcircuit and exhibit counterintuitive nonlinear phenomena. One instance of such phenomena is response reversal, whereby SST neurons show opposite responses to top-down modulation via VIP depending on the presence of bottom-up sensory input, indicating that the network may function in different regimes under different stimulation conditions. Combining analytical and computational approaches, we demonstrate that model networks with multiple interneuron subtypes and experimentally identified short-term plasticity mechanisms can implement response reversal. Surprisingly, despite not directly affecting SST and VIP activity, PV-to-E short-term depression has a decisive impact on SST response reversal. We show how response reversal relates to inhibition stabilization and the paradoxical effect in the presence of several short-term plasticity mechanisms demonstrating that response reversal coincides with a change in the indispensability of SST for network stabilization. In summary, our work suggests a role of short-term plasticity mechanisms in generating nonlinear phenomena in networks with multiple interneuron subtypes and makes several experimentally testable predictions.

Logarithmically scaled, gamma distributed neuronal spiking

Naturally log-scaled quantities abound in the nervous system. Distributions of these quantities have non-intuitive properties, which have implications for data analysis and the understanding of neural circuits. Here, we review the log-scaled statistics of neuronal spiking and the relevant analytical probability distributions. Recent work using log-scaling revealed that interspike intervals of forebrain neurons segregate into discrete modes reflecting spiking at different timescales and are each well-approximated by a gamma distribution. Each neuron spends most of the time in an irregular spiking 'ground state' with the longest intervals, which determines the mean firing rate of the neuron. Across the entire neuronal population, firing rates are log-scaled and well approximated by the gamma distribution, with a small number of highly active neurons and an overabundance of low rate neurons (the 'dark matter'). These results are intricately linked to a heterogeneous balanced operating regime, which confers upon neuronal circuits multiple computational advantages and has evolutionarily ancient origins.

A circuit mechanism for independent modulation of excitatory and inhibitory firing rates after sensory deprivation

The cortex is particularly vulnerable to perturbations during sensitive periods, such as the critical period when manipulating sensory experience can induce long-lasting changes in brain structure. Depriving rodents of vision in one eye (known as monocular deprivation [MD]) reduces network activity over two days, whereby inhibitory neurons decrease their firing rates one day after MD, while excitatory neurons are delayed by an additional day. We use spiking networks to mechanistically dissect the requirements for this independent firing-rate regulation after sensory deprivation. We find that in networks stabilized by recurrent inhibition, at least two interneuron subtypes (parvalbumin-expressing and somatostatin-expressing interneurons) are necessary to dynamically alter the circuit response after deprivation and generalize the result across sensory cortices.

Diverse interneuron subtypes shape sensory processing in mature cortical circuits. During development, sensory deprivation evokes powerful synaptic plasticity that alters circuitry, but how different inhibitory subtypes modulate circuit dynamics in response to this plasticity remains unclear. We investigate how deprivation-induced synaptic changes affect excitatory and inhibitory firing rates in a microcircuit model of the sensory cortex with multiple interneuron subtypes. We find that with a single interneuron subtype (parvalbumin-expressing [PV]), excitatory and inhibitory firing rates can only be comodulated—increased or decreased together. To explain the experimentally observed independent modulation, whereby one firing rate increases and the other decreases, requires strong feedback from a second interneuron subtype (somatostatin-expressing [SST]). Our model applies to the visual and somatosensory cortex, suggesting a general mechanism across sensory cortices. Therefore, we provide a mechanistic explanation for the differential role of interneuron subtypes in regulating firing rates, contributing to the already diverse roles they serve in the cortex.

Why Firing Rate Distributions Are Important for Understanding Spinal Central Pattern Generators

Networks in the spinal cord, which are responsible for the generation of rhythmic movements, commonly known as central pattern generators (CPGs), have remained elusive for decades. Although it is well-known that many spinal neurons are rhythmically active, little attention has been given to the distribution of firing rates across the population. Here, we argue that firing rate distributions can provide an important clue to the organization of the CPGs. The data that can be gleaned from the sparse literature indicate a firing rate distribution, which is skewed toward zero with a long tail, akin to a normal distribution on a log-scale, i.e., a “log-normal” distribution. Importantly, such a shape is difficult to unite with the widespread assumption of modules composed of recurrently connected excitatory neurons. Spinal modules with recurrent excitation has the propensity to quickly escalate their firing rate and reach the maximum, hence equalizing the spiking activity across the population. The population distribution of firing rates hence would consist of a narrow peak near the maximum. This is incompatible with experiments, that show wide distributions and a peak close to zero. A way to resolve this puzzle is to include recurrent inhibition internally in each CPG modules. Hence, we investigate the impact of recurrent inhibition in a model and find that the firing rate distributions are closer to the experimentally observed. We therefore propose that recurrent inhibition is a crucial element in motor circuits, and suggest that future models of motor circuits should include recurrent inhibition as a mandatory element.

Effects of degree distributions in random networks of type-I neurons

We consider large networks of theta neurons and use the Ott-Antonsen ansatz to derive degree-based mean-field equations governing the expected dynamics of the networks. Assuming random connectivity, we investigate the effects of varying the widths of the in- and out-degree distributions on the dynamics of excitatory or inhibitory synaptically coupled networks and gap junction coupled networks. For synaptically coupled networks, the dynamics are independent of the out-degree distribution. Broadening the in-degree distribution destroys oscillations in inhibitory networks and decreases the range of bistability in excitatory networks. For gap junction coupled neurons, broadening the degree distribution varies the values of parameters at which there is an onset of collective oscillations. Many of the results are shown to also occur in networks of more realistic neurons.

Balanced networks under spike-time dependent plasticity

The dynamics of local cortical networks are irregular, but correlated. Dynamic excitatory–inhibitory balance is a plausible mechanism that generates such irregular activity, but it remains unclear how balance is achieved and maintained in plastic neural networks. In particular, it is not fully understood how plasticity induced changes in the network affect balance, and in turn, how correlated, balanced activity impacts learning. How do the dynamics of balanced networks change under different plasticity rules? How does correlated spiking activity in recurrent networks change the evolution of weights, their eventual magnitude, and structure across the network? To address these questions, we develop a theory of spike–timing dependent plasticity in balanced networks. We show that balance can be attained and maintained under plasticity–induced weight changes. We find that correlations in the input mildly affect the evolution of synaptic weights. Under certain plasticity rules, we find an emergence of correlations between firing rates and synaptic weights. Under these rules, synaptic weights converge to a stable manifold in weight space with their final configuration dependent on the initial state of the network. Lastly, we show that our framework can also describe the dynamics of plastic balanced networks when subsets of neurons receive targeted optogenetic input.

Animals are able to learn complex tasks through changes in individual synapses between cells. Such changes lead to the coevolution of neural activity patterns and the structure of neural connectivity, but the consequences of these interactions are not fully understood. We consider plasticity in model neural networks which achieve an average balance between the excitatory and inhibitory synaptic inputs to different cells, and display cortical–like, irregular activity. We extend the theory of balanced networks to account for synaptic plasticity and show which rules can maintain balance, and which will drive the network into a different state. This theory of plasticity can provide insights into the relationship between stimuli, network dynamics, and synaptic circuitry.

Dynamics of Structured Networks of Winfree Oscillators

Winfree oscillators are phase oscillator models of neurons, characterized by their phase response curve and pulsatile interaction function. We use the Ott/Antonsen ansatz to study large heterogeneous networks of Winfree oscillators, deriving low-dimensional differential equations which describe the evolution of the expected state of networks of oscillators. We consider the effects of correlations between an oscillator's in-degree and out-degree, and between the in- and out-degrees of an “upstream” and a “downstream” oscillator (degree assortativity). We also consider correlated heterogeneity, where some property of an oscillator is correlated with a structural property such as degree. We finally consider networks with parameter assortativity, coupling oscillators according to their intrinsic frequencies. The results show how different types of network structure influence its overall dynamics.

The effects of within-neuron degree correlations in networks of spiking neurons

We consider the effects of correlations between the in- and out-degrees of individual neurons on the dynamics of a network of neurons. By using theta neurons, we can derive a set of coupled differential equations for the expected dynamics of neurons with the same in-degree. A Gaussian copula is used to introduce correlations between a neuron's in- and out-degree, and numerical bifurcation analysis is used determine the effects of these correlations on the network's dynamics. For excitatory coupling, we find that inducing positive correlations has a similar effect to increasing the coupling strength between neurons, while for inhibitory coupling it has the opposite effect. We also determine the propensity of various two- and three-neuron motifs to occur as correlations are varied and give a plausible explanation for the observed changes in dynamics.