Criticality enhances the multilevel reliability of stimulus responses in cortical neural networks

Learning Fixed Points of Recurrent Neural Networks by Reparameterizing the Network Model

In computational neuroscience, recurrent neural networks are widely used to model neural activity and learning. In many studies, fixed points of recurrent neural networks are used to model neural responses to static or slowly changing stimuli, such as visual cortical responses to static visual stimuli. These applications raise the question of how to train the weights in a recurrent neural network to minimize a loss function evaluated on fixed points. In parallel, training fixed points is a central topic in the study of deep equilibrium models in machine learning. A natural approach is to use gradient descent on the Euclidean space of weights. We show that this approach can lead to poor learning performance due in part to singularities that arise in the loss surface. We use a reparameterization of the recurrent network model to derive two alternative learning rules that produce more robust learning dynamics. We demonstrate that these learning rules avoid singularities and learn more effectively than standard gradient descent. The new learning rules can be interpreted as steepest descent and gradient descent, respectively, under a non-Euclidean metric on the space of recurrent weights. Our results question the common, implicit assumption that learning in the brain should be expected to follow the negative Euclidean gradient of synaptic weights.

The recovery of parabolic avalanches in spatially subsampled neuronal networks at criticality

Scaling relationships are key in characterizing complex systems at criticality. In the brain, they are evident in neuronal avalanches-scale-invariant cascades of neuronal activity quantified by power laws. Avalanches manifest at the cellular level as cascades of neuronal groups that fire action potentials simultaneously. Such spatiotemporal synchronization is vital to theories on brain function yet avalanche synchronization is often underestimated when only a fraction of neurons is observed. Here, we investigate biases from fractional sampling within a balanced network of excitatory and inhibitory neurons with all-to-all connectivity and critical branching process dynamics. We focus on how mean avalanche size scales with avalanche duration. For parabolic avalanches, this scaling is quadratic, quantified by the scaling exponent, χ = 2 , reflecting rapid spatial expansion of simultaneous neuronal firing over short durations. However, in networks sampled fractionally, χ is significantly lower. We demonstrate that applying temporal coarse-graining and increasing a minimum threshold for coincident firing restores χ = 2 , even when as few as 0.1% of neurons are sampled. This correction crucially depends on the network being critical and fails for near sub- and supercritical branching dynamics. Using cellular 2-photon imaging, our approach robustly identifies χ = 2 over a wide parameter regime in ongoing neuronal activity from frontal cortex of awake mice. In contrast, the common 'crackling noise' approach fails to determine χ under similar sampling conditions at criticality. Our findings overcome scaling bias from fractional sampling and demonstrate rapid, spatiotemporal synchronization of neuronal assemblies consistent with scale-invariant, parabolic avalanches at criticality.

Excitation-Inhibition Balance, Neural Criticality, and Activities in Neuronal Circuits

Neural activities in local circuits exhibit complex and multilevel dynamic features. Individual neurons spike irregularly, which is believed to originate from receiving balanced amounts of excitatory and inhibitory inputs, known as the excitation-inhibition balance. The spatial-temporal cascades of clustered neuronal spikes occur in variable sizes and durations, manifested as neural avalanches with scale-free features. These may be explained by the neural criticality hypothesis, which posits that neural systems operate around the transition between distinct dynamic states. Here, we summarize the experimental evidence for and the underlying theory of excitation-inhibition balance and neural criticality. Furthermore, we review recent studies of excitatory-inhibitory networks with synaptic kinetics as a simple solution to reconcile these two apparently distinct theories in a single circuit model. This provides a more unified understanding of multilevel neural activities in local circuits, from spontaneous to stimulus-response dynamics.

Dysfunctions of multiscale dynamic brain functional networks in subjective cognitive decline

Subjective cognitive decline is potentially the earliest symptom of Alzheimer's disease, whose objective neurological basis remains elusive. To explore the potential biomarkers for subjective cognitive decline, we developed a novel deep learning method based on multiscale dynamical brain functional networks to identify subjective cognitive declines. We retrospectively constructed an internal data set (with 112 subjective cognitive decline and 64 healthy control subjects) to develop and internally validate the deep learning model. Conventional deep learning methods based on static and dynamic brain functional networks are compared. After the model is established, we prospectively collect an external data set (26 subjective cognitive decline and 12 healthy control subjects) for testing. Meanwhile, our method provides monitoring of the transitions between normal and abnormal (subjective cognitive decline-related) dynamical functional network states. The features of abnormal dynamical functional network states are quantified by network and variability metrics and associated with individual cognitions. Our method achieves an area under the receiver operating characteristic curve of 0.807 ± 0.046 in the internal validation data set and of 0.707 (P = 0.007) in the external testing data set, which shows improvements compared to conventional methods. The method further suggests that, at the local level, the abnormal dynamical functional network states are characterized by decreased connectivity strength and increased connectivity variability at different spatial scales. At the network level, the abnormal states are featured by scale-specifically altered modularity and all-scale decreased efficiency. Low tendencies to stay in abnormal states and high state transition variabilities are significantly associated with high general, language and executive functions. Overall, our work supports the deficits in multiscale brain dynamical functional networks detected by the deep learning method as reliable and meaningful neural alternation underpinning subjective cognitive decline.

Multilevel and multifaceted brain response features in spiking, ERP and ERD: experimental observation and simultaneous generation in a neuronal network model with excitation-inhibition balance

Brain as a dynamic system responds to stimulations with specific patterns affected by its inherent ongoing dynamics. The patterns are manifested across different levels of organization-from spiking activity of neurons to collective oscillations in local field potential (LFP) and electroencephalogram (EEG). The multilevel and multifaceted response activities show patterns seemingly distinct and non-comparable from each other, but they should be coherently related because they are generated from the same underlying neural dynamic system. A coherent understanding of the interrelationships between different levels/aspects of activity features is important for understanding the complex brain functions. Here, based on analysis of data from human EEG, monkey LFP and neuronal spiking, we demonstrated that the brain response activities from different levels of neural system are highly coherent: the external stimulus simultaneously generated event-related potentials, event-related desynchronization, and variation in neuronal spiking activities that precisely match with each other in the temporal unfolding. Based on a biologically plausible but generic network of conductance-based integrate-and-fire excitatory and inhibitory neurons with dense connections, we showed that the multiple key features can be simultaneously produced at critical dynamical regimes supported by excitation-inhibition (E-I) balance. The elucidation of the inherent coherency of various neural response activities and demonstration of a simple dynamical neural circuit system having the ability to simultaneously produce multiple features suggest the plausibility of understanding high-level brain function and cognition from elementary and generic neuronal dynamics.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09889-w.

Structure and function in artificial, zebrafish and human neural networks

Network science provides a set of tools for the characterization of the structure and functional behavior of complex systems. Yet a major problem is to quantify how the structural domain is related to the dynamical one. In other words, how the diversity of dynamical states of a system can be predicted from the static network structure? Or the reverse problem: starting from a set of signals derived from experimental recordings, how can one discover the network connections or the causal relations behind the observed dynamics? Despite the advances achieved over the last two decades, many challenges remain concerning the study of the structure-dynamics interplay of complex systems. In neuroscience, progress is typically constrained by the low spatio-temporal resolution of experiments and by the lack of a universal inferring framework for empirical systems. To address these issues, applications of network science and artificial intelligence to neural data have been rapidly growing. In this article, we review important recent applications of methods from those fields to the study of the interplay between structure and functional dynamics of human and zebrafish brain. We cover the selection of topological features for the characterization of brain networks, inference of functional connections, dynamical modeling, and close with applications to both the human and zebrafish brain. This review is intended to neuroscientists who want to become acquainted with techniques from network science, as well as to researchers from the latter field who are interested in exploring novel application scenarios in neuroscience.

Quasiuniversal scaling in mouse-brain neuronal activity stems from edge-of-instability critical dynamics

The brain is in a state of perpetual reverberant neural activity, even in the absence of specific tasks or stimuli. Shedding light on the origin and functional significance of such a dynamical state is essential to understanding how the brain transmits, processes, and stores information. An inspiring, albeit controversial, conjecture proposes that some statistical characteristics of empirically observed neuronal activity can be understood by assuming that brain networks operate in a dynamical regime with features, including the emergence of scale invariance, resembling those seen typically near phase transitions. Here, we present a data-driven analysis based on simultaneous high-throughput recordings of the activity of thousands of individual neurons in various regions of the mouse brain. To analyze these data, we construct a unified theoretical framework that synergistically combines a phenomenological renormalization group approach and techniques that infer the general dynamical state of a neural population, while designing complementary tools. This strategy allows us to uncover strong signatures of scale invariance that are "quasiuniversal" across brain regions and experiments, revealing that all the analyzed areas operate, to a greater or lesser extent, near the edge of instability.

Differential associations between scale-free neural dynamics and different levels of cognitive ability

As indicators of cognitive function, scale-free neural dynamics are gaining increasing attention in cognitive neuroscience. Although the functional relevance of scale-free dynamics has been extensively reported, one fundamental question about its association with cognitive ability remains unanswered: is the association universal across a wide spectrum of cognitive abilities or confined to specific domains? Based on dual-process theory, we designed two categories of tasks to analyze two types of cognitive processes-automatic and controlled-and examined their associations with scale-free neural dynamics characterized from resting-state electroencephalography (EEG) recordings obtained from a large sample of human adults (N = 102). Our results showed that resting-state scale-free neural dynamics did not predict individuals' behavioral performance in tasks that primarily engaged the automatic process but did so in tasks that primarily engaged the controlled process. In addition, by fitting the scale-free parameters separately in different frequency bands, we found that the cognitive association of scale-free dynamics was more strongly manifested in higher-band EEG spectrum. Our findings indicate that resting-state scale-free dynamics are not universal neural indicators for all cognitive abilities but are mainly associated with high-level cognition that entails controlled processes. This finding is compatible with the widely claimed role of scale-free dynamics in reflecting properties of complex dynamic systems.