Structural Plasticity Denoises Responses and Improves Learning Speed

STDP-Driven Rewiring in Spiking Neural Networks under Stimulus-Induced and Spontaneous Activity

Mathematical and computer simulation of learning in living neural networks have typically focused on changes in the efficiency of synaptic connections represented by synaptic weights in the models. Synaptic plasticity is believed to be the cellular basis for learning and memory. In spiking neural networks composed of dynamical spiking units, a biologically relevant learning rule is based on the so-called spike-timing-dependent plasticity or STDP. However, experimental data suggest that synaptic plasticity is only a part of brain circuit plasticity, which also includes homeostatic and structural plasticity. A model of structural plasticity proposed in this study is based on the activity-dependent appearance and disappearance of synaptic connections. The results of the research indicate that such adaptive rewiring enables the consolidation of the effects of STDP in response to a local external stimulation of a neural network. Subsequently, a vector field approach is used to demonstrate the successive "recording" of spike paths in both functional connectome and synaptic connectome, and finally in the anatomical connectome of the network. Moreover, the findings suggest that the adaptive rewiring could stabilize network dynamics over time in the context of activity patterns' reproducibility. A universal measure of such reproducibility introduced in this article is based on similarity between time-consequent patterns of the special vector fields characterizing both functional and anatomical connectomes.

Estimated Gray Matter Volume Rapidly Changes after a Short Motor Task

Skill learning induces changes in estimates of gray matter volume (GMV) in the human brain, commonly detectable with magnetic resonance imaging (MRI). Rapid changes in GMV estimates while executing tasks may however confound between- and within-subject differences. Fluctuations in arterial blood flow are proposed to underlie this apparent task-related tissue plasticity. To test this hypothesis, we acquired multiple repetitions of structural T1-weighted and functional blood-oxygen level-dependent (BOLD) MRI measurements from 51 subjects performing a finger-tapping task (FTT; á 2 min) repeatedly for 30-60 min. Estimated GMV was decreased in motor regions during FTT compared with rest. Motor-related BOLD signal changes did not overlap nor correlate with GMV changes. Nearly simultaneous BOLD signals cannot fully explain task-induced changes in T1-weighted images. These sensitive and behavior-related GMV changes pose serious questions to reproducibility across studies, and morphological investigations during skill learning can also open new avenues on how to study rapid brain plasticity.

Computational benefits of structural plasticity, illustrated in songbirds

The plasticity of nervous systems allows animals to quickly adapt to a changing environment. In particular, the structural plasticity of brain networks is often critical to the development of the central nervous system and the acquisition of complex behaviors. As an example, structural plasticity is central to the development of song-related brain circuits and may be critical for song acquisition in juvenile songbirds. Here, we review current evidences for structural plasticity and their significance from a computational point of view. We start by reviewing evidence for structural plasticity across species and categorizing them along the spatial axes as well as the along the time course during development. We introduce the vocal learning circuitry in zebra finches, as a useful example of structural plasticity, and use this specific case to explore the possible contributions of structural plasticity to computational models. Finally, we discuss current modeling studies incorporating structural plasticity and unexplored questions which are raised by such models.

Higher VO2max is associated with thicker cortex and lower grey matter blood flow in older adults

VO₂max (maximal oxygen consumption), a validated measure of aerobic fitness, has been associated with better cerebral artery compliance and measures of brain morphology, such as higher cortical thickness (CT) in frontal, temporal and cingular cortices, and larger grey matter volume (GMV) of the middle temporal gyrus, hippocampus, orbitofrontal cortex and cingulate cortex. Single sessions of physical exercise can promptly enhance cognitive performance and brain activity during executive tasks. However, the immediate effects of exercise on macro-scale properties of the brain’s grey matter remain unclear. We investigated the impact of one session of moderate-intensity physical exercise, compared with rest, on grey matter volume, cortical thickness, working memory performance, and task-related brain activity in older adults. Cross-sectional associations between brain measures and VO₂max were also tested. Exercise did not induce statistically significant changes in brain activity, grey matter volume, or cortical thickness. Cardiovascular fitness, measured by VO₂max, was associated with lower grey matter blood flow in the left hippocampus and thicker cortex in the left superior temporal gyrus. Cortical thickness was reduced at post-test independent of exercise/rest. Our findings support that (1) fitter individuals may need lower grey matter blood flow to meet metabolic oxygen demand, and (2) have thicker cortex.

Simulation of Spiking Neural P Systems with Sparse Matrix-Vector Operations

To date, parallel simulation algorithms for spiking neural P (SNP) systems are based on a matrix representation. This way, the simulation is implemented with linear algebra operations, which can be easily parallelized on high performance computing platforms such as GPUs. Although it has been convenient for the first generation of GPU-based simulators, such as CuSNP, there are some bottlenecks to sort out. For example, the proposed matrix representations of SNP systems lead to very sparse matrices, where the majority of values are zero. It is known that sparse matrices can compromise the performance of algorithms since they involve a waste of memory and time. This problem has been extensively studied in the literature of parallel computing. In this paper, we analyze some of these ideas and apply them to represent some variants of SNP systems. We also provide a new simulation algorithm based on a novel compressed representation for sparse matrices. We also conclude which SNP system variant better suits our new compressed matrix representation.

Structural plasticity on an accelerated analog neuromorphic hardware system

In computational neuroscience, as well as in machine learning, neuromorphic devices promise an accelerated and scalable alternative to neural network simulations. Their neural connectivity and synaptic capacity depend on their specific design choices, but is always intrinsically limited. Here, we present a strategy to achieve structural plasticity that optimizes resource allocation under these constraints by constantly rewiring the pre- and postsynaptic partners while keeping the neuronal fan-in constant and the connectome sparse. In particular, we implemented this algorithm on the analog neuromorphic system BrainScaleS-2. It was executed on a custom embedded digital processor located on chip, accompanying the mixed-signal substrate of spiking neurons and synapse circuits. We evaluated our implementation in a simple supervised learning scenario, showing its ability to optimize the network topology with respect to the nature of its training data, as well as its overall computational efficiency.

Emergence of Stable Synaptic Clusters on Dendrites Through Synaptic Rewiring

The connectivity structure of neuronal networks in cortex is highly dynamic. This ongoing cortical rewiring is assumed to serve important functions for learning and memory. We analyze in this article a model for the self-organization of synaptic inputs onto dendritic branches of pyramidal cells. The model combines a generic stochastic rewiring principle with a simple synaptic plasticity rule that depends on local dendritic activity. In computer simulations, we find that this synaptic rewiring model leads to synaptic clustering, that is, temporally correlated inputs become locally clustered on dendritic branches. This empirical finding is backed up by a theoretical analysis which shows that rewiring in our model favors network configurations with synaptic clustering. We propose that synaptic clustering plays an important role in the organization of computation and memory in cortical circuits: we find that synaptic clustering through the proposed rewiring mechanism can serve as a mechanism to protect memories from subsequent modifications on a medium time scale. Rewiring of synaptic connections onto specific dendritic branches may thus counteract the general problem of catastrophic forgetting in neural networks.

A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing Applications

Inspired from the computational efficiency of the biological brain, spiking neural networks (SNNs) emulate biological neural networks, neural codes, dynamics, and circuitry. SNNs show great potential for the implementation of unsupervised learning using in-memory computing. Here, we report an algorithmic optimization that improves energy efficiency of online learning with SNNs on emerging non-volatile memory (eNVM) devices. We develop a pruning method for SNNs by exploiting the output firing characteristics of neurons. Our pruning method can be applied during network training, which is different from previous approaches in the literature that employ pruning on already-trained networks. This approach prevents unnecessary updates of network parameters during training. This algorithmic optimization can complement the energy efficiency of eNVM technology, which offers a unique in-memory computing platform for the parallelization of neural network operations. Our SNN maintains ~90% classification accuracy on the MNIST dataset with up to ~75% pruning, significantly reducing the number of weight updates. The SNN and pruning scheme developed in this work can pave the way toward applications of eNVM based neuro-inspired systems for energy efficient online learning in low power applications.

Structural Plasticity on the SpiNNaker Many-Core Neuromorphic System

The structural organization of cortical areas is not random, with topographic maps commonplace in sensory processing centers. This topographical organization allows optimal wiring between neurons, multimodal sensory integration, and performs input dimensionality reduction. In this work, a model of topographic map formation is implemented on the SpiNNaker neuromorphic platform, running in realtime using point neurons, and making use of both synaptic rewiring and spike-timing dependent plasticity (STDP). In agreement with Bamford et al. (2010), we demonstrate that synaptic rewiring refines an initially rough topographic map over and beyond the ability of STDP, and that input selectivity learnt through STDP is embedded into the network connectivity through rewiring. Moreover, we show the presented model can be used to generate topographic maps between layers of neurons with minimal initial connectivity, and stabilize mappings which would otherwise be unstable through the inclusion of lateral inhibition.

Diffusion markers of dendritic density and arborization in gray matter predict differences in intelligence

Previous research has demonstrated that individuals with higher intelligence are more likely to have larger gray matter volume in brain areas predominantly located in parieto-frontal regions. These findings were usually interpreted to mean that individuals with more cortical brain volume possess more neurons and thus exhibit more computational capacity during reasoning. In addition, neuroimaging studies have shown that intelligent individuals, despite their larger brains, tend to exhibit lower rates of brain activity during reasoning. However, the microstructural architecture underlying both observations remains unclear. By combining advanced multi-shell diffusion tensor imaging with a culture-fair matrix-reasoning test, we found that higher intelligence in healthy individuals is related to lower values of dendritic density and arborization. These results suggest that the neuronal circuitry associated with higher intelligence is organized in a sparse and efficient manner, fostering more directed information processing and less cortical activity during reasoning.

Network Design and the Brain

Neural circuits have evolved to accommodate similar information processing challenges as those faced by engineered systems. Here, we compare neural versus engineering strategies for constructing networks. During circuit development, synapses are overproduced and then pruned back over time, whereas in engineered networks, connections are initially sparse and are then added over time. We provide a computational perspective on these two different approaches, including discussion of how and why they are used, insights that one can provide the other, and areas for future joint investigation. By thinking algorithmically about the goals, constraints, and optimization principles used by neural circuits, we can develop brain-derived strategies for enhancing network design, while also stimulating experimental hypotheses about circuit development and function.