Associative memory model with long-tail-distributed Hebbian synaptic connections

Dynamic Pattern Recognition Model Based on Neural Network Response to Signal Fluctuation

We have proposed a model of dynamic retrieval in associative memory based on temporal input/output correlations under a stimulus-response open scheme of neural networks. This mechanism is different from that of the conventional stationary Hopfield model in which the input signal is used only as information for the initial state of the network. Building upon the fundamental properties of the proposed model, in this paper, we newly evaluate the dependence of identification performance on the signal fluctuation level and on the number of stored patterns by introducing an accuracy rate for known (stored) and unknown (non-stored) patterns, based on the network correlation to the input signal with fluctuation. The results indicate that the dynamic scheme of network response to a fluctuating signal leads to increased efficacy and usefulness.

A microcircuit model involving parvalbumin, somatostatin, and vasoactive intestinal polypeptide inhibitory interneurons for the modulation of neuronal oscillation during visual processing

Various subtypes of inhibitory interneurons contact one another to organize cortical networks. Most cortical inhibitory interneurons express 1 of 3 genes: parvalbumin (PV), somatostatin (SOM), or vasoactive intestinal polypeptide (VIP). This diversity of inhibition allows the flexible regulation of neuronal responses within and between cortical areas. However, the exact roles of these interneuron subtypes and of excitatory pyramidal (Pyr) neurons in regulating neuronal network activity and establishing perception (via interactions between feedforward sensory and feedback attentional signals) remain largely unknown. To explore the regulatory roles of distinct neuronal types in cortical computation, we developed a computational microcircuit model with biologically plausible visual cortex layers 2/3 that combined Pyr neurons and the 3 inhibitory interneuron subtypes to generate network activity. In simulations with our model, inhibitory signals from PV and SOM neurons preferentially induced neuronal firing at gamma (30-80 Hz) and beta (20-30 Hz) frequencies, respectively, in agreement with observed physiological results. Furthermore, our model indicated that rapid inhibition from VIP to SOM subtypes underlies marked attentional modulation for low-gamma frequency (30-50 Hz) in Pyr neuron responses. Our results suggest the distinct but cooperative roles of inhibitory interneuron subtypes in the establishment of visual perception.

Effect of steady-state response versus excitatory/inhibitory balance on spiking synchronization in neural networks with log-normal synaptic weight distribution

Synchronization of neural activity, especially at the gamma band, contributes to perceptual functions. In several psychiatric disorders, deficits of perceptual functions are reflected in synchronization abnormalities. Plausible cause of this impairment is an alteration in the balance between excitation and inhibition (E/I balance); a disruption in the E/I balance leads to abnormal neural interactions reminiscent of pathological states. Moreover, the local lateral excitatory-excitatory synaptic connections in the cortex exhibit excitatory postsynaptic potentials (EPSPs) that follow a log-normal amplitude distribution. This long-tailed distribution is considered an important factor for the emergence of spatiotemporal neural activity. In this context, we hypothesized that manipulating the EPSP distribution under abnormal E/I balance conditions would provide insights into psychiatric disorders characterized by deficits in perceptual functions, potentially revealing the mechanisms underlying pathological neural behaviors. In this study, we evaluated the synchronization of neural activity with external periodic stimuli in spiking neural networks in cases of both E/I balance and imbalance with or without a long-tailed EPSP amplitude distribution. The results showed that external stimuli of a high frequency lead to a decrease in the degree of synchronization with an increasing ratio of excitatory to inhibitory neurons in the presence, but not in the absence, of high-amplitude EPSPs. This monotonic reduction can be interpreted as an autonomous, strong-EPSP-dependent spiking activity selectively interfering with the responses to external stimuli. This observation is consistent with pathological findings. Thus, our modeling approach has potential to improve the understanding of the steady-state response in both healthy and pathological states.

Long-Tailed Characteristic of Spiking Pattern Alternation Induced by Log-Normal Excitatory Synaptic Distribution

Studies of structural connectivity at the synaptic level show that in synaptic connections of the cerebral cortex, the excitatory postsynaptic potential (EPSP) in most synapses exhibits sub-mV values, while a small number of synapses exhibit large EPSPs ( >~1.0 [mV]). This means that the distribution of EPSP fits a log-normal distribution. While not restricting structural connectivity, skewed and long-tailed distributions have been widely observed in neural activities, such as the occurrences of spiking rates and the size of a synchronously spiking population. Many studies have been modeled this long-tailed EPSP neural activity distribution; however, its causal factors remain controversial. This study focused on the long-tailed EPSP distributions and interlateral synaptic connections primarily observed in the cortical network structures, thereby having constructed a spiking neural network consistent with these features. Especially, we constructed two coupled modules of spiking neural networks with excitatory and inhibitory neural populations with a log-normal EPSP distribution. We evaluated the spiking activities for different input frequencies and with/without strong synaptic connections. These coupled modules exhibited intermittent intermodule-alternative behavior, given moderate input frequency and the existence of strong synaptic and intermodule connections. Moreover, the power analysis, multiscale entropy analysis, and surrogate data analysis revealed that the long-tailed EPSP distribution and intermodule connections enhanced the complexity of spiking activity at large temporal scales and induced nonlinear dynamics and neural activity that followed the long-tailed distribution.

Computational roles of intrinsic synaptic dynamics

Conventional theories assume that long-term information storage in the brain is implemented by modifying synaptic efficacy. Recent experimental findings challenge this view by demonstrating that dendritic spine sizes, or their corresponding synaptic weights, are highly volatile even in the absence of neural activity. Here, we review previous computational works on the roles of these intrinsic synaptic dynamics. We first present the possibility for neuronal networks to sustain stable performance in their presence, and we then hypothesize that intrinsic dynamics could be more than mere noise to withstand, but they may improve information processing in the brain.

Temporal-specific complexity of spiking patterns in spontaneous activity induced by a dual complex network structure

Temporal fluctuation of neural activity in the brain has an important function in optimal information processing. Spontaneous activity is a source of such fluctuation. The distribution of excitatory postsynaptic potentials (EPSPs) between cortical pyramidal neurons can follow a log-normal distribution. Recent studies have shown that networks connected by weak synapses exhibit characteristics of a random network, whereas networks connected by strong synapses have small-world characteristics of small path lengths and large cluster coefficients. To investigate the relationship between temporal complexity spontaneous activity and structural network duality in synaptic connections, we executed a simulation study using the leaky integrate-and-fire spiking neural network with log-normal synaptic weight distribution for the EPSPs and duality of synaptic connectivity, depending on synaptic weight. We conducted multiscale entropy analysis of the temporal spiking activity. Our simulation demonstrated that, when strong synaptic connections approach a small-world network, specific spiking patterns arise during irregular spatio-temporal spiking activity, and the complexity at the large temporal scale (i.e., slow frequency) is enhanced. Moreover, we confirmed through a surrogate data analysis that slow temporal dynamics reflect a deterministic process in the spiking neural networks. This modelling approach may improve the understanding of the spatio-temporal complex neural activity in the brain.

Pattern Classification by Spiking Neural Networks Combining Self-Organized and Reward-Related Spike-Timing-Dependent Plasticity

Many recent studies have applied to spike neural networks with spike-timing-dependent plasticity (STDP) to machine learning problems. The learning abilities of dopamine-modulated STDP (DA-STDP) for reward-related synaptic plasticity have also been gathering attention. Following these studies, we hypothesize that a network structure combining self-organized STDP and reward-related DA-STDP can solve the machine learning problem of pattern classification. Therefore, we studied the ability of a network in which recurrent spiking neural networks are combined with STDP for non-supervised learning, with an output layer joined by DA-STDP for supervised learning, to perform pattern classification. We confirmed that this network could perform pattern classification using the STDP effect for emphasizing features of the input spike pattern and DA-STDP supervised learning. Therefore, our proposed spiking neural network may prove to be a useful approach for machine learning problems.

Routes to Chaos Induced by a Discontinuous Resetting Process in a Hybrid Spiking Neuron Model

Several hybrid spiking neuron models combining continuous spike generation mechanisms and discontinuous resetting processes following spiking have been proposed. The Izhikevich neuron model, for example, can reproduce many spiking patterns. This model clearly possesses various types of bifurcations and routes to chaos under the effect of a state-dependent jump in the resetting process. In this study, we focus further on the relation between chaotic behaviour and the state-dependent jump, approaching the subject by comparing spiking neuron model versions with and without the resetting process. We first adopt a continuous two-dimensional spiking neuron model in which the orbit in the spiking state does not exhibit divergent behaviour. We then insert the resetting process into the model. An evaluation using the Lyapunov exponent with a saltation matrix and a characteristic multiplier of the Poincar'e map reveals that two types of chaotic behaviour (i.e. bursting chaotic spikes and near-period-two chaotic spikes) are induced by the resetting process. In addition, we confirm that this chaotic bursting state is generated from the periodic spiking state because of the slow- and fast-scale dynamics that arise when jumping to the hyperpolarization and depolarization regions, respectively.

Hebbian Wiring Plasticity Generates Efficient Network Structures for Robust Inference with Synaptic Weight Plasticity

In the adult mammalian cortex, a small fraction of spines are created and eliminated every day, and the resultant synaptic connection structure is highly nonrandom, even in local circuits. However, it remains unknown whether a particular synaptic connection structure is functionally advantageous in local circuits, and why creation and elimination of synaptic connections is necessary in addition to rich synaptic weight plasticity. To answer these questions, we studied an inference task model through theoretical and numerical analyses. We demonstrate that a robustly beneficial network structure naturally emerges by combining Hebbian-type synaptic weight plasticity and wiring plasticity. Especially in a sparsely connected network, wiring plasticity achieves reliable computation by enabling efficient information transmission. Furthermore, the proposed rule reproduces experimental observed correlation between spine dynamics and task performance.

A Lognormal Recurrent Network Model for Burst Generation during Hippocampal Sharp Waves

The strength of cortical synapses distributes lognormally, with a long tail of strong synapses. Various properties of neuronal activity, such as the average firing rates of neurons, the rate and magnitude of spike bursts, the magnitude of population synchrony, and the correlations between presynaptic and postsynaptic spikes, also obey lognormal-like distributions reported in the rodent hippocampal CA1 and CA3 areas. Theoretical models have demonstrated how such a firing rate distribution emerges from neural network dynamics. However, how the other properties also display lognormal patterns remain unknown. Because these features are likely to originate from neural dynamics in CA3, we model a recurrent neural network with the weights of recurrent excitatory connections distributed lognormally to explore the underlying mechanisms and their functional implications. Using multi-timescale adaptive threshold neurons, we construct a low-frequency spontaneous firing state of bursty neurons. This state well replicates the observed statistical properties of population synchrony in hippocampal pyramidal cells. Our results show that the lognormal distribution of synaptic weights consistently accounts for the observed long-tailed features of hippocampal activity. Furthermore, our model demonstrates that bursts spread over the lognormal network much more effectively than single spikes, implying an advantage of spike bursts in information transfer. This efficiency in burst propagation is not found in neural network models with Gaussian-weighted recurrent excitatory synapses. Our model proposes a potential network mechanism to generate sharp waves in CA3 and associated ripples in CA1 because bursts occur in CA3 pyramidal neurons most frequently during sharp waves.

A neural network model of reliably optimized spike transmission

We studied the detailed structure of a neuronal network model in which the spontaneous spike activity is correctly optimized to match the experimental data and discuss the reliability of the optimized spike transmission. Two stochastic properties of the spontaneous activity were calculated: the spike-count rate and synchrony size. The synchrony size, expected to be an important factor for optimization of spike transmission in the network, represents a percentage of observed coactive neurons within a time bin, whose probability approximately follows a power-law. We systematically investigated how these stochastic properties could matched to those calculated from the experimental data in terms of the log-normally distributed synaptic weights between excitatory and inhibitory neurons and synaptic background activity induced by the input current noise in the network model. To ensure reliably optimized spike transmission, the synchrony size as well as spike-count rate were simultaneously optimized. This required changeably balanced log-normal distributions of synaptic weights between excitatory and inhibitory neurons and appropriately amplified synaptic background activity. Our results suggested that the inhibitory neurons with a hub-like structure driven by intensive feedback from excitatory neurons were a key factor in the simultaneous optimization of the spike-count rate and synchrony size, regardless of different spiking types between excitatory and inhibitory neurons.

Interplay between short- and long-term plasticity in cell-assembly formation

Various hippocampal and neocortical synapses of mammalian brain show both short-term plasticity and long-term plasticity, which are considered to underlie learning and memory by the brain. According to Hebb’s postulate, synaptic plasticity encodes memory traces of past experiences into cell assemblies in cortical circuits. However, it remains unclear how the various forms of long-term and short-term synaptic plasticity cooperatively create and reorganize such cell assemblies. Here, we investigate the mechanism in which the three forms of synaptic plasticity known in cortical circuits, i.e., spike-timing-dependent plasticity (STDP), short-term depression (STD) and homeostatic plasticity, cooperatively generate, retain and reorganize cell assemblies in a recurrent neuronal network model. We show that multiple cell assemblies generated by external stimuli can survive noisy spontaneous network activity for an adequate range of the strength of STD. Furthermore, our model predicts that a symmetric temporal window of STDP, such as observed in dopaminergic modulations on hippocampal neurons, is crucial for the retention and integration of multiple cell assemblies. These results may have implications for the understanding of cortical memory processes.

Dense neuron clustering explains connectivity statistics in cortical microcircuits

Local cortical circuits appear highly non-random, but the underlying connectivity rule remains elusive. Here, we analyze experimental data observed in layer 5 of rat neocortex and suggest a model for connectivity from which emerge essential observed non-random features of both wiring and weighting. These features include lognormal distributions of synaptic connection strength, anatomical clustering, and strong correlations between clustering and connection strength. Our model predicts that cortical microcircuits contain large groups of densely connected neurons which we call clusters. We show that such a cluster contains about one fifth of all excitatory neurons of a circuit which are very densely connected with stronger than average synapses. We demonstrate that such clustering plays an important role in the network dynamics, namely, it creates bistable neural spiking in small cortical circuits. Furthermore, introducing local clustering in large-scale networks leads to the emergence of various patterns of persistent local activity in an ongoing network activity. Thus, our results may bridge a gap between anatomical structure and persistent activity observed during working memory and other cognitive processes.

Genetic basis of neuronal individuality in the mammalian brain

The mammalian brain is a complex multicellular system involving enormous numbers of neurons. The neuron is the basic functional unit of the brain, and neurons are organized by specialized intercellular connections into circuits with many other neurons. Physiological studies have revealed that individual neurons have remarkably selective response properties, and this individuality is a fundamental requirement for building complex and functionally diverse neural networks. Recent molecular biological studies have revealed genetic bases for neuronal individuality in the mammalian brain. For example, in the rodent olfactory epithelium, individual olfactory neurons express only one type of odorant receptor (OR) out of the over 1000 ORs encoded in the genome. The expressed OR determines the neuron's selective chemosensory response and specifies its axonal targeting to a particular olfactory glomerulus in the olfactory bulb. Neuronal diversity can also be generated in individual cells by the independent and stochastic expression of autosomal alleles, which leads to functional heterozygosity among neurons. Among the many genes that show autosomal stochastic monoallelic expression, approximately 50 members of the clustered protocadherins (Pcdhs) are stochastically expressed in individual neurons in distinct combinations. The clustered Pcdhs belong to a large subfamily of the cadherin superfamily of homophilic cell-adhesion proteins. Loss-of-function analyses show that the clustered Pcdhs have critical functions in the accuracy of axonal projections, synaptic formation, dendritic arborization, and neuronal survival. In addition, cis-tetramers, composed of heteromultimeric clustered Pcdh members, represent selective binding units for cell-cell interactions, and provide exponential numbers of possible cell-surface relationships between individual neurons. The extensive molecular diversity of neuronal cell-surface proteins affects neurons’ individual properties and connectivities. The molecular features of the diverse clustered Pcdh molecules suggest that they provide a genetic basis for neuronal individuality and appropriate neuronal wiring in the brain.