Hierarchical Bayesian Inference and Learning in Spiking Neural Networks

On a framework of data assimilation for hyperparameter estimation of spiking neuronal networks

When handling real-world data modeled by a complex network dynamical system, the number of the parameters is often much more than the size of the data. Therefore, in many cases, it is impossible to estimate these parameters and the exact value of each parameter is frequently less interesting than the distribution of the parameters. In this paper, we aim to estimate the distribution of the parameters in the mesoscopic neuronal network model from the macroscopic experimental data, for example, the BOLD (blood oxygen level dependent) signal. Herein, we assume that the parameters of the neurons and synapses are inhomogeneous but independently and identically distributed from certain distributions with unknown hyperparameters. Thus, we estimate these hyperparameters of the distributions of the parameters, instead of estimating the parameters themselves. We formulate this problem under the framework of data assimilation and hierarchical Bayesian method and present an efficient method named Hierarchical Data Assimilation (HDA) to conduct the statistical inference on the neuronal network model with the BOLD signal data simulated by the hemodynamic model. We consider the Leaky Integral-Fire (LIF) neuronal networks with four synapses and show that the proposed algorithm can estimate the BOLD signals and the hyperparameters with high preciseness. In addition, we discuss the influence on the performance of the algorithm configuration and the LIF network model setup.

Human-Level Control Through Directly Trained Deep Spiking Q-Networks

As the third-generation neural networks, spiking neural networks (SNNs) have great potential on neuromorphic hardware because of their high energy efficiency. However, deep spiking reinforcement learning (DSRL), that is, the reinforcement learning (RL) based on SNNs, is still in its preliminary stage due to the binary output and the nondifferentiable property of the spiking function. To address these issues, we propose a deep spiking Q -network (DSQN) in this article. Specifically, we propose a directly trained DSRL architecture based on the leaky integrate-and-fire (LIF) neurons and deep Q -network (DQN). Then, we adapt a direct spiking learning algorithm for the DSQN. We further demonstrate the advantages of using LIF neurons in DSQN theoretically. Comprehensive experiments have been conducted on 17 top-performing Atari games to compare our method with the state-of-the-art conversion method. The experimental results demonstrate the superiority of our method in terms of performance, stability, generalization and energy efficiency. To the best of our knowledge, our work is the first one to achieve state-of-the-art performance on multiple Atari games with the directly trained SNN.

A Brain-Inspired Approach for Probabilistic Estimation and Efficient Planning in Precision Physical Interaction

This article presents a novel structure of spiking neural networks (SNNs) to simulate the joint function of multiple brain regions in handling precision physical interactions. This task desires efficient movement planning while considering contact prediction and fast radial compensation. Contact prediction demands the cognitive memory of the interaction model, and we novelly propose a double recurrent network to imitate the hippocampus, addressing the spatiotemporal property of the distribution. Radial contact response needs rich spatial information, and we use a cerebellum-inspired module to achieve temporally dynamic prediction. We also use a block-based feedforward network to plan movements, behaving like the prefrontal cortex. These modules are integrated to realize the joint cognitive function of multiple brain regions in prediction, controlling, and planning. We present an appropriate controller and planner to generate teaching signals and provide a feasible network initialization for reinforcement learning, which modifies synapses in accordance with reality. The experimental results demonstrate the validity of the proposed method.

Effective Transfer Learning Algorithm in Spiking Neural Networks

As the third generation of neural networks, spiking neural networks (SNNs) have gained much attention recently because of their high energy efficiency on neuromorphic hardware. However, training deep SNNs requires many labeled data that are expensive to obtain in real-world applications, as traditional artificial neural networks (ANNs). In order to address this issue, transfer learning has been proposed and widely used in traditional ANNs, but it has limited use in SNNs. In this article, we propose an effective transfer learning framework for deep SNNs based on the domain in-variance representation. Specifically, we analyze the rationality of centered kernel alignment (CKA) as a domain distance measurement relative to maximum mean discrepancy (MMD) in deep SNNs. In addition, we study the feature transferability across different layers by testing on the Office-31, Office-Caltech-10, and PACS datasets. The experimental results demonstrate the transferability of SNNs and show the effectiveness of the proposed transfer learning framework by using CKA in SNNs.

Bayesian continual learning via spiking neural networks

Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps toward the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.

How to Analyze the Neurodynamic Characteristics of Pulse-Coupled Neural Networks? A Theoretical Analysis and Case Study of Intersecting Cortical Model

The intersecting cortical model (ICM), initially designed for image processing, is a special case of the biologically inspired pulse-coupled neural-network (PCNN) models. Although the ICM has been widely used, few studies concern the internal activities and firing conditions of the neuron, which may lead to an invalid model in the application. Furthermore, the lack of theoretical analysis has led to inappropriate parameter settings and consequent limitations on ICM applications. To address this deficiency, we first study the continuous firing condition of ICM neurons to determine the restrictions that exist between network parameters and the input signal. Second, we investigate the neuron pulse period to understand the neural firing mechanism. Third, we derive the relationship between the continuous firing condition and the neural pulse period, and the relationship can prove the validity of the continuous firing condition and the neural pulse period as well. A solid understanding of the neural firing mechanism is helpful in setting appropriate parameters and in providing a theoretical basis for widespread applications to use the ICM model effectively. Extensive experiments of numerical tests with a common image reveal the rationality of our theoretical results.

Distributed and Time-Delayed k-Winner-Take-All Network for Competitive Coordination of Multiple Robots

In this article, a distributed and time-delayed k-winner-take-all (DT-kWTA) network is established and analyzed for competitively coordinated task assignment of a multirobot system. It is considered and designed from the following three aspects. First, a network is built based on a k-winner-take-all (kWTA) competitive algorithm that selects k maximum values from the inputs. Second, a distributed control strategy is used to improve the network in terms of communication load and computational burden. Third, the time-delayed problem prevalent in arbitrary causal systems (especially, in networks) is taken into account in the proposed network. This work combines distributed kWTA competition network with time delay for the first time, thus enabling it to better handle realistic applications than previous work. In addition, it theoretically derives the maximum delay allowed by the network and proves the convergence and robustness of the network. The results are applied to a multirobot system to conduct its robots' competitive coordination to complete the given task.

Toward Efficient Processing and Learning With Spikes: New Approaches for Multispike Learning

Spikes are the currency in central nervous systems for information transmission and processing. They are also believed to play an essential role in low-power consumption of the biological systems, whose efficiency attracts increasing attentions to the field of neuromorphic computing. However, efficient processing and learning of discrete spikes still remain a challenging problem. In this article, we make our contributions toward this direction. A simplified spiking neuron model is first introduced with the effects of both synaptic input and firing output on the membrane potential being modeled with an impulse function. An event-driven scheme is then presented to further improve the processing efficiency. Based on the neuron model, we propose two new multispike learning rules which demonstrate better performance over other baselines on various tasks, including association, classification, and feature detection. In addition to efficiency, our learning rules demonstrate high robustness against the strong noise of different types. They can also be generalized to different spike coding schemes for the classification task, and notably, the single neuron is capable of solving multicategory classifications with our learning rules. In the feature detection task, we re-examine the ability of unsupervised spike-timing-dependent plasticity with its limitations being presented, and find a new phenomenon of losing selectivity. In contrast, our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied. Moreover, our rules cannot only detect features but also discriminate them. The improved performance of our methods would contribute to neuromorphic computing as a preferable choice.

Brain-Inspired Hardware Solutions for Inference in Bayesian Networks

The implementation of inference (i.e., computing posterior probabilities) in Bayesian networks using a conventional computing paradigm turns out to be inefficient in terms of energy, time, and space, due to the substantial resources required by floating-point operations. A departure from conventional computing systems to make use of the high parallelism of Bayesian inference has attracted recent attention, particularly in the hardware implementation of Bayesian networks. These efforts lead to several implementations ranging from digital circuits, mixed-signal circuits, to analog circuits by leveraging new emerging nonvolatile devices. Several stochastic computing architectures using Bayesian stochastic variables have been proposed, from FPGA-like architectures to brain-inspired architectures such as crossbar arrays. This comprehensive review paper discusses different hardware implementations of Bayesian networks considering different devices, circuits, and architectures, as well as a more futuristic overview to solve existing hardware implementation problems.

Epilepsy Signal Recognition Using Online Transfer TSK Fuzzy Classifier Underlying Classification Error and Joint Distribution Consensus Regularization

In this study, an online transfer TSK fuzzy classifier O-T-TSK-FC is proposed for recognizing epilepsy signals. Compared with most of the existing transfer learning models, O-T-TSK-FC enjoys its merits from the following three aspects: 1) Since different patients often response to the same neuronal firing stimulation in different neural manners, the labeled data in the source domain cannot accurately represent the primary EEG data in the target domain. Therefore, we design an objective function which can integrate with subject-specific data in the target domain to induce the target predictive function. 2) A new regularization used for knowledge transfer is proposed from the perspective of error consensus, and its rationality is explained from the perspective of probability density estimation. 3) Clustering is used to partition source domains so as to reduce the computation of O-T-TSK-FC without affecting its performance. Based on the EEG signals collected from Bonn University, six different online scenarios for transfer learning are constructed. Experimental results on them show that O-T-TSK-FC performs better than benchmarking algorithms and robustly.

Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation

Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields. Nevertheless, it remains unclear how probabilistic inference can be implemented by a network of spiking neurons in the brain. Previous studies have tried to relate the inference equation of binary Markov random fields to the dynamic equation of spiking neural networks through belief propagation algorithm and reparameterization, but they are valid only for Markov random fields with limited network structure. In this paper, we propose a spiking neural network model that can implement inference of arbitrary binary Markov random fields. Specifically, we design a spiking recurrent neural network and prove that its neuronal dynamics are mathematically equivalent to the inference process of Markov random fields by adopting mean-field theory. Furthermore, our mean-field approach unifies previous works. Theoretical analysis and experimental results, together with the application to image denoising, demonstrate that our proposed spiking neural network can get comparable results to that of mean-field inference.