Dynamic parallelism for synaptic updating in GPU-accelerated spiking neural network simulations

Dynamic control of eye-head gaze shifts by a spiking neural network model of the superior colliculus

INTRODUCTION

To reorient gaze (the eye's direction in space) towards a target is an overdetermined problem, as infinitely many combinations of eye- and head movements can specify the same gaze-displacement vector. Yet, behavioral measurements show that the primate gaze-control system selects a specific contribution of eye- and head movements to the saccade, which depends on the initial eye-in-head orientation. Single-unit recordings in the primate superior colliculus (SC) during head-unrestrained gaze shifts have further suggested that cells may encode the instantaneous trajectory of a desired straight gaze path in a feedforward way by the total cumulative number of spikes in the neural population, and that the instantaneous gaze kinematics are thus determined by the neural firing rates. The recordings also indicated that the latter is modulated by the initial eye position. We recently proposed a conceptual model that accounts for many of the observed properties of eye-head gaze shifts and on the potential role of the SC in gaze control.

METHODS

Here, we extend and test the model by incorporating a spiking neural network of the SC motor map, the output of which drives the eye-head motor control circuitry by linear cumulative summation of individual spike effects of each recruited SC neuron. We propose a simple neural mechanism on SC cells that explains the modulatory influence of feedback from an initial eye-in-head position signal on their spiking activity. The same signal also determines the onset delay of the head movement with respect to the eye. Moreover, the downstream eye- and head burst generators were taken to be linear, as our earlier work had indicated that much of the non-linear main-sequence kinematics of saccadic eye movements may be due to neural encoding at the collicular level, rather than at the brainstem.

RESULTS AND DISCUSSION

We investigate how the spiking activity of the SC population drives gaze to the intended target location within a dynamic local gaze-velocity feedback circuit that yields realistic eye- and head-movement kinematics and dynamic SC gaze-movement fields.

Brian2CUDA: Flexible and Efficient Simulation of Spiking Neural Network Models on GPUs

Graphics processing units (GPUs) are widely available and have been used with great success to accelerate scientific computing in the last decade. These advances, however, are often not available to researchers interested in simulating spiking neural networks, but lacking the technical knowledge to write the necessary low-level code. Writing low-level code is not necessary when using the popular Brian simulator, which provides a framework to generate efficient CPU code from high-level model definitions in Python. Here, we present Brian2CUDA, an open-source software that extends the Brian simulator with a GPU backend. Our implementation generates efficient code for the numerical integration of neuronal states and for the propagation of synaptic events on GPUs, making use of their massively parallel arithmetic capabilities. We benchmark the performance improvements of our software for several model types and find that it can accelerate simulations by up to three orders of magnitude compared to Brian's CPU backend. Currently, Brian2CUDA is the only package that supports Brian's full feature set on GPUs, including arbitrary neuron and synapse models, plasticity rules, and heterogeneous delays. When comparing its performance with Brian2GeNN, another GPU-based backend for the Brian simulator with fewer features, we find that Brian2CUDA gives comparable speedups, while being typically slower for small and faster for large networks. By combining the flexibility of the Brian simulator with the simulation speed of GPUs, Brian2CUDA enables researchers to efficiently simulate spiking neural networks with minimal effort and thereby makes the advancements of GPU computing available to a larger audience of neuroscientists.

EDEN: A High-Performance, General-Purpose, NeuroML-Based Neural Simulator

Modern neuroscience employs in silico experimentation on ever-increasing and more detailed neural networks. The high modeling detail goes hand in hand with the need for high model reproducibility, reusability and transparency. Besides, the size of the models and the long timescales under study mandate the use of a simulation system with high computational performance, so as to provide an acceptable time to result. In this work, we present EDEN (Extensible Dynamics Engine for Networks), a new general-purpose, NeuroML-based neural simulator that achieves both high model flexibility and high computational performance, through an innovative model-analysis and code-generation technique. The simulator runs NeuroML-v2 models directly, eliminating the need for users to learn yet another simulator-specific, model-specification language. EDEN's functional correctness and computational performance were assessed through NeuroML models available on the NeuroML-DB and Open Source Brain model repositories. In qualitative experiments, the results produced by EDEN were verified against the established NEURON simulator, for a wide range of models. At the same time, computational-performance benchmarks reveal that EDEN runs from one to nearly two orders-of-magnitude faster than NEURON on a typical desktop computer, and does so without additional effort from the user. Finally, and without added user effort, EDEN has been built from scratch to scale seamlessly over multiple CPUs and across computer clusters, when available.

A spiking neural network model of the Superior Colliculus that is robust to changes in the spatial-temporal input

Previous studies have indicated that the location of a large neural population in the Superior Colliculus (SC) motor map specifies the amplitude and direction of the saccadic eye-movement vector, while the saccade trajectory and velocity profile are encoded by the population firing rates. We recently proposed a simple spiking neural network model of the SC motor map, based on linear summation of individual spike effects of each recruited neuron, which accounts for many of the observed properties of SC cells in relation to the ensuing eye movement. However, in the model, the cortical input was kept invariant across different saccades. Electrical microstimulation and reversible lesion studies have demonstrated that the saccade properties are quite robust against large changes in supra-threshold SC activation, but that saccade amplitude and peak eye-velocity systematically decrease at low input strengths. These features were not accounted for by the linear spike-vector summation model. Here we show that the model’s input projection strengths and intra-collicular lateral connections can be tuned to generate saccades and neural spiking patterns that closely follow the experimental results.

N4Sim: The first Nervous NaNoNetwork Simulator with Synaptic Molecular Communications

The unconventional nature of molecular communication necessitates contributions from a host of scientific fields making the simulator design for such systems to be quite challenging. The nervous system is one of the largest and most important nanonetworks of the body. Several molecular and nano communication simulators exist in literature along with a few neural network simulators, however, most existing simulators are not specific for the nervous system since they ignore the synaptic diffusion because of the computational complexity required to model it. Additionally, information and communication theoretical (ICT) analysis of the system is not directly supported by existing neural network simulators. In this work, we present and describe Neural NaNoNetwork Simulator, N4Sim, which can resolve these issues in existing simulators. We describe key components of the simulator and methods to solve the synaptic communication in a fast and efficient manner. Our model for the synaptic communication channel is comparable in accuracy to those achieved by Monte Carlo simulations while using a fraction of time and processing resources. The presented simulator opens a large set of design options for applications in nervous system.

Synthesis and Analysis of the Fixed-Point Hodgkin–Huxley Neuron Model

In many tasks related to realistic neurons and neural network simulation, the performance of desktop computers is nowhere near enough. To overcome this obstacle, researchers are developing FPGA-based simulators that naturally use fixed-point arithmetic. In these implementations, little attention is usually paid to the choice of numerical method for the discretization of the continuous neuron model. In our study, the implementation accuracy of a neuron described by simplified Hodgkin–Huxley equations in fixed-point arithmetic is under investigation. The principle of constructing a fixed-point neuron model with various numerical methods is described. Interspike diagrams and refractory period analysis are used for the experimental study of the synthesized discrete maps of the simplified Hodgkin–Huxley neuron model. We show that the explicit midpoint method is much better suited to simulate the neuron dynamics on an FPGA than the explicit Euler method which is in common use.

Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines

Liquid state machine (LSM), a bio-inspired computing model consisting of the input sparsely connected to a randomly interlinked reservoir (or liquid) of spiking neurons followed by a readout layer, finds utility in a range of applications varying from robot control and sequence generation to action, speech, and image recognition. LSMs stand out among other Recurrent Neural Network (RNN) architectures due to their simplistic structure and lower training complexity. Plethora of recent efforts have been focused toward mimicking certain characteristics of biological systems to enhance the performance of modern artificial neural networks. It has been shown that biological neurons are more likely to be connected to other neurons in the close proximity, and tend to be disconnected as the neurons are spatially far apart. Inspired by this, we propose a group of locally connected neuron reservoirs, or an ensemble of liquids approach, for LSMs. We analyze how the segmentation of a single large liquid to create an ensemble of multiple smaller liquids affects the latency and accuracy of an LSM. In our analysis, we quantify the ability of the proposed ensemble approach to provide an improved representation of the input using the Separation Property (SP) and Approximation Property (AP). Our results illustrate that the ensemble approach enhances class discrimination (quantified as the ratio between the SP and AP), leading to better accuracy in speech and image recognition tasks, when compared to a single large liquid. Furthermore, we obtain performance benefits in terms of improved inference time and reduced memory requirements, due to lowered number of connections and the freedom to parallelize the liquid evaluation process.

Electrical stimulation in a spiking neural network model of monkey superior colliculus

The superior colliculus (SC) generates saccades by recruiting a population of cells in its topographically organized motor map. Supra-threshold electrical stimulation in the SC produces a normometric saccade with little effect of the stimulation parameters. Moreover, the kinematics of electrically evoked saccades strongly resemble natural, visual-evoked saccades. These findings support models in which the saccade vector is determined by a center-of-gravity computation of activated neurons, while trajectory and kinematics arise in brainstem-cerebellar feedback circuits. Recent single-unit recordings, however, have indicated that the SC population also specifies the instantaneous saccade kinematics, supporting an alternative model, in which the saccade trajectory results from dynamic summation of movement effects of all SC spike trains. Here we reconcile the linear summation model with stimulation results, by assuming that the electric field directly activates a relatively small set of neurons around the electrode tip, which subsequently sets up a large population response through lateral synaptic interactions.

Microstimulation in a spiking neural network model of the midbrain superior colliculus

The midbrain superior colliculus (SC) generates a rapid saccadic eye movement to a sensory stimulus by recruiting a population of cells in its topographically organized motor map. Supra-threshold electrical microstimulation in the SC reveals that the site of stimulation produces a normometric saccade vector with little effect of the stimulation parameters. Moreover, electrically evoked saccades (E-saccades) have kinematic properties that strongly resemble natural, visual-evoked saccades (V-saccades). These findings support models in which the saccade vector is determined by a center-of-gravity computation of activated neurons, while its trajectory and kinematics arise from downstream feedback circuits in the brainstem. Recent single-unit recordings, however, have indicated that the SC population also specifies instantaneous kinematics. These results support an alternative model, in which the desired saccade trajectory, including its kinematics, follows from instantaneous summation of movement effects of all SC spike trains. But how to reconcile this model with microstimulation results? Although it is thought that microstimulation activates a large population of SC neurons, the mechanism through which it arises is unknown. We developed a spiking neural network model of the SC, in which microstimulation directly activates a relatively small set of neurons around the electrode tip, which subsequently sets up a large population response through lateral synaptic interactions. We show that through this mechanism the population drives an E-saccade with near-normal kinematics that are largely independent of the stimulation parameters. Only at very low stimulus intensities the network recruits a population with low firing rates, resulting in abnormally slow saccades.

A structure-time parallel implementation of spike-based deep learning

Motivated by the recent progress of deep spiking neural networks (SNNs), we propose a structure-time parallel strategy based on layered structure and one-time computation over a time window to speed up the prominent spike-based deep learning algorithm named broadcast alignment. Furthermore, a well-designed deep hierarchical model based on the parallel broadcast alignment is proposed for object recognition. The parallel broadcast alignment achieves a significant 137× speedup compared to its original implementation on MNIST dataset. The object recognition model achieves higher accuracy than that of the latest spiking deep convolutional neural networks on the ETH-80 dataset. The proposed parallel strategy and the object recognition model will facilitate both the simulation of deep SNNs for studying spiking neural dynamics and also the applications of spike-based deep learning in real-world problems.

Double Stimulation in a Spiking Neural Network Model of the Midbrain Superior Colliculus

The midbrain superior colliculus (SC) is a crucial sensorimotor interface in the generation of rapid saccadic gaze shifts. For every saccade it recruits a large population of cells in its vectorial motor map. Supra-threshold electrical microstimulation in the SC reveals that the stimulated site produces the saccade vector specified by the motor map. Electrically evoked saccades (E-saccades) have kinematic properties that strongly resemble natural, visual-evoked saccades (V-saccades), with little influence of the stimulation parameters. Moreover, synchronous stimulation at two sites yields eye movements that resemble a weighted vector average of the individual stimulation effects. Single-unit recordings have indicated that the SC population acts as a vectorial pulse generator by specifying the instantaneous gaze-kinematics through dynamic summation of the movement effects of all SC spike trains. But how to reconcile the a-specific stimulation pulses with these intricate saccade properties? We recently developed a spiking neural network model of the SC, in which microstimulation initially activates a relatively small set of (~50) neurons around the electrode tip, which subsequently sets up a large population response (~5,000 neurons) through lateral synaptic interactions. Single-site microstimulation in this network thus produces the saccade properties and firing rate profiles as seen in single-unit recording experiments. We here show that this mechanism also accounts for many results of simultaneous double stimulation at different SC sites. The resulting E-saccade trajectories resemble a weighted average of the single-site effects, in which stimulus current strength of the electrode pulses serve as weighting factors. We discuss under which conditions the network produces effects that deviate from experimental results.