Real-time simulation of biologically realistic stochastic neurons in VLSI

Stochastic Resonance in Organic Electronic Devices

Stochastic Resonance (SR) is a phenomenon in which noise improves the performance of a system. With the addition of noise, a weak input signal to a nonlinear system, which may exceed its threshold, is transformed into an output signal. In the other words, noise-driven signal transfer is achieved. SR has been observed in nonlinear response systems, such as biological and artificial systems, and this review will focus mainly on examples of previous studies of mathematical models and experimental realization of SR using poly(hexylthiophene)-based organic field-effect transistors (OFETs). This phenomenon may contribute to signal processing with low energy consumption. However, the generation of SR requires a noise source. Therefore, the focus is on OFETs using materials such as organic materials with unstable electrical properties and critical elements due to unidirectional signal transmission, such as neural synapses. It has been reported that SR can be observed in OFETs by application of external noise. However, SR does not occur under conditions where the input signal exceeds the OFET threshold without external noise. Here, we present an example of a study that analyzes the behavior of SR in OFET systems and explain how SR can be made observable. At the same time, the role of internal noise in OFETs will be explained.

Hodgkin-Huxley Neuron and FPAA Dynamics

We present the experimental silicon results on the dynamics of a Hodgkin-Huxley neuron implemented on a reconfigurable platform. The circuit has been inspired by the similarity between biology and silicon, by modeling ion channels and their time constants. Another significant motivation behind this paper is to make the system available to circuit designers as well as users in the neuroscience community. The open-source tool infrastructure and a remote system ease the accessibility of our system to a number of users. We demonstrate the reproducibility of the results by replicating the dynamics across different boards along with responses from different inputs and with different parameters. The reconfigurability enables one to make use of a single primary design to obtain a variety of results. The measurements are taken from the system compiled on a field programmable analog array fabricated on a 350-nm process.

Leaky Integrate-and-Fire Neuron Circuit Based on Floating-Gate Integrator

The artificial spiking neural network (SNN) is promising and has been brought to the notice of the theoretical neuroscience and neuromorphic engineering research communities. In this light, we propose a new type of artificial spiking neuron based on leaky integrate-and-fire (LIF) behavior. A distinctive feature of the proposed FG-LIF neuron is the use of a floating-gate (FG) integrator rather than a capacitor-based one. The relaxation time of the charge on the FG relies mainly on the tunnel barrier profile, e.g., barrier height and thickness (rather than the area). This opens up the possibility of large-scale integration of neurons. The circuit simulation results offered biologically plausible spiking activity (<100 Hz) with a capacitor of merely 6 fF, which is hosted in an FG metal-oxide-semiconductor field-effect transistor. The FG-LIF neuron also has the advantage of low operation power (<30 pW/spike). Finally, the proposed circuit was subject to possible types of noise, e.g., thermal noise and burst noise. The simulation results indicated remarkable distributional features of interspike intervals that are fitted to Gamma distribution functions, similar to biological neurons in the neocortex.

A Spiking Neural Simulator Integrating Event-Driven and Time-Driven Computation Schemes Using Parallel CPU-GPU Co-Processing: A Case Study

Time-driven simulation methods in traditional CPU architectures perform well and precisely when simulating small-scale spiking neural networks. Nevertheless, they still have drawbacks when simulating large-scale systems. Conversely, event-driven simulation methods in CPUs and time-driven simulation methods in graphic processing units (GPUs) can outperform CPU time-driven methods under certain conditions. With this performance improvement in mind, we have developed an event-and-time-driven spiking neural network simulator suitable for a hybrid CPU-GPU platform. Our neural simulator is able to efficiently simulate bio-inspired spiking neural networks consisting of different neural models, which can be distributed heterogeneously in both small layers and large layers or subsystems. For the sake of efficiency, the low-activity parts of the neural network can be simulated in CPU using event-driven methods while the high-activity subsystems can be simulated in either CPU (a few neurons) or GPU (thousands or millions of neurons) using time-driven methods. In this brief, we have undertaken a comparative study of these different simulation methods. For benchmarking the different simulation methods and platforms, we have used a cerebellar-inspired neural-network model consisting of a very dense granular layer and a Purkinje layer with a smaller number of cells (according to biological ratios). Thus, this cerebellar-like network includes a dense diverging neural layer (increasing the dimensionality of its internal representation and sparse coding) and a converging neural layer (integration) similar to many other biologically inspired and also artificial neural networks.

Hardware friendly probabilistic spiking neural network with long-term and short-term plasticity

This paper proposes a probabilistic spiking neural network (PSNN) with unimodal weight distribution, possessing long- and short-term plasticity. The proposed algorithm is derived by both the arithmetic gradient decent calculation and bioinspired algorithms. The algorithm is benchmarked by the Iris and Wisconsin breast cancer (WBC) data sets. The network features fast convergence speed and high accuracy. In the experiment, the PSNN took not more than 40 epochs for convergence. The average testing accuracy for Iris and WBC data is 96.7% and 97.2%, respectively. To test the usefulness of the PSNN to real world application, the PSNN was also tested with the odor data, which was collected by our self-developed electronic nose (e-nose). Compared with the algorithm (K-nearest neighbor) that has the highest classification accuracy in the e-nose for the same odor data, the classification accuracy of the PSNN is only 1.3% less but the memory requirement can be reduced at least 40%. All the experiments suggest that the PSNN is hardware friendly. First, it requires only nine-bits weight resolution for training and testing. Second, the PSNN can learn complex data sets with a little number of neurons that in turn reduce the cost of VLSI implementation. In addition, the algorithm is insensitive to synaptic noise and the parameter variation induced by the VLSI fabrication. Therefore, the algorithm can be implemented by either software or hardware, making it suitable for wider application.

Dynamical system design for silicon neurons using phase reduction approach

In the present paper, we apply a computer-aided phase reduction approach to dynamical system design for silicon neurons (SiNs). Firstly, we briefly review the dynamical system design for SiNs. Secondly, we summarize the phase response properties of circuit models of previous SiNs to clarify design criteria in our approach. From a viewpoint of the phase reduction theory, as a case study, we show how to tune circuit parameters of the resonate-and-fire neuron (RFN) circuit as a hybrid type SiN. Finally, we demonstrate delay-induced synchronization in a silicon spiking neural network that consists of the RFN circuits.

Design of silicon brains in the nano-CMOS era: spiking neurons, learning synapses and neural architecture optimization

We present a design framework for neuromorphic architectures in the nano-CMOS era. Our approach to the design of spiking neurons and STDP learning circuits relies on parallel computational structures where neurons are abstracted as digital arithmetic logic units and communication processors. Using this approach, we have developed arrays of silicon neurons that scale to millions of neurons in a single state-of-the-art Field Programmable Gate Array (FPGA). We demonstrate the validity of the design methodology through the implementation of cortical development in a circuit of spiking neurons, STDP synapses, and neural architecture optimization.

Asynchronous cellular automaton-based neuron: theoretical analysis and on-FPGA learning

A generalized asynchronous cellular automaton-based neuron model is a special kind of cellular automaton that is designed to mimic the nonlinear dynamics of neurons. The model can be implemented as an asynchronous sequential logic circuit and its control parameter is the pattern of wires among the circuit elements that is adjustable after implementation in a field-programmable gate array (FPGA) device. In this paper, a novel theoretical analysis method for the model is presented. Using this method, stabilities of neuron-like orbits and occurrence mechanisms of neuron-like bifurcations of the model are clarified theoretically. Also, a novel learning algorithm for the model is presented. An equivalent experiment shows that an FPGA-implemented learning algorithm enables an FPGA-implemented model to automatically reproduce typical nonlinear responses and occurrence mechanisms observed in biological and model neurons.

Synaptic variability in a cortical neuromorphic circuit

Variable behavior has been observed in several mechanisms found in biological neurons, resulting in changes in neural behavior that might be useful to capture in neuromorphic circuits. This paper presents a neuromorphic cortical neuron with synaptic neurotransmitter-release variability, which is designed to be used in neural networks as part of the Biomimetic Real-Time Cortex project. This neuron has been designed and simulated using carbon nanotube (CNT) transistors, which is one of several nanotechnologies under consideration to meet the challenges of scale presented by the cortex. Some research results suggest that some instances of variability are stochastic, while others indicate that some instances of variability are chaotic. In this paper, both possible sources of variability are considered by embedding either Gaussian noise or a chaotic signal into the neuromorphic or synaptic circuit and observing the simulation results. In order to embed chaotic behavior into the neuromorphic circuit, a chaotic signal generator circuit is presented, implemented with CNT transistors that could be embedded in the electronic neural circuit, and simulated using CNT SPICE models. The circuit uses a chaotic piecewise linear 1-D map implemented by switched-current circuits. The simulation results presented in this paper illustrate that neurotransmitter-release variability plays a beneficial role in the reliability of spike generation. In an examination of this reliability, the precision of spike timing in the CNT circuit simulations is found to be dependent on stimulus (postsynaptic potential) transients. Postsynaptic potentials with low neurotransmitter release variability or without neurotransmitter release variability produce imprecise spike trains, whereas postsynaptic potentials with high neurotransmitter-release variability produce spike trains with reproducible timing.

Six networks on a universal neuromorphic computing substrate

In this study, we present a highly configurable neuromorphic computing substrate and use it for emulating several types of neural networks. At the heart of this system lies a mixed-signal chip, with analog implementations of neurons and synapses and digital transmission of action potentials. Major advantages of this emulation device, which has been explicitly designed as a universal neural network emulator, are its inherent parallelism and high acceleration factor compared to conventional computers. Its configurability allows the realization of almost arbitrary network topologies and the use of widely varied neuronal and synaptic parameters. Fixed-pattern noise inherent to analog circuitry is reduced by calibration routines. An integrated development environment allows neuroscientists to operate the device without any prior knowledge of neuromorphic circuit design. As a showcase for the capabilities of the system, we describe the successful emulation of six different neural networks which cover a broad spectrum of both structure and functionality.

Systematic computation of nonlinear cellular and molecular dynamics with low-power CytoMimetic circuits: a simulation study

This paper presents a novel method for the systematic implementation of low-power microelectronic circuits aimed at computing nonlinear cellular and molecular dynamics. The method proposed is based on the Nonlinear Bernoulli Cell Formalism (NBCF), an advanced mathematical framework stemming from the Bernoulli Cell Formalism (BCF) originally exploited for the modular synthesis and analysis of linear, time-invariant, high dynamic range, logarithmic filters. Our approach identifies and exploits the striking similarities existing between the NBCF and coupled nonlinear ordinary differential equations (ODEs) typically appearing in models of naturally encountered biochemical systems. The resulting continuous-time, continuous-value, low-power CytoMimetic electronic circuits succeed in simulating fast and with good accuracy cellular and molecular dynamics. The application of the method is illustrated by synthesising for the first time microelectronic CytoMimetic topologies which simulate successfully: 1) a nonlinear intracellular calcium oscillations model for several Hill coefficient values and 2) a gene-protein regulatory system model. The dynamic behaviours generated by the proposed CytoMimetic circuits are compared and found to be in very good agreement with their biological counterparts. The circuits exploit the exponential law codifying the low-power subthreshold operation regime and have been simulated with realistic parameters from a commercially available CMOS process. They occupy an area of a fraction of a square-millimetre, while consuming between 1 and 12 microwatts of power. Simulations of fabrication-related variability results are also presented.

Adaptive visual and auditory map alignment in barn owl superior colliculus and its neuromorphic implementation

Adaptation is one of the most important phenomena in biology. A young barn owl can adapt to imposed environmental changes, such as artificial visual distortion caused by wearing a prism. This adjustment process has been modeled mathematically and the model replicates the sensory map realignment of barn owl superior colliculus (SC) through axonogenesis and synaptogenesis. This allows the biological mechanism to be transferred to an artificial computing system and thereby imbue it with a new form of adaptability to the environment. The model is demonstrated in a real-time robot environment. Results of the experiments are compared with and without prism distortion of vision, and show improved adaptability for the robot. However, the computation speed of the embedded system in the robot is slow. A digital and analog mixed signal very-large-scale integration (VLSI) circuit has been fabricated to implement adaptive sensory pathway changes derived from the SC model at higher speed. VLSI experimental results are consistent with simulation results.

Programming time-multiplexed reconfigurable hardware using a scalable neuromorphic compiler

Scalability and connectivity are two key challenges in designing neuromorphic hardware that can match biological levels. In this paper, we describe a neuromorphic system architecture design that addresses an approach to meet these challenges using traditional complementary metal-oxide-semiconductor (CMOS) hardware. A key requirement in realizing such neural architectures in hardware is the ability to automatically configure the hardware to emulate any neural architecture or model. The focus for this paper is to describe the details of such a programmable front-end. This programmable front-end is composed of a neuromorphic compiler and a digital memory, and is designed based on the concept of synaptic time-multiplexing (STM). The neuromorphic compiler automatically translates any given neural architecture to hardware switch states and these states are stored in digital memory to enable desired neural architectures. STM enables our proposed architecture to address scalability and connectivity using traditional CMOS hardware. We describe the details of the proposed design and the programmable front-end, and provide examples to illustrate its capabilities. We also provide perspectives for future extensions and potential applications.

Neuromorphic control of stepping pattern generation: a dynamic model with analog circuit implementation

Animals such as stick insects can adaptively walk on complex terrains by dynamically adjusting their stepping motion patterns. Inspired by the coupled Matsuoka and resonate-and-fire neuron models, we present a nonlinear oscillation model as the neuromorphic central pattern generator (CPG) for rhythmic stepping pattern generation. This dynamic model can also be used to actuate the motoneurons on a leg joint with adjustable driving frequencies and duty cycles by changing a few of the model parameters while operating such that different stepping patterns can be generated. A novel mixed-signal integrated circuit design of this dynamic model is subsequently implemented, which, although simplified, shares the equivalent output performance in terms of the adjustable frequency and duty cycle. Three identical CPG models being used to drive three joints can make an arthropod leg of three degrees of freedom. With appropriate initial circuit parameter settings, and thus suitable phase lags among joints, the leg is expected to walk on a complex terrain with adaptive steps. The adaptation is associated with the circuit parameters mediated both by the higher level nervous system and the lower level sensory signals. The model is realized using a 0.3- complementary metal-oxide-semiconductor process and the results are reported.

Tunable neuromimetic integrated system for emulating cortical neuron models

Nowadays, many software solutions are currently available for simulating neuron models. Less conventional than software-based systems, hardware-based solutions generally combine digital and analog forms of computation. In previous work, we designed several neuromimetic chips, including the Galway chip that we used for this paper. These silicon neurons are based on the Hodgkin–Huxley formalism and they are optimized for reproducing a large variety of neuron behaviors thanks to tunable parameters. Due to process variation and device mismatch in analog chips, we use a full-custom fitting method in voltage-clamp mode to tune our neuromimetic integrated circuits. By comparing them with experimental electrophysiological data of these cells, we show that the circuits can reproduce the main firing features of cortical cell types. In this paper, we present the experimental measurements of our system which mimic the four most prominent biological cells: fast spiking, regular spiking, intrinsically bursting, and low-threshold spiking neurons into analog neuromimetic integrated circuit dedicated to cortical neuron simulations. This hardware and software platform will allow to improve the hybrid technique, also called “dynamic-clamp,” that consists of connecting artificial and biological neurons to study the function of neuronal circuits.

Analysis and compensation of the effects of analog VLSI arithmetic on the LMS algorithm

Analog very large scale integration implementations of neural networks can compute using a fraction of the size and power required by their digital counterparts. However, intrinsic limitations of analog hardware, such as device mismatch, charge leakage, and noise, reduce the accuracy of analog arithmetic circuits, degrading the performance of large-scale adaptive systems. In this paper, we present a detailed mathematical analysis that relates different parameters of the hardware limitations to specific effects on the convergence properties of linear perceptrons trained with the least-mean-square (LMS) algorithm. Using this analysis, we derive design guidelines and introduce simple on-chip calibration techniques to improve the accuracy of analog neural networks with a small cost in die area and power dissipation. We validate our analysis by evaluating the performance of a mixed-signal complementary metal-oxide-semiconductor implementation of a 32-input perceptron trained with LMS.