A digital implementation of neuron–astrocyte interaction for neuromorphic applications

Efficient digital design of the nonlinear behavior of Hindmarsh-Rose neuron model in large-scale neural population

Spiking networks, as the third generation of neural networks, are of great interest today due to their low power consumption in cognitive processes. This important characteristic has caused the hardware implementation techniques of spiking networks in the form of neuromorphic systems attract a lot of attention. For the first time, the focus is on the digital implementation based on CORDIC approximation of the Hindmarsh-Rose (HR) neuron so that the hardware implementation cost is lower than previous studies. If the digital design of a neuron is done efficient, the possibility of implementing a population of neurons is provided for the feasibility of low-consumption implementation of high-level cognitive processes in hardware, which is considered in this paper through edge detector, noise removal and image magnification spiking networks based on the proposed CORDIC_HR model. While using less hardware resources, the proposed HR neuron model follows the behavior of the original neuron model in the time domain with much less error than previous study. Also, the complex nonlinear behavior of the original and the proposed model of HR neuron through the bifurcation diagram, phase space and nullcline space analysis under different system parameters was investigated and the good follow-up of the proposed model was confirmed from the original model. In addition to the fact that the individual behavior of the original and the proposed neurons is the same, the functional and behavioral performance of the randomly connected neuronal population of original and proposed neuron model is equal. In general, the main contribution of the paper is in presenting an efficient hardware model, which consumes less hardware resources, follows the behavior of the original model with high accuracy, and has an acceptable performance in image processing applications such as noise removal and edge detection.

Astrocyte strategies in the energy-efficient brain

Astrocytes generate ATP through glycolysis and mitochondrion respiration, using glucose, lactate, fatty acids, amino acids, and ketone bodies as metabolic fuels. Astrocytic mitochondria also participate in neuronal redox homeostasis and neurotransmitter recycling. In this essay, we aim to integrate the multifaceted evidence about astrocyte bioenergetics at the cellular and systems levels, with a focus on mitochondrial oxidation. At the cellular level, the use of fatty acid β-oxidation and the existence of molecular switches for the selection of metabolic mode and fuels are examined. At the systems level, we discuss energy audits of astrocytes and how astrocytic Ca2+ signaling might contribute to the higher performance and lower energy consumption of the brain as compared to engineered circuits. We finish by examining the neural-circuit dysregulation and behavior impairment associated with alterations of astrocytic mitochondria. We conclude that astrocytes may contribute to brain energy efficiency by coupling energy, redox, and computational homeostasis in neural circuits.

Transition behavior of the seizure dynamics modulated by the astrocyte inositol triphosphate noise

Epilepsy is a neurological disorder with recurrent seizures, which convey complex dynamical characteristics including chaos and randomness. Until now, the underlying mechanism has not been fully elucidated, especially the bistable property beneath the epileptic random induction phenomena in certain conditions. Inspired by the recent finding that astrocyte GTPase-activating protein (G-protein)-coupled receptors could be involved in stochastic epileptic seizures, we proposed a neuron-astrocyte network model, incorporating the noise of the astrocytic second messenger, inositol triphosphate (IP3) that is modulated by G-protein-coupled receptor activation. Based on this model, we have statistically analyzed the transitions of epileptic seizures by performing repeatable simulation trials. Our simulation results show that the increase in the IP3 noise intensity induces depolarization-block epileptic seizures together with an increase in neuronal firing frequency, consistent with corresponding experiments. Meanwhile, the bistable states of the seizure dynamics were present under certain noise intensities, during which the neuronal firing pattern switches between regular sparse spiking and epileptic seizure states. This random presence of epileptic seizures is absent when the noise intensity continues to increase, accompanying with an increase in the epileptic depolarization block duration. The simulation results also shed light on the fact that calcium signals in astrocytes play significant roles in the pattern formations of the epileptic seizure. Our results provide a potential pathway for understanding the epileptic randomness in certain conditions.

Central Nervous System: Overall Considerations Based on Hardware Realization of Digital Spiking Silicon Neurons (DSSNs) and Synaptic Coupling

The Central Nervous System (CNS) is the part of the nervous system including the brain and spinal cord. The CNS is so named because the brain integrates the received information and influences the activity of different sections of the bodies. The basic elements of this important organ are: neurons, synapses, and glias. Neuronal modeling approach and hardware realization design for the nervous system of the brain is an important issue in the case of reproducing the same biological neuronal behaviors. This work applies a quadratic-based modeling called Digital Spiking Silicon Neuron (DSSN) to propose a modified version of the neuronal model which is capable of imitating the basic behaviors of the original model. The proposed neuron is modeled based on the primary hyperbolic functions, which can be realized in high correlation state with the main model (original one). Really, if the high-cost terms of the original model, and its functions were removed, a low-error and high-performance (in case of frequency and speed-up) new model will be extracted compared to the original model. For testing and validating the new model in hardware state, Xilinx Spartan-3 FPGA board has been considered and used. Hardware results show the high-degree of similarity between the original and proposed models (in terms of neuronal behaviors) and also higher frequency and low-cost condition have been achieved. The implementation results show that the overall saving is more than other papers and also the original model. Moreover, frequency of the proposed neuronal model is about 168 MHz, which is significantly higher than the original model frequency, 63 MHz.

Astrocytes mediate analogous memory in a multi-layer neuron–astrocyte network

Modeling the neuronal processes underlying short-term working memory remains the focus of many theoretical studies in neuroscience. In this paper, we propose a mathematical model of a spiking neural network (SNN) which simulates the way a fragment of information is maintained as a robust activity pattern for several seconds and the way it completely disappears if no other stimuli are fed to the system. Such short-term memory traces are preserved due to the activation of astrocytes accompanying the SNN. The astrocytes exhibit calcium transients at a time scale of seconds. These transients further modulate the efficiency of synaptic transmission and, hence, the firing rate of neighboring neurons at diverse timescales through gliotransmitter release. We demonstrate how such transients continuously encode frequencies of neuronal discharges and provide robust short-term storage of analogous information. This kind of short-term memory can store relevant information for seconds and then completely forget it to avoid overlapping with forthcoming patterns. The SNN is inter-connected with the astrocytic layer by local inter-cellular diffusive connections. The astrocytes are activated only when the neighboring neurons fire synchronously, e.g., when an information pattern is loaded. For illustration, we took grayscale photographs of people’s faces where the shades of gray correspond to the level of applied current which stimulates the neurons. The astrocyte feedback modulates (facilitates) synaptic transmission by varying the frequency of neuronal firing. We show how arbitrary patterns can be loaded, then stored for a certain interval of time, and retrieved if the appropriate clue pattern is applied to the input.

Emulation of Astrocyte Induced Neural Phase Synchrony in Spin-Orbit Torque Oscillator Neurons

Astrocytes play a central role in inducing concerted phase synchronized neural-wave patterns inside the brain. In this article, we demonstrate that injected radio-frequency signal in underlying heavy metal layer of spin-orbit torque oscillator neurons mimic the neuron phase synchronization effect realized by glial cells. Potential application of such phase coupling effects is illustrated in the context of a temporal "binding problem." We also present the design of a coupled neuron-synapse-astrocyte network enabled by compact neuromimetic devices by combining the concepts of local spike-timing dependent plasticity and astrocyte induced neural phase synchrony.

Entropy estimation within in vitro neural-astrocyte networks as a measure of development instability

The brain demands a significant fraction of the energy budget in an organism; in humans, it accounts for 2% of the body mass, but utilizes 20% of the total energy metabolized. This is due to the large load required for information processing; spiking demands from neurons are high but are a key component to understanding brain functioning. Astrocytic brain cells contribute to the healthy functioning of brain circuits by mediating neuronal network energy and facilitating the formation and stabilization of synaptic connectivity. During development, spontaneous activity influences synaptic formation, shaping brain circuit construction, and adverse astrocyte mutations can lead to pathological processes impacting cognitive impairment due to inefficiencies in network spiking activity. We have developed a measure that quantifies information stability within in vitro networks consisting of mixed neural-astrocyte cells. Brain cells were harvested from mice with mutations to a gene associated with the strongest known genetic risk factor for Alzheimer's disease, APOE. We calculate energy states of the networks and using these states, we present an entropy-based measure to assess changes in information stability over time. We show that during development, stability profiles of spontaneous network activity are modified by exogenous astrocytes and that network stability, in terms of the rate of change of entropy, is allele dependent.

On the Self-Repair Role of Astrocytes in STDP Enabled Unsupervised SNNs

Neuromorphic computing is emerging to be a disruptive computational paradigm that attempts to emulate various facets of the underlying structure and functionalities of the brain in the algorithm and hardware design of next-generation machine learning platforms. This work goes beyond the focus of current neuromorphic computing architectures on computational models for neuron and synapse to examine other computational units of the biological brain that might contribute to cognition and especially self-repair. We draw inspiration and insights from computational neuroscience regarding functionalities of glial cells and explore their role in the fault-tolerant capacity of Spiking Neural Networks (SNNs) trained in an unsupervised fashion using Spike-Timing Dependent Plasticity (STDP). We characterize the degree of self-repair that can be enabled in such networks with varying degree of faults ranging from 50 to 90% and evaluate our proposal on the MNIST and Fashion-MNIST datasets.

Information Transmitted From Bioinspired Neuron-Astrocyte Network Improves Cortical Spiking Network's Pattern Recognition Performance

We trained two spiking neural networks (SNNs), the cortical spiking network (CSN) and the cortical neuron-astrocyte network (CNAN), using a spike-based unsupervised method, on the MNIST and alpha-digit data sets and achieve an accuracy of 96.1% and 77.35%, respectively. We then connected CNAN to CSN by preserving maximum synchronization between them thanks to the concept of prolate spheroidal wave functions (PSWF). As a result, CSN receives additional information from CNAN without retraining. The important outcome is that CSN reaches 70.57% correct classification rate on capital letters without being trained on them. The overall contribution of transfer is 87.47%. We observed that for CSN the classifying neurons that relate to digits 0-9 of the alpha-digit data set are completely supported by the ones that relate to digits 0-9 of the MNIST data set. This means that CSN recognizes the similarity between the digits of the MNIST and alpha-digit data sets and classifies each digit of both data sets in the same class.

Optimized Real-Time Biomimetic Neural Network on FPGA for Bio-hybridization

Neurological diseases can be studied by performing bio-hybrid experiments using a real-time biomimetic Spiking Neural Network (SNN) platform. The Hodgkin-Huxley model offers a set of equations including biophysical parameters which can serve as a base to represent different classes of neurons and affected cells. Also, connecting the artificial neurons to the biological cells would allow us to understand the effect of the SNN stimulation using different parameters on nerve cells. Thus, designing a real-time SNN could useful for the study of simulations of some part of the brain. Here, we present a different approach to optimize the Hodgkin-Huxley equations adapted for Field Programmable Gate Array (FPGA) implementation. The equations of the conductance have been unified to allow the use of same functions with different parameters for all ionic channels. The low resources and high-speed implementation also include features, such as synaptic noise using the Ornstein-Uhlenbeck process and different synapse receptors including AMPA, GABAa, GABAb, and NMDA receptors. The platform allows real-time modification of the neuron parameters and can output different cortical neuron families like Fast Spiking (FS), Regular Spiking (RS), Intrinsically Bursting (IB), and Low Threshold Spiking (LTS) neurons using a Digital to Analog Converter (DAC). Gaussian distribution of the synaptic noise highlights similarities with the biological noise. Also, cross-correlation between the implementation and the model shows strong correlations, and bifurcation analysis reproduces similar behavior compared to the original Hodgkin-Huxley model. The implementation of one core of calculation uses 3% of resources of the FPGA and computes in real-time 500 neurons with 25,000 synapses and synaptic noise which can be scaled up to 15,000 using all resources. This is the first step toward neuromorphic system which can be used for the simulation of bio-hybridization and for the study of neurological disorders or the advanced research on neuroprosthesis to regain lost function.

A Digital Hardware Realization for Spiking Model of Cutaneous Mechanoreceptor

Inspired by the biology of human tactile perception, a hardware neuromorphic approach is proposed for spiking model of mechanoreceptors to encode the input force. In this way, a digital circuit is designed for a slowly adapting type I (SA-I) and fast adapting type I (FA-I) mechanoreceptors to be implemented on a low-cost digital hardware, such as field-programmable gate array (FPGA). This system computationally replicates the neural firing responses of both afferents. Then, comparative simulations are shown. The spiking models of mechanoreceptors are first simulated in MATLAB and next the digital neuromorphic circuits simulated in VIVADO are also compared to show that obtained results are in good agreement both quantitatively and qualitatively. Finally, we test the performance of the proposed digital mechanoreceptors in hardware using a prepared experimental set up. Hardware synthesis and physical realization on FPGA indicate that the digital mechanoreceptors are able to replicate essential characteristics of different firing patterns including bursting and spiking responses of the SA-I and FA-I mechanoreceptors. In addition to parallel computation, a main advantage of this method is that the mechanoreceptor digital circuits can be implemented in real-time through low-power neuromorphic hardware. This novel engineering framework is generally suitable for use in robotic and hand-prosthetic applications, so progressing the state of the art for tactile sensing.

Computational Models for Calcium-Mediated Astrocyte Functions

The computational neuroscience field has heavily concentrated on the modeling of neuronal functions, largely ignoring other brain cells, including one type of glial cell, the astrocytes. Despite the short history of modeling astrocytic functions, we were delighted about the hundreds of models developed so far to study the role of astrocytes, most often in calcium dynamics, synchronization, information transfer, and plasticity in vitro, but also in vascular events, hyperexcitability, and homeostasis. Our goal here is to present the state-of-the-art in computational modeling of astrocytes in order to facilitate better understanding of the functions and dynamics of astrocytes in the brain. Due to the large number of models, we concentrated on a hundred models that include biophysical descriptions for calcium signaling and dynamics in astrocytes. We categorized the models into four groups: single astrocyte models, astrocyte network models, neuron-astrocyte synapse models, and neuron-astrocyte network models to ease their use in future modeling projects. We characterized the models based on which earlier models were used for building the models and which type of biological entities were described in the astrocyte models. Features of the models were compared and contrasted so that similarities and differences were more readily apparent. We discovered that most of the models were basically generated from a small set of previously published models with small variations. However, neither citations to all the previous models with similar core structure nor explanations of what was built on top of the previous models were provided, which made it possible, in some cases, to have the same models published several times without an explicit intention to make new predictions about the roles of astrocytes in brain functions. Furthermore, only a few of the models are available online which makes it difficult to reproduce the simulation results and further develop the models. Thus, we would like to emphasize that only via reproducible research are we able to build better computational models for astrocytes, which truly advance science. Our study is the first to characterize in detail the biophysical and biochemical mechanisms that have been modeled for astrocytes.

SPANNER: A Self-Repairing Spiking Neural Network Hardware Architecture

Recent research has shown that a glial cell of astrocyte underpins a self-repair mechanism in the human brain, where spiking neurons provide direct and indirect feedbacks to presynaptic terminals. These feedbacks modulate the synaptic transmission probability of release (PR). When synaptic faults occur, the neuron becomes silent or near silent due to the low PR of synapses; whereby the PRs of remaining healthy synapses are then increased by the indirect feedback from the astrocyte cell. In this paper, a novel hardware architecture of Self-rePAiring spiking Neural NEtwoRk (SPANNER) is proposed, which mimics this self-repairing capability in the human brain. This paper demonstrates that the hardware can self-detect and self-repair synaptic faults without the conventional components for the fault detection and fault repairing. Experimental results show that SPANNER can maintain the system performance with fault densities of up to 40%, and more importantly SPANNER has only a 20% performance degradation when the self-repairing architecture is significantly damaged at a fault density of 80%.

Digital Implementation of the Two-Compartmental Pinsky-Rinzel Pyramidal Neuron Model

It is believed that brain-like computing system can be achieved by the fusion of electronics and neuroscience. In this way, the optimized digital hardware implementation of neurons, primary units of nervous system, play a vital role in neuromorphic applications. Moreover, one of the main features of pyramidal neurons in cortical areas is bursting activities that has a critical role in synaptic plasticity. The Pinsky-Rinzel model is a nonlinear two-compartmental model for CA3 pyramidal cell that is widely used in neuroscience. In this paper, a modified Pinsky-Rinzel pyramidal model is proposed by replacing its complex nonlinear equations with piecewise linear approximation. Next, a digital circuit is designed for the simplified model to be able to implement on a low-cost digital hardware, such as field-programmable gate array (FPGA). Both original and proposed models are simulated in MATLAB and next digital circuit simulated in Vivado is compared to show that obtained results are in good agreement. Finally, the results of physical implementation on FPGA are also illustrated. The presented circuit advances preceding designs with regards to the ability to replicate essential characteristics of different firing responses including bursting and spiking in the compartmental model. This new circuit has various applications in neuromorphic engineering, such as developing new neuroinspired chips.

Complete Neuron-Astrocyte Interaction Model: Digital Multiplierless Design and Networking Mechanism

Glial cells, also known as neuroglia or glia, are non-neuronal cells providing support and protection for neurons in the central nervous system (CNS). They also act as supportive cells in the brain. Among a variety of glial cells, the star-shaped glial cells, i.e., astrocytes, are the largest cell population in the brain. The important role of astrocyte such as neuronal synchronization, synaptic information regulation, feedback to neural activity and extracellular regulation make the astrocytes play a vital role in brain disease. This paper presents a modified complete neuron-astrocyte interaction model that is more suitable for efficient and large scale biological neural network realization on digital platforms. Simulation results show that the modified complete interaction model can reproduce biological-like behavior of the original neuron-astrocyte mechanism. The modified interaction model is investigated in terms of digital realization feasibility and cost targeting a low cost hardware implementation. Networking behavior of this interaction is investigated and compared between two cases: i) the neuron spiking mechanism without astrocyte effects, and ii) the effect of astrocyte in regulating the neurons behavior and synaptic transmission via controlling the LTP and LTD processes. Hardware implementation on FPGA shows that the modified model mimics the main mechanism of neuron-astrocyte communication with higher performance and considerably lower hardware overhead cost compared with the original interaction model.

A new bio-inspired stimulator to suppress hyper-synchronized neural firing in a cortical network

Hyper-synchronous neural oscillations are the character of several neurological diseases such as epilepsy. On the other hand, glial cells and particularly astrocytes can influence neural synchronization. Therefore, based on the recent researches, a new bio-inspired stimulator is proposed which basically is a dynamical model of the astrocyte biophysical model. The performance of the new stimulator is investigated on a large-scale, cortical network. Both excitatory and inhibitory synapses are also considered in the simulated spiking neural network. The simulation results show that the new stimulator has a good performance and is able to reduce recurrent abnormal excitability which in turn avoids the hyper-synchronous neural firing in the spiking neural network. In this way, the proposed stimulator has a demand controlled characteristic and is a good candidate for deep brain stimulation (DBS) technique to successfully suppress the neural hyper-synchronization.

Deep Artificial Neural Networks and Neuromorphic Chips for Big Data Analysis: Pharmaceutical and Bioinformatics Applications

Over the past decade, Deep Artificial Neural Networks (DNNs) have become the state-of-the-art algorithms in Machine Learning (ML), speech recognition, computer vision, natural language processing and many other tasks. This was made possible by the advancement in Big Data, Deep Learning (DL) and drastically increased chip processing abilities, especially general-purpose graphical processing units (GPGPUs). All this has created a growing interest in making the most of the potential offered by DNNs in almost every field. An overview of the main architectures of DNNs, and their usefulness in Pharmacology and Bioinformatics are presented in this work. The featured applications are: drug design, virtual screening (VS), Quantitative Structure-Activity Relationship (QSAR) research, protein structure prediction and genomics (and other omics) data mining. The future need of neuromorphic hardware for DNNs is also discussed, and the two most advanced chips are reviewed: IBM TrueNorth and SpiNNaker. In addition, this review points out the importance of considering not only neurons, as DNNs and neuromorphic chips should also include glial cells, given the proven importance of astrocytes, a type of glial cell which contributes to information processing in the brain. The Deep Artificial Neuron-Astrocyte Networks (DANAN) could overcome the difficulties in architecture design, learning process and scalability of the current ML methods.