1
|
Shinji Y, Okuno H, Hirata Y. Artificial cerebellum on FPGA: realistic real-time cerebellar spiking neural network model capable of real-world adaptive motor control. Front Neurosci 2024; 18:1220908. [PMID: 38726031 PMCID: PMC11079192 DOI: 10.3389/fnins.2024.1220908] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 04/09/2024] [Indexed: 05/12/2024] Open
Abstract
The cerebellum plays a central role in motor control and learning. Its neuronal network architecture, firing characteristics of component neurons, and learning rules at their synapses have been well understood in terms of anatomy and physiology. A realistic artificial cerebellum with mimetic network architecture and synaptic plasticity mechanisms may allow us to analyze cerebellar information processing in the real world by applying it to adaptive control of actual machines. Several artificial cerebellums have previously been constructed, but they require high-performance hardware to run in real-time for real-world machine control. Presently, we implemented an artificial cerebellum with the size of 104 spiking neuron models on a field-programmable gate array (FPGA) which is compact, lightweight, portable, and low-power-consumption. In the implementation three novel techniques are employed: (1) 16-bit fixed-point operation and randomized rounding, (2) fully connected spike information transmission, and (3) alternative memory that uses pseudo-random number generators. We demonstrate that the FPGA artificial cerebellum runs in real-time, and its component neuron models behave as those in the corresponding artificial cerebellum configured on a personal computer in Python. We applied the FPGA artificial cerebellum to the adaptive control of a machine in the real world and demonstrated that the artificial cerebellum is capable of adaptively reducing control error after sudden load changes. This is the first implementation and demonstration of a spiking artificial cerebellum on an FPGA applicable to real-world adaptive control. The FPGA artificial cerebellum may provide neuroscientific insights into cerebellar information processing in adaptive motor control and may be applied to various neuro-devices to augment and extend human motor control capabilities.
Collapse
Affiliation(s)
- Yusuke Shinji
- Department of Computer Science, Graduate School of Engineering, Chubu University, Kasugai, Japan
| | - Hirotsugu Okuno
- Faculty of Information Science and Technology, Osaka Institute of Technology, Hirakata, Japan
| | - Yutaka Hirata
- Department of Artificial Intelligence and Robotics, College of Engineering, Chubu University, Kasugai, Japan
- Center for Mathematical Science and Artificial Intelligence, Chubu University, Kasugai, Japan
- Academy of Emerging Sciences, Chubu University, Kasugai, Japan
| |
Collapse
|
2
|
Kumar G, Zhou Z, Wang Z, Kwan KM, Tin C, Ma CHE. Real-time field-programmable gate array-based closed-loop deep brain stimulation platform targeting cerebellar circuitry rescues motor deficits in a mouse model of cerebellar ataxia. CNS Neurosci Ther 2024; 30:e14638. [PMID: 38488445 PMCID: PMC10941591 DOI: 10.1111/cns.14638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 01/09/2024] [Accepted: 02/01/2024] [Indexed: 03/18/2024] Open
Abstract
AIMS The open-loop nature of conventional deep brain stimulation (DBS) produces continuous and excessive stimulation to patients which contributes largely to increased prevalence of adverse side effects. Cerebellar ataxia is characterized by abnormal Purkinje cells (PCs) dendritic arborization, loss of PCs and motor coordination, and muscle weakness with no effective treatment. We aim to develop a real-time field-programmable gate array (FPGA) prototype targeting the deep cerebellar nuclei (DCN) to close the loop for ataxia using conditional double knockout mice with deletion of PC-specific LIM homeobox (Lhx)1 and Lhx5, resulting in abnormal dendritic arborization and motor deficits. METHODS We implanted multielectrode array in the DCN and muscles of ataxia mice. The beneficial effect of open-loop DCN-DBS or closed-loop DCN-DBS was compared by motor behavioral assessments, electromyography (EMG), and neural activities (neurospike and electroencephalogram) in freely moving mice. FPGA board, which performed complex real-time computation, was used for closed-loop DCN-DBS system. RESULTS Closed-loop DCN-DBS was triggered only when symptomatic muscle EMG was detected in a real-time manner, which restored motor activities, electroencephalogram activities and neurospike properties completely in ataxia mice. Closed-loop DCN-DBS was more effective than an open-loop paradigm as it reduced the frequency of DBS. CONCLUSION Our real-time FPGA-based DCN-DBS system could be a potential clinical strategy for alleviating cerebellar ataxia and other movement disorders.
Collapse
Affiliation(s)
- Gajendra Kumar
- Department of NeuroscienceCity University of Hong KongHong KongHong Kong SAR
| | - Zhanhong Zhou
- Department of Biomedical EngineeringCity University of Hong KongHong KongHong Kong SAR
| | - Zhihua Wang
- Department of Biomedical EngineeringCity University of Hong KongHong KongHong Kong SAR
| | - Kin Ming Kwan
- School of Life Sciences, Center for Cell and Developmental Biology and State Key Laboratory of AgrobiotechnologyThe Chinese University of Hong KongHong KongHong Kong SAR
| | - Chung Tin
- Department of Biomedical EngineeringCity University of Hong KongHong KongHong Kong SAR
| | - Chi Him Eddie Ma
- Department of NeuroscienceCity University of Hong KongHong KongHong Kong SAR
| |
Collapse
|
3
|
Deng B, Fan Y, Wang J, Yang S. Auditory perception architecture with spiking neural network and implementation on FPGA. Neural Netw 2023; 165:31-42. [PMID: 37276809 DOI: 10.1016/j.neunet.2023.05.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 05/07/2023] [Accepted: 05/16/2023] [Indexed: 06/07/2023]
Abstract
Spike-based perception brings up a new research idea in the field of neuromorphic engineering. A high-performance biologically inspired flexible spiking neural network (SNN) architecture provides a novel method for the exploration of perception mechanisms and the development of neuromorphic computing systems . In this article, we present a biological-inspired spike-based SNN perception digital system that can realize robust perception. The system employs a fully paralleled pipeline scheme to improve the performance and accelerate the processing of feature extraction. An auditory perception system prototype is realized on ten Intel Cyclone field-programmable gate arrays, which can reach the maximum frequency of 107.28 MHz and the maximum throughput of 5364 Mbps. Our design also achieves the power of 5. 148 W/system and energy efficiency of 845.85 μJ. Our auditory perception implementation is also proved to have superior robustness compared with other SNN systems. We use TIMIT digit speech in noise in accuracy testing. Result shows that it achieves up to 85.75% speech recognition accuracy under obvious noise conditions (signal-to-noise ratio of 20 dB) and maintain small accuracy attenuation with the decline of the signal-to-noise ratio. The overall performance of our proposed system outperforms the state-of-the-art perception system on SNN.
Collapse
Affiliation(s)
- Bin Deng
- School of Electrical and Information Engineering, Tianjin University, China
| | - Yanrong Fan
- School of Electrical and Information Engineering, Tianjin University, China
| | - Jiang Wang
- School of Electrical and Information Engineering, Tianjin University, China
| | - Shuangming Yang
- School of Electrical and Information Engineering, Tianjin University, China.
| |
Collapse
|
4
|
Yang S, Wang J, Deng B, Azghadi MR, Linares-Barranco B. Neuromorphic Context-Dependent Learning Framework With Fault-Tolerant Spike Routing. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:7126-7140. [PMID: 34115596 DOI: 10.1109/tnnls.2021.3084250] [Citation(s) in RCA: 54] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Neuromorphic computing is a promising technology that realizes computation based on event-based spiking neural networks (SNNs). However, fault-tolerant on-chip learning remains a challenge in neuromorphic systems. This study presents the first scalable neuromorphic fault-tolerant context-dependent learning (FCL) hardware framework. We show how this system can learn associations between stimulation and response in two context-dependent learning tasks from experimental neuroscience, despite possible faults in the hardware nodes. Furthermore, we demonstrate how our novel fault-tolerant neuromorphic spike routing scheme can avoid multiple fault nodes successfully and can enhance the maximum throughput of the neuromorphic network by 0.9%-16.1% in comparison with previous studies. By utilizing the real-time computational capabilities and multiple-fault-tolerant property of the proposed system, the neuronal mechanisms underlying the spiking activities of neuromorphic networks can be readily explored. In addition, the proposed system can be applied in real-time learning and decision-making applications, brain-machine integration, and the investigation of brain cognition during learning.
Collapse
|
5
|
Vijayan A, Diwakar S. A cerebellum inspired spiking neural network as a multi-model for pattern classification and robotic trajectory prediction. Front Neurosci 2022; 16:909146. [DOI: 10.3389/fnins.2022.909146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 11/02/2022] [Indexed: 11/29/2022] Open
Abstract
Spiking neural networks were introduced to understand spatiotemporal information processing in neurons and have found their application in pattern encoding, data discrimination, and classification. Bioinspired network architectures are considered for event-driven tasks, and scientists have looked at different theories based on the architecture and functioning. Motor tasks, for example, have networks inspired by cerebellar architecture where the granular layer recodes sparse representations of the mossy fiber (MF) inputs and has more roles in motor learning. Using abstractions from cerebellar connections and learning rules of deep learning network (DLN), patterns were discriminated within datasets, and the same algorithm was used for trajectory optimization. In the current work, a cerebellum-inspired spiking neural network with dynamics of cerebellar neurons and learning mechanisms attributed to the granular layer, Purkinje cell (PC) layer, and cerebellar nuclei interconnected by excitatory and inhibitory synapses was implemented. The model’s pattern discrimination capability was tested for two tasks on standard machine learning (ML) datasets and on following a trajectory of a low-cost sensor-free robotic articulator. Tuned for supervised learning, the pattern classification capability of the cerebellum-inspired network algorithm has produced more generalized models than data-specific precision models on smaller training datasets. The model showed an accuracy of 72%, which was comparable to standard ML algorithms, such as MLP (78%), Dl4jMlpClassifier (64%), RBFNetwork (71.4%), and libSVM-linear (85.7%). The cerebellar model increased the network’s capability and decreased storage, augmenting faster computations. Additionally, the network model could also implicitly reconstruct the trajectory of a 6-degree of freedom (DOF) robotic arm with a low error rate by reconstructing the kinematic parameters. The variability between the actual and predicted trajectory points was noted to be ± 3 cm (while moving to a position in a cuboid space of 25 × 30 × 40 cm). Although a few known learning rules were implemented among known types of plasticity in the cerebellum, the network model showed a generalized processing capability for a range of signals, modulating the data through the interconnected neural populations. In addition to potential use on sensor-free or feed-forward based controllers for robotic arms and as a generalized pattern classification algorithm, this model adds implications to motor learning theory.
Collapse
|
6
|
Zhao Z, Wang Y, Zou Q, Xu T, Tao F, Zhang J, Wang X, Shi CJR, Luo J, Xie Y. The spike gating flow: A hierarchical structure-based spiking neural network for online gesture recognition. Front Neurosci 2022; 16:923587. [PMID: 36408382 PMCID: PMC9667043 DOI: 10.3389/fnins.2022.923587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Accepted: 10/03/2022] [Indexed: 01/25/2023] Open
Abstract
Action recognition is an exciting research avenue for artificial intelligence since it may be a game changer in emerging industrial fields such as robotic visions and automobiles. However, current deep learning (DL) faces major challenges for such applications because of the huge computational cost and inefficient learning. Hence, we developed a novel brain-inspired spiking neural network (SNN) based system titled spiking gating flow (SGF) for online action learning. The developed system consists of multiple SGF units which are assembled in a hierarchical manner. A single SGF unit contains three layers: a feature extraction layer, an event-driven layer, and a histogram-based training layer. To demonstrate the capability of the developed system, we employed a standard dynamic vision sensor (DVS) gesture classification as a benchmark. The results indicated that we can achieve 87.5% of accuracy which is comparable with DL, but at a smaller training/inference data number ratio of 1.5:1. Only a single training epoch is required during the learning process. Meanwhile, to the best of our knowledge, this is the highest accuracy among the non-backpropagation based SNNs. Finally, we conclude the few-shot learning (FSL) paradigm of the developed network: 1) a hierarchical structure-based network design involves prior human knowledge; 2) SNNs for content-based global dynamic feature detection.
Collapse
Affiliation(s)
- Zihao Zhao
- School of Microelectronics, Fudan University, Shanghai, China,Alibaba DAMO Academy, Shanghai, China
| | - Yanhong Wang
- School of Microelectronics, Fudan University, Shanghai, China,Alibaba DAMO Academy, Shanghai, China
| | - Qiaosha Zou
- School of Microelectronics, Fudan University, Shanghai, China
| | - Tie Xu
- Alibaba Group, Hangzhou, China
| | | | | | - Xiaoan Wang
- BrainUp Research Laboratory, Shanghai, China
| | - C.-J. Richard Shi
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, United States
| | - Junwen Luo
- Alibaba DAMO Academy, Shanghai, China,BrainUp Research Laboratory, Shanghai, China,*Correspondence: Junwen Luo
| | - Yuan Xie
- Alibaba DAMO Academy, Shanghai, China
| |
Collapse
|
7
|
Yang S, Wang J, Zhang N, Deng B, Pang Y, Azghadi MR. CerebelluMorphic: Large-Scale Neuromorphic Model and Architecture for Supervised Motor Learning. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:4398-4412. [PMID: 33621181 DOI: 10.1109/tnnls.2021.3057070] [Citation(s) in RCA: 37] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
The cerebellum plays a vital role in motor learning and control with supervised learning capability, while neuromorphic engineering devises diverse approaches to high-performance computation inspired by biological neural systems. This article presents a large-scale cerebellar network model for supervised learning, as well as a cerebellum-inspired neuromorphic architecture to map the cerebellar anatomical structure into the large-scale model. Our multinucleus model and its underpinning architecture contain approximately 3.5 million neurons, upscaling state-of-the-art neuromorphic designs by over 34 times. Besides, the proposed model and architecture incorporate 3411k granule cells, introducing a 284 times increase compared to a previous study including only 12k cells. This large scaling induces more biologically plausible cerebellar divergence/convergence ratios, which results in better mimicking biology. In order to verify the functionality of our proposed model and demonstrate its strong biomimicry, a reconfigurable neuromorphic system is used, on which our developed architecture is realized to replicate cerebellar dynamics during the optokinetic response. In addition, our neuromorphic architecture is used to analyze the dynamical synchronization within the Purkinje cells, revealing the effects of firing rates of mossy fibers on the resonance dynamics of Purkinje cells. Our experiments show that real-time operation can be realized, with a system throughput of up to 4.70 times larger than previous works with high synaptic event rate. These results suggest that the proposed work provides both a theoretical basis and a neuromorphic engineering perspective for brain-inspired computing and the further exploration of cerebellar learning.
Collapse
|
8
|
Yamazaki T, Igarashi J, Yamaura H. Human-scale Brain Simulation via Supercomputer: A Case Study on the Cerebellum. Neuroscience 2021; 462:235-246. [PMID: 33482329 DOI: 10.1016/j.neuroscience.2021.01.014] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2020] [Revised: 12/30/2020] [Accepted: 01/06/2021] [Indexed: 01/03/2023]
Abstract
Performance of supercomputers has been steadily and exponentially increasing for the past 20 years, and is expected to increase further. This unprecedented computational power enables us to build and simulate large-scale neural network models composed of tens of billions of neurons and tens of trillions of synapses with detailed anatomical connections and realistic physiological parameters. Such "human-scale" brain simulation could be considered a milestone in computational neuroscience and even in general neuroscience. Towards this milestone, it is mandatory to introduce modern high-performance computing technology into neuroscience research. In this article, we provide an introductory landscape about large-scale brain simulation on supercomputers from the viewpoints of computational neuroscience and modern high-performance computing technology for specialists in experimental as well as computational neurosciences. This introduction to modeling and simulation methods is followed by a review of various representative large-scale simulation studies conducted to date. Then, we direct our attention to the cerebellum, with a review of more simulation studies specific to that region. Furthermore, we present recent simulation results of a human-scale cerebellar network model composed of 86 billion neurons on the Japanese flagship supercomputer K (now retired). Finally, we discuss the necessity and importance of human-scale brain simulation, and suggest future directions of such large-scale brain simulation research.
Collapse
Affiliation(s)
- Tadashi Yamazaki
- Graduate School of Informatics and Engineering, The University of Electro-Communications, Japan.
| | | | - Hiroshi Yamaura
- Graduate School of Informatics and Engineering, The University of Electro-Communications, Japan
| |
Collapse
|
9
|
Luo J, Firflionis D, Turnbull M, Xu W, Walsh D, Escobedo-Cousin E, Soltan A, Ramezani R, Liu Y, Bailey R, ONeill A, Idil AS, Donaldson N, Constandinou T, Jackson A, Degenaar P. The Neural Engine: A Reprogrammable Low Power Platform for Closed-Loop Optogenetics. IEEE Trans Biomed Eng 2020; 67:3004-3015. [PMID: 32091984 DOI: 10.1109/tbme.2020.2973934] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Brain-machine Interfaces (BMI) hold great potential for treating neurological disorders such as epilepsy. Technological progress is allowing for a shift from open-loop, pacemaker-class, intervention towards fully closed-loop neural control systems. Low power programmable processing systems are therefore required which can operate within the thermal window of 2° C for medical implants and maintain long battery life. In this work, we have developed a low power neural engine with an optimized set of algorithms which can operate under a power cycling domain. We have integrated our system with a custom-designed brain implant chip and demonstrated the operational applicability to the closed-loop modulating neural activities in in-vitro and in-vivo brain tissues: the local field potentials can be modulated at required central frequency ranges. Also, both a freely-moving non-human primate (24-hour) and a rodent (1-hour) in-vivo experiments were performed to show system reliable recording performance. The overall system consumes only 2.93 mA during operation with a biological recording frequency 50 Hz sampling rate (the lifespan is approximately 56 hours). A library of algorithms has been implemented in terms of detection, suppression and optical intervention to allow for exploratory applications in different neurological disorders. Thermal experiments demonstrated that operation creates minimal heating as well as battery performance exceeding 24 hours on a freely moving rodent. Therefore, this technology shows great capabilities for both neuroscience in-vitro/in-vivo applications and medical implantable processing units.
Collapse
|
10
|
Yang S, Deng B, Wang J, Li H, Lu M, Che Y, Wei X, Loparo KA. Scalable Digital Neuromorphic Architecture for Large-Scale Biophysically Meaningful Neural Network With Multi-Compartment Neurons. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:148-162. [PMID: 30892250 DOI: 10.1109/tnnls.2019.2899936] [Citation(s) in RCA: 83] [Impact Index Per Article: 20.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
Multicompartment emulation is an essential step to enhance the biological realism of neuromorphic systems and to further understand the computational power of neurons. In this paper, we present a hardware efficient, scalable, and real-time computing strategy for the implementation of large-scale biologically meaningful neural networks with one million multi-compartment neurons (CMNs). The hardware platform uses four Altera Stratix III field-programmable gate arrays, and both the cellular and the network levels are considered, which provides an efficient implementation of a large-scale spiking neural network with biophysically plausible dynamics. At the cellular level, a cost-efficient multi-CMN model is presented, which can reproduce the detailed neuronal dynamics with representative neuronal morphology. A set of efficient neuromorphic techniques for single-CMN implementation are presented with all the hardware cost of memory and multiplier resources removed and with hardware performance of computational speed enhanced by 56.59% in comparison with the classical digital implementation method. At the network level, a scalable network-on-chip (NoC) architecture is proposed with a novel routing algorithm to enhance the NoC performance including throughput and computational latency, leading to higher computational efficiency and capability in comparison with state-of-the-art projects. The experimental results demonstrate that the proposed work can provide an efficient model and architecture for large-scale biologically meaningful networks, while the hardware synthesis results demonstrate low area utilization and high computational speed that supports the scalability of the approach.
Collapse
|
11
|
Xu T, Xiao N, Zhai X, Kwan Chan P, Tin C. Real-time cerebellar neuroprosthetic system based on a spiking neural network model of motor learning. J Neural Eng 2019; 15:016021. [PMID: 29115280 DOI: 10.1088/1741-2552/aa98e9] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Damage to the brain, as a result of various medical conditions, impacts the everyday life of patients and there is still no complete cure to neurological disorders. Neuroprostheses that can functionally replace the damaged neural circuit have recently emerged as a possible solution to these problems. Here we describe the development of a real-time cerebellar neuroprosthetic system to substitute neural function in cerebellar circuitry for learning delay eyeblink conditioning (DEC). APPROACH The system was empowered by a biologically realistic spiking neural network (SNN) model of the cerebellar neural circuit, which considers the neuronal population and anatomical connectivity of the network. The model simulated synaptic plasticity critical for learning DEC. This SNN model was carefully implemented on a field programmable gate array (FPGA) platform for real-time simulation. This hardware system was interfaced in in vivo experiments with anesthetized rats and it used neural spikes recorded online from the animal to learn and trigger conditioned eyeblink in the animal during training. MAIN RESULTS This rat-FPGA hybrid system was able to process neuronal spikes in real-time with an embedded cerebellum model of ~10 000 neurons and reproduce learning of DEC with different inter-stimulus intervals. Our results validated that the system performance is physiologically relevant at both the neural (firing pattern) and behavioral (eyeblink pattern) levels. SIGNIFICANCE This integrated system provides the sufficient computation power for mimicking the cerebellar circuit in real-time. The system interacts with the biological system naturally at the spike level and can be generalized for including other neural components (neuron types and plasticity) and neural functions for potential neuroprosthetic applications.
Collapse
Affiliation(s)
- Tao Xu
- Department of Mechanical and Biomedical Engineering, City University of Hong Kong, Hong Kong, SAR, People's Republic of China
| | | | | | | | | |
Collapse
|
12
|
Yang S, Wang J, Deng B, Liu C, Li H, Fietkiewicz C, Loparo KA. Real-Time Neuromorphic System for Large-Scale Conductance-Based Spiking Neural Networks. IEEE TRANSACTIONS ON CYBERNETICS 2019; 49:2490-2503. [PMID: 29993922 DOI: 10.1109/tcyb.2018.2823730] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The investigation of the human intelligence, cognitive systems and functional complexity of human brain is significantly facilitated by high-performance computational platforms. In this paper, we present a real-time digital neuromorphic system for the simulation of large-scale conductance-based spiking neural networks (LaCSNN), which has the advantages of both high biological realism and large network scale. Using this system, a detailed large-scale cortico-basal ganglia-thalamocortical loop is simulated using a scalable 3-D network-on-chip (NoC) topology with six Altera Stratix III field-programmable gate arrays simulate 1 million neurons. Novel router architecture is presented to deal with the communication of multiple data flows in the multinuclei neural network, which has not been solved in previous NoC studies. At the single neuron level, cost-efficient conductance-based neuron models are proposed, resulting in the average utilization of 95% less memory resources and 100% less DSP resources for multiplier-less realization, which is the foundation of the large-scale realization. An analysis of the modified models is conducted, including investigation of bifurcation behaviors and ionic dynamics, demonstrating the required range of dynamics with a more reduced resource cost. The proposed LaCSNN system is shown to outperform the alternative state-of-the-art approaches previously used to implement the large-scale spiking neural network, and enables a broad range of potential applications due to its real-time computational power.
Collapse
|
13
|
Jokar E, Abolfathi H, Ahmadi A. A Novel Nonlinear Function Evaluation Approach for Efficient FPGA Mapping of Neuron and Synaptic Plasticity Models. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:454-469. [PMID: 30802873 DOI: 10.1109/tbcas.2019.2900943] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Efficient hardware realization of spiking neural networks is of great significance in a wide variety of applications, such as high-speed modeling and simulation of large-scale neural systems. Exploiting the key features of FPGAs, this paper presents a novel nonlinear function evaluation approach, based on an effective uniform piecewise linear segmentation method, to efficiently approximate the nonlinear terms of neuron and synaptic plasticity models targeting low-cost digital implementation. The proposed approach takes advantage of a high-speed and extremely simple segment address encoder unit regardless of the number of segments, and therefore is capable of accurately approximating a given nonlinear function with a large number of straight lines. In addition, this approach can be efficiently mapped into FPGAs with minimal hardware cost. To investigate the application of the proposed nonlinear function evaluation approach in low-cost neuromorphic circuit design, it is applied to four case studies: the Izhikevich and FitzHugh-Nagumo neuron models as 2-dimensional case studies, the Hindmarsh-Rose neuron model as a relatively complex 3-dimensional model containing two nonlinear terms, and a calcium-based synaptic plasticity model capable of producing various STDP curves. Simulation and FPGA synthesis results demonstrate that the hardware proposed for each case study is capable of producing various responses remarkably similar to the original model and significantly outperforms the previously published counterparts in terms of resource utilization and maximum clock frequency.
Collapse
|
14
|
Abstract
Mathematical models of brain function are built from data covering anatomy, physiology, biophysics and behavior. In almost all cases, many possible models could fit the available data. Theoreticians make assumptions that allow them to constrain the number of possible model structures. However, a model that was more useful clinically would result if the constraints came from lesion studies in animals or clinical disorders. Here, we show a few examples of how clinical disorders have led to improvements in models. We also show a few examples of how models could lead to neural prostheses for patients. The best outcomes result when clinicians, basic scientists and theoreticians work together to understand brain function.
Collapse
Affiliation(s)
- Lance M Optican
- Laboratory of Sensorimotor Research, NEI, NIH, DHHS, Bethesda, MD, United States.
| | - Elena Pretegiani
- Laboratory of Sensorimotor Research, NEI, NIH, DHHS, Bethesda, MD, United States
| |
Collapse
|
15
|
Zhang LQ, Yao J, Gao J, Sun L, Wang LT, Sui JF. Modulation of eyeblink conditioning through sensory processing of conditioned stimulus by cortical and subcortical regions. Behav Brain Res 2019; 359:149-155. [PMID: 30385367 DOI: 10.1016/j.bbr.2018.10.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2018] [Revised: 10/25/2018] [Accepted: 10/25/2018] [Indexed: 10/28/2022]
Abstract
Classical eyeblink conditioning (EBC) is one of the simplest forms of associative learning that depends critically on the cerebellum. Using delay EBC (dEBC), a standard paradigm in which the unconditioned stimulus (US) is delayed and co-terminates with the conditioned stimulus (CS), converging lines of evidence has been accumulated and shows that the essential neural circuit mediating EBC resides in the cerebellum and brainstem. In addition to this essential circuit, multiple cerebral cortical and subcortical structures are required to modulate dEBC with suboptimal training parameters, and trace EBC (tEBC) in which a trace-interval separates the CS and US. However, it remains largely unclear why and how so many brain regions are involved for modulation of EBC. Previous research has suggested that the forebrain regions, such as medial prefrontal cortex (mPFC) and hippocampus, may be required to process weak CSs, or to realize temporal overlap between the CS and US signal inputs when the two stimuli were separated in time (i.e. during tEBC). Here, we proposed a multi-level network model for EBC modulation which focuses on sensory processing of CS. The model explains how different neural pathways projecting to pontine nucleus (PN) are involved to amplify or extend CS through heterosynaptic facilitation mechanism or "substitution effect" under different circumstances to achieve EBC. As such, our model can serve as a general framework to explain the modulating mechanism of EBC in a variety of conditions and to help understand the interaction among cerebellum, brainstem, cortical and subcortical regions in EBC modulation.
Collapse
Affiliation(s)
- Lang-Qian Zhang
- Experimental Center of Basic Medicine, College of Basic Medical Sciences, Army Medical University, 30 Gaotanyan Street, Shapingba District, Chongqing 400038, PR China; Department of Medical Technology, Chongqing Medical and Pharmaceutical College, 82 University City Road, Shapingba District, Chongqing 401331, PR China
| | - Juan Yao
- Experimental Center of Basic Medicine, College of Basic Medical Sciences, Army Medical University, 30 Gaotanyan Street, Shapingba District, Chongqing 400038, PR China
| | - Jie Gao
- State Key Laboratory of Trauma, Burns and Combined Injury, Institute of Surgery Research, Daping Hospital, Army Medical University, Chongqing 400042, PR China
| | - Lin Sun
- Experimental Center of Basic Medicine, College of Basic Medical Sciences, Army Medical University, 30 Gaotanyan Street, Shapingba District, Chongqing 400038, PR China
| | - Li-Ting Wang
- Experimental Center of Basic Medicine, College of Basic Medical Sciences, Army Medical University, 30 Gaotanyan Street, Shapingba District, Chongqing 400038, PR China.
| | - Jian-Feng Sui
- Experimental Center of Basic Medicine, College of Basic Medical Sciences, Army Medical University, 30 Gaotanyan Street, Shapingba District, Chongqing 400038, PR China.
| |
Collapse
|
16
|
Frenkel C, Lefebvre M, Legat JD, Bol D. A 0.086-mm 2 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:145-158. [PMID: 30418919 DOI: 10.1109/tbcas.2018.2880425] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Shifting computing architectures from von Neumann to event-based spiking neural networks (SNNs) uncovers new opportunities for low-power processing of sensory data in applications such as vision or sensorimotor control. Exploring roads toward cognitive SNNs requires the design of compact, low-power and versatile experimentation platforms with the key requirement of online learning in order to adapt and learn new features in uncontrolled environments. However, embedding online learning in SNNs is currently hindered by high incurred complexity and area overheads. In this paper, we present ODIN, a 0.086-mm 2 64k-synapse 256-neuron online-learning digital spiking neuromorphic processor in 28-nm FDSOI CMOS achieving a minimum energy per synaptic operation (SOP) of 12.7 pJ. It leverages an efficient implementation of the spike-driven synaptic plasticity (SDSP) learning rule for high-density embedded online learning with only 0.68 μm 2 per 4-bit synapse. Neurons can be independently configured as a standard leaky integrate-and-fire model or as a custom phenomenological model that emulates the 20 Izhikevich behaviors found in biological spiking neurons. Using a single presentation of 6k 16 × 16 MNIST training images to a single-layer fully-connected 10-neuron network with on-chip SDSP-based learning, ODIN achieves a classification accuracy of 84.5%, while consuming only 15 nJ/inference at 0.55 V using rank order coding. ODIN thus enables further developments toward cognitive neuromorphic devices for low-power, adaptive and low-cost processing.
Collapse
|
17
|
Zjajo A, Hofmann J, Christiaanse GJ, van Eijk M, Smaragdos G, Strydis C, de Graaf A, Galuzzi C, van Leuken R. A Real-Time Reconfigurable Multichip Architecture for Large-Scale Biophysically Accurate Neuron Simulation. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2018; 12:326-337. [PMID: 29570060 DOI: 10.1109/tbcas.2017.2780287] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Simulation of brain neurons in real-time using biophysically meaningful models is a prerequisite for comprehensive understanding of how neurons process information and communicate with each other, in effect efficiently complementing in-vivo experiments. State-of-the-art neuron simulators are, however, capable of simulating at most few tens/hundreds of biophysically accurate neurons in real-time due to the exponential growth in the interneuron communication costs with the number of simulated neurons. In this paper, we propose a real-time, reconfigurable, multichip system architecture based on localized communication, which effectively reduces the communication cost to a linear growth. All parts of the system are generated automatically, based on the neuron connectivity scheme. Experimental results indicate that the proposed system architecture allows the capacity of over 3000 to 19 200 (depending on the connectivity scheme) biophysically accurate neurons over multiple chips.
Collapse
|
18
|
Rahimian E, Zabihi S, Amiri M, Linares-Barranco B. Digital Implementation of the Two-Compartmental Pinsky-Rinzel Pyramidal Neuron Model. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2018; 12:47-57. [PMID: 29028209 DOI: 10.1109/tbcas.2017.2753541] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
It is believed that brain-like computing system can be achieved by the fusion of electronics and neuroscience. In this way, the optimized digital hardware implementation of neurons, primary units of nervous system, play a vital role in neuromorphic applications. Moreover, one of the main features of pyramidal neurons in cortical areas is bursting activities that has a critical role in synaptic plasticity. The Pinsky-Rinzel model is a nonlinear two-compartmental model for CA3 pyramidal cell that is widely used in neuroscience. In this paper, a modified Pinsky-Rinzel pyramidal model is proposed by replacing its complex nonlinear equations with piecewise linear approximation. Next, a digital circuit is designed for the simplified model to be able to implement on a low-cost digital hardware, such as field-programmable gate array (FPGA). Both original and proposed models are simulated in MATLAB and next digital circuit simulated in Vivado is compared to show that obtained results are in good agreement. Finally, the results of physical implementation on FPGA are also illustrated. The presented circuit advances preceding designs with regards to the ability to replicate essential characteristics of different firing responses including bursting and spiking in the compartmental model. This new circuit has various applications in neuromorphic engineering, such as developing new neuroinspired chips.
Collapse
|
19
|
Sripad A, Sanchez G, Zapata M, Pirrone V, Dorta T, Cambria S, Marti A, Krishnamourthy K, Madrenas J. SNAVA-A real-time multi-FPGA multi-model spiking neural network simulation architecture. Neural Netw 2017; 97:28-45. [PMID: 29054036 DOI: 10.1016/j.neunet.2017.09.011] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2016] [Revised: 09/07/2017] [Accepted: 09/15/2017] [Indexed: 10/18/2022]
Abstract
Spiking Neural Networks (SNN) for Versatile Applications (SNAVA) simulation platform is a scalable and programmable parallel architecture that supports real-time, large-scale, multi-model SNN computation. This parallel architecture is implemented in modern Field-Programmable Gate Arrays (FPGAs) devices to provide high performance execution and flexibility to support large-scale SNN models. Flexibility is defined in terms of programmability, which allows easy synapse and neuron implementation. This has been achieved by using a special-purpose Processing Elements (PEs) for computing SNNs, and analyzing and customizing the instruction set according to the processing needs to achieve maximum performance with minimum resources. The parallel architecture is interfaced with customized Graphical User Interfaces (GUIs) to configure the SNN's connectivity, to compile the neuron-synapse model and to monitor SNN's activity. Our contribution intends to provide a tool that allows to prototype SNNs faster than on CPU/GPU architectures but significantly cheaper than fabricating a customized neuromorphic chip. This could be potentially valuable to the computational neuroscience and neuromorphic engineering communities.
Collapse
Affiliation(s)
- Athul Sripad
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Giovanny Sanchez
- Instituto Politecnico Nacional, ESIME Culhuacan, Av. Santa Ana N 1000, Coyoacan, 04260, Distrito Federal, Mexico.
| | - Mireya Zapata
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Vito Pirrone
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Taho Dorta
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Salvatore Cambria
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Albert Marti
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Karthikeyan Krishnamourthy
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| | - Jordi Madrenas
- Dept. of Electronics Engineering, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, edif. C4, 08034 Barcelona, Catalunya, Spain
| |
Collapse
|
20
|
Nikolic K, Evans BD, Andras P, Yakovlev A, Degenaar P. Optogenetics in Silicon: A Neural Processor for Predicting Optically Active Neural Networks. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2017; 11:15-27. [PMID: 28113518 DOI: 10.1109/tbcas.2016.2571339] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
We present a reconfigurable neural processor for real-time simulation and prediction of opto-neural behaviour. We combined a detailed Hodgkin-Huxley CA3 neuron integrated with a four-state Channelrhodopsin-2 (ChR2) model into reconfigurable silicon hardware. Our architecture consists of a Field Programmable Gated Array (FPGA) with a custom-built computing data-path, a separate data management system and a memory approach based router. Advancements over previous work include the incorporation of short and long-term calcium and light-dependent ion channels in reconfigurable hardware. Also, the developed processor is computationally efficient, requiring only 0.03 ms processing time per sub-frame for a single neuron and 9.7 ms for a fully connected network of 500 neurons with a given FPGA frequency of 56.7 MHz. It can therefore be utilized for exploration of closed loop processing and tuning of biologically realistic optogenetic circuitry.
Collapse
|