1
|
Yang S, Wang J, Deng B, Azghadi MR, Linares-Barranco B. Neuromorphic Context-Dependent Learning Framework With Fault-Tolerant Spike Routing. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:7126-7140. [PMID: 34115596 DOI: 10.1109/tnnls.2021.3084250] [Citation(s) in RCA: 50] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Neuromorphic computing is a promising technology that realizes computation based on event-based spiking neural networks (SNNs). However, fault-tolerant on-chip learning remains a challenge in neuromorphic systems. This study presents the first scalable neuromorphic fault-tolerant context-dependent learning (FCL) hardware framework. We show how this system can learn associations between stimulation and response in two context-dependent learning tasks from experimental neuroscience, despite possible faults in the hardware nodes. Furthermore, we demonstrate how our novel fault-tolerant neuromorphic spike routing scheme can avoid multiple fault nodes successfully and can enhance the maximum throughput of the neuromorphic network by 0.9%-16.1% in comparison with previous studies. By utilizing the real-time computational capabilities and multiple-fault-tolerant property of the proposed system, the neuronal mechanisms underlying the spiking activities of neuromorphic networks can be readily explored. In addition, the proposed system can be applied in real-time learning and decision-making applications, brain-machine integration, and the investigation of brain cognition during learning.
Collapse
|
2
|
Hao X, Yang S, Deng B, Wang J, Wei X, Che Y. A CORDIC based real-time implementation and analysis of a respiratory central pattern generator. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2020.10.101] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|
3
|
Chen M, Zu L, Wang H, Su F. FPGA-Based Real-Time Simulation Platform for Large-Scale STN-GPe Network. IEEE Trans Neural Syst Rehabil Eng 2020; 28:2537-2547. [PMID: 32991283 DOI: 10.1109/tnsre.2020.3027546] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
The real-time simulation of large-scale subthalamic nucleus (STN)-external globus pallidus (GPe) network model is of great significance for the mechanism analysis and performance improvement of deep brain stimulation (DBS) for Parkinson's states. This paper implements the real-time simulation of a large-scale STN-GPe network containing 512 single-compartment Hodgkin-Huxley type neurons on the Altera Stratix IV field programmable gate array (FPGA) hardware platform. At the single neuron level, some resource optimization schemes such as multiplier substitution, fixed-point operation, nonlinear function approximation and function recombination are adopted, which consists the foundation of the large-scale network realization. At the network level, the simulation scale of network is expanded using module reuse method at the cost of simulation time. The correlation coefficient between the neuron firing waveform of the FPGA platform and the MATLAB software simulation waveform is 0.9756. Under the same physiological time, the simulation speed of FPGA platform is 75 times faster than the Intel Core i7-8700K 3.70 GHz CPU 32GB RAM computer simulation speed. In addition, the established platform is used to analyze the effects of temporal pattern DBS on network firing activities. The proposed large-scale STN-GPe network meets the need of real time simulation, which would be rather helpful in designing closed-loop DBS improvement strategies.
Collapse
|
4
|
Akbarzadeh-Sherbaf K, Safari S, Vahabie AH. A digital hardware implementation of spiking neural networks with binary FORCE training. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.05.044] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
|
5
|
Kwon D, Lim S, Bae JH, Lee ST, Kim H, Seo YT, Oh S, Kim J, Yeom K, Park BG, Lee JH. On-Chip Training Spiking Neural Networks Using Approximated Backpropagation With Analog Synaptic Devices. Front Neurosci 2020; 14:423. [PMID: 32733180 PMCID: PMC7358558 DOI: 10.3389/fnins.2020.00423] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2020] [Accepted: 04/07/2020] [Indexed: 12/02/2022] Open
Abstract
Hardware-based spiking neural networks (SNNs) inspired by a biological nervous system are regarded as an innovative computing system with very low power consumption and massively parallel operation. To train SNNs with supervision, we propose an efficient on-chip training scheme approximating backpropagation algorithm suitable for hardware implementation. We show that the accuracy of the proposed scheme for SNNs is close to that of conventional artificial neural networks (ANNs) by using the stochastic characteristics of neurons. In a hardware configuration, gated Schottky diodes (GSDs) are used as synaptic devices, which have a saturated current with respect to the input voltage. We design the SNN system by using the proposed on-chip training scheme with the GSDs, which can update their conductance in parallel to speed up the overall system. The performance of the on-chip training SNN system is validated through MNIST data set classification based on network size and total time step. The SNN systems achieve accuracy of 97.83% with 1 hidden layer and 98.44% with 4 hidden layers in fully connected neural networks. We then evaluate the effect of non-linearity and asymmetry of conductance response for long-term potentiation (LTP) and long-term depression (LTD) on the performance of the on-chip training SNN system. In addition, the impact of device variations on the performance of the on-chip training SNN system is evaluated.
Collapse
Affiliation(s)
- Dongseok Kwon
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Suhwan Lim
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Jong-Ho Bae
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Sung-Tae Lee
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Hyeongsu Kim
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Young-Tak Seo
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Seongbin Oh
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Jangsaeng Kim
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Kyuho Yeom
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Byung-Gook Park
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| | - Jong-Ho Lee
- Department of Electrical and Computer Engineering, Inter-University Semiconductor Research Center, Seoul National University, Seoul, South Korea
| |
Collapse
|
6
|
Scalable Implementation of Hippocampal Network on Digital Neuromorphic System towards Brain-Inspired Intelligence. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10082857] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
In this paper, an expanded digital hippocampal spurt neural network (HSNN) is innovatively proposed to simulate the mammalian cognitive system and to perform the neuroregulatory dynamics that play a critical role in the cognitive processes of the brain, such as memory and learning. The real-time computation of a large-scale peak neural network can be realized by the scalable on-chip network and parallel topology. By exploring the latest research in the field of neurons and comparing with the results of this paper, it can be found that the implementation of the hippocampal neuron model using the coordinate rotation numerical calculation algorithm can significantly reduce the cost of hardware resources. In addition, the rational use of on-chip network technology can further improve the performance of the system, and even significantly improve the network scalability on a single field programmable gate array chip. The neuromodulation dynamics are considered in the proposed system, which can replicate more relevant biological dynamics. Based on the analysis of biological theory and the theory of hardware integration, it is shown that the innovative system proposed in this paper can reproduce the biological characteristics of the hippocampal network and may be applied to brain-inspired intelligent subjects. The study in this paper will have an unexpected effect on the future research of digital neuromorphic design of spike neural network and the dynamics of the hippocampal network.
Collapse
|
7
|
Shama F, Haghiri S, Imani MA. FPGA Realization of Hodgkin-Huxley Neuronal Model. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1059-1068. [PMID: 32175866 DOI: 10.1109/tnsre.2020.2980475] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
One of the appealing cases of the neuromorphic research area is the implementation of biological neural networks. The current study offers Multiplierless Hodgkin-Huxley Model (MHHM). This modified model may reproduce various spiking behaviors, like the biological HH neurons, with high accuracy. The presented modified model, in comparison to the original HH model, due to its exact similarity to the original model, has more top performances in the case of FPGA saving and more achievable frequency (speed-up). In this approach, the proposed model has a 69 % saving in FPGA resources and also the maximum frequency of 85 MHz that is more than other similar works. In this modification, all spiking behaviors of the original model have been generated with low error calculations. To validate the MHHM neuron, this proposed model has been implemented on digital hardware FPGA. This approach demonstrates that the original HH model and the proposed model have high similarity in terms of higher performance and digital hardware cost reduction.
Collapse
|
8
|
Yavari F, Amiri M, Rahatabad FN, Falotico E, Laschi C. Spike train analysis in a digital neuromorphic system of cutaneous mechanoreceptor. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2019.09.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
9
|
Yang S, Deng B, Wang J, Li H, Lu M, Che Y, Wei X, Loparo KA. Scalable Digital Neuromorphic Architecture for Large-Scale Biophysically Meaningful Neural Network With Multi-Compartment Neurons. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:148-162. [PMID: 30892250 DOI: 10.1109/tnnls.2019.2899936] [Citation(s) in RCA: 83] [Impact Index Per Article: 20.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
Multicompartment emulation is an essential step to enhance the biological realism of neuromorphic systems and to further understand the computational power of neurons. In this paper, we present a hardware efficient, scalable, and real-time computing strategy for the implementation of large-scale biologically meaningful neural networks with one million multi-compartment neurons (CMNs). The hardware platform uses four Altera Stratix III field-programmable gate arrays, and both the cellular and the network levels are considered, which provides an efficient implementation of a large-scale spiking neural network with biophysically plausible dynamics. At the cellular level, a cost-efficient multi-CMN model is presented, which can reproduce the detailed neuronal dynamics with representative neuronal morphology. A set of efficient neuromorphic techniques for single-CMN implementation are presented with all the hardware cost of memory and multiplier resources removed and with hardware performance of computational speed enhanced by 56.59% in comparison with the classical digital implementation method. At the network level, a scalable network-on-chip (NoC) architecture is proposed with a novel routing algorithm to enhance the NoC performance including throughput and computational latency, leading to higher computational efficiency and capability in comparison with state-of-the-art projects. The experimental results demonstrate that the proposed work can provide an efficient model and architecture for large-scale biologically meaningful networks, while the hardware synthesis results demonstrate low area utilization and high computational speed that supports the scalability of the approach.
Collapse
|
10
|
Gomar S, Ahmadi M. Digital Hardware Implementation of Gaussian Wilson–Cowan Neocortex Model. IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE 2019. [DOI: 10.1109/tetci.2018.2849095] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
11
|
Yang S, Wang J, Lin Q, Deng B, Wei X, Liu C, Li H. Cost-efficient FPGA implementation of a biologically plausible dopamine neural network and its application. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.07.006] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|
12
|
Salimi-Nezhad N, Amiri M, Falotico E, Laschi C. A Digital Hardware Realization for Spiking Model of Cutaneous Mechanoreceptor. Front Neurosci 2018; 12:322. [PMID: 29937707 PMCID: PMC6003138 DOI: 10.3389/fnins.2018.00322] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2017] [Accepted: 04/25/2018] [Indexed: 11/17/2022] Open
Abstract
Inspired by the biology of human tactile perception, a hardware neuromorphic approach is proposed for spiking model of mechanoreceptors to encode the input force. In this way, a digital circuit is designed for a slowly adapting type I (SA-I) and fast adapting type I (FA-I) mechanoreceptors to be implemented on a low-cost digital hardware, such as field-programmable gate array (FPGA). This system computationally replicates the neural firing responses of both afferents. Then, comparative simulations are shown. The spiking models of mechanoreceptors are first simulated in MATLAB and next the digital neuromorphic circuits simulated in VIVADO are also compared to show that obtained results are in good agreement both quantitatively and qualitatively. Finally, we test the performance of the proposed digital mechanoreceptors in hardware using a prepared experimental set up. Hardware synthesis and physical realization on FPGA indicate that the digital mechanoreceptors are able to replicate essential characteristics of different firing patterns including bursting and spiking responses of the SA-I and FA-I mechanoreceptors. In addition to parallel computation, a main advantage of this method is that the mechanoreceptor digital circuits can be implemented in real-time through low-power neuromorphic hardware. This novel engineering framework is generally suitable for use in robotic and hand-prosthetic applications, so progressing the state of the art for tactile sensing.
Collapse
Affiliation(s)
- Nima Salimi-Nezhad
- Medical Biology Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Mahmood Amiri
- Medical Biology Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran.,The BioRobotics Institute, Scuola Superiore Sant'Anna, Pontedera, Italy
| | - Egidio Falotico
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pontedera, Italy
| | - Cecilia Laschi
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pontedera, Italy
| |
Collapse
|
13
|
Yang S, Deng B, Li H, Liu C, Wang J, Yu H, Qin Y. FPGA implementation of hippocampal spiking network and its real-time simulation on dynamical neuromodulation of oscillations. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2017.12.031] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
14
|
Yang S, Wei X, Wang J, Deng B, Liu C, Yu H, Li H. Efficient hardware implementation of the subthalamic nucleus–external globus pallidus oscillation system and its dynamics investigation. Neural Netw 2017; 94:220-238. [DOI: 10.1016/j.neunet.2017.07.012] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2017] [Revised: 05/26/2017] [Accepted: 07/13/2017] [Indexed: 12/20/2022]
|
15
|
A real-time FPGA implementation of a biologically inspired central pattern generator network. Neurocomputing 2017. [DOI: 10.1016/j.neucom.2017.03.028] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
16
|
Yang S, Deng B, Wang J, Li H, Liu C, Fietkiewicz C, Loparo KA. Efficient implementation of a real-time estimation system for thalamocortical hidden Parkinsonian properties. Sci Rep 2017; 7:40152. [PMID: 28065938 PMCID: PMC5220381 DOI: 10.1038/srep40152] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2016] [Accepted: 12/01/2016] [Indexed: 12/13/2022] Open
Abstract
Real-time estimation of dynamical characteristics of thalamocortical cells, such as dynamics of ion channels and membrane potentials, is useful and essential in the study of the thalamus in Parkinsonian state. However, measuring the dynamical properties of ion channels is extremely challenging experimentally and even impossible in clinical applications. This paper presents and evaluates a real-time estimation system for thalamocortical hidden properties. For the sake of efficiency, we use a field programmable gate array for strictly hardware-based computation and algorithm optimization. In the proposed system, the FPGA-based unscented Kalman filter is implemented into a conductance-based TC neuron model. Since the complexity of TC neuron model restrains its hardware implementation in parallel structure, a cost efficient model is proposed to reduce the resource cost while retaining the relevant ionic dynamics. Experimental results demonstrate the real-time capability to estimate thalamocortical hidden properties with high precision under both normal and Parkinsonian states. While it is applied to estimate the hidden properties of the thalamus and explore the mechanism of the Parkinsonian state, the proposed method can be useful in the dynamic clamp technique of the electrophysiological experiments, the neural control engineering and brain-machine interface studies.
Collapse
Affiliation(s)
- Shuangming Yang
- School of Electrical Engineering and Automation, Tianjin University, 300072, Tianjin, China
| | - Bin Deng
- School of Electrical Engineering and Automation, Tianjin University, 300072, Tianjin, China
| | - Jiang Wang
- School of Electrical Engineering and Automation, Tianjin University, 300072, Tianjin, China
| | - Huiyan Li
- School of Automation and Electrical Engineering, Tianjin University of Technology and Educations, 300222, Tianjin, China
| | - Chen Liu
- School of Electrical Engineering and Automation, Tianjin University, 300072, Tianjin, China.,Department of Electrical Engineering and Computer Science, Case Western Reserve University, 44106, Cleveland, Ohio, USA
| | - Chris Fietkiewicz
- Department of Electrical Engineering and Computer Science, Case Western Reserve University, 44106, Cleveland, Ohio, USA
| | - Kenneth A Loparo
- Department of Electrical Engineering and Computer Science, Case Western Reserve University, 44106, Cleveland, Ohio, USA
| |
Collapse
|
17
|
|
18
|
|