1
|
Camuñas-Mesa LA, Linares-Barranco B, Serrano-Gotarredona T. Neuromorphic Spiking Neural Networks and Their Memristor-CMOS Hardware Implementations. MATERIALS (BASEL, SWITZERLAND) 2019; 12:E2745. [PMID: 31461877 PMCID: PMC6747825 DOI: 10.3390/ma12172745] [Citation(s) in RCA: 41] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/05/2019] [Revised: 08/02/2019] [Accepted: 08/10/2019] [Indexed: 11/17/2022]
Abstract
Inspired by biology, neuromorphic systems have been trying to emulate the human brain for decades, taking advantage of its massive parallelism and sparse information coding. Recently, several large-scale hardware projects have demonstrated the outstanding capabilities of this paradigm for applications related to sensory information processing. These systems allow for the implementation of massive neural networks with millions of neurons and billions of synapses. However, the realization of learning strategies in these systems consumes an important proportion of resources in terms of area and power. The recent development of nanoscale memristors that can be integrated with Complementary Metal-Oxide-Semiconductor (CMOS) technology opens a very promising solution to emulate the behavior of biological synapses. Therefore, hybrid memristor-CMOS approaches have been proposed to implement large-scale neural networks with learning capabilities, offering a scalable and lower-cost alternative to existing CMOS systems.
Collapse
Affiliation(s)
- Luis A Camuñas-Mesa
- Instituto de Microelectrónica de Sevilla (IMSE-CNM), CSIC and Universidad de Sevilla, 41092 Sevilla, Spain.
| | - Bernabé Linares-Barranco
- Instituto de Microelectrónica de Sevilla (IMSE-CNM), CSIC and Universidad de Sevilla, 41092 Sevilla, Spain
| | - Teresa Serrano-Gotarredona
- Instituto de Microelectrónica de Sevilla (IMSE-CNM), CSIC and Universidad de Sevilla, 41092 Sevilla, Spain
| |
Collapse
|
2
|
Scheerlinck C, Barnes N, Mahony R. Asynchronous Spatial Image Convolutions for Event Cameras. IEEE Robot Autom Lett 2019. [DOI: 10.1109/lra.2019.2893427] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
3
|
Cassidy AS, Georgiou J, Andreou AG. Design of silicon brains in the nano-CMOS era: spiking neurons, learning synapses and neural architecture optimization. Neural Netw 2013; 45:4-26. [PMID: 23886551 DOI: 10.1016/j.neunet.2013.05.011] [Citation(s) in RCA: 80] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2012] [Revised: 05/20/2013] [Accepted: 05/21/2013] [Indexed: 11/30/2022]
Abstract
We present a design framework for neuromorphic architectures in the nano-CMOS era. Our approach to the design of spiking neurons and STDP learning circuits relies on parallel computational structures where neurons are abstracted as digital arithmetic logic units and communication processors. Using this approach, we have developed arrays of silicon neurons that scale to millions of neurons in a single state-of-the-art Field Programmable Gate Array (FPGA). We demonstrate the validity of the design methodology through the implementation of cortical development in a circuit of spiking neurons, STDP synapses, and neural architecture optimization.
Collapse
Affiliation(s)
- Andrew S Cassidy
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA.
| | | | | |
Collapse
|
4
|
Linares-Barranco B, Serrano-Gotarredona T, Serrano-Gotarredona R. Compact low-power calibration mini-DACs for neural arrays with programmable weights. ACTA ACUST UNITED AC 2012; 14:1207-16. [PMID: 18244572 DOI: 10.1109/tnn.2003.816370] [Citation(s) in RCA: 42] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This paper considers the viability of compact low-resolution low-power mini digital-to-analog converters (mini-DACs) for use in large arrays of neural type cells, where programmable weights are required. Transistors are biased in weak inversion in order to yield small currents and low power consumptions, a necessity when building large size arrays. One important drawback of weak inversion operation is poor matching between transistors. The resulting effective precision of a fabricated array of 50 DACs turned out to be 47% (1.1 bits), due to transistor mismatch. However, it is possible to combine them two by two in order to build calibrated DACs, thus compensating for inter-DAC mismatch. It is shown experimentally that the precision can be improved easily by a factor of 10 (4.8% or 4.4 bits), which makes these DACs viable for low-resolution applications such as massive arrays of neural processing circuits. A design methodology is provided, and illustrated through examples, to obtain calibrated mini-DACs of a given target precision. As an example application, we show simulation results of using this technique to calibrate an array of digitally controlled integrate-and-fire neurons.
Collapse
|
5
|
Garg V, Shekhar R, Harris JG. Spiking neuron computation with the time machine. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2012; 6:142-155. [PMID: 23852979 DOI: 10.1109/tbcas.2011.2179544] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
The Time Machine (TM) is a spike-based computation architecture that represents synaptic weights in time. This choice of weight representation allows the use of virtual synapses, providing an excellent tradeoff in terms of flexibility, arbitrary weight connections and hardware usage compared to dedicated synapse architectures. The TM supports an arbitrary number of synapses and is limited only by the number of simultaneously active synapses to each neuron. SpikeSim, a behavioral hardware simulator for the architecture, is described along with example algorithms for edge detection and objection recognition. The TM can implement traditional spike-based processing as well as recently developed time mode operations where step functions serve as the input and output of each neuron block. A custom hybrid digital/analog implementation and a fully digital realization of the TM are discussed. An analog chip with 32 neurons, 1024 synapses and an address event representation (AER) block has been fabricated in 0.5 μm technology. A fully digital field-programmable gate array (FPGA)-based implementation of the architecture has 6,144 neurons and 100,352 simultaneously active synapses. Both implementations utilize a digital controller for routing spikes that can process up to 34 million synapses per second.
Collapse
Affiliation(s)
- Vaibhav Garg
- Texas Instruments Incorpoarted, Dallas, TX 75266, USA.
| | | | | |
Collapse
|
6
|
Selective change driven imaging: a biomimetic visual sensing strategy. SENSORS 2011; 11:11000-20. [PMID: 22346684 PMCID: PMC3274326 DOI: 10.3390/s111111000] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/15/2011] [Revised: 11/15/2011] [Accepted: 11/18/2011] [Indexed: 12/02/2022]
Abstract
Selective Change Driven (SCD) Vision is a biologically inspired strategy for acquiring, transmitting and processing images that significantly speeds up image sensing. SCD vision is based on a new CMOS image sensor which delivers, ordered by the absolute magnitude of its change, the pixels that have changed after the last time they were read out. Moreover, the traditional full frame processing hardware and programming methodology has to be changed, as a part of this biomimetic approach, to a new processing paradigm based on pixel processing in a data flow manner, instead of full frame image processing.
Collapse
|
7
|
Perez-Carrasco JA, Acha B, Serrano C, Camunas-Mesa L, Serrano-Gotarredona T, Linares-Barranco B. Fast vision through frameless event-based sensing and convolutional processing: application to texture recognition. ACTA ACUST UNITED AC 2010; 21:609-20. [PMID: 20181543 DOI: 10.1109/tnn.2009.2039943] [Citation(s) in RCA: 31] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted.
Collapse
|
8
|
Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, Gomez-Rodriguez F, Camunas-Mesa L, Berner R, Rivas-Perez M, Delbruck T, Liu SC, Douglas R, Hafliger P, Jimenez-Moreno G, Civit Ballcels A, Serrano-Gotarredona T, Acosta-Jimenez AJ, Linares-Barranco B. CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking. ACTA ACUST UNITED AC 2009; 20:1417-38. [PMID: 19635693 DOI: 10.1109/tnn.2009.2023653] [Citation(s) in RCA: 265] [Impact Index Per Article: 17.7] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
9
|
Serrano-Gotarredona R, Serrano-Gotarredona T, Acosta-Jimenez A, Serrano-Gotarredona C, Perez-Carrasco J, Linares-Barranco B, Linares-Barranco A, Jimenez-Moreno G, Civit-Ballcels A. On Real-Time AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing. ACTA ACUST UNITED AC 2008. [DOI: 10.1109/tnn.2008.2000163] [Citation(s) in RCA: 54] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
10
|
Meng Y, Shi BE. Adaptive gain control for spike-based map communication in a neuromorphic vision system. IEEE TRANSACTIONS ON NEURAL NETWORKS 2008; 19:1010-21. [PMID: 18541501 DOI: 10.1109/tnn.2007.915113] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
To support large numbers of model neurons, neuromorphic vision systems are increasingly adopting a distributed architecture, where different arrays of neurons are located on different chips or processors. Spike-based protocols are used to communicate activity between processors. The spike activity in the arrays depends on the input statistics as well as internal parameters such as time constants and gains. In this paper, we investigate strategies for automatically adapting these parameters to maintain a constant firing rate in response to changes in the input statistics. We find that under the constraint of maintaining a fixed firing rate, a strategy based upon updating the gain alone performs as well as an optimal strategy where both the gain and the time constant are allowed to vary. We discuss how to choose the time constant and propose an adaptive gain control mechanism whose operation is robust to changes in the input statistics. Our experimental results on a mobile robotic platform validate the analysis and efficacy of the proposed strategy.
Collapse
Affiliation(s)
- Yicong Meng
- Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong.
| | | |
Collapse
|
11
|
|
12
|
Costas-Santos J, Serrano-Gotarredona T, Serrano-Gotarredona R, Linares-Barranco B. A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2007.900179] [Citation(s) in RCA: 83] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
13
|
Chicca E, Whatley AM, Lichtsteiner P, Dante V, Delbruck T, Del Giudice P, Douglas RJ, Indiveri G. A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2007.893509] [Citation(s) in RCA: 93] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
14
|
Chan V, Liu SC, van Schaik A. AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/tcsi.2006.887979] [Citation(s) in RCA: 181] [Impact Index Per Article: 10.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
15
|
Serrano-Gotarredona R, Serrano-Gotarredona T, Acosta-Jimenez A, Linares-Barranco B. A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems. ACTA ACUST UNITED AC 2006. [DOI: 10.1109/tcsi.2006.883843] [Citation(s) in RCA: 78] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
16
|
Linares-Barranco A, Jimenez-Moreno G, Linares-Barranco B, Civit-Balcells A. On algorithmic rate-coded AER generation. ACTA ACUST UNITED AC 2006; 17:771-88. [PMID: 16722179 DOI: 10.1109/tnn.2006.872253] [Citation(s) in RCA: 48] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This paper addresses the problem of converting a conventional video stream based on sequences of frames into the spike event-based representation known as the address-event-representation (AER). In this paper we concentrate on rate-coded AER. The problem is addressed as an algorithmic problem, in which different methods are proposed, implemented and tested through software algorithms. The proposed algorithms are comparatively evaluated according to different criteria. Emphasis is put on the potential of such algorithms for a) doing the frame-based to event-based representation in real time, and b) that the resulting event streams ressemble as much as possible those generated naturally by rate-coded address-event VLSI chips, such as silicon AER retinae. It is found that simple and straightforward algorithms tend to have high potential for real time but produce event distributions that differ considerably from those obtained in AER VLSI chips. On the other hand, sophisticated algorithms that yield better event distributions are not efficient for real time operations. The methods based on linear-feedback-shift-register (LFSR) pseudorandom number generation is a good compromise, which is feasible for real time and yield reasonably well distributed events in time. Our software experiments, on a 1.6-GHz Pentium IV, show that at 50% AER bus load the proposed algorithms require between 0.011 and 1.14 ms per 8 bit-pixel per frame. One of the proposed LFSR methods is implemented in real time hardware using a prototyping board that includes a VirtexE 300 FPGA. The demonstration hardware is capable of transforming frames of 64 x 64 pixels of 8-bit depth at a frame rate of 25 frames per second, producing spike events at a peak rate of 10(7) events per second.
Collapse
|
17
|
Choi T, Merolla P, Arthur J, Boahen K, Shi B. Neuromorphic implementation of orientation hypercolumns. ACTA ACUST UNITED AC 2005. [DOI: 10.1109/tcsi.2005.849136] [Citation(s) in RCA: 91] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
18
|
Choi T, Shi B, Boahen K. An ON–OFF Orientation Selective Address Event Representation Image Transceiver Chip. ACTA ACUST UNITED AC 2004. [DOI: 10.1109/tcsi.2003.822551] [Citation(s) in RCA: 54] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
19
|
Liu SC, Kramer J, Indiveri G, Delbrück T, Burg T, Douglas R. Orientation-selective aVLSI spiking neurons. Neural Netw 2001; 14:629-43. [PMID: 11665759 DOI: 10.1016/s0893-6080(01)00054-5] [Citation(s) in RCA: 78] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
We describe a programmable multi-chip VLSI neuronal system that can be used for exploring spike-based information processing models. The system consists of a silicon retina, a PIC microcontroller, and a transceiver chip whose integrate-and-fire neurons are connected in a soft winner-take-all architecture. The circuit on this multi-neuron chip approximates a cortical microcircuit. The neurons can be configured for different computational properties by the virtual connections of a selected set of pixels on the silicon retina. The virtual wiring between the different chips is effected by an event-driven communication protocol that uses asynchronous digital pulses, similar to spikes in a neuronal system. We used the multi-chip spike-based system to synthesize orientation-tuned neurons using both a feedforward model and a feedback model. The performance of our analog hardware spiking model matched the experimental observations and digital simulations of continuous-valued neurons. The multi-chip VLSI system has advantages over computer neuronal models in that it is real-time, and the computational time does not scale with the size of the neuronal network.
Collapse
Affiliation(s)
- S C Liu
- Institute of Neuroinformatics, ETH/UNIZ, Zurich, Switzerland
| | | | | | | | | | | |
Collapse
|
20
|
Goldberg DH, Cauwenberghs G, Andreou AG. Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons. Neural Netw 2001; 14:781-93. [PMID: 11665770 DOI: 10.1016/s0893-6080(01)00057-0] [Citation(s) in RCA: 91] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
We present a scheme for implementing highly-connected, reconfigurable networks of integrate-and-fire neurons in VLSI. Neural activity is encoded by spikes, where the address of an active neuron is communicated through an asynchronous request and acknowledgement cycle. We employ probabilistic transmission of spikes to implement continuous-valued synaptic weights, and memory-based look-up tables to implement arbitrary interconnection topologies. The scheme is modular and scalable, and lends itself to the implementation of multi-chip network architectures. Results from a prototype system with 1024 analog VLSI integrate-and-fire neurons, each with up to 128 probabilistic synapses, demonstrate these concepts in an image processing task.
Collapse
Affiliation(s)
- D H Goldberg
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA.
| | | | | |
Collapse
|