Fully parallel write/read in resistive synaptic array for accelerating on-chip learning

Hardware implementation of backpropagation using progressive gradient descent for in situ training of multilayer neural networks

Neural network training can be slow and energy-expensive due to the frequent transfer of weight data between digital memory and processing units. Neuromorphic systems can accelerate neural networks by performing multiply-accumulate operations in parallel using nonvolatile analog memory. However, executing the widely used backpropagation training algorithm in multilayer neural networks requires information-and therefore storage-of the partial derivatives of the weight values preventing suitable and scalable implementation in hardware. Here, we propose a hardware implementation of the backpropagation algorithm that progressively updates each layer using in situ stochastic gradient descent, avoiding this storage requirement. We experimentally demonstrate the in situ error calculation and the proposed progressive backpropagation method in a multilayer hardware-implemented neural network. We confirm identical learning characteristics and classification performance compared to conventional backpropagation in software. We show that our approach can be scaled to large and deep neural networks, enabling highly efficient training of advanced artificial intelligence computing systems.

Retention-aware zero-shifting technique for Tiki-Taka algorithm-based analog deep learning accelerator

We present the fabrication of 4 K-scale electrochemical random-access memory (ECRAM) cross-point arrays for analog neural network training accelerator and an electrical characteristic of an 8 × 8 ECRAM array with a 100% yield, showing excellent switching characteristics, low cycle-to-cycle, and device-to-device variations. Leveraging the advances of the ECRAM array, we showcase its efficacy in neural network training using the Tiki-Taka version 2 algorithm (TTv2) tailored for non-ideal analog memory devices. Through an experimental study using ECRAM devices, we investigate the influence of retention characteristics on the training performance of TTv2, revealing that the relative location of the retention convergence point critically determines the available weight range and, consequently, affects the training accuracy. We propose a retention-aware zero-shifting technique designed to optimize neural network training performance, particularly in scenarios involving cross-point devices with limited retention times. This technique ensures robust and efficient analog neural network training despite the practical constraints posed by analog cross-point devices.

Synergistically Modulating Conductive Filaments in Ion-Based Memristors for Enhanced Analog In-Memory Computing

Memristors offer a promising solution to address the performance and energy challenges faced by conventional von Neumann computer systems. Yet, stochastic ion migration in conductive filament often leads to an undesired performance tradeoff between memory window, retention, and endurance. Herein, a robust memristor based on oxygen-rich SnO2 nanoflowers switching medium, enabled by seed-mediated wet chemistry, to overcome the ion migration issue for enhanced analog in-memory computing is reported. Notably, the interplay between the oxygen vacancy (Vo) and Ag ions (Ag+) in the Ag/SnO2/p++-Si memristor can efficiently modulate the formation and abruption of conductive filaments, thereby resulting in a high on/off ratio (>106), long memory retention (10-year extrapolation), and low switching variability (SV = 6.85%). Multiple synaptic functions, such as paired-pulse facilitation, long-term potentiation/depression, and spike-time dependent plasticity, are demonstrated. Finally, facilitated by the symmetric analog weight updating and multiple conductance states, a high image recognition accuracy of ≥ 91.39% is achieved, substantiating its feasibility for analog in-memory computing. This study highlights the significance of synergistically modulating conductive filaments in optimizing performance trade-offs, balancing memory window, retention, and endurance, which demonstrates techniques for regulating ion migration, rendering them a promising approach for enabling cutting-edge neuromorphic applications.

Analyzing Various Structural and Temperature Characteristics of Floating Gate Field Effect Transistors Applicable to Fine-Grain Logic-in-Memory Devices

Although the von Neumann architecture-based computing system has been used for a long time, its limitations in data processing, energy consumption, etc. have led to research on various devices and circuit systems suitable for logic-in-memory (LiM) computing applications. In this paper, we analyze the temperature-dependent device and circuit characteristics of the floating gate field effect transistor (FGFET) source drain barrier (SDB) and FGFET central shallow barrier (CSB) identified in previous papers, and their applicability to LiM applications is specifically confirmed. These FGFETs have the advantage of being much more compatible with existing silicon-based complementary metal oxide semiconductor (CMOS) processes compared to devices using new materials such as ferroelectrics for LiM computing. Utilizing the 32 nm technology node, the leading-edge node where the planar metal oxide semiconductor field effect transistor structure is applied, FGFET devices were analyzed in TCAD, and an environment for analyzing circuits in HSPICE was established. To seamlessly connect FGFET-based devices and circuit analyses, compact models of FGFET-SDB and -CSBs were developed and applied to the design of ternary content-addressable memory (TCAM) and full adder (FA) circuits for LiM. In addition, depression and potential for application of FGFET devices to neural networks were analyzed. The temperature-dependent characteristics of the TCAM and FA circuits with FGFETs were analyzed as an indicator of energy and delay time, and the appropriate number of CSBs should be applied.

Emerging memristive artificial neuron and synapse devices for the neuromorphic electronics era

Growth of data eases the way to access the world but requires increasing amounts of energy to store and process. Neuromorphic electronics has emerged in the last decade, inspired by biological neurons and synapses, with in-memory computing ability, extenuating the 'von Neumann bottleneck' between the memory and processor and offering a promising solution to reduce the efforts both in data storage and processing, thanks to their multi-bit non-volatility, biology-emulated characteristics, and silicon compatibility. This work reviews the recent advances in emerging memristive devices for artificial neuron and synapse applications, including memory and data-processing ability: the physics and characteristics are discussed first, i.e., valence changing, electrochemical metallization, phase changing, interfaced-controlling, charge-trapping, ferroelectric tunnelling, and spin-transfer torquing. Next, we propose a universal benchmark for the artificial synapse and neuron devices on spiking energy consumption, standby power consumption, and spike timing. Based on the benchmark, we address the challenges, suggest the guidelines for intra-device and inter-device design, and provide an outlook for the neuromorphic applications of resistive switching-based artificial neuron and synapse devices.

Device-Algorithm Co-Optimization for an On-Chip Trainable Capacitor-Based Synaptic Device with IGZO TFT and Retention-Centric Tiki-Taka Algorithm

Analog in-memory computing synaptic devices are widely studied for efficient implementation of deep learning. However, synaptic devices based on resistive memory have difficulties implementing on-chip training due to the lack of means to control the amount of resistance change and large device variations. To overcome these shortcomings, silicon complementary metal-oxide semiconductor (Si-CMOS) and capacitor-based charge storage synapses are proposed, but it is difficult to obtain sufficient retention time due to Si-CMOS leakage currents, resulting in a deterioration of training accuracy. Here, a novel 6T1C synaptic device using only n-type indium gaIlium zinc oxide thin film transistor (IGZO TFT) with low leakage current and a capacitor is proposed, allowing not only linear and symmetric weight update but also sufficient retention time and parallel on-chip training operations. In addition, an efficient and realistic training algorithm to compensate for any remaining device non-idealities such as drifting references and long-term retention loss is proposed, demonstrating the importance of device-algorithm co-optimization.

Compute in-Memory with Non-Volatile Elements for Neural Networks: A Review from a Co-Design Perspective

Deep learning has become ubiquitous, touching daily lives across the globe. Today, traditional computer architectures are stressed to their limits in efficiently executing the growing complexity of data and models. Compute-in-memory (CIM) can potentially play an important role in developing efficient hardware solutions that reduce data movement from compute-unit to memory, known as the von Neumann bottleneck. At its heart is a cross-bar architecture with nodal non-volatile-memory elements that performs an analog multiply-and-accumulate operation, enabling the matrix-vector-multiplications repeatedly used in all neural network workloads. The memory materials can significantly influence final system-level characteristics and chip performance, including speed, power, and classification accuracy. With an over-arching co-design viewpoint, this review assesses the use of cross-bar based CIM for neural networks, connecting the material properties and the associated design constraints and demands to application, architecture, and performance. Both digital and analog memory are considered, assessing the status for training and inference, and providing metrics for the collective set of properties non-volatile memory materials will need to demonstrate for a successful CIM technology.

Post-silicon nano-electronic device and its application in brain-inspired chips

As information technology is moving toward the era of big data, the traditional Von-Neumann architecture shows limitations in performance. The field of computing has already struggled with the latency and bandwidth required to access memory (“the memory wall”) and energy dissipation (“the power wall”). These challenging issues, such as “the memory bottleneck,” call for significant research investments to develop a new architecture for the next generation of computing systems. Brain-inspired computing is a new computing architecture providing a method of high energy efficiency and high real-time performance for artificial intelligence computing. Brain-inspired neural network system is based on neuron and synapse. The memristive device has been proposed as an artificial synapse for creating neuromorphic computer applications. In this study, post-silicon nano-electronic device and its application in brain-inspired chips are surveyed. First, we introduce the development of neural networks and review the current typical brain-inspired chips, including brain-inspired chips dominated by analog circuit and brain-inspired chips of the full-digital circuit, leading to the design of brain-inspired chips based on post-silicon nano-electronic device. Then, through the analysis of N kinds of post-silicon nano-electronic devices, the research progress of constructing brain-inspired chips using post-silicon nano-electronic device is expounded. Lastly, the future of building brain-inspired chips based on post-silicon nano-electronic device has been prospected.

Synaptic Segmented Transistor with Improved Linearity by Schottky Junctions and Accelerated Speed by Double-Layered Nitride

Neuromorphic devices have been extensively studied to overcome the limitations of a von Neumann system for artificial intelligence. A synaptic device is one of the most important components in the hardware integration for a neuromorphic system because a number of synaptic devices can be connected to a neuron with compactness as high as possible. Therefore, synaptic devices using silicon-based memory, which are advantageous for a high packing density and mass production due to matured fabrication technologies, have attracted considerable attention. In this study, a segmented transistor devoted to an artificial synapse is proposed for the first time to improve the linearity of the potentiation and depression (P/D). It is a complementary metal oxide semiconductor (CMOS)-compatible device that harnesses both non-ohmic Schottky junctions of the source and drain for improved weight linearity and double-layered nitride for enhanced speed. It shows three distinct and unique segments in drain current-gate voltage transfer characteristics induced by Schottky junctions. In addition, the different stoichiometries of Si_xN_y for a double-layered nitride is utilized as a charge trap layer for boosting the operation speed. This work can bring the industry potentially one step closer to realizing the mass production of hardware-based synaptic devices in the future.

Neural sampling machine with stochastic synapse allows brain-like learning and inference

Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Brain-inspired probabilistic models of neural network can explicitly handle the uncertainty in data and allow adaptive learning on the fly. However, their implementation in a compact, low-power hardware remains a challenge. In this work, we introduce a novel hardware fabric that can implement a new class of stochastic neural network called Neural Sampling Machine (NSM) by exploiting the stochasticity in the synaptic connections for approximate Bayesian inference. We experimentally demonstrate an in silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor (FeFET)-based analog weight cell with a two-terminal stochastic selector element. We show that the stochastic switching characteristic of the selector between the insulator and the metallic states resembles the multiplicative synaptic noise of the NSM. We perform network-level simulations to highlight the salient features offered by the stochastic NSM such as performing autonomous weight normalization for continual online learning and Bayesian inferencing. We show that the stochastic NSM can not only perform highly accurate image classification with 98.25% accuracy on standard MNIST dataset, but also estimate the uncertainty in prediction (measured in terms of the entropy of prediction) when the digits of the MNIST dataset are rotated. Building such a probabilistic hardware platform that can support neuroscience inspired models can enhance the learning and inference capability of the current artificial intelligence (AI).

Neural sampling machines make use of noise to perform learning. Here, Dutta et al. present a hybrid stochastic synapse composed out of a ferroelectric transistor combined with a stochastic selector exhibiting multiplicative synaptic noise required for implementing a neural sample machine.

Exploiting Non-idealities of Resistive Switching Memories for Efficient Machine Learning

Novel computing architectures based on resistive switching memories (also known as memristors or RRAMs) have been shown to be promising approaches for tackling the energy inefficiency of deep learning and spiking neural networks. However, resistive switch technology is immature and suffers from numerous imperfections, which are often considered limitations on implementations of artificial neural networks. Nevertheless, a reasonable amount of variability can be harnessed to implement efficient probabilistic or approximate computing. This approach turns out to improve robustness, decrease overfitting and reduce energy consumption for specific applications, such as Bayesian and spiking neural networks. Thus, certain non-idealities could become opportunities if we adapt machine learning methods to the intrinsic characteristics of resistive switching memories. In this short review, we introduce some key considerations for circuit design and the most common non-idealities. We illustrate the possible benefits of stochasticity and compression with examples of well-established software methods. We then present an overview of recent neural network implementations that exploit the imperfections of resistive switching memory, and discuss the potential and limitations of these approaches.

Zinc oxide and indium-gallium-zinc-oxide bi-layer synaptic device with highly linear long-term potentiation and depression characteristics

The electrical properties, resistive switching behavior, and long-term potentiation/depression (LTP/LTD) in a single indium-gallium-zinc-oxide (IGZO) and bi-layer IGZO/ZnO (ZnO: zinc oxide) memristors were investigated for synapse application. The use of the oxide bi-layer memristors, in particular, improved electrical properties such as stability, memristor reliability, and an increase in synaptic weight states. The set voltage of bi-layer IGZO/ZnO memristors was 0.9 V, and the reset voltage was around - 0.7 V, resulting in a low-operating voltage for neuromorphic systems. The oxygen vacancies in the X-ray photoelectron spectroscopy analysis played a role in the modulation of the high-resistance state (HRS) (oxygen-deficient) and the low-resistance state (oxygen-rich) region. The V_RESET of the bi-layer IGZO/ZnO memristors was lower than that of a single IGZO, which implied that oxygen-vacancy filaments could be easily ruptured due to the higher oxygen vacancy peak HRS layer. The nonlinearity of the LTP and LTD characteristics in a bi-layer IGZO/ZnO memristor was 6.77% and 11.49%, respectively, compared to those of 20.03% and 51.1% in a single IGZO memristor, respectively. Therefore, the extra ZnO layer in the bi-layer memristor with IGZO was potentially significant and essential to achieve a small set voltage and a reset voltage, and the switching behavior to form the conductive path.

Neural Network Training Acceleration With RRAM-Based Hybrid Synapses

Hardware neural network (HNN) based on analog synapse array excels in accelerating parallel computations. To implement an energy-efficient HNN with high accuracy, high-precision synaptic devices and fully-parallel array operations are essential. However, existing resistive memory (RRAM) devices can represent only a finite number of conductance states. Recently, there have been attempts to compensate device nonidealities using multiple devices per weight. While there is a benefit, it is difficult to apply the existing parallel updating scheme to the synaptic units, which significantly increases updating process’s cost in terms of computation speed, energy, and complexity. Here, we propose an RRAM-based hybrid synaptic unit consisting of a “big” synapse and a “small” synapse, and a related training method. Unlike previous attempts, array-wise fully-parallel learning is possible with our proposed architecture with a simple array selection logic. To experimentally verify the hybrid synapse, we exploit Mo/TiO_x RRAM, which shows promising synaptic properties and areal dependency of conductance precision. By realizing the intrinsic gain via proportionally scaled device area, we show that the big and small synapse can be implemented at the device-level without modifications to the operational scheme. Through neural network simulations, we confirm that RRAM-based hybrid synapse with the proposed learning method achieves maximum accuracy of 97 %, comparable to floating-point implementation (97.92%) of the software even with only 50 conductance states in each device. Our results promise training efficiency and inference accuracy by using existing RRAM devices.

Hybrid oxide brain-inspired neuromorphic devices for hardware implementation of artificial intelligence

The state-of-the-art artificial intelligence technologies mainly rely on deep learning algorithms based on conventional computers with classical von Neumann computing architectures, where the memory and processing units are separated resulting in an enormous amount of energy and time consumed in the data transfer process. Inspired by the human brain acting like an ultra-highly efficient biological computer, neuromorphic computing is proposed as a technology for hardware implementation of artificial intelligence. Artificial synapses are the main component of a neuromorphic computing architecture. Memristors are considered to be a relatively ideal candidate for artificial synapse applications due to their high scalability and low power consumption. Oxides are most widely used in memristors due to the ease of fabrication and high compatibility with complementary metal-oxide-semiconductor processes. However, oxide memristors suffer from unsatisfactory stability and reliability. Oxide-based hybrid structures can effectively improve the device stability and reliability, therefore providing a promising prospect for the application of oxide memristors to neuromorphic computing. This work reviews the recent advances in the development of hybrid oxide memristive synapses. The discussion is organized according to the blending schemes as well as the working mechanisms of hybrid oxide memristors.

Modeling-Based Design of Memristive Devices for Brain-Inspired Computing

Resistive switching random access memory (RRAM) has emerged for non-volatile memory application with the features of simple structure, low cost, high density, high speed, low power, and CMOS compatibility. In recent years, RRAM technology has made significant progress in brain-inspired computing paradigms by exploiting its unique physical characteristics, which attempts to eliminate the energy-intensive and time-consuming data transfer between the processing unit and the memory unit. The design of RRAM-based computing paradigms, however, requires a detailed description of the dominant physical effects correlated with the resistive switching processes to realize the interaction and optimization between devices and algorithms or architectures. This work provides an overview of the current progress on device-level resistive switching behaviors with detailed insights into the physical effects in the resistive switching layer and the multifunctional assistant layer. Then the circuit-level physics-based compact models will be reviewed in terms of typical binary RRAM and the emerging analog synaptic RRAM, which act as an interface between the device and circuit design. After that, the interaction between device and system performances will finally be addressed by reviewing the specific applications of brain-inspired computing systems including neuromorphic computing, in-memory logic, and stochastic computing.

A 2D material-based floating gate device with linear synaptic weight update

Neuromorphic computing is of great interest among researchers interested in overcoming the von Neumann computing bottleneck. A synaptic device, one of the key components to realize a neuromorphic system, has a weight that indicates the strength of the connection between two neurons, and updating this weight must have linear and symmetric characteristics. Especially, a transistor-type device has a gate terminal, separating the processes of reading and updating the conductivity, used as a synaptic weight to prevent sneak path current issues during synaptic operations. In this study, we fabricate a top-gated flash memory device based on two-dimensional (2D) materials, MoS2 and graphene, as a channel and a floating gate, respectively, and Al2O3 and HfO2 to increase the tunneling efficiency. We demonstrate the linear weight updates and repeatable characteristics of applying negative/positive pulses, and also emulate spike timing-dependent plasticity (STDP), one of the learning rules in a spiking neural network (SNN).

Hysteresis Modulation on Van der Waals-Based Ferroelectric Field-Effect Transistor by Interfacial Passivation Technique and Its Application in Optic Neural Networks

2D semiconductor-based ferroelectric field effect transistors (FeFETs) have been considered as a promising artificial synaptic device for implementation of neuromorphic computing systems. However, an inevitable problem, interface traps at the 2D semiconductor/ferroelectric oxide interface, suppresses ferroelectric characteristics, and causes a critical degradation on the performance of 2D-based FeFETs. Here, hysteresis modulation method using self-assembly monolayer (SAM) material for interface trap passivation on 2D-based FeFET is presented. Through effectively passivation of interface traps by SAM layer, the hysteresis of the proposed device changes from interface traps-dependent to polarization-dependent direction. The reduction of interface trap density is clearly confirmed through the result of calculation using the subthreshold swing of the device. Furthermore, excellent optic-neural synaptic characteristics are successfully implemeted, including linear and symmetric potentiation and depression, and multilevel conductance. This work identifies the potential of passivation effect for 2D-based FeFETs to accelerate the development of neuromorphic computing systems.

Emerging Memristive Artificial Synapses and Neurons for Energy-Efficient Neuromorphic Computing

Memristors have recently attracted significant interest due to their applicability as promising building blocks of neuromorphic computing and electronic systems. The dynamic reconfiguration of memristors, which is based on the history of applied electrical stimuli, can mimic both essential analog synaptic and neuronal functionalities. These can be utilized as the node and terminal devices in an artificial neural network. Consequently, the ability to understand, control, and utilize fundamental switching principles and various types of device architectures of the memristor is necessary for achieving memristor-based neuromorphic hardware systems. Herein, a wide range of memristors and memristive-related devices for artificial synapses and neurons is highlighted. The device structures, switching principles, and the applications of essential synaptic and neuronal functionalities are sequentially presented. Moreover, recent advances in memristive artificial neural networks and their hardware implementations are introduced along with an overview of the various learning algorithms. Finally, the main challenges of the memristive synapses and neurons toward high-performance and energy-efficient neuromorphic computing are briefly discussed. This progress report aims to be an insightful guide for the research on memristors and neuromorphic-based computing.

Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy

Neural networks based on memristive devices have achieved great progress recently. However, memristive synapses with nonlinearity and asymmetry seriously limit the classification accuracy. Moreover, insufficient number of training samples in many cases also have negative effect on the classification accuracy of neural networks due to overfitting. In this work, dropout neuronal units are developed based on stochastic volatile memristive devices of Ag/Ta2O5:Ag/Pt. The memristive neural network using the dropout neuronal units effectively solves the problem of overfitting and mitigates the negative effects of the nonideality of memristive synapses, eventually achieves a classification accuracy comparable to the theoretical limit. The stochastic and volatile switching performances of the Ag/Ta2O5:Ag/Pt device are attributed to the stochastical rupture of the Ag filament under high electrical stress in the Ta2O5 layer, according to the TEM observation and the kinetic Monte Carlo simulation.

Sensory Adaptation and Neuromorphic Phototransistors Based on CsPb(Br1-xIx)3 Perovskite and MoS2 Hybrid Structure

Sensory adaptation is an essential part of biological neural systems for sustaining human life. Using the light-induced halide phase segregation of CsPb(Br_1-xI_x)₃ perovskite, we introduce neuromorphic phototransistors that emulate human sensory adaptation. The phototransistor based on a hybrid structure of perovskite and transition-metal dichalcogenide (TMD) emulates the sensory adaptation in response to a continuous light stimulus, similar to the neural system. The underlying mechanism for the sensory adaptation is the halide segregation of the mixed halide perovskites. The phase separation under visible-light illumination leads to the segregation of I and Br into separate iodide- and bromide-rich domains, significantly changing the photocurrent in the phototransistors. The devices are reversible upon the removal of the light stimulation, resulting in near-complete recovery of the photosensitivity before the phase segregation (sensitivity recovery of 96.65% for 5 min rest time). The proposed phototransistor based on the perovskite-TMD hybrid structure can be applied to other neuromorphic devices such as neuromorphic photonic devices, intelligent sensors, and selective light-detecting image sensors.

Supervised Learning in All FeFET-Based Spiking Neural Network: Opportunities and Challenges

The two possible pathways toward artificial intelligence (AI)-(i) neuroscience-oriented neuromorphic computing [like spiking neural network (SNN)] and (ii) computer science driven machine learning (like deep learning) differ widely in their fundamental formalism and coding schemes (Pei et al., 2019). Deviating from traditional deep learning approach of relying on neuronal models with static nonlinearities, SNNs attempt to capture brain-like features like computation using spikes. This holds the promise of improving the energy efficiency of the computing platforms. In order to achieve a much higher areal and energy efficiency compared to today's hardware implementation of SNN, we need to go beyond the traditional route of relying on CMOS-based digital or mixed-signal neuronal circuits and segregation of computation and memory under the von Neumann architecture. Recently, ferroelectric field-effect transistors (FeFETs) are being explored as a promising alternative for building neuromorphic hardware by utilizing their non-volatile nature and rich polarization switching dynamics. In this work, we propose an all FeFET-based SNN hardware that allows low-power spike-based information processing and co-localized memory and computing (a.k.a. in-memory computing). We experimentally demonstrate the essential neuronal and synaptic dynamics in a 28 nm high-K metal gate FeFET technology. Furthermore, drawing inspiration from the traditional machine learning approach of optimizing a cost function to adjust the synaptic weights, we implement a surrogate gradient (SG) learning algorithm on our SNN platform that allows us to perform supervised learning on MNIST dataset. As such, we provide a pathway toward building energy-efficient neuromorphic hardware that can support traditional machine learning algorithms. Finally, we undertake synergistic device-algorithm co-design by accounting for the impacts of device-level variation (stochasticity) and limited bit precision of on-chip synaptic weights (available analog states) on the classification accuracy.

Parallel weight update protocol for a carbon nanotube synaptic transistor array for accelerating neuromorphic computing

Brain-inspired neuromorphic computing has the potential to overcome the inherent inefficiency of the conventional von Neumann architecture by using the massively parallel processing power of artificial neural networks. Neuromorphic parallel processing can be implemented naturally using the crossbar geometry of synaptic device arrays with Ohm's and Kirchhoff's laws. However, selective and parallel weight updates of the synaptic crossbar array are still very challenging due to the unavoidable crosstalk between adjacent devices and sneak path currents. Here, we experimentally demonstrate a weight update protocol in a carbon nanotube synaptic transistor array, where selective and parallel weight updates can be executed by exploiting the individually controllable three terminals of the synaptic device via a localized carrier trapping mechanism. The trained 9 × 8 synaptic array solves four different convolution operations simultaneously for the feature extraction of an image. The massive parallelism and robustness of the weight update protocol are important features toward effective manipulation of big data through neuromorphic computing systems.

A Novel Resistive Switching Identification Method through Relaxation Characteristics for Sneak-path-constrained Selectorless RRAM application

Resistive random access memory (RRAM) is a leading candidate in the race towards emerging nonvolatile memory technologies. The sneak path current (SPC) problem is one of the main difficulties in crossbar memory configurations. RRAM devices with desirable properties such as a selectorless, 1R-only architecture with self-rectifying behavior are potential SPC solutions. In this work, the intrinsic nonlinear (NL) characteristics and relaxation characteristics of bilayer high-k/low-k stacked RRAMs are presented. The intrinsic nonlinearity reliability of bilayer selectorless 1R-only RRAM without additional switches has been studied for their ability to effectively suppress SPC in RRAM arrays. The relaxation properties with resistive switching identification method by utilizing the activation energy (Ea) extraction methodology is demonstrated, which provides insights and design guidance for non-uniform bilayer selectorless 1R-only RRAM array applications.

Unsupervised Learning on Resistive Memory Array Based Spiking Neural Networks

Spiking Neural Networks (SNNs) offer great potential to promote both the performance and efficiency of real-world computing systems, considering the biological plausibility of SNNs. The emerging analog Resistive Random Access Memory (RRAM) devices have drawn increasing interest as potential neuromorphic hardware for implementing practical SNNs. In this article, we propose a novel training approach (called greedy training) for SNNs by diluting spike events on the temporal dimension with necessary controls on input encoding phase switching, endowing SNNs with the ability to cooperate with the inevitable conductance variations of RRAM devices. The SNNs could utilize Spike-Timing-Dependent Plasticity (STDP) as the unsupervised learning rule, and this plasticity has been observed on our one-transistor-one-resistor (1T1R) RRAM devices under voltage pulses with designed waveforms. We have also conducted handwritten digit recognition task simulations on MNIST dataset. The results show that the unsupervised SNNs trained by the proposed method could mitigate the requirement for the number of gradual levels of RRAM devices, and also have immunity to both cycle-to-cycle and device-to-device RRAM conductance variations. Unsupervised SNNs trained by the proposed methods could cooperate with real RRAM devices with non-ideal behaviors better, promising high feasibility of RRAM array based neuromorphic systems for online training.

Nanochannel-Based Transport in an Interfacial Memristor Can Emulate the Analog Weight Modulation of Synapses

By exploiting novel transport phenomena such as ion selectivity at the nanoscale, it has been shown that nanochannel systems can exhibit electrically controllable conductance, suggesting their potential use in neuromorphic devices. However, several critical features of biological synapses, particularly their conductance modulation, which is both memorable and gradual, have rarely been reported in these types of systems due to the fast flow property of typical inorganic electrolytes. In this work, we demonstrate that electrically manipulating the nanochannel conductance can result in nonvolatile conductance tuning capable of mimicking the analog behavior of synapses by introducing a room-temperature ionic liquid (IL) and a KCl solution into the two ends of a nanochannel system. The gradual conductance-tuning mechanism is identified through fluorescence measurements as the voltage-induced movement of the interface between the immiscible IL and KCl solution, while the successful memorization of the conductance tuning is ascribed to the large viscosity of the IL. We applied a nanochannel-based synapse to a handwritten digit-recognition task, reaching an accuracy of 94%. These promising results provide important guidance for the future design of nanochannel-based neuromorphic devices and the manipulation of nanochannel transport for computing.

Solving matrix equations in one step with cross-point resistive arrays

Conventional digital computers can execute advanced operations by a sequence of elementary Boolean functions of 2 or more bits. As a result, complicated tasks such as solving a linear system or solving a differential equation require a large number of computing steps and an extensive use of memory units to store individual bits. To accelerate the execution of such advanced tasks, in-memory computing with resistive memories provides a promising avenue, thanks to analog data storage and physical computation in the memory. Here, we show that a cross-point array of resistive memory devices can directly solve a system of linear equations, or find the matrix eigenvectors. These operations are completed in just one single step, thanks to the physical computing with Ohm's and Kirchhoff's laws, and thanks to the negative feedback connection in the cross-point circuit. Algebraic problems are demonstrated in hardware and applied to classical computing tasks, such as ranking webpages and solving the Schrödinger equation in one step.

Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain

Neuromorphic engineering (NE) encompasses a diverse range of approaches to information processing that are inspired by neurobiological systems, and this feature distinguishes neuromorphic systems from conventional computing systems. The brain has evolved over billions of years to solve difficult engineering problems by using efficient, parallel, low-power computation. The goal of NE is to design systems capable of brain-like computation. Numerous large-scale neuromorphic projects have emerged recently. This interdisciplinary field was listed among the top 10 technology breakthroughs of 2014 by the MIT Technology Review and among the top 10 emerging technologies of 2015 by the World Economic Forum. NE has two-way goals: one, a scientific goal to understand the computational properties of biological neural systems by using models implemented in integrated circuits (ICs); second, an engineering goal to exploit the known properties of biological systems to design and implement efficient devices for engineering applications. Building hardware neural emulators can be extremely useful for simulating large-scale neural models to explain how intelligent behavior arises in the brain. The principal advantages of neuromorphic emulators are that they are highly energy efficient, parallel and distributed, and require a small silicon area. Thus, compared to conventional CPUs, these neuromorphic emulators are beneficial in many engineering applications such as for the porting of deep learning algorithms for various recognitions tasks. In this review article, we describe some of the most significant neuromorphic spiking emulators, compare the different architectures and approaches used by them, illustrate their advantages and drawbacks, and highlight the capabilities that each can deliver to neural modelers. This article focuses on the discussion of large-scale emulators and is a continuation of a previous review of various neural and synapse circuits (Indiveri et al., 2011). We also explore applications where these emulators have been used and discuss some of their promising future applications.

Self-controlled multilevel writing architecture for fast training in neuromorphic RRAM applications

Memristor crossbar arrays naturally accelerate neural networks applications by carrying out parallel multiply-add operations. Due to the abrupt SET operation characterizing most RRAM devices, on-chip training usually requires either from iterative write/read stages, large and variation-sensitive circuitry, or both, to achieve multilevel capabilities. This paper presents a self-controlled architecture to program multilevel devices with a short and fixed operation duration. We rely on an ad hoc scheme to self-control the abrupt SET, choking the writing stimulus as the cell addresses the desired level. To achieve this goal, we make use of the voltage divider concept by placing a variable resistive load in series with the target cell. We validated the proposal against thorough simulations using RRAM cells fitting extremely fast physical devices and a commercial 40 nm CMOS technology, both exhibiting variability. For every case the proposed architecture allowed progressive and almost-linear resistive levels in each [Formula: see text] and [Formula: see text] crossbars structures.

Equivalent-accuracy accelerated neural-network training using analogue memory

Neural-network training can be slow and energy intensive, owing to the need to transfer the weight data for the network between conventional digital memory chips and processor chips. Analogue non-volatile memory can accelerate the neural-network training algorithm known as backpropagation by performing parallelized multiply-accumulate operations in the analogue domain at the location of the weight data. However, the classification accuracies of such in situ training using non-volatile-memory hardware have generally been less than those of software-based training, owing to insufficient dynamic range and excessive weight-update asymmetry. Here we demonstrate mixed hardware-software neural-network implementations that involve up to 204,900 synapses and that combine long-term storage in phase-change memory, near-linear updates of volatile capacitors and weight-data transfer with 'polarity inversion' to cancel out inherent device-to-device variations. We achieve generalization accuracies (on previously unseen data) equivalent to those of software-based training on various commonly used machine-learning test datasets (MNIST, MNIST-backrand, CIFAR-10 and CIFAR-100). The computational energy efficiency of 28,065 billion operations per second per watt and throughput per area of 3.6 trillion operations per second per square millimetre that we calculate for our implementation exceed those of today's graphical processing units by two orders of magnitude. This work provides a path towards hardware accelerators that are both fast and energy efficient, particularly on fully connected neural-network layers.

SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations

Although several types of architecture combining memory cells and transistors have been used to demonstrate artificial synaptic arrays, they usually present limited scalability and high power consumption. Transistor-free analog switching devices may overcome these limitations, yet the typical switching process they rely on-formation of filaments in an amorphous medium-is not easily controlled and hence hampers the spatial and temporal reproducibility of the performance. Here, we demonstrate analog resistive switching devices that possess desired characteristics for neuromorphic computing networks with minimal performance variations using a single-crystalline SiGe layer epitaxially grown on Si as a switching medium. Such epitaxial random access memories utilize threading dislocations in SiGe to confine metal filaments in a defined, one-dimensional channel. This confinement results in drastically enhanced switching uniformity and long retention/high endurance with a high analog on/off ratio. Simulations using the MNIST handwritten recognition data set prove that epitaxial random access memories can operate with an online learning accuracy of 95.1%.

A nondestructive approach to study resistive switching mechanism in metal oxide based on defect photoluminescence mapping

The mechanism of resistive switching in metal oxides is a widely studied topic with interest in both fundamental physics and the practical need to improve device characteristics for memory based applications. Various experimental approaches were employed to reveal the different aspects of resistive switching; however, there is still a debate on the switching mechanism due to the lack of nondestructive microscopic characterization tools to monitor the oxygen vacancies. In this study, a novel approach using photoluminescence (PL) mapping was developed to study switching dynamics in metal oxides. By monitoring the emission properties with a confocal PL system, information regarding the switching mechanism can be obtained. The nondestructive nature of this approach allowed us to make comparisons between different switching conditions and endurance cycles. SrTiO₃ based switching devices were used in the study. The distribution of oxygen vacancies can be positioned by mapping the integrated intensity of oxygen vacancy emission on a transparent top electrode, and both interface switching and filament switching can be distinguished. Moreover, the endurance study revealed a sudden rise in the emission intensity correlated with the device failure, which indicates an abrupt increase in the localized density of oxygen vacancies that results in an irreversible set process for the conductive filament.

Tuning analog resistive switching and plasticity in bilayer transition metal oxide based memristive synapses

The existence of rich suboxide phases is favorable for increasing the number of weight states in transition metal oxide synapses.