Integration and Co-design of Memristive Devices and Algorithms for Artificial Intelligence

Memristive Monte Carlo DropConnect crossbar array enabled by device and algorithm co-design

Device and algorithm co-design aims to develop energy-efficient hardware that directly implements complex algorithms and optimizes algorithms to match the hardware's characteristics. Specifically, neuromorphic computing algorithms are constantly growing in complexity, necessitating an ongoing search for hardware implementations capable of handling these intricate algorithms. Here, we present a memristive Monte Carlo DropConnect (MC-DC) crossbar array developed through a hardware algorithm co-design approach. To implement the MC-DC neural network, stochastic switching and analog memory characteristics are required, and we achieved them using Ag-based diffusive selectors and Ru-based electrochemical metalization (ECM) memristors, respectively. The devices were integrated with a one-selector one-memristor (1S1M) structure, and their well-matched operating voltages and currents enabled stochastic readout and deterministic analog programming. With the integrated hardware, we successfully demonstrated the MC-DC operation. Additionally, the selector allowed for the control of switching polarity, and by understanding this hardware characteristic, we were able to modify the algorithm to fit it and further improve the network performance.

Reservoir computing with a random memristor crossbar array

Physical implementations of reservoir computing (RC) based on the emerging memristors have become promising candidates of unconventional computing paradigms. Traditionally, sequential approaches by time-multiplexing volatile memristors have been prevalent because of their low hardware overhead. However, they suffer from the problem of speed degradation and fall short of capturing the spatial relationship between the time-domain inputs. Here, we explore a new avenue for RC using memristor crossbar arrays with device-to-device variations, which serve as physical random weight matrices of the reservoir layers, enabling faster computation thanks to the parallelism of matrix-vector multiplication as an intensive operation in RC. To achieve this new RC architecture, ultralow-current, self-selective memristors are fabricated and integrated without the need of transistors, showing greater potential of high scalability and three-dimensional integrability compared to the previous realizations. The information processing ability of our RC system is demonstrated in asks of recognizing digit images and waveforms. This work indicates that the 'nonidealities' of the emerging memristor devices and circuits are a useful source of inspiration for new computing paradigms.

Shaping high-performance wearable robots for human motor and sensory reconstruction and enhancement

Most wearable robots such as exoskeletons and prostheses can operate with dexterity, while wearers do not perceive them as part of their bodies. In this perspective, we contend that integrating environmental, physiological, and physical information through multi-modal fusion, incorporating human-in-the-loop control, utilizing neuromuscular interface, employing flexible electronics, and acquiring and processing human-robot information with biomechatronic chips, should all be leveraged towards building the next generation of wearable robots. These technologies could improve the embodiment of wearable robots. With optimizations in mechanical structure and clinical training, the next generation of wearable robots should better facilitate human motor and sensory reconstruction and enhancement.

Multifunctional HfAlO thin film: Ferroelectric tunnel junction and resistive random access memory

This study presents findings indicating that the ferroelectric tunnel junction (FTJ) or resistive random-access memory (RRAM) in one cell can be intentionally selected depending on the application. The HfAlO film annealed at 700 °C shows stable FTJ characteristics and can be converted into RRAM by forming a conductive filament inside the same cell, that is, the process of intentionally forming a conductive filament is the result of defect generation and redistribution, and applying compliance current prior to a hard breakdown event of the dielectric film enables subsequent RRAM operation. The converted RRAM demonstrated good memory performance. Through current-voltage fitting, it was confirmed that the two resistance states of the FTJ and RRAM had different transport mechanisms. In the RRAM, the 1/f noise power of the high-resistance state (HRS) was about ten times higher than that of the low-resistance state (LRS). This is because the noise components increase due to the additional current paths in the HRS. The 1/f noise power according to resistance states in the FTJ was exactly the opposite result from the case of the RRAM. This is because the noise component due to the Poole-Frenkel emission is added to the noise component due to the tunneling current in the LRS. In addition, we confirmed the potentiation and depression characteristics of the two devices and further evaluated the accuracy of pattern recognition through a simulation by considering a dataset from the Modified National Institute of Standards and Technology.

Generative complex networks within a dynamic memristor with intrinsic variability

Artificial neural networks (ANNs) have gained considerable momentum in the past decade. Although at first the main task of the ANN paradigm was to tune the connection weights in fixed-architecture networks, there has recently been growing interest in evolving network architectures toward the goal of creating artificial general intelligence. Lagging behind this trend, current ANN hardware struggles for a balance between flexibility and efficiency but cannot achieve both. Here, we report on a novel approach for the on-demand generation of complex networks within a single memristor where multiple virtual nodes are created by time multiplexing and the non-trivial topological features, such as small-worldness, are generated by exploiting device dynamics with intrinsic cycle-to-cycle variability. When used for reservoir computing, memristive complex networks can achieve a noticeable increase in memory capacity a and respectable performance boost compared to conventional reservoirs trivially implemented as fully connected networks. This work expands the functionality of memristors for ANN computing.

Interfacing Biology and Electronics with Memristive Materials

Memristive technologies promise to have a large impact on modern electronics, particularly in the areas of reconfigurable computing and artificial intelligence (AI) hardware. Meanwhile, the evolution of memristive materials alongside the technological progress is opening application perspectives also in the biomedical field, particularly for implantable and lab-on-a-chip devices where advanced sensing technologies generate a large amount of data. Memristive devices are emerging as bioelectronic links merging biosensing with computation, acting as physical processors of analog signals or in the framework of advanced digital computing architectures. Recent developments in the processing of electrical neural signals, as well as on transduction and processing of chemical biomarkers of neural and endocrine functions, are reviewed. It is concluded with a critical perspective on the future applicability of memristive devices as pivotal building blocks in bio-AI fusion concepts and bionic schemes.

Recent progress in three-terminal artificial synapses based on 2D materials: from mechanisms to applications

Synapses are essential for the transmission of neural signals. Synaptic plasticity allows for changes in synaptic strength, enabling the brain to learn from experience. With the rapid development of neuromorphic electronics, tremendous efforts have been devoted to designing and fabricating electronic devices that can mimic synapse operating modes. This growing interest in the field will provide unprecedented opportunities for new hardware architectures for artificial intelligence. In this review, we focus on research of three-terminal artificial synapses based on two-dimensional (2D) materials regulated by electrical, optical and mechanical stimulation. In addition, we systematically summarize artificial synapse applications in various sensory systems, including bioplastic bionics, logical transformation, associative learning, image recognition, and multimodal pattern recognition. Finally, the current challenges and future perspectives involving integration, power consumption and functionality are outlined.

Short-Term Memory Characteristics of IGZO-Based Three-Terminal Devices

A three-terminal synaptic transistor enables more accurate controllability over the conductance compared with traditional two-terminal synaptic devices for the synaptic devices in hardware-oriented neuromorphic systems. In this work, we fabricated IGZO-based three-terminal devices comprising HfAlO_x and CeO_x layers to demonstrate the synaptic operations. The chemical compositions and thicknesses of the devices were verified by transmission electron microscopy and energy dispersive spectroscopy in cooperation. The excitatory post-synaptic current (EPSC), paired-pulse facilitation (PPF), short-term potentiation (STP), and short-term depression (STD) of the synaptic devices were realized for the short-term memory behaviors. The IGZO-based three-terminal synaptic transistor could thus be controlled appropriately by the amplitude, width, and interval time of the pulses for implementing the neuromorphic systems.

A Unified Current-Voltage Model for Metal Oxide-Based Resistive Random-Access Memory

Resistive random-access memory (RRAM) is essential for developing neuromorphic devices, and it is still a competitive candidate for future memory devices. In this paper, a unified model is proposed to describe the entire electrical characteristics of RRAM devices, which exhibit two different resistive switching phenomena. To enhance the performance of the model by reflecting the physical properties such as the length index of the undoped area during the switching operation, the Voltage ThrEshold Adaptive Memristor (VTEAM) model and the tungsten-based model are combined to represent two different resistive switching phenomena. The accuracy of the I-V relationship curve tails of the device is improved significantly by adjusting the ranges of unified internal state variables. Furthermore, the unified model describes a variety of electrical characteristics and yields continuous results by using the device's current-voltage relationship without dividing its fitting conditions. The unified model describes the optimized electrical characteristics that reflect the electrical behavior of the device.

Compact SPICE Model of Memristor with Barrier Modulated Considering Short- and Long-Term Memory Characteristics by IGZO Oxygen Content

This paper introduces a compact SPICE model of a two-terminal memory with a Pd/Ti/IGZO/p⁺-Si structure. In this paper, short- and long-term components are systematically separated and applied in each model. Such separations are conducted by the applied bias and oxygen flow rate (OFR) during indium gallium zinc oxide (IGZO) deposition. The short- and long-term components in the potentiation and depression curves are modeled by considering the process (OFR of IGZO) and bias conditions. The compact SPICE model with the physical mechanism of SiO₂ modulation is introduced, which can be useful for optimizing the specification of memristor devices.

Complementary Resistive Switching Behavior in Tetraindolyl Derivative-Based Memory Devices

Complementary resistive switching (CRS) devices are more advantageous compared to bipolar resistive switching (BRS) devices for memory applications as they can minimize the sneak path problem observed in the case of BRS having a crossbar array structure. Here, we report the CRS behavior of 1,4-bis(di(1H-indol-3-yl)methyl)benzene (Indole1) molecules. Our earlier study revealed that Au/Indole1/Indium tin oxide (ITO) devices showed BRS under ambient conditions. However, the present investigations revealed that when the device is exposed to 353 K or higher temperatures, dynamic evolution of the Au/Indole1/ITO device from BRS to CRS occurred with a very good memory window (∼10³), data retention (5.1 × 10³ s), stability (50 days), and device yield (∼ 60%). This work explores the application possibility of indole derivatives toward future ultradense resistive random access memory.

All-Printed Flexible Memristor with Metal–Non-Metal-Doped TiO2 Nanoparticle Thin Films

A memristor is a fundamental electronic device that operates like a biological synapse and is considered as the solution of classical von Neumann computers. Here, a fully printed and flexible memristor is fabricated by depositing a thin film of metal–non-metal (chromium-nitrogen)-doped titanium dioxide (TiO₂). The resulting device exhibited enhanced performance with self-rectifying and forming free bipolar switching behavior. Doping was performed to bring stability in the performance of the memristor by controlling the defects and impurity levels. The forming free memristor exhibited characteristic behavior of bipolar resistive switching with a high on/off ratio (2.5 × 10³), high endurance (500 cycles), long retention time (5 × 10³ s) and low operating voltage (±1 V). Doping the thin film of TiO₂ with metal–non-metal had a significant effect on the switching properties and conduction mechanism as it directly affected the energy bandgap by lowering it from 3.2 eV to 2.76 eV. Doping enhanced the mobility of charge carriers and eased the process of filament formation by suppressing its randomness between electrodes under the applied electric field. Furthermore, metal–non-metal-doped TiO₂ thin film exhibited less switching current and improved non-linearity by controlling the surface defects.

Exploiting Non-idealities of Resistive Switching Memories for Efficient Machine Learning

Novel computing architectures based on resistive switching memories (also known as memristors or RRAMs) have been shown to be promising approaches for tackling the energy inefficiency of deep learning and spiking neural networks. However, resistive switch technology is immature and suffers from numerous imperfections, which are often considered limitations on implementations of artificial neural networks. Nevertheless, a reasonable amount of variability can be harnessed to implement efficient probabilistic or approximate computing. This approach turns out to improve robustness, decrease overfitting and reduce energy consumption for specific applications, such as Bayesian and spiking neural networks. Thus, certain non-idealities could become opportunities if we adapt machine learning methods to the intrinsic characteristics of resistive switching memories. In this short review, we introduce some key considerations for circuit design and the most common non-idealities. We illustrate the possible benefits of stochasticity and compression with examples of well-established software methods. We then present an overview of recent neural network implementations that exploit the imperfections of resistive switching memory, and discuss the potential and limitations of these approaches.

Effects of the Operating Ambiance and Active Layer Treatments on the Performance of Magnesium Fluoride Based Bipolar RRAM

This study investigates switching characteristics of the magnesium fluoride (MgFx)-based bipolar resistive random-access memory (RRAM) devices at different operating ambiances (open-air and vacuum). Operating ambiances alter the elemental composition of the amorphous MgFx active layer and Ti/MgFx interface region, which affects the overall device performance. The experimental results indicate that filament type resistive switching takes place at the interface of Ti/MgFx and trap-controlled space charge limited conduction (SCLC) mechanisms is dominant in both the low and high resistance states in the bulk MgFx layer. RRAM device performances at different operating ambiances are also altered by MgFx active layer treatments (air exposure and annealing). Devices show the better uniformity, stability, and a higher on/off current ratio in vacuum compared to an open-air environment. The Ti/MgFx/Pt memory devices have great potential for future vacuum electronic applications.

Asymmetric carrier transport in flexible interface-type memristor enables artificial synapses with sub-femtojoule energy consumption

Flexible and transparent artificial synapses with extremely low energy consumption have potential for use in brain-like neuromorphic electronics. However, most of the transparent materials for flexible memristive artificial synapses were reported to show picojoule-scale high energy consumption with kiloohm-scale low resistance, which limits the scalability for parallel operation. Here, we report on a flexible memristive artificial synapse based on Cs₃Cu₂I₅ with energy consumption as low as 10.48 aJ (= 10.48 × 10^-18 J) μm^-2 and resistance as high as 243 MΩ for writing pulses. Interface-type resistive switching at the Schottky junction between p-type Cu₃Cs₂I₅ and Au is verified, where migration of iodide vacancies and asymmetric carrier transport owing to the effective hole mass is three times heavier than effective electron mass are found to play critical roles in controlling the conductance, leading to high resistance. There was little difference in synaptic weight updates with high linearity and 250 states before and after bending the flexible device. Moreover, the MNIST-based recognition rate of over 90% is maintained upon bending, indicative of a promising candidate for highly efficient flexible artificial synapses.

Memristor-CMOS Hybrid Neuron Circuit with Nonideal-Effect Correction Related to Parasitic Resistance for Binary-Memristor-Crossbar Neural Networks

Voltages and currents in a memristor crossbar can be significantly affected due to nonideal effects such as parasitic source, line, and neuron resistance. These nonideal effects related to the parasitic resistance can cause the degradation of the neural network’s performance realized with the nonideal memristor crossbar. To avoid performance degradation due to the parasitic-resistance-related nonideal effects, adaptive training methods were proposed previously. However, the complicated training algorithm could add a heavy computational burden to the neural network hardware. Especially, the hardware and algorithmic burden can be more serious for edge intelligence applications such as Internet of Things (IoT) sensors. In this paper, a memristor-CMOS hybrid neuron circuit is proposed for compensating the parasitic-resistance-related nonideal effects during not the training phase but the inference one, where the complicated adaptive training is not needed. Moreover, unlike the previous linear correction method performed by the external hardware, the proposed correction circuit can be included in the memristor crossbar to minimize the power and hardware overheads for compensating the nonideal effects. The proposed correction circuit has been verified to be able to restore the degradation of source and output voltages in the nonideal crossbar. For the source voltage, the average percentage error of the uncompensated crossbar is as large as 36.7%. If the correction circuit is used, the percentage error in the source voltage can be reduced from 36.7% to 7.5%. For the output voltage, the average percentage error of the uncompensated crossbar is as large as 65.2%. The correction circuit can improve the percentage error in the output voltage from 65.2% to 8.6%. Almost the percentage error can be reduced to ~1/7 if the correction circuit is used. The nonideal memristor crossbar with the correction circuit has been tested for MNIST and CIFAR-10 datasets in this paper. For MNIST, the uncompensated and compensated crossbars indicate the recognition rate of 90.4% and 95.1%, respectively, compared to 95.5% of the ideal crossbar. For CIFAR-10, the nonideal crossbars without and with the nonideal-effect correction show the rate of 85.3% and 88.1%, respectively, compared to the ideal crossbar achieving the rate as large as 88.9%.

Adaptive Extreme Edge Computing for Wearable Devices

Wearable devices are a fast-growing technology with impact on personal healthcare for both society and economy. Due to the widespread of sensors in pervasive and distributed networks, power consumption, processing speed, and system adaptation are vital in future smart wearable devices. The visioning and forecasting of how to bring computation to the edge in smart sensors have already begun, with an aspiration to provide adaptive extreme edge computing. Here, we provide a holistic view of hardware and theoretical solutions toward smart wearable devices that can provide guidance to research in this pervasive computing era. We propose various solutions for biologically plausible models for continual learning in neuromorphic computing technologies for wearable sensors. To envision this concept, we provide a systematic outline in which prospective low power and low latency scenarios of wearable sensors in neuromorphic platforms are expected. We successively describe vital potential landscapes of neuromorphic processors exploiting complementary metal-oxide semiconductors (CMOS) and emerging memory technologies (e.g., memristive devices). Furthermore, we evaluate the requirements for edge computing within wearable devices in terms of footprint, power consumption, latency, and data size. We additionally investigate the challenges beyond neuromorphic computing hardware, algorithms and devices that could impede enhancement of adaptive edge computing in smart wearable devices.

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate in situ training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network's synaptic weights during parallel in situ (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first in situ parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.

Minimization of the Line Resistance Impact on Memdiode-Based Simulations of Multilayer Perceptron Arrays Applied to Pattern Recognition

In this paper, we extend the application of the Quasi-Static Memdiode model to the realistic SPICE simulation of memristor-based single (SLPs) and multilayer perceptrons (MLPs) intended for large dataset pattern recognition. By considering ex-situ training and the classification of the hand-written characters of the MNIST database, we evaluate the degradation of the inference accuracy due to the interconnection resistances for MLPs involving up to three hidden neural layers. Two approaches to reduce the impact of the line resistance are considered and implemented in our simulations, they are the inclusion of an iterative calibration algorithm and the partitioning of the synaptic layers into smaller blocks. The obtained results indicate that MLPs are more sensitive to the line resistance effect than SLPs and that partitioning is the most effective way to minimize the impact of high line resistance values.