Silicon synaptic transistor for hardware-based spiking neural network and neuromorphic system

Highly biomimetic spiking neuron using SiGe heterojunction bipolar transistors for energy-efficient neuromorphic systems

We demonstrate a highly biomimetic spiking neuron capable of fast and energy-efficient neuronal oscillation dynamics. Our simple neuron circuit is constructed using silicon-germanium heterojunction based bipolar transistors (HBTs) with nanowire structure. The HBT has a hysteresis window with steep switching characteristics and high current margin in the low voltage range, which enables a high spiking frequency (~ 245 kHz) with low energy consumption (≤ 1.37 pJ/spike). Also, gated structure achieves a stable balance in the activity of the neural system by incorporating both excitatory and inhibitory signal. Furthermore, inhibition of multiple strengths can be realized by adjusting the integration time according to the amplitude of the inhibitory signal. In addition, the spiking frequency can be tuned by mutually controlling the hysteresis window in the HBTs. These results ensure the sparse activity and homeostasis of neural networks.

Technology and Integration Roadmap for Optoelectronic Memristor

Optoelectronic memristors (OMs) have emerged as a promising optoelectronic Neuromorphic computing paradigm, opening up new opportunities for neurosynaptic devices and optoelectronic systems. These OMs possess a range of desirable features including minimal crosstalk, high bandwidth, low power consumption, zero latency, and the ability to replicate crucial neurological functions such as vision and optical memory. By incorporating large-scale parallel synaptic structures, OMs are anticipated to greatly enhance high-performance and low-power in-memory computing, effectively overcoming the limitations of the von Neumann bottleneck. However, progress in this field necessitates a comprehensive understanding of suitable structures and techniques for integrating low-dimensional materials into optoelectronic integrated circuit platforms. This review aims to offer a comprehensive overview of the fundamental performance, mechanisms, design of structures, applications, and integration roadmap of optoelectronic synaptic memristors. By establishing connections between materials, multilayer optoelectronic memristor units, and monolithic optoelectronic integrated circuits, this review seeks to provide insights into emerging technologies and future prospects that are expected to drive innovation and widespread adoption in the near future.

Novel training method for metal-oxide memristive synapse device to overcome trade-off between linearity and dynamic range

Synapse devices are essential for the hardware implementation of neuromorphic computing systems. However, it is difficult to realize ideal synapse devices because of issues such as nonlinear conductance change (linearity) and a small number of conductance states (dynamic range). In this study, the correlation between the linearity and dynamic range was investigated. Consequently, we found a trade-off relationship between the linearity and dynamic range and proposed a novel training method to overcome this trade-off.

Synaptic transistors based on a tyrosine-rich peptide for neuromorphic computing

In this article, we propose an artificial synaptic device based on a proton-conducting peptide material. By using the redox-active property of tyrosine, the Tyr-Tyr-Ala-Cys-Ala-Tyr-Tyr peptide film was utilized as a gate insulator that shows synaptic plasticity owing to the formation of proton electric double layers. The ion gating effects on the transfer characteristics and temporal current responses are shown. Further, timing-dependent responses, including paired-pulse facilitation, synaptic potentiation, and transition from short-term plasticity to long-term plasticity, have been demonstrated for the electrical emulation of biological synapses in the human brain. Herein, we provide a novel material platform that is bio-inspired and biocompatible for use in brain-mimetic electronic devices.

Architecture and Process Integration Overview of 3D NAND Flash Technologies

In the past few decades, NAND flash memory has been one of the most successful nonvolatile storage technologies, and it is commonly used in electronic devices because of its high scalability and reliable switching properties. To overcome the scaling limit of planar NAND flash arrays, various three-dimensional (3D) architectures of NAND flash memory and their process integration methods have been investigated in both industry and academia and adopted in commercial mass production. In this paper, 3D NAND flash technologies are reviewed in terms of their architecture and fabrication methods, and the advantages and disadvantages of the architectures are compared.

Core-Shell Dual-Gate Nanowire Charge-Trap Memory for Synaptic Operations for Neuromorphic Applications

This work showcases the physical insights of a core-shell dual-gate (CSDG) nanowire transistor as an artificial synaptic device with short/long-term potentiation and long-term depression (LTD) operation. Short-term potentiation (STP) is a temporary potentiation of a neural network, and it can be transformed into long-term potentiation (LTP) through repetitive stimulus. In this work, floating body effects and charge trapping are utilized to show the transition from STP to LTP while de-trapping the holes from the nitride layer shows the LTD operation. Furthermore, linearity and symmetry in conductance are achieved through optimal device design and biases. In a system-level simulation, with CSDG nanowire transistor a recognition accuracy of up to 92.28% is obtained in the Modified National Institute of Standards and Technology (MNIST) pattern recognition task. Complementary metal-oxide-semiconductor (CMOS) compatibility and high recognition accuracy makes the CSDG nanowire transistor a promising candidate for the implementation of neuromorphic hardware.

3-bit multilevel operation with accurate programming scheme in TiOx/Al2O3memristor crossbar array for quantized neuromorphic system

As interest in artificial intelligence (AI) and relevant hardware technologies has been developed rapidly, algorithms and network structures have become significantly complicated, causing serious power consumption issues because an enormous amount of computation is required. Neuromorphic computing, a hardware AI technology with memory devices, has emerged to solve this problem. For this application, multilevel operations of synaptic devices are important to imitate floating point weight values in software AI technologies. Furthermore, weight transfer methods to desired weight targets must be arranged for off-chip training. From this point of view, we fabricate 32 × 32 memristor crossbar array and verify the 3-bit multilevel operations. The programming accuracy is verified for 3-bit quantized levels by applying a reset-voltage-control programming scheme to the fabricated TiO_x/Al₂O₃-based memristor array. After that, a synapse composed of two differential memristors and a fully-connected neural network for modified national institute of standards and technology (MNIST) pattern recognition are constructed. The trained weights are post-training quantized in consideration of the 3-bit characteristics of the memristor. Finally, the effect of programming error on classification accuracy is verified based on the measured data, and we obtained 98.12% classification accuracy for MNIST data with the programming accuracy of 1.79% root-mean-square-error. These results imply that the proposed reset-voltage-control programming scheme can be utilized for a precise tuning, and expected to contribute for the development of a neuromorphic system capable of highly precise weight transfer.

Quantized Weight Transfer Method Using Spike-Timing-Dependent Plasticity for Hardware Spiking Neural Network

A hardware-based spiking neural network (SNN) has attracted many researcher’s attention due to its energy-efficiency. When implementing the hardware-based SNN, offline training is most commonly used by which trained weights by a software-based artificial neural network (ANN) are transferred to synaptic devices. However, it is time-consuming to map all the synaptic weights as the scale of the neural network increases. In this paper, we propose a method for quantized weight transfer using spike-timing-dependent plasticity (STDP) for hardware-based SNN. STDP is an online learning algorithm for SNN, but we utilize it as the weight transfer method. Firstly, we train SNN using the Modified National Institute of Standards and Technology (MNIST) dataset and perform weight quantization. Next, the quantized weights are mapped to the synaptic devices using STDP, by which all the synaptic weights connected to a neuron are transferred simultaneously, reducing the number of pulse steps. The performance of the proposed method is confirmed, and it is demonstrated that there is little reduction in the accuracy at more than a certain level of quantization, but the number of pulse steps for weight transfer substantially decreased. In addition, the effect of the device variation is verified.

Neuromorphic Computing Using NAND Flash Memory Architecture With Pulse Width Modulation Scheme

A novel operation scheme is proposed for high-density and highly robust neuromorphic computing based on NAND flash memory architecture. Analog input is represented with time-encoded input pulse by pulse width modulation (PWM) circuit, and 4-bit synaptic weight is represented with adjustable conductance of NAND cells. Pulse width modulation scheme for analog input value and proposed operation scheme is suitably applicable to the conventional NAND flash architecture to implement a neuromorphic system without additional change of memory architecture. Saturated current-voltage characteristic of NAND cells eliminates the effect of serial resistance of adjacent cells where a pass bias is applied in a synaptic string and IR drop of metal wire resistance. Multiply-accumulate (MAC) operation of 4-bit weight and width-modulated input can be performed in a single input step without additional logic operation. Furthermore, the effect of quantization training (QT) on the classification accuracy is investigated compared with post-training quantization (PTQ) with 4-bit weight. Lastly, a sufficiently low current variance of NAND cells obtained by the read-verify-write (RVW) scheme achieves satisfying accuracies of 98.14 and 89.6% for the MNIST and CIFAR10 images, respectively.

An artificial sensory neuron with visual-haptic fusion

Human behaviors are extremely sophisticated, relying on the adaptive, plastic and event-driven network of sensory neurons. Such neuronal system analyzes multiple sensory cues efficiently to establish accurate depiction of the environment. Here, we develop a bimodal artificial sensory neuron to implement the sensory fusion processes. Such a bimodal artificial sensory neuron collects optic and pressure information from the photodetector and pressure sensors respectively, transmits the bimodal information through an ionic cable, and integrates them into post-synaptic currents by a synaptic transistor. The sensory neuron can be excited in multiple levels by synchronizing the two sensory cues, which enables the manipulating of skeletal myotubes and a robotic hand. Furthermore, enhanced recognition capability achieved on fused visual/haptic cues is confirmed by simulation of a multi-transparency pattern recognition task. Our biomimetic design has the potential to advance technologies in cyborg and neuromorphic systems by endowing them with supramodal perceptual capabilities.

A carbon-based memristor design for associative learning activities and neuromorphic computing

Carbon quantum dots (QDs) have attracted significant interest due to their excellent electronic properties and wide application prospects. However, the application of carbon QDs has been rarely reported in memristors. Here, a memristor model with carbon conductive filaments (CFs) is proposed for the first time based on carbon quantum dots. The CF-based devices exhibited excellent resistive switching performance, in particular a narrow range of SET and RESET voltages and good power efficiency and retention properties. These devices could also emulate important biological synapse performances, such as the transition from short-term plasticity (STP) to long-term potentiation (LTP) behaviors, long-term depression (LTD) behavior, and four types of spike-timing-dependent plasticity (STDP) learning rules. Interestingly, Pavlovian associative learning functions were also reliably demonstrated in the memristor device (MD). The digit recognition ability of the MDs was evaluated though a single-layer perceptron model, in which the recognition accuracy of digits reached 92.63% after 250 training iterations. The transmission electron microscopy (TEM) results evidenced that the carbon CF was found in the MD at the "ON" state. Thus, this new carbon CF-based mechanism for memristors provides a new idea for achieving better neuromorphic MDs and applications.

Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks

Spiking neural networks (SNNs) are considered as the third generation of artificial neural networks, having the potential to improve the energy efficiency of conventional computing systems. Although the firing rate of a spiking neuron is an approximation of rectified linear unit (ReLU) activation in an analog-valued neural network (ANN), there remain many challenges to be overcome owing to differences in operation between ANNs and SNNs. Unlike actual biological and biophysical processes, various hardware implementations of neurons and SNNs do not allow the membrane potential to fall below the resting potential—in other words, neurons must allow the sub-resting membrane potential. Because there occur an excitatory post-synaptic potential (EPSP) as well as an inhibitory post-synaptic potential (IPSP), negatively valued synaptic weights in SNNs induce the sub-resting membrane potential at some time point. If a membrane is not allowed to hold the sub-resting potential, errors will accumulate over time, resulting in inaccurate inference operations. This phenomenon is not observed in ANNs given their use of only spatial synaptic integration, but it can cause serious performance degradation in SNNs. In this paper, we demonstrate the impact of the sub-resting membrane potential on accurate inference operations in SNNs. Moreover, several important considerations for a hardware SNN that can maintain the sub-resting membrane potential are discussed. All of the results in this paper indicate that it is essential for neurons to allow the sub-resting membrane potential in order to realize high-performance SNNs.

Artificial 2D van der Waals Synapse Devices via Interfacial Engineering for Neuromorphic Systems

Despite extensive investigations of a wide variety of artificial synapse devices aimed at realizing a neuromorphic hardware system, the identification of a physical parameter that modulates synaptic plasticity is still required. In this context, a novel two-dimensional architecture consisting of a NbSe₂/WSe₂/Nb₂O₅ heterostructure placed on an SiO₂/p+ Si substrate was designed to overcome the limitations of the conventional silicon-based complementary metal-oxide semiconductor technology. NbSe₂, WSe₂, and Nb₂O₅ were used as the metal electrode, active channel, and conductance-modulating layer, respectively. Interestingly, it was found that the post-synaptic current was successfully modulated by the thickness of the interlayer Nb₂O₅, with a thicker interlayer inducing a higher synapse spike current and a stronger interaction in the sequential pulse mode. Introduction of the Nb₂O₅ interlayer can facilitate the realization of reliable and controllable synaptic devices for brain-inspired integrated neuromorphic systems.

3-D Synapse Array Architecture Based on Charge-Trap Flash Memory for Neuromorphic Application

In order to address a fundamental bottleneck of conventional digital computers, there is recently a tremendous upsurge of investigations on hardware-based neuromorphic systems. To emulate the functionalities of artificial neural networks, various synaptic devices and their 2-D cross-point array structures have been proposed. In our previous work, we proposed the 3-D synapse array architecture based on a charge-trap flash (CTF) memory. It has the advantages of high-density integration of 3-D stacking technology and excellent reliability characteristics of mature CTF device technology. This paper examines some issues of the 3-D synapse array architecture. Also, we propose an improved structure and programming method compared to the previous work. The synaptic characteristics of the proposed method are closely examined and validated through a technology computer-aided design (TCAD) device simulation and a system-level simulation for the pattern recognition task. The proposed technology will be the promising solution for high-performance and high-reliability of neuromorphic hardware systems.

Robust Ag/ZrO2/WS2/Pt Memristor for Neuromorphic Computing

The development of the information age has made resistive random access memory (RRAM) a critical nanoscale memristor device (MD). However, due to the randomness of the area formed by the conductive filaments (CFs), the RRAM MD still suffers from a problem of insufficient reliability. In this study, the memristor of Ag/ZrO₂/WS₂/Pt structure is proposed for the first time, and a layer of two-dimensional (2D) WS₂ nanosheets was inserted into the MD to form 2D material and oxide double-layer MD (2DOMD) to improve the reliability of single-layer devices. The results indicate that the electrochemical metallization memory cell exhibits a highly stable memristive switching and concentrated ON- and OFF-state voltage distribution, high speed (∼10 ns), and robust endurance (>10⁹ cycles). This result is superior to MDs with a single-layer ZrO₂ or WS₂ film because two layers have different ion transport rates, thereby limiting the rupture/rejuvenation of CFs to the bilayer interface region, which can greatly reduce the randomness of CFs in MDs. Moreover, we used the handwritten recognition dataset (i.e., the Modified National Institute of Standards and Technology (MNIST) database) for neuromorphic simulations. Furthermore, biosynaptic functions and plasticity, including spike-timing-dependent plasticity and paired-pulse facilitation, have been successfully achieved. By incorporating 2D materials and oxides into a double-layer MD, the practical application of RRAM MD can be significantly enhanced to facilitate the development of artificial synapses for brain-enhanced computing systems in the future.

Solving Overlapping Pattern Issues in On-Chip Learning of Bio-Inspired Neuromorphic System with Synaptic Transistors

Recently, bio-inspired neuromorphic systems have been attracting widespread interest thanks to their energy-efficiency compared to conventional von Neumann architecture computing systems. Previously, we reported a silicon synaptic transistor with an asymmetric dual-gate structure for the direct connection between synaptic devices and neuron circuits. In this study, we study a hardware-based spiking neural network for pattern recognition using a binary modified National Institute of Standards and Technology (MNIST) dataset with a device model. A total of three systems were compared with regard to learning methods, and it was confirmed that the feature extraction of each pattern is the most crucial factor to avoiding overlapping pattern issues and obtaining a high pattern classification ability.

A Spiking Neural Network Based on the Model of VO2—Neuron

In this paper, we present an electrical circuit of a leaky integrate-and-fire neuron with one VO2 switch, which models the properties of biological neurons. Based on VO2 neurons, a two-layer spiking neural network consisting of nine input and three output neurons is modeled in the SPICE simulator. The network contains excitatory and inhibitory couplings, and implements the winner-takes-all principle in pattern recognition. Using a supervised Spike-Timing-Dependent Plasticity training method and a timing method of information coding, the network was trained to recognize three patterns with dimensions of 3 × 3 pixels. The neural network is able to recognize up to 105 images per second, and has the potential to increase the recognition speed further.

A Neuromorphic Device Implemented on a Salmon-DNA Electrolyte and its Application to Artificial Neural Networks

A bioinspired neuromorphic device operating as synapse and neuron simultaneously, which is fabricated on an electrolyte based on Cu2+-doped salmon deoxyribonucleic acid (S-DNA) is reported. Owing to the slow Cu2+ diffusion through the base pairing sites in the S-DNA electrolyte, the synaptic operation of the S-DNA device features special long-term plasticity with negative and positive nonlinearity values for potentiation and depression (αp and αd), respectively, which consequently improves the learning/recognition efficiency of S-DNA-based neural networks. Furthermore, the representative neuronal operation, "integrate-and-fire," is successfully emulated in this device by adjusting the duration time of the input voltage stimulus. In particular, by applying a Cu2+ doping technique to the S-DNA neuromorphic device, the characteristics for synaptic weight updating are enhanced (|αp|: 31→20, |αd|: 11→18, weight update margin: 33→287 nS) and also the threshold conditions for neuronal firing (amplitude and number of stimulus pulses) are modulated. The improved synaptic characteristics consequently increase the Modified National Institute of Standards and Technology (MNIST) pattern recognition rate from 38% to 44% (single-layer perceptron model) and from 89.42% to 91.61% (multilayer perceptron model). This neuromorphic device technology based on S-DNA is expected to contribute to the successful implementation of a future neuromorphic system that simultaneously satisfies high integration density and remarkable recognition accuracy.

Nonvolatile Memory Materials for Neuromorphic Intelligent Machines

Recent progress in deep learning extends the capability of artificial intelligence to various practical tasks, making the deep neural network (DNN) an extremely versatile hypothesis. While such DNN is virtually built on contemporary data centers of the von Neumann architecture, physical (in part) DNN of non-von Neumann architecture, also known as neuromorphic computing, can remarkably improve learning and inference efficiency. Particularly, resistance-based nonvolatile random access memory (NVRAM) highlights its handy and efficient application to the multiply-accumulate (MAC) operation in an analog manner. Here, an overview is given of the available types of resistance-based NVRAMs and their technological maturity from the material- and device-points of view. Examples within the strategy are subsequently addressed in comparison with their benchmarks (virtual DNN in deep learning). A spiking neural network (SNN) is another type of neural network that is more biologically plausible than the DNN. The successful incorporation of resistance-based NVRAM in SNN-based neuromorphic computing offers an efficient solution to the MAC operation and spike timing-based learning in nature. This strategy is exemplified from a material perspective. Intelligent machines are categorized according to their architecture and learning type. Also, the functionality and usefulness of NVRAM-based neuromorphic computing are addressed.

Analog Synaptic Behavior of a Silicon Nitride Memristor

In this paper, we present a synapse function using analog resistive-switching behaviors in a SiN_x-based memristor with a complementary metal-oxide-semiconductor compatibility and expandability to three-dimensional crossbar array architecture. A progressive conductance change is attainable as a result of the gradual growth and dissolution of the conducting path, and the series resistance of the AlO_y layer in the Ni/SiN_x/AlO_y/TiN memristor device enhances analog switching performance by reducing current overshoot. A continuous and smooth gradual reset switching transition can be observed with a compliance current limit (>100 μA), and is highly suitable for demonstrating synaptic characteristics. Long-term potentiation and long-term depression are obtained by means of identical pulse responses. Moreover, symmetric and linear synaptic behaviors are significantly improved by optimizing pulse response conditions, which is verified by a neural network simulation. Finally, we display the spike-timing-dependent plasticity with the multipulse scheme. This work provides a possible way to mimic biological synapse function for energy-efficient neuromorphic systems by using a conventional passive SiN_x layer as an active dielectric.