Memristor-CMOS Hybrid Neuron Circuit with Nonideal-Effect Correction Related to Parasitic Resistance for Binary-Memristor-Crossbar Neural Networks

Memristor Crossbar Circuits Implementing Equilibrium Propagation for On-Device Learning

Equilibrium propagation (EP) has been proposed recently as a new neural network training algorithm based on a local learning concept, where only local information is used to calculate the weight update of the neural network. Despite the advantages of local learning, numerical iteration for solving the EP dynamic equations makes the EP algorithm less practical for realizing edge intelligence hardware. Some analog circuits have been suggested to solve the EP dynamic equations physically, not numerically, using the original EP algorithm. However, there are still a few problems in terms of circuit implementation: for example, the need for storing the free-phase solution and the lack of essential peripheral circuits for calculating and updating synaptic weights. Therefore, in this paper, a new analog circuit technique is proposed to realize the EP algorithm in practical and implementable hardware. This work has two major contributions in achieving this objective. First, the free-phase and nudge-phase solutions are calculated by the proposed analog circuits simultaneously, not at different times. With this process, analog voltage memories or digital memories with converting circuits between digital and analog domains for storing the free-phase solution temporarily can be eliminated in the proposed EP circuit. Second, a simple EP learning rule relying on a fixed amount of conductance change per programming pulse is newly proposed and implemented in peripheral circuits. The modified EP learning rule can make the weight update circuit practical and implementable without requiring the use of a complicated program verification scheme. The proposed memristor conductance update circuit is simulated and verified for training synaptic weights on memristor crossbars. The simulation results showed that the proposed EP circuit could be used for realizing on-device learning in edge intelligence hardware.

Area-Efficient Mapping of Convolutional Neural Networks to Memristor Crossbars Using Sub-Image Partitioning

Memristor crossbars can be very useful for realizing edge-intelligence hardware, because the neural networks implemented by memristor crossbars can save significantly more computing energy and layout area than the conventional CMOS (complementary metal-oxide-semiconductor) digital circuits. One of the important operations used in neural networks is convolution. For performing the convolution by memristor crossbars, the full image should be partitioned into several sub-images. By doing so, each sub-image convolution can be mapped to small-size unit crossbars, of which the size should be defined as 128 × 128 or 256 × 256 to avoid the line resistance problem caused from large-size crossbars. In this paper, various convolution schemes with 3D, 2D, and 1D kernels are analyzed and compared in terms of neural network's performance and overlapping overhead. The neural network's simulation indicates that the 2D + 1D kernels can perform the sub-image convolution using a much smaller number of unit crossbars with less rate loss than the 3D kernels. When the CIFAR-10 dataset is tested, the mapping of sub-image convolution of 2D + 1D kernels to crossbars shows that the number of unit crossbars can be reduced almost by 90% and 95%, respectively, for 128 × 128 and 256 × 256 crossbars, compared with the 3D kernels. On the contrary, the rate loss of 2D + 1D kernels can be less than 2%. To improve the neural network's performance more, the 2D + 1D kernels can be combined with 3D kernels in one neural network. When the normalized ratio of 2D + 1D layers is around 0.5, the neural network's performance indicates very little rate loss compared to when the normalized ratio of 2D + 1D layers is zero. However, the number of unit crossbars for the normalized ratio = 0.5 can be reduced by half compared with that for the normalized ratio = 0.

Reservoir Computing-Based Design of ZnO Memristor-Type Digital Identification Circuits

Reservoir Computing (RC) is a network architecture inspired by biological neural systems that maps time-dimensional input features to a high-dimensional space for computation. The key to hardware implementation of the RC system is whether sufficient reservoir states can be generated. In this paper, a laboratory-prepared zinc oxide (ZnO) memristor is reported and modeled. The device is found to have nonlinear dynamic responses and characteristics of simulating neurosynaptic long-term potentiation (LTP) and long-term depression (LTD). Based on this, a novel two-level RC structure based on the ZnO memristor is proposed. Novel synaptic encoding is used to maintain stress activity based on the characteristics of after-discharge and proneness to fatigue during synaptic transmission. This greatly alleviates the limitations of the self-attenuating characteristic reservoir of the duration and interval of the input signal. This makes the reservoir, in combination with a fully connected neural network, an ideal system for time series classification. The experimental results show that the recognition rate for the complete MNIST dataset is 95.08% when 35 neurons are present as hidden layers while achieving low training consumption.

Transient Response and Firing Behaviors of Memristive Neuron Circuit

The signal transmission mechanism of the Resistor-Capacitor (RC) circuit is similar to the intracellular and extracellular signal propagating mechanism of the neuron. Thus, the RC circuit can be utilized as the circuit model of the neuron cell membrane. However, resistors are electronic components with the fixed-resistance and have no memory properties. A memristor is a promising neuro-morphological electronic device with nonvolatile, switching, and nonlinear characteristics. First of all, we consider replacing the resistor in the RC neuron circuit with a memristor, which is named the Memristor-Capacitor (MC) circuit, then the MC neuron model is constructed. We compare the charging and discharging processes between the RC and MC neuron circuits. Secondly, two models are compared under the different external stimuli. Finally, the synchronous and asynchronous activities of the RC and MC neuron circuits are performed. Extensive experimental results suggest that the charging and discharging speed of the MC neuron circuit is faster than that of the RC neuron circuit. Given sufficient time and proper external stimuli, the RC and MC neuron circuits can produce the action potentials. The synchronous and asynchronous phenomena in the two neuron circuits reproduce nonlinear dynamic behaviors of the biological neurons.

A Robust Memristor Based on Epitaxial Vertically Aligned Nanostructured BaTiO3 -CeO2 Films on Silicon

With the exploration of ferroelectric materials, researchers have a strong desire to explore the next generation of non-volatile ferroelectric memory with silicon-based epitaxy, high-density storage, and algebraic operations. Herein, a silicon-based memristor with an epitaxial vertically aligned nanostructures BaTiO₃ -CeO₂ film based on La_0.67 Sr_0.33 MnO₃ /SrTiO₃ /Si substrate is reported. The ferroelectric polarization reversal is optimized through the continuous exploring of growth temperature, and the epitaxial structure is obtained, thus it improves the resistance characteristic, the multi-value storage function of five states is achieved, and the robust endurance characteristic can reach 10⁹ cycles. In the synapse plasticity modulated by pulse voltage process, the function of the spiking-time-dependent plasticity and paired-pulse facilitation is simulated successfully. More importantly, the algebraic operations of addition, subtraction, multiplication, and division are realized by using fast speed pulse of the width ≈50 ns. Subsequently, a convolutional neural network is constructed for identifying the CIFAR-10 dataset, to simulate the performance of the device; the online and offline learning recognition rate reach 90.03% and 92.55%, respectively. Overall, this study paves the way for memristors with silicon-based epitaxial ferroelectric films to realize multi-value storage, algebraic operations, and neural computing chip applications.