Analogue pattern recognition with stochastic switching binary CMOS-integrated memristive devices

Blooming and pruning: learning from mistakes with memristive synapses

Blooming and pruning is one of the most important developmental mechanisms of the biological brain in the first years of life, enabling it to adapt its network structure to the demands of the environment. The mechanism is thought to be fundamental for the development of cognitive skills. Inspired by this, Chialvo and Bak proposed in 1999 a learning scheme that learns from mistakes by eliminating from the initial surplus of synaptic connections those that lead to an undesirable outcome. Here, this idea is implemented in a neuromorphic circuit scheme using CMOS integrated HfO2-based memristive devices. The implemented two-layer neural network learns in a self-organized manner without positive reinforcement and exploits the inherent variability of the memristive devices. This approach provides hardware, local, and energy-efficient learning. A combined experimental and simulation-based parameter study is presented to find the relevant system and device parameters leading to a compact and robust memristive neuromorphic circuit that can handle association tasks.

Research progress in architecture and application of RRAM with computing-in-memory

The development of new technologies has led to an explosion of data, while the computation ability of traditional computers is approaching its upper limit. The dominant system architecture is the von Neumann architecture, with the processing and storage units working independently. The data migrate between them via buses, reducing computing speed and increasing energy loss. Research is underway to increase computing power, such as developing new chips and adopting new system architectures. Computing-in-memory (CIM) technology allows data to be computed directly on the memory, changing the current computation-centric architecture and designing a new storage-centric architecture. Resistive random access memory (RRAM) is one of the advanced memories which has appeared in recent years. RRAM can change its resistance with electrical signals at both ends, and the state will be preserved after power-down. It has potential in logic computing, neural networks, brain-like computing, and fused technology of sense-storage-computing. These advanced technologies promise to break the performance bottleneck of traditional architectures and dramatically increase computing power. This paper introduces the basic concepts of computing-in-memory technology and the principle and applications of RRAM and finally gives a conclusion about these new technologies.

Effects of stochastic forces on the nonlinear behaviour of a silicon nitride membrane nanoelectromechanical resonator

In this work, we present the effects of stochastic force generated by white noise on the nonlinear dynamics of a circular silicon nitride membrane. By tuning the membrane to the Duffing nonlinear region, detected signals switching between low- and high-amplitudes have been observed. They are generated by noise-assisted random jumps between bistable states at room temperature and exhibit high sensitivity to the driving frequency. Through artificially heating different mechanical vibration modes by external input of white noise, the switching rate exhibits exponential dependence on the effective temperature and follows with Kramer's law. Furthermore, both the measured switching rate and activation energy exhibit sensitivity to the width of the hysteresis window in nonlinear response and the driving force, which is in qualitative agreement with the theoretical descriptions. Besides, white noise-induced hysteresis window squeezing and bifurcation point shifting have also been observed, which are attributed to the stochastic force modulation of the spring constant of the membrane. These studies are carried out in an all-electric operating scheme at room temperature, paving the way for the exploration of probability distribution-based functional elements that can be massively integrated on-chip.

Recent progress in three-terminal artificial synapses based on 2D materials: from mechanisms to applications

Synapses are essential for the transmission of neural signals. Synaptic plasticity allows for changes in synaptic strength, enabling the brain to learn from experience. With the rapid development of neuromorphic electronics, tremendous efforts have been devoted to designing and fabricating electronic devices that can mimic synapse operating modes. This growing interest in the field will provide unprecedented opportunities for new hardware architectures for artificial intelligence. In this review, we focus on research of three-terminal artificial synapses based on two-dimensional (2D) materials regulated by electrical, optical and mechanical stimulation. In addition, we systematically summarize artificial synapse applications in various sensory systems, including bioplastic bionics, logical transformation, associative learning, image recognition, and multimodal pattern recognition. Finally, the current challenges and future perspectives involving integration, power consumption and functionality are outlined.

Toward Reflective Spiking Neural Networks Exploiting Memristive Devices

The design of modern convolutional artificial neural networks (ANNs) composed of formal neurons copies the architecture of the visual cortex. Signals proceed through a hierarchy, where receptive fields become increasingly more complex and coding sparse. Nowadays, ANNs outperform humans in controlled pattern recognition tasks yet remain far behind in cognition. In part, it happens due to limited knowledge about the higher echelons of the brain hierarchy, where neurons actively generate predictions about what will happen next, i.e., the information processing jumps from reflex to reflection. In this study, we forecast that spiking neural networks (SNNs) can achieve the next qualitative leap. Reflective SNNs may take advantage of their intrinsic dynamics and mimic complex, not reflex-based, brain actions. They also enable a significant reduction in energy consumption. However, the training of SNNs is a challenging problem, strongly limiting their deployment. We then briefly overview new insights provided by the concept of a high-dimensional brain, which has been put forward to explain the potential power of single neurons in higher brain stations and deep SNN layers. Finally, we discuss the prospect of implementing neural networks in memristive systems. Such systems can densely pack on a chip 2D or 3D arrays of plastic synaptic contacts directly processing analog information. Thus, memristive devices are a good candidate for implementing in-memory and in-sensor computing. Then, memristive SNNs can diverge from the development of ANNs and build their niche, cognitive, or reflective computations.

Exploiting Non-idealities of Resistive Switching Memories for Efficient Machine Learning

Novel computing architectures based on resistive switching memories (also known as memristors or RRAMs) have been shown to be promising approaches for tackling the energy inefficiency of deep learning and spiking neural networks. However, resistive switch technology is immature and suffers from numerous imperfections, which are often considered limitations on implementations of artificial neural networks. Nevertheless, a reasonable amount of variability can be harnessed to implement efficient probabilistic or approximate computing. This approach turns out to improve robustness, decrease overfitting and reduce energy consumption for specific applications, such as Bayesian and spiking neural networks. Thus, certain non-idealities could become opportunities if we adapt machine learning methods to the intrinsic characteristics of resistive switching memories. In this short review, we introduce some key considerations for circuit design and the most common non-idealities. We illustrate the possible benefits of stochasticity and compression with examples of well-established software methods. We then present an overview of recent neural network implementations that exploit the imperfections of resistive switching memory, and discuss the potential and limitations of these approaches.

The Image Identification Application with HfO2-Based Replaceable 1T1R Neural Networks

This paper mainly studies the hardware implementation of a fully connected neural network based on the 1T1R (one-transistor-one-resistor) array and its application in handwritten digital image recognition. The 1T1R arrays are prepared by connecting the memristor and nMOSFET in series, and a single-layer and a double-layer fully connected neural network are established. The recognition accuracy of 8 × 8 handwritten digital images reaches 95.19%. By randomly replacing the devices with failed devices, it is found that the stuck-off devices have little effect on the accuracy of the network, but the stuck-on devices will cause a sharp reduction of accuracy. By using the measured conductivity adjustment range and precision data of the memristor, the relationship between the recognition accuracy of the network and the number of hidden neurons is simulated. The simulation results match the experimental results. Compared with the neural network based on the precision of 32-bit floating point, the difference is lower than 1%.

Stochastic binary synapses having sigmoidal cumulative distribution functions for unsupervised learning with spike timing-dependent plasticity

Spike timing-dependent plasticity (STDP), which is widely studied as a fundamental synaptic update rule for neuromorphic hardware, requires precise control of continuous weights. From the viewpoint of hardware implementation, a simplified update rule is desirable. Although simplified STDP with stochastic binary synapses was proposed previously, we find that it leads to degradation of memory maintenance during learning, which is unfavourable for unsupervised online learning. In this work, we propose a stochastic binary synaptic model where the cumulative probability of the weight change evolves in a sigmoidal fashion with potentiation or depression trials, which can be implemented using a pair of switching devices consisting of serially connected multiple binary memristors. As a benchmark test we perform simulations of unsupervised learning of MNIST images with a two-layer network and show that simplified STDP in combination with this model can outperform conventional rules with continuous weights not only in memory maintenance but also in recognition accuracy. Our method achieves 97.3% in recognition accuracy, which is higher than that reported with standard STDP in the same framework. We also show that the high performance of our learning rule is robust against device-to-device variability of the memristor's probabilistic behaviour.

Utilizing the Switching Stochasticity of HfO2/TiOx-Based ReRAM Devices and the Concept of Multiple Device Synapses for the Classification of Overlapping and Noisy Patterns

With the arrival of the Internet of Things (IoT) and the challenges arising from Big Data, neuromorphic chip concepts are seen as key solutions for coping with the massive amount of unstructured data streams by moving the computation closer to the sensors, the so-called "edge computing." Augmenting these chips with emerging memory technologies enables these edge devices with non-volatile and adaptive properties which are desirable for low power and online learning operations. However, an energy- and area-efficient realization of these systems requires disruptive hardware changes. Memristor-based solutions for these concepts are in the focus of research and industry due to their low-power and high-density online learning potential. Specifically, the filamentary-type valence change mechanism (VCM memories) have shown to be a promising candidate In consequence, physical models capturing a broad spectrum of experimentally observed features such as the pronounced cycle-to-cycle (c2c) and device-to-device (d2d) variability are required for accurate evaluation of the proposed concepts. In this study, we present an in-depth experimental analysis of d2d and c2c variability of filamentary-type bipolar switching HfO2/TiOx nano-sized crossbar devices and match the experimentally observed variabilities to our physically motivated JART VCM compact model. Based on this approach, we evaluate the concept of parallel operation of devices as a synapse both experimentally and theoretically. These parallel synapses form a synaptic array which is at the core of neuromorphic chips. We exploit the c2c variability of these devices for stochastic online learning which has shown to increase the effective bit precision of the devices. Finally, we demonstrate that stochastic switching features for a pattern classification task that can be employed in an online learning neural network.

Neuromorphic on-chip recognition of saliva samples of COPD and healthy controls using memristive devices

Chronic Obstructive Pulmonary Disease (COPD) is a life-threatening lung disease, affecting millions of people worldwide. Implementation of Machine Learning (ML) techniques is crucial for the effective management of COPD in home-care environments. However, shortcomings of cloud-based ML tools in terms of data safety and energy efficiency limit their integration with low-power medical devices. To address this, energy efficient neuromorphic platforms can be used for the hardware-based implementation of ML methods. Therefore, a memristive neuromorphic platform is presented in this paper for the on-chip recognition of saliva samples of COPD patients and healthy controls. Results of its performance evaluations showed that the digital neuromorphic chip is capable of recognizing unseen COPD samples with accuracy and sensitivity values of 89% and 86%, respectively. Integration of this technology into personalized healthcare devices will enable the better management of chronic diseases such as COPD.