Threshold switching memristor-based stochastic neurons for probabilistic computing

Leveraging volatile memristors in neuromorphic computing: from materials to system implementation

Inspired by the functions of biological neural networks, volatile memristors are essential for implementing neuromorphic computing. These devices enable large-scale and energy-efficient data processing by emulating neural functionalities through dynamic resistance changes. The threshold switching characteristics of volatile memristors, which are driven by various mechanisms in materials ranging from oxides to chalcogenides, make them versatile and suitable for neuromorphic computing systems. Understanding these mechanisms and selecting appropriate devices for specific applications are crucial for optimizing the performance. However, the existing literature lacks a comprehensive review of switching mechanisms, their compatibility with different applications, and a deeper exploration of the spatiotemporal processing capabilities and inherent stochasticity of volatile memristors. This review begins with a detailed analysis of the operational principles and material characteristics of volatile memristors. Their diverse applications are then explored, emphasizing their role in crossbar arrays, artificial receptors, and neurons. Furthermore, the potential of volatile memristors in artificial inference systems and reservoir computing is discussed, due to their spatiotemporal processing capabilities. Hardware security applications and probabilistic computing are also examined, where the inherent stochasticity of the devices can improve the system robustness and adaptability. To conclude, the suitability of different switching mechanisms for various applications is evaluated, and future perspectives for the development and implementation of volatile memristors are presented. This review aims to fill the gaps in existing research and highlight the potential of volatile memristors to drive innovation in neuromorphic computing, paving the way for more efficient and powerful computational paradigms.

A stochastic photo-responsive memristive neuron for an in-sensor visual system based on a restricted Boltzmann machine

In-sensor computing has gained attention as a solution to overcome the von Neumann computing bottlenecks inherent in conventional sensory systems. This attention is due to the ability of sensor elements to directly extract meaningful information from external signals, thereby simplifying complex data. The advantage of in-sensor computing can be maximized with the sampling principle of a restricted Boltzmann machine (RBM) to extract significant features. In this study, a stochastic photo-responsive neuron is developed using a TiN/In-Ga-Zn-O/TiN optoelectronic memristor and an Ag/HfO2/Pt threshold-switching memristor, which can be configured as an input neuron in an in-sensor RBM. It demonstrates a sigmoidal switching probability depending on light intensity. The stochastic properties allow for the simultaneous exploration of various neuron states within the network, making identifying optimal features in complex images easier. Based on semi-empirical simulations, high recognition accuracies of 90.9% and 95.5% are achieved using handwritten digit and face image datasets, respectively. In addition, the in-sensor RBM effectively reconstructs abnormal face images, indicating that integrating in-sensor computing with probabilistic neural networks can lead to reliable and efficient image recognition under unpredictable real-world conditions.

Memristive Monte Carlo DropConnect crossbar array enabled by device and algorithm co-design

Device and algorithm co-design aims to develop energy-efficient hardware that directly implements complex algorithms and optimizes algorithms to match the hardware's characteristics. Specifically, neuromorphic computing algorithms are constantly growing in complexity, necessitating an ongoing search for hardware implementations capable of handling these intricate algorithms. Here, we present a memristive Monte Carlo DropConnect (MC-DC) crossbar array developed through a hardware algorithm co-design approach. To implement the MC-DC neural network, stochastic switching and analog memory characteristics are required, and we achieved them using Ag-based diffusive selectors and Ru-based electrochemical metalization (ECM) memristors, respectively. The devices were integrated with a one-selector one-memristor (1S1M) structure, and their well-matched operating voltages and currents enabled stochastic readout and deterministic analog programming. With the integrated hardware, we successfully demonstrated the MC-DC operation. Additionally, the selector allowed for the control of switching polarity, and by understanding this hardware characteristic, we were able to modify the algorithm to fit it and further improve the network performance.

Mott memristor based stochastic neurons for probabilistic computing

Many studies suggest that probabilistic spiking in biological neural systems is beneficial as it aids learning and provides Bayesian inference-like dynamics. If appropriately utilised, noise and stochasticity in nanoscale devices can benefit neuromorphic systems. In this paper, we build a stochastic leaky integrate and fire (LIF) neuron, utilising a Mott memristor's inherent stochastic switching dynamics. We demonstrate that the developed LIF neuron is capable of biological neural dynamics. We leverage these characteristics of the proposed LIF neuron by integrating it into a population-coded spiking neural network and a spiking restricted Boltzmann machine (sRBM), thereby showcasing its ability to implement probabilistic learning and inference. The sRBM achieves a software-comparable accuracy of 87.13%. Unlike CMOS-based probabilistic neurons, our design does not require any external noise sources. The designed neurons are highly energy efficient and ultra-compact, requiring only three components: a resistor, a capacitor and a memristor device.

Ultralow Energy Consumption Angstrom-Fluidic Memristor

The emergence of nanofluidic memristors has made a giant leap to mimic the neuromorphic functions of biological neurons. Here, we report neuromorphic signaling using Angstrom-scale funnel-shaped channels with poly-l-lysine (PLL) assembled at nano-openings. We found frequency-dependent current-voltage characteristics under sweeping voltage, which represents a diode in low frequencies, but it showed pinched current hysteresis as frequency increases. The current hysteresis is strongly dependent on pH values but weakly dependent on salt concentration. We attributed the current hysteresis to the entropy barrier of PLL molecules entering and exiting the Angstrom channels, resulting in reversible voltage-gated open-close state transitions. We successfully emulated the synaptic adaptation of Hebbian learning using voltage spikes and obtained a minimum energy consumption of 2-23 fJ in each spike per channel. Our findings pave a new way to mimic neuronal functions by Angstrom channels in low energy consumption.

Filament-free memristors for computing

Memristors have attracted increasing attention due to their tremendous potential to accelerate data-centric computing systems. The dynamic reconfiguration of memristive devices in response to external electrical stimuli can provide highly desirable novel functionalities for computing applications when compared with conventional complementary-metal-oxide-semiconductor (CMOS)-based devices. Those most intensively studied and extensively reviewed memristors in the literature so far have been filamentary type memristors, which typically exhibit a relatively large variability from device to device and from switching cycle to cycle. On the other hand, filament-free switching memristors have shown a better uniformity and attractive dynamical properties, which can enable a variety of new computing paradigms but have rarely been reviewed. In this article, a wide range of filament-free switching memristors and their corresponding computing applications are reviewed. Various junction structures, switching properties, and switching principles of filament-free memristors are surveyed and discussed. Furthermore, we introduce recent advances in different computing schemes and their demonstrations based on non-filamentary memristors. This Review aims to present valuable insights and guidelines regarding the key computational primitives and implementations enabled by these filament-free switching memristors.

Memristors based on NdNiO3 nanocrystals film as sensory neurons for neuromorphic computing

By mimicking the behavior of the human brain, artificial neural systems offer the possibility to further improve computing efficiency and solve the von Neumann bottleneck. In particular, neural systems with perceptual capability expand the application field and lay a good foundation for the construction of perceptual storage and computational systems. However, research on neurons with perceptual functions is still relatively scarce, with most works focusing on optoelectronic synapses. The neuron is important for neuromorphic computing systems because neurons output excitatory or inhibitory stimuli to regulate the weight of synapses. Therefore, the construction of sensory neurons is crucial to expand the application range of brain-like neural computing. Here, an artificial sensory neuron is proposed, which is constructed using a photosensitive bipolar threshold switching memristor based on NdNiO3 (NNO) nanocrystals. These metallic phase nanocrystals can not only enhance the local electric field, but also act as a reservoir for defects (VoS) to guide the growth of conductive filaments and stabilize the performance of the device. They present stable bipolar threshold switching behavior with a low 120 nW set power, and the operating voltages decreased in light due to photocarrier action. A leaky integrate firing (LIF) neuron has been realized, which achieved key biological neuron functions, such as all-or-nothing spiking, threshold-driven firing, refractory period, and spiking frequency modulation. The LIF neurons receiving optical inputs have the properties of an artificial sensory neuron. It could regulate the spiking output frequency at different light densities, which could be used for a ship approaching a port. This work provides a promising hardware implementation towards constructing high-performance artificial intelligence to assist ships at night in a sensory system.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

Memristor-based neural networks: a bridge from device to artificial intelligence

Since the beginning of the 21st century, there is no doubt that the importance of artificial intelligence has been highlighted in many fields, among which the memristor-based artificial neural network technology is expected to break through the limitation of von Neumann so as to realize the replication of the human brain by enabling strong parallel computing ability and efficient data processing and become an important way towards the next generation of artificial intelligence. A new type of nanodevice, namely memristor, which is based on the variability of its resistance value, not only has very important applications in nonvolatile information storage, but also presents obsessive progressiveness in highly integrated circuits, making it one of the most promising circuit components in the post-Moore era. In particular, memristors can effectively simulate neural synapses and build neural networks; thus, they can be applied for the preparation of various artificial intelligence systems. This study reviews the research progress of memristors in artificial neural networks in detail and highlights the structural advantages and frontier applications of neural networks based on memristors. Finally, some urgent problems and challenges in current research are summarized and corresponding solutions and future development trends are put forward.

Memristor-Based Signal Processing for Compressed Sensing

With the rapid progress of artificial intelligence, various perception networks were constructed to enable Internet of Things (IoT) applications, thereby imposing formidable challenges to communication bandwidth and information security. Memristors, which exhibit powerful analog computing capabilities, emerged as a promising solution expected to address these challenges by enabling the development of the next-generation high-speed digital compressed sensing (CS) technologies for edge computing. However, the mechanisms and fundamental properties of memristors for achieving CS remain unclear, and the underlying principles for selecting different implementation methods based on various application scenarios have yet to be elucidated. A comprehensive overview of memristor-based CS techniques is currently lacking. In this article, we systematically presented CS requirements on device performance and hardware implementation. The relevant models were analyzed and discussed from the mechanism level to elaborate the memristor CS system scientifically. In addition, the method of deploying CS hardware using the powerful signal processing capabilities and unique performance of memristors was further reviewed. Subsequently, the potential of memristors in all-in-one compression and encryption was anticipated. Finally, existing challenges and future outlooks for memristor-based CS systems were discussed.

Stochastic photonic spiking neuron for Bayesian inference with unsupervised learning

Stochasticity is an inherent feature of biological neural activities. We propose a noise-injection scheme to implement a GHz-rate stochastic photonic spiking neuron (S-PSN). The firing-probability encoding is experimentally demonstrated and exploited for Bayesian inference with unsupervised learning. In a breast diagnosis task, the stochastic photonic spiking neural network (S-PSNN) can not only achieve a classification accuracy of 96.6%, but can also evaluate the diagnosis uncertainty with prediction entropies. As a result, the misdiagnosis rate is reduced by 80% compared to that of a conventional deterministic photonic spiking neural network (D-PSNN) for the same task. The GHz-rate S-PSN endows the neuromorphic photonics with high-speed Bayesian inference for reliable information processing in error-critical scenarios.

Reconfigurable neuromorphic memristor network for ultralow-power smart textile electronics

Neuromorphic computing memristors are attractive to construct low-power- consumption electronic textiles due to the intrinsic interwoven architecture and promising applications in wearable electronics. Developing reconfigurable fiber-based memristors is an efficient method to realize electronic textiles that capable of neuromorphic computing function. However, the previously reported artificial synapse and neuron need different materials and configurations, making it difficult to realize multiple functions in a single device. Herein, a textile memristor network of Ag/MoS2/HfAlOx/carbon nanotube with reconfigurable characteristics was reported, which can achieve both nonvolatile synaptic plasticity and volatile neuron functions. In addition, a single reconfigurable memristor can realize integrate-and-fire function, exhibiting significant advantages in reducing the complexity of neuron circuits. The firing energy consumption of fiber-based memristive neuron is 1.9 fJ/spike (femtojoule-level), which is at least three orders of magnitude lower than that of the reported biological and artificial neuron (picojoule-level). The ultralow energy consumption makes it possible to create an electronic neural network that reduces the energy consumption compared to human brain. By integrating the reconfigurable synapse, neuron and heating resistor, a smart textile system is successfully constructed for warm fabric application, providing a unique functional reconfiguration pathway toward the next-generation in-memory computing textile system.

Spintronic Integrate-Fire-Reset Neuron with Stochasticity for Neuromorphic Computing

Spintronics has been recently extended to neuromorphic computing because of its energy efficiency and scalability. However, a biorealistic spintronic neuron with probabilistic "spiking" and a spontaneous reset functionality has not been demonstrated yet. Here, we propose a biorealistic spintronic neuron device based on the heavy metal (HM)/ferromagnet (FM)/antiferromagnet (AFM) spin-orbit torque (SOT) heterostructure. The spintronic neuron can autoreset itself after firing due to the exchange bias of the AFM. The firing process is inherently stochastic because of the competition between the SOT and AFM pinning effects. We also implement a restricted Boltzmann machine (RBM) and stochastic integration multilayer perceptron (SI-MLP) using our proposed neuron. Despite the bit-width limitation, the proposed spintronic model can achieve an accuracy of 97.38% in pattern recognition, which is even higher than the baseline accuracy (96.47%). Our results offer a spintronic device solution to emulate biologically realistic spiking neurons.

A Memristor-Based High-Resolution A/D Converter

Based on the voltage threshold adaptive memristor (VTEAM) model, this paper proposes a circuit design of a memristor-based A/D converter, which can achieve high-resolution conversion by simple configuration. For this A/D converter, there are the input voltage stage and the reference voltage stage in one conversion. According to the memristance change in the two stages, the input analog voltage is converted as the corresponding digital value. In the input voltage stage, the memristance increases from the initial memristance. Meanwhile, the counter rises its value from zero to the maximum. Next, the memristance returns to the initial memristance in the reference voltage stage. At the same time, the counting value starts to increase from zero again. Then, the input analog voltage is mapped to the eventual counting value of the reference voltage stage. The simulations of the memristor-based A/D converter demonstrate that it has good conversion performance. The proposed memristor-based A/D converter not only has more brilliant performance than the CMOS A/D converter, but also has the advantages over existing memristor-based A/D converters of anti-interference ability and high resolution.

Controllable SiOx Nanorod Memristive Neuron for Probabilistic Bayesian Inference

Modern artificial neural network technology using a deterministic computing framework is faced with a critical challenge in dealing with massive data that are largely unstructured and ambiguous. This challenge demands the advances of an elementary physical device for tackling these uncertainties. Here, we designed and fabricated a SiO_x nanorod memristive device by employing the glancing angle deposition (GLAD) technique, suggesting a controllable stochastic artificial neuron that can mimic the fundamental integrate-and-fire signaling and stochastic dynamics of a biological neuron. The nanorod structure provides the random distribution of multiple nanopores all across the active area, capable of forming a multitude of Si filaments at many SiO_x nanorod edges after the electromigration process, leading to a stochastic switching event with very high dynamic range (≈5.15 × 10¹⁰ ) and low energy (≈4.06 pJ). Different probabilistic activation (ProbAct) functions in a sigmoid form are implemented, showing its controllability with low variation by manufacturing and electrical programming schemes. Furthermore, as an application prospect, based on the suggested memristive neuron, we demonstrated the self-resting neural operation with the local circuit configuration and revealed probabilistic Bayesian inferences for genetic regulatory networks with low normalized mean squared errors (≈2.41 × 10^-2 ) and its robustness to the ProbAct variation.

Brain-Inspired Hardware Solutions for Inference in Bayesian Networks

The implementation of inference (i.e., computing posterior probabilities) in Bayesian networks using a conventional computing paradigm turns out to be inefficient in terms of energy, time, and space, due to the substantial resources required by floating-point operations. A departure from conventional computing systems to make use of the high parallelism of Bayesian inference has attracted recent attention, particularly in the hardware implementation of Bayesian networks. These efforts lead to several implementations ranging from digital circuits, mixed-signal circuits, to analog circuits by leveraging new emerging nonvolatile devices. Several stochastic computing architectures using Bayesian stochastic variables have been proposed, from FPGA-like architectures to brain-inspired architectures such as crossbar arrays. This comprehensive review paper discusses different hardware implementations of Bayesian networks considering different devices, circuits, and architectures, as well as a more futuristic overview to solve existing hardware implementation problems.

Artificial Visual Perception Nervous System Based on Low-Dimensional Material Photoelectric Memristors

The visual perception system is the most important system for human learning since it receives over 80% of the learning information from the outside world. With the exponential growth of artificial intelligence technology, there is a pressing need for high-energy and area-efficiency visual perception systems capable of processing efficiently the received natural information. Currently, memristors with their elaborate dynamics, excellent scalability, and information (e.g., visual, pressure, sound, etc.) perception ability exhibit tremendous potential for the application of visual perception. Here, we propose a fully memristor-based artificial visual perception nervous system (AVPNS) which consists of a quantum-dot-based photoelectric memristor and a nanosheet-based threshold-switching (TS) memristor. We use a photoelectric and a TS memristor to implement the synapse and leaky integrate-and-fire (LIF) neuron functions, respectively. With the proposed AVPNS we successfully demonstrate the biological image perception, integration and fire, as well as the biosensitization process. Furthermore, the self-regulation process of a speed meeting control system in driverless automobiles can be accurately and conceptually emulated by this system. Our work shows that the functions of the biological visual nervous system may be systematically emulated by a memristor-based hardware system, thus expanding the spectrum of memristor applications in artificial intelligence.

Artificial Neurons Based on Ag/V2C/W Threshold Switching Memristors

Artificial synapses and neurons are two critical, fundamental bricks for constructing hardware neural networks. Owing to its high-density integration, outstanding nonlinearity, and modulated plasticity, memristors have attracted emerging attention on emulating biological synapses and neurons. However, fabricating a low-power and robust memristor-based artificial neuron without extra electrical components is still a challenge for brain-inspired systems. In this work, we demonstrate a single two-dimensional (2D) MXene(V₂C)-based threshold switching (TS) memristor to emulate a leaky integrate-and-fire (LIF) neuron without auxiliary circuits, originating from the Ag diffusion-based filamentary mechanism. Moreover, our V₂C-based artificial neurons faithfully achieve multiple neural functions including leaky integration, threshold-driven fire, self-relaxation, and linear strength-modulated spike frequency characteristics. This work demonstrates that three-atom-type MXene (e.g., V₂C) memristors may provide an efficient method to construct the hardware neuromorphic computing systems.

Memristor-based biomimetic compound eye for real-time collision detection

The lobula giant movement detector (LGMD) is the movement-sensitive, wide-field visual neuron positioned in the third visual neuropile of lobula. LGMD neuron can anticipate collision and trigger avoidance efficiently owing to the earlier occurring firing peak before collision. Vision chips inspired by the LGMD have been successfully implemented in very-large-scale-integration (VLSI) system. However, transistor-based chips and single devices to simulate LGMD neurons make them bulky, energy-inefficient and complicated. The devices with relatively compact structure and simple operation mode to mimic the escape response of LGMD neuron have not been realized yet. Here, the artificial LGMD visual neuron is implemented using light-mediated threshold switching memristor. The non-monotonic response to light flow field originated from the formation and break of Ag conductive filaments is analogue to the escape response of LGMD neuron. Furthermore, robot navigation with obstacle avoidance capability and biomimetic compound eyes with wide field-of-view (FoV) detection capability are demonstrated.

Inherent stochasticity during insulator-metal transition in VO2

Vanadium dioxide (VO2), which exhibits a near-room-temperature insulator-metal transition, has great potential in applications of neuromorphic computing devices. Although its volatile switching property, which could emulate neuron spiking, has been studied widely, nanoscale studies of the structural stochasticity across the phase transition are still lacking. In this study, using in situ transmission electron microscopy and ex situ resistive switching measurement, we successfully characterized the structural phase transition between monoclinic and rutile VO2 at local areas in planar VO2/TiO2 device configuration under external biasing. After each resistive switching, different VO2 monoclinic crystal orientations are observed, forming different equilibrium states. We have evaluated a statistical cycle-to-cycle variation, demonstrated a stochastic nature of the volatile resistive switching, and presented an approach to study in-plane structural anisotropy. Our microscopic studies move a big step forward toward understanding the volatile switching mechanisms and the related applications of VO2 as the key material of neuromorphic computing.

Emerging Artificial Neuron Devices for Probabilistic Computing

In recent decades, artificial intelligence has been successively employed in the fields of finance, commerce, and other industries. However, imitating high-level brain functions, such as imagination and inference, pose several challenges as they are relevant to a particular type of noise in a biological neuron network. Probabilistic computing algorithms based on restricted Boltzmann machine and Bayesian inference that use silicon electronics have progressed significantly in terms of mimicking probabilistic inference. However, the quasi-random noise generated from additional circuits or algorithms presents a major challenge for silicon electronics to realize the true stochasticity of biological neuron systems. Artificial neurons based on emerging devices, such as memristors and ferroelectric field-effect transistors with inherent stochasticity can produce uncertain non-linear output spikes, which may be the key to make machine learning closer to the human brain. In this article, we present a comprehensive review of the recent advances in the emerging stochastic artificial neurons (SANs) in terms of probabilistic computing. We briefly introduce the biological neurons, neuron models, and silicon neurons before presenting the detailed working mechanisms of various SANs. Finally, the merits and demerits of silicon-based and emerging neurons are discussed, and the outlook for SANs is presented.