Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses

Three-Dimensional Integrated Synaptic Devices Based on a Silver-Cluster Conduction Mechanism with High Thermostability

During the operation of synaptic devices based on traditional conductive filament (CF) models, the formation and dissolution of CFs are usually uncertain. Moreover, when the device is operated for a long time, the CFs may dissolve due to both the Joule heat generated by the device itself and the thermal coupling between the devices. These problems seriously reduce the reliability and stability of the synaptic device. Here, an artificial synapse device based on polyimide-molybdenum disulfide quantum dot (MoS2 QD) nanocomposites is presented. Research has shown that MoS2 QDs doped into the active layer can effectively induce the reduction of Ag ions into Ag atoms, leading to the formation of Ag clusters and thereby achieving control over the growth of the CFs. Therefore, the device is capable of stably realizing various basic synaptic functions. Moreover, the long-term potentiation/long-term depression (LTP/LTD) of this device shows good linearity. In addition, due to the change in the shape of the CFs, the highly integrated devices with a three-dimensional (3D) stacked structure can operate normally even in a high-temperature environment of 110 °C. Finally, the synaptic characteristics of the devices on learning and inference tests show that their recognition rates are approximately 90.75% (room temperature) and 90.63% (110 °C).

Unravelling the operation of organic artificial neurons for neuromorphic bioelectronics

Organic artificial neurons operating in liquid environments are crucial components in neuromorphic bioelectronics. However, the current understanding of these neurons is limited, hindering their rational design and development for realistic neuronal emulation in biological settings. Here we combine experiments, numerical non-linear simulations, and analytical tools to unravel the operation of organic artificial neurons. This comprehensive approach elucidates a broad spectrum of biorealistic behaviors, including firing properties, excitability, wetware operation, and biohybrid integration. The non-linear simulations are grounded in a physics-based framework, accounting for ion type and ion concentration in the electrolytic medium, organic mixed ionic-electronic parameters, and biomembrane features. The derived analytical expressions link the neurons spiking features with material and physical parameters, bridging closer the domains of artificial neurons and neuroscience. This work provides streamlined and transferable guidelines for the design, development, engineering, and optimization of organic artificial neurons, advancing next generation neuronal networks, neuromorphic electronics, and bioelectronics.

BiœmuS: A new tool for neurological disorders studies through real-time emulation and hybridization using biomimetic Spiking Neural Network

Characterization and modeling of biological neural networks has emerged as a field driving significant advancements in our understanding of brain function and related pathologies. As of today, pharmacological treatments for neurological disorders remain limited, pushing the exploration of promising alternative approaches such as electroceutics. Recent research in bioelectronics and neuromorphic engineering have fostered the development of the new generation of neuroprostheses for brain repair. However, achieving their full potential necessitates a deeper understanding of biohybrid interaction. In this study, we present a novel real-time, biomimetic, cost-effective and user-friendly neural network capable of real-time emulation for biohybrid experiments. Our system facilitates the investigation and replication of biophysically detailed neural network dynamics while prioritizing cost-efficiency, flexibility and ease of use. We showcase the feasibility of conducting biohybrid experiments using standard biophysical interfaces and a variety of biological cells as well as real-time emulation of diverse network configurations. We envision our system as a crucial step towards the development of neuromorphic-based neuroprostheses for bioelectrical therapeutics, enabling seamless communication with biological networks on a comparable timescale. Its embedded real-time functionality enhances practicality and accessibility, amplifying its potential for real-world applications in biohybrid experiments.

High-performance one-dimensional halide perovskite crossbar memristors and synapses for neuromorphic computing

Despite impressive demonstrations of memristive behavior with halide perovskites, no clear pathway for material and device design exists for their applications in neuromorphic computing. Present approaches are limited to single element structures, fall behind in terms of switching reliability and scalability, and fail to map out the analog programming window of such devices. Here, we systematically design and evaluate robust pyridinium-templated one-dimensional halide perovskites as crossbar memristive materials for artificial neural networks. We compare two halide perovskite 1D inorganic lattices, namely (propyl)pyridinium and (benzyl)pyridinium lead iodide. The absence of conjugated, electron-rich substituents in PrPyr+ prevents edge-to-face type π-stacking, leading to enhanced electronic isolation of the 1D iodoplumbate chains in (PrPyr)[PbI3], and hence, superior resistive switching performance compared to (BnzPyr)[PbI3]. We report outstanding resistive switching behaviours in (PrPyr)[PbI3] on the largest flexible crossbar implementation (16 × 16) to date - on/off ratio (>105), long term retention (105 s) and high endurance (2000 cycles). Finally, we put forth a universal approach to comprehensively map the analog programming window of halide perovskite memristive devices - a critical prerequisite for weighted synaptic connections in artificial neural networks. This consequently facilitates the demonstration of accurate handwritten digit recognition from the MNIST database based on spike-timing-dependent plasticity of halide perovskite memristive synapses.

DenRAM: neuromorphic dendritic architecture with RRAM for efficient temporal processing with delays

An increasing number of studies are highlighting the importance of spatial dendritic branching in pyramidal neurons in the neocortex for supporting non-linear computation through localized synaptic integration. In particular, dendritic branches play a key role in temporal signal processing and feature detection. This is accomplished thanks to coincidence detection (CD) mechanisms enabled by the presence of synaptic delays that align temporally disparate inputs for effective integration. Computational studies on spiking neural networks further highlight the significance of delays for achieving spatio-temporal pattern recognition with pure feed-forward neural networks, without the need of resorting to recurrent architectures. In this work, we present "DenRAM", the first realization of a feed-forward spiking neural network with dendritic compartments, implemented using analog electronic circuits integrated into a 130 nm technology node and coupled with Resistive Random Access Memory (RRAM) technology. DenRAM's dendritic circuits use RRAM devices to implement both delays and synaptic weights in the network. By configuring the RRAM devices to reproduce bio-realistic timescales, and by exploiting their heterogeneity we experimentally demonstrate DenRAM's ability to replicate synaptic delay profiles, and to efficiently implement CD for spatio-temporal pattern recognition. To validate the architecture, we conduct comprehensive system-level simulations on two representative temporal benchmarks, demonstrating DenRAM's resilience to analog hardware noise, and its superior accuracy compared to recurrent architectures with an equivalent number of parameters. DenRAM not only brings rich temporal processing capabilities to neuromorphic architectures, but also reduces the memory footprint of edge devices, warrants high accuracy on temporal benchmarks, and represents a significant step-forward in low-power real-time signal processing technologies.

A Hybrid CMOS-Memristor Spiking Neural Network Supporting Multiple Learning Rules

Artificial intelligence (AI) is changing the way computing is performed to cope with real-world, ill-defined tasks for which traditional algorithms fail. AI requires significant memory access, thus running into the von Neumann bottleneck when implemented in standard computing platforms. In this respect, low-latency energy-efficient in-memory computing can be achieved by exploiting emerging memristive devices, given their ability to emulate synaptic plasticity, which provides a path to design large-scale brain-inspired spiking neural networks (SNNs). Several plasticity rules have been described in the brain and their coexistence in the same network largely expands the computational capabilities of a given circuit. In this work, starting from the electrical characterization and modeling of the memristor device, we propose a neuro-synaptic architecture that co-integrates in a unique platform with a single type of synaptic device to implement two distinct learning rules, namely, the spike-timing-dependent plasticity (STDP) and the Bienenstock-Cooper-Munro (BCM). This architecture, by exploiting the aforementioned learning rules, successfully addressed two different tasks of unsupervised learning.

Imitating Synapse Behavior: Exploiting Off-Current in TPBi-Doped Small Molecule Phototransistors for Broadband Wavelength Recognition

Phototransistors have gained significant attention in diverse applications such as photodetectors, image sensors, and neuromorphic devices due to their ability to control electrical characteristics through photoresponse. The choice of photoactive materials in phototransistor research significantly impacts its development. In this study, we propose a novel device that emulates artificial synaptic behavior by leveraging the off-current of a phototransistor. We utilize a p-type organic semiconductor, dinaphtho[2,3-b:2',3'- f]thieno[3,2-b]thiophene (DNTT), as the channel material and dope it with the organic semiconductor 2,2',2″-(1,3,5-benzinetriyl)-tris(1-phenyl-1-H-benzimidazole) (TPBi) on the DNTT transistor. Under light illumination, the general DNTT transistor shows no change in off-current, except at 400 nm wavelength, whereas the TPBi-doped DNTT phototransistor exhibits increased off-current across all wavelength bands. Notably, DNTT phototransistors demonstrate broad photoresponse characteristics in the wavelength range of 400-1000 nm. We successfully simulate artificial synaptic behavior by differentiating the level of off-current and achieving a recognition rate of over 70% across all wavelength bands.

Mosaic: in-memory computing and routing for small-world spike-based neuromorphic systems

The brain's connectivity is locally dense and globally sparse, forming a small-world graph-a principle prevalent in the evolution of various species, suggesting a universal solution for efficient information routing. However, current artificial neural network circuit architectures do not fully embrace small-world neural network models. Here, we present the neuromorphic Mosaic: a non-von Neumann systolic architecture employing distributed memristors for in-memory computing and in-memory routing, efficiently implementing small-world graph topologies for Spiking Neural Networks (SNNs). We've designed, fabricated, and experimentally demonstrated the Mosaic's building blocks, using integrated memristors with 130 nm CMOS technology. We show that thanks to enforcing locality in the connectivity, routing efficiency of Mosaic is at least one order of magnitude higher than other SNN hardware platforms. This is while Mosaic achieves a competitive accuracy in a variety of edge benchmarks. Mosaic offers a scalable approach for edge systems based on distributed spike-based computing and in-memory routing.

In-situ electro-responsive through-space coupling enabling foldamers as volatile memory elements

Voltage-gated processing units are fundamental components for non-von Neumann architectures like memristor and electric synapses, on which nanoscale molecular electronics have possessed great potentials. Here, tailored foldamers with furan‒benzene stacking (f-Fu) and thiophene‒benzene stacking (f-Th) are designed to decipher electro-responsive through-space interaction, which achieve volatile memory behaviors via quantum interference switching in single-molecule junctions. f-Fu exhibits volatile turn-on feature while f-Th performs stochastic turn-off feature with low voltages as 0.2 V. The weakened orbital through-space mixing induced by electro-polarization dominates stacking malposition and quantum interference switching. f-Fu possesses higher switching probability and faster responsive time, while f-Th suffers incomplete switching and longer responsive time. High switching ratios of up to 91 for f-Fu is realized by electrochemical gating. These findings provide evidence and interpretation of the electro-responsiveness of non-covalent interaction at single-molecule level and offer design strategies of molecular non-von Neumann architectures like true random number generator.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

Synaptic turnover promotes efficient learning in bio-realistic spiking neural networks

While artificial machine learning systems achieve superhuman performance in specific tasks such as language processing, image and video recognition, they do so use extremely large datasets and huge amounts of power. On the other hand, the brain remains superior in several cognitively challenging tasks while operating with the energy of a small lightbulb. We use a biologically constrained spiking neural network model to explore how the neural tissue achieves such high efficiency and assess its learning capacity on discrimination tasks. We found that synaptic turnover, a form of structural plasticity, which is the ability of the brain to form and eliminate synapses continuously, increases both the speed and the performance of our network on all tasks tested. Moreover, it allows accurate learning using a smaller number of examples. Importantly, these improvements are most significant under conditions of resource scarcity, such as when the number of trainable parameters is halved and when the task difficulty is increased. Our findings provide new insights into the mechanisms that underlie efficient learning in the brain and can inspire the development of more efficient and flexible machine learning algorithms.

Short-Term Memory Characteristics of IGZO-Based Three-Terminal Devices

A three-terminal synaptic transistor enables more accurate controllability over the conductance compared with traditional two-terminal synaptic devices for the synaptic devices in hardware-oriented neuromorphic systems. In this work, we fabricated IGZO-based three-terminal devices comprising HfAlOx and CeOx layers to demonstrate the synaptic operations. The chemical compositions and thicknesses of the devices were verified by transmission electron microscopy and energy dispersive spectroscopy in cooperation. The excitatory post-synaptic current (EPSC), paired-pulse facilitation (PPF), short-term potentiation (STP), and short-term depression (STD) of the synaptic devices were realized for the short-term memory behaviors. The IGZO-based three-terminal synaptic transistor could thus be controlled appropriately by the amplitude, width, and interval time of the pulses for implementing the neuromorphic systems.

A CMOS-memristor hybrid system for implementing stochastic binary spike timing-dependent plasticity

This paper describes a fully experimental hybrid system in which a [Formula: see text] memristive crossbar spiking neural network (SNN) was assembled using custom high-resistance state memristors with analogue CMOS neurons fabricated in 180 nm CMOS technology. The custom memristors used NMOS selector transistors, made available on a second 180 nm CMOS chip. One drawback is that memristors operate with currents in the micro-amperes range, while analogue CMOS neurons may need to operate with currents in the pico-amperes range. One possible solution was to use a compact circuit to scale the memristor-domain currents down to the analogue CMOS neuron domain currents by at least 5-6 orders of magnitude. Here, we proposed using an on-chip compact current splitter circuit based on MOS ladders to aggressively attenuate the currents by over 5 orders of magnitude. This circuit was added before each neuron. This paper describes the proper experimental operation of an SNN circuit using a [Formula: see text] 1T1R synaptic crossbar together with four post-synaptic CMOS circuits, each with a 5-decade current attenuator and an integrate-and-fire neuron. It also demonstrates one-shot winner-takes-all training and stochastic binary spike-timing-dependent-plasticity learning using this small system. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Conductance Switching in Liquid Crystal-Inspired Self-Assembled Monolayer Junctions

We present the prototype of a ferroelectric tunnel junction (FTJ), which is based on a self-assembled monolayer (SAM) of small, functional molecules. These molecules have a structure similar to those of liquid crystals, and they are embedded between two solid-state electrodes. The SAM, which is deposited through a short sequence of simple fabrication steps, is extremely thin (3.4 ± 0.5 nm) and highly uniform. The functionality of the FTJ is ingrained in the chemical structure of the SAM components: a conformationally flexible dipole that can be reversibly reoriented in an electrical field. Thus, the SAM acts as an electrically switchable tunnel barrier. Fabricated stacks of Al/Al₂O₃/SAM/Pb/Ag with such a polar SAM show pronounced hysteretic, reversible conductance switching at voltages in the range of ±2-3 V, with a conductance ratio of the low and the high resistive states of up to 100. The switching mechanism is analyzed using a combination of quantum chemical, molecular dynamics, and tunneling resistance calculation methods. In contrast to more common, inorganic material-based FTJs, our approach using SAMs of small organic molecules allows for a high degree of functional complexity and diversity to be integrated by synthetic standard methods, while keeping the actual device fabrication process robust and simple. We expect that this technology can be further developed toward a level that would then allow its application in the field of information storage and processing, in particular for in-memory and neuromorphic computing architectures.

Palimpsest memories stored in memristive synapses

Biological synapses store multiple memories on top of each other in a palimpsest fashion and at different time scales. Palimpsest consolidation is facilitated by the interaction of hidden biochemical processes governing synaptic efficacy during varying lifetimes. This arrangement allows idle memories to be temporarily overwritten without being forgotten, while previously unseen memories are used in the short term. While embedded artificial intelligence can greatly benefit from this functionality, a practical demonstration in hardware is missing. Here, we show how the intrinsic properties of metal-oxide volatile memristors emulate the processes supporting biological palimpsest consolidation. Our memristive synapses exhibit an expanded doubled capacity and protect a consolidated memory while up to hundreds of uncorrelated short-term memories temporarily overwrite it, without requiring specialized instructions. We further demonstrate this technology in the context of visual working memory. This showcases how emerging memory technologies can efficiently expand the capabilities of artificial intelligence hardware toward more generalized learning memories.

Memristive technologies for data storage, computation, encryption, and radio-frequency communication

Memristive devices, which combine a resistor with memory functions such that voltage pulses can change their resistance (and hence their memory state) in a nonvolatile manner, are beginning to be implemented in integrated circuits for memory applications. However, memristive devices could have applications in many other technologies, such as non-von Neumann in-memory computing in crossbar arrays, random number generation for data security, and radio-frequency switches for mobile communications. Progress toward the integration of memristive devices in commercial solid-state electronic circuits and other potential applications will depend on performance and reliability challenges that still need to be addressed, as described here.

NeuroPack: An Algorithm-Level Python-Based Simulator for Memristor-Empowered Neuro-Inspired Computing

Emerging two-terminal nanoscale memory devices, known as memristors, have demonstrated great potential for implementing energy-efficient neuro-inspired computing architectures over the past decade. As a result, a wide range of technologies have been developed that, in turn, are described via distinct empirical models. This diversity of technologies requires the establishment of versatile tools that can enable designers to translate memristors’ attributes in novel neuro-inspired topologies. In this study, we present NeuroPack, a modular, algorithm-level Python-based simulation platform that can support studies of memristor neuro-inspired architectures for performing online learning or offline classification. The NeuroPack environment is designed with versatility being central, allowing the user to choose from a variety of neuron models, learning rules, and memristor models. Its hierarchical structure empowers NeuroPack to predict any memristor state changes and the corresponding neural network behavior across a variety of design decisions and user parameter options. The use of NeuroPack is demonstrated herein via an application example of performing handwritten digit classification with the MNIST dataset and an existing empirical model for metal-oxide memristors.

Memristive LIF Spiking Neuron Model and Its Application in Morse Code

The leaky integrate-and-fire (LIF) spiking model can successively mimic the firing patterns and information propagation of a biological neuron. It has been applied in neural networks, cognitive computing, and brain-inspired computing. Due to the resistance variability and the natural storage capacity of the memristor, the LIF spiking model with a memristor (MLIF) is presented in this article to simulate the function and working mode of neurons in biological systems. First, the comparison between the MLIF spiking model and the LIF spiking model is conducted. Second, it is experimentally shown that a single memristor could mimic the function of the integration and filtering of the dendrite and emulate the function of the integration and firing of the soma. Finally, the feasibility of the proposed MLIF spiking model is verified by the generation and recognition of Morse code. The experimental results indicate that the presented MLIF model efficiently performs good biological frequency adaptation, high firing frequency, and rich spiking patterns. A memristor can be used as the dendrite and the soma, and the MLIF spiking model can emulate the axon. The constructed single neuron can efficiently complete the generation and propagation of firing patterns.

Flexible Neural Network Realized by the Probabilistic SiOx Memristive Synaptic Array for Energy-Efficient Image Learning

The human brain's neural networks are sparsely connected via tunable and probabilistic synapses, which may be essential for performing energy-efficient cognitive and intellectual functions. In this sense, the implementation of a flexible neural network with probabilistic synapses is a first step toward realizing the ultimate energy-efficient computing framework. Here, inspired by the efficient threshold-tunable and probabilistic rod-to-rod bipolar synapses in the human visual system, a 16 × 16 crossbar array comprising the vertical form of gate-tunable probabilistic SiO_x memristive synaptic barristor utilizing the Si/graphene heterojunction is designed and fabricated. Controllable stochastic switching dynamics in this array are achieved via various input voltage pulse schemes. In particular, the threshold tunability via electrostatic gating enables the efficient in situ alteration of the probabilistic switching activation (P_Act ) from 0 to 1.0, and can even modulate the degree of the P_Act change. A drop-connected algorithm based on the P_Act is constructed and used to successfully classify the shapes of several fashion items. The suggested approach can decrease the learning energy by up to ≈2,116 times relative to that of the conventional all-to-all connected network while exhibiting a high recognition accuracy of ≈93 %.

Hardware Demonstration of SRDP Neuromorphic Computing with Online Unsupervised Learning Based on Memristor Synapses

Neuromorphic computing has shown great advantages towards cognitive tasks with high speed and remarkable energy efficiency. Memristor is considered as one of the most promising candidates for the electronic synapse of the neuromorphic computing system due to its scalability, power efficiency and capability to simulate biological behaviors. Several memristor-based hardware demonstrations have been explored to achieve the capacity of unsupervised learning with the spike-rate-dependent plasticity (SRDP) learning rule. However, the learning capacity is limited and few of the memristor-based hardware demonstrations have explored the online unsupervised learning at the network level with an SRDP algorithm. Here, we construct a memristor-based hardware system and demonstrate the online unsupervised learning of SRDP networks. The neuromorphic system consists of multiple memristor arrays as the synapse and the discrete CMOS circuit unit as the neuron. Unsupervised learning and online weight update of 10 MNIST handwritten digits are realized by the constructed SRDP networks, and the recognition accuracy is above 90% with 20% device variation. This work paves the way towards the realization of large-scale and efficient networks for more complex tasks.

An Artificial Nerve Capable of UV-Perception, NIR-Vis Switchable Plasticity Modulation, and Motion State Monitoring

The first flexible organic-heterojunction neuromorphic transistor (OHNT) that senses broadband light, including near-ultraviolet (NUV), visible (vis), and near-infrared (NIR), and processes multiplexed-neurotransmission signals is demonstrated. For UV perception, electrical energy consumption down to 536 aJ per synaptic event is demonstrated, at least one order of magnitude lower than current UV-sensitive synaptic devices. For NIR- and vis-perception, switchable plasticity by alternating light sources is yielded for recognition and memory. The device emulates multiplexed neurochemical transition of different neurotransmitters such as dopamine and noradrenaline to form short-term and long-term responses. These facilitate the first realization of human-integrated motion state monitoring and processing using a synaptic hardware, which is then used for real-time heart monitoring of human movement. Motion state analysis with the 96% accuracy is then achieved by artificial neural network. This work provides important support to future biomedical electronics and neural prostheses.

Reconfigurable Stochastic neurons based on tin oxide/MoS2 hetero-memristors for simulated annealing and the Boltzmann machine

Neuromorphic hardware implementation of Boltzmann Machine using a network of stochastic neurons can allow non-deterministic polynomial-time (NP) hard combinatorial optimization problems to be efficiently solved. Efficient implementation of such Boltzmann Machine with simulated annealing desires the statistical parameters of the stochastic neurons to be dynamically tunable, however, there has been limited research on stochastic semiconductor devices with controllable statistical distributions. Here, we demonstrate a reconfigurable tin oxide (SnO_x)/molybdenum disulfide (MoS₂) heterogeneous memristive device that can realize tunable stochastic dynamics in its output sampling characteristics. The device can sample exponential-class sigmoidal distributions analogous to the Fermi-Dirac distribution of physical systems with quantitatively defined tunable "temperature" effect. A BM composed of these tunable stochastic neuron devices, which can enable simulated annealing with designed "cooling" strategies, is conducted to solve the MAX-SAT, a representative in NP-hard combinatorial optimization problems. Quantitative insights into the effect of different "cooling" strategies on improving the BM optimization process efficiency are also provided.

Stochastic binary synapses having sigmoidal cumulative distribution functions for unsupervised learning with spike timing-dependent plasticity

Spike timing-dependent plasticity (STDP), which is widely studied as a fundamental synaptic update rule for neuromorphic hardware, requires precise control of continuous weights. From the viewpoint of hardware implementation, a simplified update rule is desirable. Although simplified STDP with stochastic binary synapses was proposed previously, we find that it leads to degradation of memory maintenance during learning, which is unfavourable for unsupervised online learning. In this work, we propose a stochastic binary synaptic model where the cumulative probability of the weight change evolves in a sigmoidal fashion with potentiation or depression trials, which can be implemented using a pair of switching devices consisting of serially connected multiple binary memristors. As a benchmark test we perform simulations of unsupervised learning of MNIST images with a two-layer network and show that simplified STDP in combination with this model can outperform conventional rules with continuous weights not only in memory maintenance but also in recognition accuracy. Our method achieves 97.3% in recognition accuracy, which is higher than that reported with standard STDP in the same framework. We also show that the high performance of our learning rule is robust against device-to-device variability of the memristor's probabilistic behaviour.

Hybrid memristor-CMOS neurons for in-situ learning in fully hardware memristive spiking neural networks

Spiking neural network, inspired by the human brain, consisting of spiking neurons and plastic synapses, is a promising solution for highly efficient data processing in neuromorphic computing. Recently, memristor-based neurons and synapses are becoming intriguing candidates to build spiking neural networks in hardware, owing to the close resemblance between their device dynamics and the biological counterparts. However, the functionalities of memristor-based neurons are currently very limited, and a hardware demonstration of fully memristor-based spiking neural networks supporting in-situ learning is very challenging. Here, a hybrid spiking neuron combining a memristor with simple digital circuits is designed and implemented in hardware to enhance neuron functions. The hybrid neuron with memristive dynamics not only realizes the basic leaky integrate-and-fire neuron function but also enables the in-situ tuning of the connected synaptic weights. Finally, a fully hardware spiking neural network with the hybrid neurons and memristive synapses is experimentally demonstrated for the first time, and in-situ Hebbian learning is achieved with this network. This work opens up a way towards the implementation of spiking neurons, supporting in-situ learning for future neuromorphic computing systems.

Avalanches and edge-of-chaos learning in neuromorphic nanowire networks

The brain's efficient information processing is enabled by the interplay between its neuro-synaptic elements and complex network structure. This work reports on the neuromorphic dynamics of nanowire networks (NWNs), a unique brain-inspired system with synapse-like memristive junctions embedded within a recurrent neural network-like structure. Simulation and experiment elucidate how collective memristive switching gives rise to long-range transport pathways, drastically altering the network's global state via a discontinuous phase transition. The spatio-temporal properties of switching dynamics are found to be consistent with avalanches displaying power-law size and life-time distributions, with exponents obeying the crackling noise relationship, thus satisfying criteria for criticality, as observed in cortical neuronal cultures. Furthermore, NWNs adaptively respond to time varying stimuli, exhibiting diverse dynamics tunable from order to chaos. Dynamical states at the edge-of-chaos are found to optimise information processing for increasingly complex learning tasks. Overall, these results reveal a rich repertoire of emergent, collective neural-like dynamics in NWNs, thus demonstrating the potential for a neuromorphic advantage in information processing.

Finite-Time and Fixed-Time Synchronization of Coupled Memristive Neural Networks With Time Delay

This article is devoted to analyzing the finite-time and fixed-time synchronization of coupled memristive neural networks with time delays. The synchronization is leaderless rather than leader-follower as the tracking targets are uncertain. By designing a proper controller and using the Lyapunov method, several sufficient conditions are obtained to achieve the finite-time and fixed-time synchronization of coupled memristive neural networks by introducing a class of special auxiliary matrices. Moreover, the settling times can be estimated for finite-time synchronization that depends on the initial values as well as fixed-time synchronization that is uniformly bounded for any initial values. Finally, two examples are presented to substantiate the effectiveness of the theoretical results.

EqSpike: spike-driven equilibrium propagation for neuromorphic implementations

Finding spike-based learning algorithms that can be implemented within the local constraints of neuromorphic systems, while achieving high accuracy, remains a formidable challenge. Equilibrium propagation is a promising alternative to backpropagation as it only involves local computations, but hardware-oriented studies have so far focused on rate-based networks. In this work, we develop a spiking neural network algorithm called EqSpike, compatible with neuromorphic systems, which learns by equilibrium propagation. Through simulations, we obtain a test recognition accuracy of 97.6% on the MNIST handwritten digits dataset (Mixed National Institute of Standards and Technology), similar to rate-based equilibrium propagation, and comparing favorably to alternative learning techniques for spiking neural networks. We show that EqSpike implemented in silicon neuromorphic technology could reduce the energy consumption of inference and training, respectively, by three orders and two orders of magnitude compared to graphics processing units. Finally, we also show that during learning, EqSpike weight updates exhibit a form of spike-timing-dependent plasticity, highlighting a possible connection with biology.

Threshold switching memristor-based stochastic neurons for probabilistic computing

Biological neurons exhibit dynamic excitation behavior in the form of stochastic firing, rather than stiffly giving out spikes upon reaching a fixed threshold voltage, which empowers the brain to perform probabilistic inference in the face of uncertainty. However, owing to the complexity of the stochastic firing process in biological neurons, the challenge of fabricating and applying stochastic neurons with bio-realistic dynamics to probabilistic scenarios remains to be fully addressed. In this work, a novel CuS/GeSe conductive-bridge threshold switching memristor is fabricated and singled out to realize electronic stochastic neurons, which is ascribed to the similarity between the stochastic switching behavior observed in the device and that of biological ion channels. The corresponding electric circuit of a stochastic neuron is then constructed and the probabilistic firing capacity of the neuron is utilized to implement Bayesian inference in a spiking neural network (SNN). The application prospects are demonstrated on the example of a tumor diagnosis task, where common fatal diagnostic errors of a conventional artificial neural network are successfully circumvented. Moreover, in comparison to deterministic neuron-based SNNs, the stochastic neurons enable SNNs to deliver an estimate of the uncertainty in their predictions, and the fidelity of the judgement is drastically improved by 81.2%.

Synaptic Iontronic Devices for Brain-Mimicking Functions: Fundamentals and Applications

Inspired by the information transmission mechanism in the central nervous systems of life, synapse-mimicking devices have been designed and fabricated for the purpose of breaking the bottleneck of von Neumann architecture and realizing the construction of effective hardware-based artificial intelligence. In this case, synaptic iontronic devices, dealing with current information with ions instead of electrons, have attracted enormous scientific interests owing to their unique characteristics provided by ions, such as the designability of charge carriers and the diversity of chemical regulation. Herein, the basic conception, working mechanism, performance metrics, and advanced applications of synaptic iontronic devices based on three-terminal transistors and two-terminal memristors are systematically reviewed and comprehensively discussed. This Review provides a prospect on how to realize artificial synaptic functions based on the regulation of ions and raises a series of further challenges unsolved in this area.

Neuromorphic Engineering: From Biological to Spike-Based Hardware Nervous Systems

The human brain is a sophisticated, high-performance biocomputer that processes multiple complex tasks in parallel with high efficiency and remarkably low power consumption. Scientists have long been pursuing an artificial intelligence (AI) that can rival the human brain. Spiking neural networks based on neuromorphic computing platforms simulate the architecture and information processing of the intelligent brain, providing new insights for building AIs. The rapid development of materials engineering, device physics, chip integration, and neuroscience has led to exciting progress in neuromorphic computing with the goal of overcoming the von Neumann bottleneck. Herein, fundamental knowledge related to the structures and working principles of neurons and synapses of the biological nervous system is reviewed. An overview is then provided on the development of neuromorphic hardware systems, from artificial synapses and neurons to spike-based neuromorphic computing platforms. It is hoped that this review will shed new light on the evolution of brain-like computing.

Emerging Memristive Artificial Synapses and Neurons for Energy-Efficient Neuromorphic Computing

Memristors have recently attracted significant interest due to their applicability as promising building blocks of neuromorphic computing and electronic systems. The dynamic reconfiguration of memristors, which is based on the history of applied electrical stimuli, can mimic both essential analog synaptic and neuronal functionalities. These can be utilized as the node and terminal devices in an artificial neural network. Consequently, the ability to understand, control, and utilize fundamental switching principles and various types of device architectures of the memristor is necessary for achieving memristor-based neuromorphic hardware systems. Herein, a wide range of memristors and memristive-related devices for artificial synapses and neurons is highlighted. The device structures, switching principles, and the applications of essential synaptic and neuronal functionalities are sequentially presented. Moreover, recent advances in memristive artificial neural networks and their hardware implementations are introduced along with an overview of the various learning algorithms. Finally, the main challenges of the memristive synapses and neurons toward high-performance and energy-efficient neuromorphic computing are briefly discussed. This progress report aims to be an insightful guide for the research on memristors and neuromorphic-based computing.

Effects of top electrode material in hafnium-oxide-based memristive systems on highly-doped Si

This work provides useful insights into the development of HfO2-based memristive systems with a p-type silicon bottom electrode that are compatible with the complementary metal-oxide-semiconductor technology. The results obtained reveal the importance of the top electrode selection to achieve unique device characteristics. The Ag/HfO2/Si devices have exhibited a larger memory window and self-compliance characteristics. On the other hand, the Au/HfO2/Si devices have displayed substantial cycle-to-cycle variation in the ON-state conductance. These device characteristics can be used as an indicator for the design of resistive-switching devices in various scenes such as, memory, security, and sensing. The current-voltage (I-V) characteristics of Ag/HfO2/Si and Au/HfO2/Si devices under positive and negative bias conditions have provided valuable information on the ON and OFF states of the devices and the underlying resistive switching mechanisms. Repeatable, low-power, and forming-free bipolar resistive switching is obtained with both device structures, with the Au/HfO2/Si devices displaying a poorer device-to-device reproducibility. Furthermore, the Au/HfO2/Si devices have exhibited N-type negative differential resistance (NDR), suggesting Joule-heating activated migration of oxygen vacancies to be responsible for the SET process in the unstable unipolar mode.

Graphene memristive synapses for high precision neuromorphic computing

Memristive crossbar architectures are evolving as powerful in-memory computing engines for artificial neural networks. However, the limited number of non-volatile conductance states offered by state-of-the-art memristors is a concern for their hardware implementation since trained weights must be rounded to the nearest conductance states, introducing error which can significantly limit inference accuracy. Moreover, the incapability of precise weight updates can lead to convergence problems and slowdown of on-chip training. In this article, we circumvent these challenges by introducing graphene-based multi-level (>16) and non-volatile memristive synapses with arbitrarily programmable conductance states. We also show desirable retention and programming endurance. Finally, we demonstrate that graphene memristors enable weight assignment based on k-means clustering, which offers greater computing accuracy when compared with uniform weight quantization for vector matrix multiplication, an essential component for any artificial neural network.

Nano-engineering and functionalization of hybrid Au-MexOy-TiO2 (Me = W, Ga) hetero-interfaces for optoelectronic receptors and nociceptors

Bio-inspired nano-electronic devices are key instruments for the development of advanced artificial intelligence systems, which will shape the future of humanoid nano-robotics. An emerging demand is realized for an accurate reception of environmental stimuli via visual perception, processing and realization of optical signals. The present study demonstrates the capability of functionalized all-oxide heterostructured two-dimensional (2D) plasmonic devices for the self-adaptive recognition of visual optical pulses. Specifically, the nano-engineering of the metal/semiconductor interface and co-modulation of heterostructured 2D semiconductor hetero-interfaces of Au/WO3 : TiO2 and Au/Ga2O3 : TiO2 facilitated the receptive and nociceptive detection of visible light pulses. A decrease in the dark current of the Au/WO3 : TiO2 unit resulted in the development of sensitive visible light photoreceptors. Furthermore, the modulation of charge transfers at the Au/Ga2O3 : TiO2 hetero-interfaces were the key parameter to determine the optical reception characteristics and nociceptive performance of all-oxide optoelectronic devices. Specifically, the rapid thermal annealing (RTA) of 2D Ga2O3 in N2 atmosphere ensured the modulation of charge transfer at Au/Ga2O3 : TiO2 hetero-interfaces in plasmonic devices. Thus, hetero-interface engineering enabled the effective control of charge transfer at 2D hetero-interfaces for an adaptive perception of visible optical pulses. Consequently, the fabricated sensitive Au/Ga2O3 (N2) : TiO2 bio-inspired unit emulated the optical functionalities of corneal nociceptors.

Engineering Complex Synaptic Behaviors in a Single Device: Emulating Consolidation of Short-term Memory to Long-term Memory in Artificial Synapses via Dielectric Band Engineering

As one of the key neuronal activities associated with memory in the human brain, memory consolidation is the process of the transition of short-term memory (STM) to long-term memory (LTM), which transforms an external stimulus to permanently stored information. Here, we report the emulation of this complex synaptic function, consolidation of STM to LTM, in a single-crystal indium phosphide (InP) field effect transistor (FET)-based artificial synapse. This behavior is achieved via the dielectric band and charge trap lifetime engineering in a dielectric gate heterostructure of aluminum oxide and titanium oxide. We analyze the behavior of these complex synaptic functions by engineering a variety of action potential parameters, and the devices exhibit good endurance, long retention time (>10⁵ s), and high uniformity. Uniquely, this approach utilizes growth and device fabrication techniques which are scalable and back-end CMOS compatible, making this InP synaptic device a potential building block for neuromorphic computing.

Monitoring PSA levels as chemical state-variables in metal-oxide memristors

Medical interventions increasingly rely on biosensors that can provide reliable quantitative information. A longstanding bottleneck in realizing this, is various non-idealities that generate offsets and variable responses across sensors. Current mitigation strategies involve the calibration of sensors, performed in software or via auxiliary compensation circuitry thus constraining real-time operation and integration efforts. Here, we show that bio-functionalized metal-oxide memristors can be utilized for directly transducing biomarker concentration levels to discrete memory states. The introduced chemical state-variable is found to be dependent on the devices’ initial resistance, with its response to chemical stimuli being more pronounced for higher resistive states. We leverage this attribute along with memristors’ inherent state programmability for calibrating a biosensing array to render a homogeneous response across all cells. Finally, we demonstrate the application of this technology in detecting Prostate Specific Antigen in clinically relevant levels (ng/ml), paving the way towards applications in large multi-panel assays.

Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy

Neural networks based on memristive devices have achieved great progress recently. However, memristive synapses with nonlinearity and asymmetry seriously limit the classification accuracy. Moreover, insufficient number of training samples in many cases also have negative effect on the classification accuracy of neural networks due to overfitting. In this work, dropout neuronal units are developed based on stochastic volatile memristive devices of Ag/Ta2O5:Ag/Pt. The memristive neural network using the dropout neuronal units effectively solves the problem of overfitting and mitigates the negative effects of the nonideality of memristive synapses, eventually achieves a classification accuracy comparable to the theoretical limit. The stochastic and volatile switching performances of the Ag/Ta2O5:Ag/Pt device are attributed to the stochastical rupture of the Ag filament under high electrical stress in the Ta2O5 layer, according to the TEM observation and the kinetic Monte Carlo simulation.

Memory devices and applications for in-memory computing

Traditional von Neumann computing systems involve separate processing and memory units. However, data movement is costly in terms of time and energy and this problem is aggravated by the recent explosive growth in highly data-centric applications related to artificial intelligence. This calls for a radical departure from the traditional systems and one such non-von Neumann computational approach is in-memory computing. Hereby certain computational tasks are performed in place in the memory itself by exploiting the physical attributes of the memory devices. Both charge-based and resistance-based memory devices are being explored for in-memory computing. In this Review, we provide a broad overview of the key computational primitives enabled by these memory devices as well as their applications spanning scientific computing, signal processing, optimization, machine learning, deep learning and stochastic computing.

Polaritonic Neuromorphic Computing Outperforms Linear Classifiers

Machine learning software applications are ubiquitous in many fields of science and society for their outstanding capability to solve computationally vast problems like the recognition of patterns and regularities in big data sets. In spite of these impressive achievements, such processors are still based on the so-called von Neumann architecture, which is a bottleneck for faster and power-efficient neuromorphic computation. Therefore, one of the main goals of research is to conceive physical realizations of artificial neural networks capable of performing fully parallel and ultrafast operations. Here we show that lattices of exciton-polariton condensates accomplish neuromorphic computing with outstanding accuracy thanks to their high optical nonlinearity. We demonstrate that our neural network significantly increases the recognition efficiency compared with the linear classification algorithms on one of the most widely used benchmarks, the MNIST problem, showing a concrete advantage from the integration of optical systems in neural network architectures.

Perovskite neural trees

Trees are used by animals, humans and machines to classify information and make decisions. Natural tree structures displayed by synapses of the brain involves potentiation and depression capable of branching and is essential for survival and learning. Demonstration of such features in synthetic matter is challenging due to the need to host a complex energy landscape capable of learning, memory and electrical interrogation. We report experimental realization of tree-like conductance states at room temperature in strongly correlated perovskite nickelates by modulating proton distribution under high speed electric pulses. This demonstration represents physical realization of ultrametric trees, a concept from number theory applied to the study of spin glasses in physics that inspired early neural network theory dating almost forty years ago. We apply the tree-like memory features in spiking neural networks to demonstrate high fidelity object recognition, and in future can open new directions for neuromorphic computing and artificial intelligence.

Neurohybrid Memristive CMOS-Integrated Systems for Biosensors and Neuroprosthetics

Here we provide a perspective concept of neurohybrid memristive chip based on the combination of living neural networks cultivated in microfluidic/microelectrode system, metal-oxide memristive devices or arrays integrated with mixed-signal CMOS layer to control the analog memristive circuits, process the decoded information, and arrange a feedback stimulation of biological culture as parts of a bidirectional neurointerface. Our main focus is on the state-of-the-art approaches for cultivation and spatial ordering of the network of dissociated hippocampal neuron cells, fabrication of a large-scale cross-bar array of memristive devices tailored using device engineering, resistive state programming, or non-linear dynamics, as well as hardware implementation of spiking neural networks (SNNs) based on the arrays of memristive devices and integrated CMOS electronics. The concept represents an example of a brain-on-chip system belonging to a more general class of memristive neurohybrid systems for a new-generation robotics, artificial intelligence, and personalized medicine, discussed in the framework of the proposed roadmap for the next decade period.

A novel memristor-based dynamical system with multi-wing attractors and symmetric periodic bursting

This paper presents a novel memristor-based dynamical system with circuit implementation, which has a 2×3-wing, 2×2-wing, and 2×1-wing non-Shilnikov type of chaotic attractors. The system has two index-2 saddle-focus equilibria, symmetrical with respect to the x-axis. The system is analyzed with bifurcation diagrams and Lyapunov exponents, demonstrating its complex dynamical behaviors: the system reaches the chaotic state from the periodic state through alternating period-doubling bifurcations and then from the chaotic state back to the periodic state through inverse bifurcations, as one parameter changes. It shows two interesting phenomena: a jump-switching periodic state and jump-switching chaotic state. Also, the system can sustain chaos with a constant Lyapunov spectrum in some initial conditions and a parameter set. In addition, a class of symmetric periodic bursting phenomena is surprisingly observed under a particular set of parameters, and its generation mechanism is revealed through bifurcation analysis. Finally, the circuit implementation verifies the theoretical analysis and the jump-switching numerical simulation results.

Inhibitory Plasticity: From Molecules to Computation and Beyond

Synaptic plasticity is the cellular and molecular counterpart of learning and memory and, since its first discovery, the analysis of the mechanisms underlying long-term changes of synaptic strength has been almost exclusively focused on excitatory connections. Conversely, inhibition was considered as a fixed controller of circuit excitability. Only recently, inhibitory networks were shown to be finely regulated by a wide number of mechanisms residing in their synaptic connections. Here, we review recent findings on the forms of inhibitory plasticity (IP) that have been discovered and characterized in different brain areas. In particular, we focus our attention on the molecular pathways involved in the induction and expression mechanisms leading to changes in synaptic efficacy, and we discuss, from the computational perspective, how IP can contribute to the emergence of functional properties of brain circuits.

Memristive synapses connect brain and silicon spiking neurons

Brain function relies on circuits of spiking neurons with synapses playing the key role of merging transmission with memory storage and processing. Electronics has made important advances to emulate neurons and synapses and brain-computer interfacing concepts that interlink brain and brain-inspired devices are beginning to materialise. We report on memristive links between brain and silicon spiking neurons that emulate transmission and plasticity properties of real synapses. A memristor paired with a metal-thin film titanium oxide microelectrode connects a silicon neuron to a neuron of the rat hippocampus. Memristive plasticity accounts for modulation of connection strength, while transmission is mediated by weighted stimuli through the thin film oxide leading to responses that resemble excitatory postsynaptic potentials. The reverse brain-to-silicon link is established through a microelectrode-memristor pair. On these bases, we demonstrate a three-neuron brain-silicon network where memristive synapses undergo long-term potentiation or depression driven by neuronal firing rates.

Emerging neuromorphic devices

Artificial intelligence (AI) has the ability of revolutionizing our lives and society in a radical way, by enabling machine learning in the industry, business, health, transportation, and many other fields. The ability to recognize objects, faces, and speech, requires, however, exceptional computational power and time, which is conflicting with the current difficulties in transistor scaling due to physical and architectural limitations. As a result, to accelerate the progress of AI, it is necessary to develop materials, devices, and systems that closely mimic the human brain. In this work, we review the current status and challenges on the emerging neuromorphic devices for brain-inspired computing. First, we provide an overview of the memory device technologies which have been proposed for synapse and neuron circuits in neuromorphic systems. Then, we describe the implementation of synaptic learning in the two main types of neural networks, namely the deep neural network and the spiking neural network (SNN). Bio-inspired learning, such as the spike-timing dependent plasticity scheme, is shown to enable unsupervised learning processes which are typical of the human brain. Hardware implementations of SNNs for the recognition of spatial and spatio-temporal patterns are also shown to support the cognitive computation in silico. Finally, we explore the recent advances in reproducing bio-neural processes via device physics, such as insulating-metal transitions, nanoionics drift/diffusion, and magnetization flipping in spintronic devices. By harnessing the device physics in emerging materials, neuromorphic engineering with advanced functionality, higher density and better energy efficiency can be developed.

Hidden Bursting Firings and Bifurcation Mechanisms in Memristive Neuron Model With Threshold Electromagnetic Induction

Memristors can be employed to mimic biological neural synapses or to describe electromagnetic induction effects. To exhibit the threshold effect of electromagnetic induction, this paper presents a threshold flux-controlled memristor and examines its frequency-dependent pinched hysteresis loops. Using an electromagnetic induction current generated by the threshold memristor to replace the external current in 2-D Hindmarsh-Rose (HR) neuron model, a 3-D memristive HR (mHR) neuron model with global hidden oscillations is established and the corresponding numerical simulations are performed. It is found that due to no equilibrium point, the obtained mHR neuron model always operates in hidden bursting firing patterns, including coexisting hidden bursting firing patterns with bistability also. In addition, the model exhibits complex dynamics of the actual neuron electrical activities, which acts like the 3-D HR neuron model, indicating its feasibility. In particular, by constructing the fold and Hopf bifurcation sets of the fast-scale subsystem, the bifurcation mechanisms of hidden bursting firings are expounded. Finally, circuit experiments on hardware breadboards are deployed and the captured results well match with the numerical results, validating the physical mechanism of biological neuron and the reliability of electronic neuron.