Neural sampling machine with stochastic synapse allows brain-like learning and inference

Extreme value statistics of nerve transmission delay

Delays in nerve transmission are an important topic in the field of neuroscience. Spike signals fired or received by the dendrites of a neuron travel from the axon to a presynaptic cell. The spike signal then triggers a chemical reaction at the synapse, wherein a presynaptic cell transfers neurotransmitters to the postsynaptic cell, regenerates electrical signals via a chemical reaction through ion channels, and transmits them to neighboring neurons. In the context of describing the complex physiological reaction process as a stochastic process, this study aimed to show that the distribution of the maximum time interval of spike signals follows extreme-order statistics. By considering the statistical variance in the time constant of the leaky Integrate-and-Fire model, a deterministic time evolution model for spike signals, we enabled randomness in the time interval of the spike signals. When the time constant follows an exponential distribution function, the time interval of the spike signal also follows an exponential distribution. In this case, our theory and simulations confirmed that the histogram of the maximum time interval follows the Gumbel distribution, one of the three forms of extreme-value statistics. We further confirmed that the histogram of the maximum time interval followed a Fréchet distribution when the time interval of the spike signal followed a Pareto distribution. These findings confirm that nerve transmission delay can be described using extreme value statistics and can therefore be used as a new indicator of transmission delay.

Memristive Monte Carlo DropConnect crossbar array enabled by device and algorithm co-design

Device and algorithm co-design aims to develop energy-efficient hardware that directly implements complex algorithms and optimizes algorithms to match the hardware's characteristics. Specifically, neuromorphic computing algorithms are constantly growing in complexity, necessitating an ongoing search for hardware implementations capable of handling these intricate algorithms. Here, we present a memristive Monte Carlo DropConnect (MC-DC) crossbar array developed through a hardware algorithm co-design approach. To implement the MC-DC neural network, stochastic switching and analog memory characteristics are required, and we achieved them using Ag-based diffusive selectors and Ru-based electrochemical metalization (ECM) memristors, respectively. The devices were integrated with a one-selector one-memristor (1S1M) structure, and their well-matched operating voltages and currents enabled stochastic readout and deterministic analog programming. With the integrated hardware, we successfully demonstrated the MC-DC operation. Additionally, the selector allowed for the control of switching polarity, and by understanding this hardware characteristic, we were able to modify the algorithm to fit it and further improve the network performance.

Organic-Inorganic Hybrid Synaptic Transistors: Methyl-Silsesquioxanes-Based Electric Double Layer for Enhanced Synaptic Functionality and CMOS Compatibility

Electrical double-layer (EDL) synaptic transistors based on organic materials exhibit low thermal and chemical stability and are thus incompatible with complementary metal oxide semiconductor (CMOS) processes involving high-temperature operations. This paper proposes organic-inorganic hybrid synaptic transistors using methyl silsesquioxane (MSQ) as the electrolyte. MSQ, derived from the combination of inorganic silsesquioxanes and the organic methyl (-CH3) group, exhibits exceptional thermal and chemical stability, thus ensuring compatibility with CMOS processes. We fabricated Al/MSQ electrolyte/Pt capacitors, exhibiting a substantial capacitance of 1.89 µF/cm2 at 10 Hz. MSQ-based EDL synaptic transistors demonstrated various synaptic behaviors, such as excitatory post-synaptic current, paired-pulse facilitation, signal pass filtering, and spike-number-dependent plasticity. Additionally, we validated synaptic functions such as information storage and synapse weight adjustment, simulating brain synaptic operations through potentiation and depression. Notably, these synaptic operations demonstrated stability over five continuous operation cycles. Lastly, we trained a multi-layer artificial deep neural network (DNN) using a handwritten Modified National Institute of Standards and Technology image dataset. The DNN achieved an impressive recognition rate of 92.28%. The prepared MSQ-based EDL synaptic transistors, with excellent thermal/chemical stability, synaptic functionality, and compatibility with CMOS processes, harbor tremendous potential as materials for next-generation artificial synapse components.

Novel Three-Dimensional Artificial Neural Network Based on an Eight-Layer Vertical Memristor with an Ultrahigh Rectify Ratio (>107) and an Ultrahigh Nonlinearity (>105) for Neuromorphic Computing

In recent years, memristors have successfully demonstrated their significant potential in artificial neural networks (ANNs) and neuromorphic computing. Nonetheless, ANNs constructed by crossbar arrays suffer from cross-talk issues and low integration densities. Here, we propose an eight-layer three-dimensional (3D) vertical crossbar memristor with an ultrahigh rectify ratio (RR > 107) and an ultrahigh nonlinearity (>105) to overcome these limitations, which enables it to reach a >1 Tb array size without reading failure. Furthermore, the proposed 3D RRAM shows advanced endurance (>1010 cycles), retention (>104 s), and uniformity. In addition, several synaptic functions observed in the human brain were mimicked. On the basis of the advanced performance, we constructed a novel 3D ANN, whose learning efficiency and recognition accuracy were enhanced significantly compared with those of conventional single-layer ANNs. These findings hold promise for the development of highly efficient, precise, integrated, and stable VLSI neuromorphic computing systems.

Synchronization of Complex Dynamical Networks with Stochastic Links Dynamics

The mean square synchronization problem of the complex dynamical network (CDN) with the stochastic link dynamics is investigated. In contrast to previous literature, the CDN considered in this paper can be viewed as consisting of two subsystems coupled to each other. One subsystem consists of all nodes, referred to as the nodes subsystem, and the other consists of all links, referred to as the network topology subsystem, where the weighted values can quantitatively reflect changes in the network's topology. Based on the above understanding of CDN, two vector stochastic differential equations with Brownian motion are used to model the dynamic behaviors of nodes and links, respectively. The control strategy incorporates not only the controller in the nodes but also the coupling term in the links, through which the CDN is synchronized in the mean-square sense. Meanwhile, the dynamic stochastic signal is proposed in this paper, which is regarded as the auxiliary reference tracking target of links, such that the links can track the reference target asymptotically when synchronization occurs in nodes. This implies that the eventual topological structure of CDN is stochastic. Finally, a comparison simulation example confirms the superiority of the control strategy in this paper.

Generative complex networks within a dynamic memristor with intrinsic variability

Artificial neural networks (ANNs) have gained considerable momentum in the past decade. Although at first the main task of the ANN paradigm was to tune the connection weights in fixed-architecture networks, there has recently been growing interest in evolving network architectures toward the goal of creating artificial general intelligence. Lagging behind this trend, current ANN hardware struggles for a balance between flexibility and efficiency but cannot achieve both. Here, we report on a novel approach for the on-demand generation of complex networks within a single memristor where multiple virtual nodes are created by time multiplexing and the non-trivial topological features, such as small-worldness, are generated by exploiting device dynamics with intrinsic cycle-to-cycle variability. When used for reservoir computing, memristive complex networks can achieve a noticeable increase in memory capacity a and respectable performance boost compared to conventional reservoirs trivially implemented as fully connected networks. This work expands the functionality of memristors for ANN computing.

Stochastic transition in synchronized spiking nanooscillators

This work reports that synchronization of Mott material-based nanoscale coupled spiking oscillators can be drastically different from that in conventional harmonic oscillators. We investigated the synchronization of spiking nanooscillators mediated by thermal interactions due to the close physical proximity of the devices. Controlling the driving voltage enables in-phase 1:1 and 2:1 integer synchronization modes between neighboring oscillators. Transition between these two integer modes occurs through an unusual stochastic synchronization regime instead of the loss of spiking coherence. In the stochastic synchronization regime, random length spiking sequences belonging to the 1:1 and 2:1 integer modes are intermixed. The occurrence of this stochasticity is an important factor that must be taken into account in the design of large-scale spiking networks for hardware-level implementation of novel computational paradigms such as neuromorphic and stochastic computing.

Probabilistic Neural Computing with Stochastic Devices

The brain has effectively proven a powerful inspiration for the development of computing architectures in which processing is tightly integrated with memory, communication is event-driven, and analog computation can be performed at scale. These neuromorphic systems increasingly show an ability to improve the efficiency and speed of scientific computing and artificial intelligence applications. Herein, it is proposed that the brain's ubiquitous stochasticity represents an additional source of inspiration for expanding the reach of neuromorphic computing to probabilistic applications. To date, many efforts exploring probabilistic computing have focused primarily on one scale of the microelectronics stack, such as implementing probabilistic algorithms on deterministic hardware or developing probabilistic devices and circuits with the expectation that they will be leveraged by eventual probabilistic architectures. A co-design vision is described by which large numbers of devices, such as magnetic tunnel junctions and tunnel diodes, can be operated in a stochastic regime and incorporated into a scalable neuromorphic architecture that can impact a number of probabilistic computing applications, such as Monte Carlo simulations and Bayesian neural networks. Finally, a framework is presented to categorize increasingly advanced hardware-based probabilistic computing technologies.

Enhanced regularization for on-chip training using analog and temporary memory weights

In-memory computing techniques are used to accelerate artificial neural network (ANN) training and inference tasks. Memory technology and architectural innovations allow efficient matrix-vector multiplications, gradient calculations, and updates to network weights. However, on-chip learning for edge devices is quite challenging due to the frequent updates. Here, we propose using an analog and temporary on-chip memory (ATOM) cell with controllable retention timescales for implementing the weights of an on-chip training task. Measurement results for Read-Write timescales are presented for an ATOM cell fabricated in GlobalFoundries' 45 nm RFSOI technology. The effect of limited retention and its variability is evaluated for training a fully connected neural network with a variable number of layers for the MNIST hand-written digit recognition task. Our studies show that weight decay due to temporary memory can have benefits equivalent to regularization, achieving a ∼33% reduction in the validation error (from 3.6% to 2.4%). We also show that the controllability of the decay timescale can be advantageous in achieving a further ∼26% reduction in the validation error. This strongly suggests the utility of temporary memory during learning before on-chip non-volatile memories can take over for the storage and inference tasks using the neural network weights. We thus propose an algorithm-circuit codesign in the form of temporary analog memory for high-performing on-chip learning of ANNs.

A sensory memory processing system with multi-wavelength synaptic-polychromatic light emission for multi-modal information recognition

Realizing multi-modal information recognition tasks which can process external information efficiently and comprehensively is an urgent requirement in the field of artificial intelligence. However, it remains a challenge to achieve simple structure and high-performance multi-modal recognition demonstrations owing to the complex execution module and separation of memory processing based on the traditional complementary metal oxide semiconductor (CMOS) architecture. Here, we propose an efficient sensory memory processing system (SMPS), which can process sensory information and generate synapse-like and multi-wavelength light-emitting output, realizing diversified utilization of light in information processing and multi-modal information recognition. The SMPS exhibits strong robustness in information encoding/transmission and the capability of visible information display through the multi-level color responses, which can implement the multi-level pain warning process of organisms intuitively. Furthermore, different from the conventional multi-modal information processing system that requires independent and complex circuit modules, the proposed SMPS with unique optical multi-information parallel output can realize efficient multi-modal information recognition of dynamic step frequency and spatial positioning simultaneously with the accuracy of 99.5% and 98.2%, respectively. Therefore, the SMPS proposed in this work with simple component, flexible operation, strong robustness, and highly efficiency is promising for future sensory-neuromorphic photonic systems and interactive artificial intelligence.

Synapse-Mimetic Hardware-Implemented Resistive Random-Access Memory for Artificial Neural Network

Memristors mimic synaptic functions in advanced electronics and image sensors, thereby enabling brain-inspired neuromorphic computing to overcome the limitations of the von Neumann architecture. As computing operations based on von Neumann hardware rely on continuous memory transport between processing units and memory, fundamental limitations arise in terms of power consumption and integration density. In biological synapses, chemical stimulation induces information transfer from the pre- to the post-neuron. The memristor operates as resistive random-access memory (RRAM) and is incorporated into the hardware for neuromorphic computing. Hardware composed of synaptic memristor arrays is expected to lead to further breakthroughs owing to their biomimetic in-memory processing capabilities, low power consumption, and amenability to integration; these aspects satisfy the upcoming demands of artificial intelligence for higher computational loads. Among the tremendous efforts toward achieving human-brain-like electronics, layered 2D materials have demonstrated significant potential owing to their outstanding electronic and physical properties, facile integration with other materials, and low-power computing. This review discusses the memristive characteristics of various 2D materials (heterostructures, defect-engineered materials, and alloy materials) used in neuromorphic computing for image segregation or pattern recognition. Neuromorphic computing, the most powerful artificial networks for complicated image processing and recognition, represent a breakthrough in artificial intelligence owing to their enhanced performance and lower power consumption compared with von Neumann architectures. A hardware-implemented CNN with weight control based on synaptic memristor arrays is expected to be a promising candidate for future electronics in society, offering a solution based on non-von Neumann hardware. This emerging paradigm changes the computing algorithm using entirely hardware-connected edge computing and deep neural networks.

SET Kinetics of Ag/HfO2-Based Diffusive Memristors under Various Counter-Electrode Materials

The counter-electrode (CE) material in electrochemical metallization memory (ECM) cells plays a crucial role in the switching process by affecting the reactions at the CE/electrolyte interface. This is due to the different electrocatalytic activity of the CE material towards reduction-oxidation reactions, which determines the metal ion concentration in the electrolyte and ultimately impacts the switching kinetics. In this study, the focus is laid on Pt, TiN, and W, which are relevant in standard chip technology. For these, the influence of CE metal on the switching kinetics of Ag/HfO₂-based volatile ECM cells is investigated. Rectangular voltage pulses of different amplitudes were applied, and the SET times were analyzed from the transient curves. The results show that CE material has a significant effect on the SET kinetics, with differences being observed depending on the voltage regime. The formation of interfacial oxides at the CE/electrolyte interface, particularly for non-noble metals, is also discussed in relation to the findings. Overall, this work highlights the important role of the CE material in the switching process of Ag/HfO₂-based diffusive memristors and the importance of considering interfacial oxide formation in the design of these devices.