Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array

Memristive Synapses with Photoelectric Plasticity Realized in ZnO1-x/AlOy Heterojunction

With the end of Moore's law in sight, new computing architectures are urgently needed to satisfy the increasing demands for big data processing. Neuromorphic architectures with photoelectric learning capability are good candidates for energy-efficient computing for recognition and classification tasks. In this work, artificial synapses based on the ZnO_1-x/AlO_y heterojunction were fabricated and the photoelectric plasticity was investigated. Versatile synaptic functions such as photoelectric short-term/long-term plasticity, paired-pulse facilitation, neuromorphic facilitation, and depression were emulated based on the inherent persistent photoconductivity and volatile resistive switching characteristics of the device. It is found that the naturally formed AlO_y layer provides traps for photogenerated holes, resulting in a significant persistent photoconductivity effect. Moreover, the resistive switching can be attributed to the electron trapping/detrapping at the trapping sites in the AlO_y layer.

Recent Advances on Neuromorphic Systems Using Phase-Change Materials

Realization of brain-like computer has always been human's ultimate dream. Today, the possibility of having this dream come true has been significantly boosted due to the advent of several emerging non-volatile memory devices. Within these innovative technologies, phase-change memory device has been commonly regarded as the most promising candidate to imitate the biological brain, owing to its excellent scalability, fast switching speed, and low energy consumption. In this context, a detailed review concerning the physical principles of the neuromorphic circuit using phase-change materials as well as a comprehensive introduction of the currently available phase-change neuromorphic prototypes becomes imperative for scientists to continuously progress the technology of artificial neural networks. In this paper, we first present the biological mechanism of human brain, followed by a brief discussion about physical properties of phase-change materials that recently receive a widespread application on non-volatile memory field. We then survey recent research on different types of neuromorphic circuits using phase-change materials in terms of their respective geometrical architecture and physical schemes to reproduce the biological events of human brain, in particular for spike-time-dependent plasticity. The relevant virtues and limitations of these devices are also evaluated. Finally, the future prospect of the neuromorphic circuit based on phase-change technologies is envisioned.

Flexible Ionic-Electronic Hybrid Oxide Synaptic TFTs with Programmable Dynamic Plasticity for Brain-Inspired Neuromorphic Computing

Emulation of biological synapses is necessary for future brain-inspired neuromorphic computational systems that could look beyond the standard von Neuman architecture. Here, artificial synapses based on ionic-electronic hybrid oxide-based transistors on rigid and flexible substrates are demonstrated. The flexible transistors reported here depict a high field-effect mobility of ≈9 cm² V^-1 s^-1 with good mechanical performance. Comprehensive learning abilities/synaptic rules like paired-pulse facilitation, excitatory and inhibitory postsynaptic currents, spike-time-dependent plasticity, consolidation, superlinear amplification, and dynamic logic are successfully established depicting concurrent processing and memory functionalities with spatiotemporal correlation. The results present a fully solution processable approach to fabricate artificial synapses for next-generation transparent neural circuits.

Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity

Brain-inspired computation can revolutionize information technology by introducing machines capable of recognizing patterns (images, speech, video) and interacting with the external world in a cognitive, humanlike way. Achieving this goal requires first to gain a detailed understanding of the brain operation, and second to identify a scalable microelectronic technology capable of reproducing some of the inherent functions of the human brain, such as the high synaptic connectivity (~10⁴) and the peculiar time-dependent synaptic plasticity. Here we demonstrate unsupervised learning and tracking in a spiking neural network with memristive synapses, where synaptic weights are updated via brain-inspired spike timing dependent plasticity (STDP). The synaptic conductance is updated by the local time-dependent superposition of pre- and post-synaptic spikes within a hybrid one-transistor/one-resistor (1T1R) memristive synapse. Only 2 synaptic states, namely the low resistance state (LRS) and the high resistance state (HRS), are sufficient to learn and recognize patterns. Unsupervised learning of a static pattern and tracking of a dynamic pattern of up to 4 × 4 pixels are demonstrated, paving the way for intelligent hardware technology with up-scaled memristive neural networks.

Brain-Inspired Photonic Neuromorphic Devices using Photodynamic Amorphous Oxide Semiconductors and their Persistent Photoconductivity

The combination of a neuromorphic architecture and photonic computing may open up a new era for computational systems owing to the possibility of attaining high bandwidths and the low-computation-power requirements. Here, the demonstration of photonic neuromorphic devices based on amorphous oxide semiconductors (AOSs) that mimic major synaptic functions, such as short-term memory/long-term memory, spike-timing-dependent plasticity, and neural facilitation, is reported. The synaptic functions are successfully emulated using the inherent persistent photoconductivity (PPC) characteristic of AOSs. Systematic analysis of the dynamics of photogenerated carriers for various AOSs is carried out to understand the fundamental mechanisms underlying the photoinduced carrier-generation and relaxation behaviors, and to search for a proper channel material for photonic neuromorphic devices. It is found that the activation energy for the neutralization of ionized oxygen vacancies has a significant influence on the photocarrier-generation and time-variant recovery behaviors of AOSs, affecting the PPC behavior.

Memristive Devices with Highly Repeatable Analog States Boosted by Graphene Quantum Dots

Memristive devices, having a huge potential as artificial synapses for low-power neural networks, have received tremendous attention recently. Despite great achievements in demonstration of plasticity and learning functions, little progress has been made in the repeatable analog resistance states of memristive devices, which is, however, crucial for achieving controllable synaptic behavior. The controllable behavior of synapse is highly desired in building neural networks as it helps reduce training epochs and diminish error probability. Fundamentally, the poor repeatability of analog resistance states is closely associated with the random formation of conductive filaments, which consists of oxygen vacancies. In this work, graphene quantum dots (GQDs) are introduced into memristive devices. By virtue of the abundant oxygen anions released from GQDs, the GQDs can serve as nano oxygen-reservoirs and enhance the localization of filament formation. As a result, analog resistance states with highly tight distribution are achieved with nearly 85% reduction in variations. In addition the insertion of GQDs can alter the energy band alignment and boost the tunneling current, which leads to significant reduction in both switching voltages and their distribution variations. This work may pave the way for achieving artificial neural networks with accurate and efficient learning capability.

Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning

Emerging brain-inspired architectures call for devices that can emulate the functionality of biological synapses in order to implement new efficient computational schemes able to solve ill-posed problems. Various devices and solutions are still under investigation and, in this respect, a challenge is opened to the researchers in the field. Indeed, the optimal candidate is a device able to reproduce the complete functionality of a synapse, i.e., the typical synaptic process underlying learning in biological systems (activity-dependent synaptic plasticity). This implies a device able to change its resistance (synaptic strength, or weight) upon proper electrical stimuli (synaptic activity) and showing several stable resistive states throughout its dynamic range (analog behavior). Moreover, it should be able to perform spike timing dependent plasticity (STDP), an associative homosynaptic plasticity learning rule based on the delay time between the two firing neurons the synapse is connected to. This rule is a fundamental learning protocol in state-of-art networks, because it allows unsupervised learning. Notwithstanding this fact, STDP-based unsupervised learning has been proposed several times mainly for binary synapses rather than multilevel synapses composed of many binary memristors. This paper proposes an HfO₂-based analog memristor as a synaptic element which performs STDP within a small spiking neuromorphic network operating unsupervised learning for character recognition. The trained network is able to recognize five characters even in case incomplete or noisy images are displayed and it is robust to a device-to-device variability of up to ±30%.

Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses

In an increasingly data-rich world the need for developing computing systems that cannot only process, but ideally also interpret big data is becoming continuously more pressing. Brain-inspired concepts have shown great promise towards addressing this need. Here we demonstrate unsupervised learning in a probabilistic neural network that utilizes metal-oxide memristive devices as multi-state synapses. Our approach can be exploited for processing unlabelled data and can adapt to time-varying clusters that underlie incoming data by supporting the capability of reversible unsupervised learning. The potential of this work is showcased through the demonstration of successful learning in the presence of corrupted input data and probabilistic neurons, thus paving the way towards robust big-data processors.

Physical Realization of a Supervised Learning System Built with Organic Memristive Synapses

Multiple modern applications of electronics call for inexpensive chips that can perform complex operations on natural data with limited energy. A vision for accomplishing this is implementing hardware neural networks, which fuse computation and memory, with low cost organic electronics. A challenge, however, is the implementation of synapses (analog memories) composed of such materials. In this work, we introduce robust, fastly programmable, nonvolatile organic memristive nanodevices based on electrografted redox complexes that implement synapses thanks to a wide range of accessible intermediate conductivity states. We demonstrate experimentally an elementary neural network, capable of learning functions, which combines four pairs of organic memristors as synapses and conventional electronics as neurons. Our architecture is highly resilient to issues caused by imperfect devices. It tolerates inter-device variability and an adaptable learning rule offers immunity against asymmetries in device switching. Highly compliant with conventional fabrication processes, the system can be extended to larger computing systems capable of complex cognitive tasks, as demonstrated in complementary simulations.

Training and operation of an integrated neuromorphic network based on metal-oxide memristors

Despite much progress in semiconductor integrated circuit technology, the extreme complexity of the human cerebral cortex, with its approximately 10(14) synapses, makes the hardware implementation of neuromorphic networks with a comparable number of devices exceptionally challenging. To provide comparable complexity while operating much faster and with manageable power dissipation, networks based on circuits combining complementary metal-oxide-semiconductors (CMOSs) and adjustable two-terminal resistive devices (memristors) have been developed. In such circuits, the usual CMOS stack is augmented with one or several crossbar layers, with memristors at each crosspoint. There have recently been notable improvements in the fabrication of such memristive crossbars and their integration with CMOS circuits, including first demonstrations of their vertical integration. Separately, discrete memristors have been used as artificial synapses in neuromorphic networks. Very recently, such experiments have been extended to crossbar arrays of phase-change memristive devices. The adjustment of such devices, however, requires an additional transistor at each crosspoint, and hence these devices are much harder to scale than metal-oxide memristors, whose nonlinear current-voltage curves enable transistor-free operation. Here we report the experimental implementation of transistor-free metal-oxide memristor crossbars, with device variability sufficiently low to allow operation of integrated neural networks, in a simple network: a single-layer perceptron (an algorithm for linear classification). The network can be taught in situ using a coarse-grain variety of the delta rule algorithm to perform the perfect classification of 3 × 3-pixel black/white images into three classes (representing letters). This demonstration is an important step towards much larger and more complex memristive neuromorphic networks.

Unsupervised Learning by Spike Timing Dependent Plasticity in Phase Change Memory (PCM) Synapses

We present a novel one-transistor/one-resistor (1T1R) synapse for neuromorphic networks, based on phase change memory (PCM) technology. The synapse is capable of spike-timing dependent plasticity (STDP), where gradual potentiation relies on set transition, namely crystallization, in the PCM, while depression is achieved via reset or amorphization of a chalcogenide active volume. STDP characteristics are demonstrated by experiments under variable initial conditions and number of pulses. Finally, we support the applicability of the 1T1R synapse for learning and recognition of visual patterns by simulations of fully connected neuromorphic networks with 2 or 3 layers with high recognition efficiency. The proposed scheme provides a feasible low-power solution for on-line unsupervised machine learning in smart reconfigurable sensors.

Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits

The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2- x /Pt memristors and CMOS integrated circuit components.

Relation between bandgap and resistance drift in amorphous phase change materials

Memory based on phase change materials is currently the most promising candidate for bridging the gap in access time between memory and storage in traditional memory hierarchy. However, multilevel storage is still hindered by the so-called resistance drift commonly related to structural relaxation of the amorphous phase. Here, we present the temporal evolution of infrared spectra measured on amorphous thin films of the three phase change materials Ag₄In₃Sb₆₇Te₂₆, GeTe and the most popular Ge₂Sb₂Te₅. A widening of the bandgap upon annealing accompanied by a decrease of the optical dielectric constant ε_∞ is observed for all three materials. Quantitative comparison with experimental data for the apparent activation energy of conduction reveals that the temporal evolution of bandgap and activation energy can be decoupled. The case of Ag₄In₃Sb₆₇Te₂₆, where the increase of activation energy is significantly smaller than the bandgap widening, demonstrates the possibility to identify new phase change materials with reduced resistance drift.

Carbon Nanotube Synaptic Transistor Network for Pattern Recognition

Inspired by the human brain, a neuromorphic system combining complementary metal-oxide semiconductor (CMOS) and adjustable synaptic devices may offer new computing paradigms by enabling massive neural-network parallelism. In particular, synaptic devices, which are capable of emulating the functions of biological synapses, are used as the essential building blocks for an information storage and processing system. However, previous synaptic devices based on two-terminal resistive devices remain challenging because of their variability and specific physical mechanisms of resistance change, which lead to a bottleneck in the implementation of a high-density synaptic device network. Here we report that a three-terminal synaptic transistor based on carbon nanotubes can provide reliable synaptic functions that encode relative timing and regulate weight change. In addition, using system-level simulations, the developed synaptic transistor network associated with CMOS circuits can perform unsupervised learning for pattern recognition using a simplified spike-timing-dependent plasticity scheme.

Fully parallel write/read in resistive synaptic array for accelerating on-chip learning

A neuro-inspired computing paradigm beyond the von Neumann architecture is emerging and it generally takes advantage of massive parallelism and is aimed at complex tasks that involve intelligence and learning. The cross-point array architecture with synaptic devices has been proposed for on-chip implementation of the weighted sum and weight update in the learning algorithms. In this work, forming-free, silicon-process-compatible Ta/TaOx/TiO2/Ti synaptic devices are fabricated, in which >200 levels of conductance states could be continuously tuned by identical programming pulses. In order to demonstrate the advantages of parallelism of the cross-point array architecture, a novel fully parallel write scheme is designed and experimentally demonstrated in a small-scale crossbar array to accelerate the weight update in the training process, at a speed that is independent of the array size. Compared to the conventional row-by-row write scheme, it achieves >30× speed-up and >30× improvement in energy efficiency as projected in a large-scale array. If realistic synaptic device characteristics such as device variations are taken into an array-level simulation, the proposed array architecture is able to achieve ∼95% recognition accuracy of MNIST handwritten digits, which is close to the accuracy achieved by software using the ideal sparse coding algorithm.

Projected phase-change memory devices

Nanoscale memory devices, whose resistance depends on the history of the electric signals applied, could become critical building blocks in new computing paradigms, such as brain-inspired computing and memcomputing. However, there are key challenges to overcome, such as the high programming power required, noise and resistance drift. Here, to address these, we present the concept of a projected memory device, whose distinguishing feature is that the physical mechanism of resistance storage is decoupled from the information-retrieval process. We designed and fabricated projected memory devices based on the phase-change storage mechanism and convincingly demonstrate the concept through detailed experimentation, supported by extensive modelling and finite-element simulations. The projected memory devices exhibit remarkably low drift and excellent noise performance. We also demonstrate active control and customization of the programming characteristics of the device that reliably realize a multitude of resistance states.

Tunnel junction based memristors as artificial synapses

We prepared magnesia, tantalum oxide, and barium titanate based tunnel junction structures and investigated their memristive properties. The low amplitudes of the resistance change in these types of junctions are the major obstacle for their use. Here, we increased the amplitude of the resistance change from 10% up to 100%. Utilizing the memristive properties, we looked into the use of the junction structures as artificial synapses. We observed analogs of long-term potentiation, long-term depression and spike-time dependent plasticity in these simple two terminal devices. Finally, we suggest a possible pathway of these devices toward their integration in neuromorphic systems for storing analog synaptic weights and supporting the implementation of biologically plausible learning mechanisms.

Associative memory realized by a reconfigurable memristive Hopfield neural network

Although synaptic behaviours of memristors have been widely demonstrated, implementation of an even simple artificial neural network is still a great challenge. In this work, we demonstrate the associative memory on the basis of a memristive Hopfield network. Different patterns can be stored into the memristive Hopfield network by tuning the resistance of the memristors, and the pre-stored patterns can be successfully retrieved directly or through some associative intermediate states, being analogous to the associative memory behaviour. Both single-associative memory and multi-associative memories can be realized with the memristive Hopfield network.

A 2-transistor/1-resistor artificial synapse capable of communication and stochastic learning in neuromorphic systems

Resistive (or memristive) switching devices based on metal oxides find applications in memory, logic and neuromorphic computing systems. Their small area, low power operation, and high functionality meet the challenges of brain-inspired computing aiming at achieving a huge density of active connections (synapses) with low operation power. This work presents a new artificial synapse scheme, consisting of a memristive switch connected to 2 transistors responsible for gating the communication and learning operations. Spike timing dependent plasticity (STDP) is achieved through appropriate shaping of the pre-synaptic and the post synaptic spikes. Experiments with integrated artificial synapses demonstrate STDP with stochastic behavior due to (i) the natural variability of set/reset processes in the nanoscale switch, and (ii) the different response of the switch to a given stimulus depending on the initial state. Experimental results are confirmed by model-based simulations of the memristive switching. Finally, system-level simulations of a 2-layer neural network and a simplified STDP model show random learning and recognition of patterns.

Hardware implementation of associative memory characteristics with analogue-type resistive-switching device

We have investigated the analogue memory characteristics of an oxide-based resistive-switching device under an electrical pulse to mimic biological spike-timing-dependent plasticity synapse characteristics. As a synaptic device, a TiN/Pr0.7Ca0.3MnO3-based resistive-switching device exhibiting excellent analogue memory characteristics was used to control the synaptic weight by applying various pulse amplitudes and cycles. Furthermore, potentiation and depression characteristics with the same spikes can be achieved by applying negative and positive pulses, respectively. By adopting complementary metal-oxide-semiconductor devices as neurons and TiN/PCMO devices as synapses, we implemented neuromorphic hardware that mimics associative memory characteristics in real time for the first time. Owing to their excellent scalability, resistive-switching devices, shows promise for future high-density neuromorphic applications.