A Photoelectric Spiking Neuron for Visual Depth Perception

Leveraging volatile memristors in neuromorphic computing: from materials to system implementation

Inspired by the functions of biological neural networks, volatile memristors are essential for implementing neuromorphic computing. These devices enable large-scale and energy-efficient data processing by emulating neural functionalities through dynamic resistance changes. The threshold switching characteristics of volatile memristors, which are driven by various mechanisms in materials ranging from oxides to chalcogenides, make them versatile and suitable for neuromorphic computing systems. Understanding these mechanisms and selecting appropriate devices for specific applications are crucial for optimizing the performance. However, the existing literature lacks a comprehensive review of switching mechanisms, their compatibility with different applications, and a deeper exploration of the spatiotemporal processing capabilities and inherent stochasticity of volatile memristors. This review begins with a detailed analysis of the operational principles and material characteristics of volatile memristors. Their diverse applications are then explored, emphasizing their role in crossbar arrays, artificial receptors, and neurons. Furthermore, the potential of volatile memristors in artificial inference systems and reservoir computing is discussed, due to their spatiotemporal processing capabilities. Hardware security applications and probabilistic computing are also examined, where the inherent stochasticity of the devices can improve the system robustness and adaptability. To conclude, the suitability of different switching mechanisms for various applications is evaluated, and future perspectives for the development and implementation of volatile memristors are presented. This review aims to fill the gaps in existing research and highlight the potential of volatile memristors to drive innovation in neuromorphic computing, paving the way for more efficient and powerful computational paradigms.

A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO2 Mott Memristor for the Spike-Based Electronic Retina

The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO2 Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 × 107) and high flexibility (bending radius ∼5 mm). Then, we put forward an all-spike electronic retina architecture that comprises one layer of ASPs and one layer of artificial optical nerves (AONs) to process the spike information. Each AON consists of a single Mott memristor connected in series with a neuro-transistor that is a multiple-input floating-gate MOS transistor. Simulation results demonstrate that the all-spike electronic retina can successfully segment images with high Shannon entropy, thus laying the foundation for the development of a spike-based neuromorphic vision system.

A stochastic photo-responsive memristive neuron for an in-sensor visual system based on a restricted Boltzmann machine

In-sensor computing has gained attention as a solution to overcome the von Neumann computing bottlenecks inherent in conventional sensory systems. This attention is due to the ability of sensor elements to directly extract meaningful information from external signals, thereby simplifying complex data. The advantage of in-sensor computing can be maximized with the sampling principle of a restricted Boltzmann machine (RBM) to extract significant features. In this study, a stochastic photo-responsive neuron is developed using a TiN/In-Ga-Zn-O/TiN optoelectronic memristor and an Ag/HfO2/Pt threshold-switching memristor, which can be configured as an input neuron in an in-sensor RBM. It demonstrates a sigmoidal switching probability depending on light intensity. The stochastic properties allow for the simultaneous exploration of various neuron states within the network, making identifying optimal features in complex images easier. Based on semi-empirical simulations, high recognition accuracies of 90.9% and 95.5% are achieved using handwritten digit and face image datasets, respectively. In addition, the in-sensor RBM effectively reconstructs abnormal face images, indicating that integrating in-sensor computing with probabilistic neural networks can lead to reliable and efficient image recognition under unpredictable real-world conditions.

Toward a Brain-Neuromorphics Interface

Brain-computer interfaces (BCIs) that enable human-machine interaction have immense potential in restoring or augmenting human capabilities. Traditional BCIs are realized based on complementary metal-oxide-semiconductor (CMOS) technologies with complex, bulky, and low biocompatible circuits, and suffer with the low energy efficiency of the von Neumann architecture. The brain-neuromorphics interface (BNI) would offer a promising solution to advance the BCI technologies and shape the interactions with machineries. Neuromorphic devices and systems are able to provide substantial computation power with extremely high energy-efficiency by implementing in-materia computing such as in situ vector-matrix multiplication (VMM) and physical reservoir computing. Recent progresses on integrating neuromorphic components with sensing and/or actuating modules, give birth to the neuromorphic afferent nerve, efferent nerve, sensorimotor loop, and so on, which has advanced the technologies for future neurorobotics by achieving sophisticated sensorimotor capabilities as the biological system. With the development on the compact artificial spiking neuron and bioelectronic interfaces, the seamless communication between a BNI and a bioentity is reasonably expectable. In this review, the upcoming BNIs are profiled by introducing the brief history of neuromorphics, reviewing the recent progresses on related areas, and discussing the future advances and challenges that lie ahead.

Hemispherical Retina Emulated by Plasmonic Optoelectronic Memristors with All-Optical Modulation for Neuromorphic Stereo Vision

Binocular stereo vision relies on imaging disparity between two hemispherical retinas, which is essential to acquire image information in three dimensional environment. Therefore, retinomorphic electronics with structural and functional similarities to biological eyes are always highly desired to develop stereo vision perception system. In this work, a hemispherical optoelectronic memristor array based on Ag-TiO2 nanoclusters/sodium alginate film is developed to realize binocular stereo vision. All-optical modulation induced by plasmonic thermal effect and optical excitation in Ag-TiO2 nanoclusters is exploited to realize in-pixel image sensing and storage. Wide field of view (FOV) and spatial angle detection are experimentally demonstrated owing to the device arrangement and incident-angle-dependent characteristics in hemispherical geometry. Furthermore, depth perception and motion detection based on binocular disparity have been realized by constructing two retinomorphic memristive arrays. The results demonstrated in this work provide a promising strategy to develop all-optically controlled memristor and promote the future development of binocular vision system with in-sensor architecture.

Optically Tunable Electrical Oscillations in Oxide-Based Memristors for Neuromorphic Computing

The application of hardware-based neural networks can be enhanced by integrating sensory neurons and synapses that enable direct input from external stimuli. This work reports direct optical control of an oscillatory neuron based on volatile threshold switching in V3O5. The devices exhibit electroforming-free operation with switching parameters that can be tuned by optical illumination. Using temperature-dependent electrical measurements, conductive atomic force microscopy (C-AFM), in situ thermal imaging, and lumped element modelling, it is shown that the changes in switching parameters, including threshold and hold voltages, arise from overall conductivity increase of the oxide film due to the contribution of both photoconductive and bolometric characteristics of V3O5, which eventually affects the oscillation dynamics. Furthermore, V3O5 is identified as a new bolometric material with a temperature coefficient of resistance (TCR) as high as -4.6% K-1 at 423 K. The utility of these devices is illustrated by demonstrating in-sensor reservoir computing with reduced computational effort and an optical encoding layer for spiking neural network (SNN), respectively, using a simulated array of devices.

A Reconfigurable Bipolar Image Sensor for High-Efficiency Dynamic Vision Recognition

Dynamic vision perception and processing (DVPP) is in high demand by booming edge artificial intelligence. However, existing imaging systems suffer from low efficiency or low compatibility with advanced machine vision techniques. Here, we propose a reconfigurable bipolar image sensor (RBIS) for in-sensor DVPP based on a two-dimensional WSe2/GeSe heterostructure device. Owing to the gate-tunable and reversible built-in electric field, its photoresponse shows bipolarity as being positive or negative. High-efficiency DVPP incorporating front-end RBIS and back-end CNN is then demonstrated. It shows a high recognition accuracy of over 94.9% on the derived DVS128 data set and requires much fewer neural network parameters than that without RBIS. Moreover, we demonstrate an optimized device with a vertically stacked structure and a stable nonvolatile bipolarity, which enables more efficient DVPP hardware. Our work demonstrates the potential of fabricating DVPP devices with a simple structure, high efficiency, and outputs compatible with advanced algorithms.

Ovonic threshold switching-based artificial afferent neurons for thermal in-sensor computing

Artificial afferent neurons in the sensory nervous system inspired by biology have enormous potential for efficiently perceiving and processing environmental information. However, the previously reported artificial afferent neurons suffer from two prominent challenges: considerable power consumption and limited scalability efficiency. Herein, addressing these challenges, a bioinspired artificial thermal afferent neuron based on a N-doped SiTe ovonic threshold switching (OTS) device is presented for the first time. The engineered OTS device shows remarkable uniformity and robust endurance, ensuring the reliability and efficacy of the artificial afferent neurons. A substantially decreased leakage current of the SiTe OTS device by nitrogen doping results in ultra-low power consumption less than 0.3 nJ per spike for artificial afferent neurons. The inherent temperature response exhibited by N-doped SiTe OTS materials allows us to construct a highly compact artificial thermal afferent neuron over a wide temperature range. An edge detection task is performed to further verify its thermal perceptual computing function. Our work provides an insight into OTS-based artificial afferent neurons for electronic skin and sensory neurorobotics.

Bio-Inspired Sensory Receptors for Artificial-Intelligence Perception

In the era of artificial intelligence (AI), there is a growing interest in replicating human sensory perception. Selective and sensitive bio-inspired sensory receptors with synaptic plasticity have recently gained significant attention in developing energy-efficient AI perception. Various bio-inspired sensory receptors and their applications in AI perception are reviewed here. The critical challenges for the future development of bio-inspired sensory receptors are outlined, emphasizing the need for innovative solutions to overcome hurdles in sensor design, integration, and scalability. AI perception can revolutionize various fields, including human-machine interaction, autonomous systems, medical diagnostics, environmental monitoring, industrial optimization, and assistive technologies. As advancements in bio-inspired sensing continue to accelerate, the promise of creating more intelligent and adaptive AI systems becomes increasingly attainable, marking a significant step forward in the evolution of human-like sensory perception.

An artificial visual neuron with multiplexed rate and time-to-first-spike coding

Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.

Memristor-Based Bionic Tactile Devices: Opening the Door for Next-Generation Artificial Intelligence

Bioinspired tactile devices can effectively mimic and reproduce the functions of the human tactile system, presenting significant potential in the field of next-generation wearable electronics. In particular, memristor-based bionic tactile devices have attracted considerable attention due to their exceptional characteristics of high flexibility, low power consumption, and adaptability. These devices provide advanced wearability and high-precision tactile sensing capabilities, thus emerging as an important research area within bioinspired electronics. This paper delves into the integration of memristors with other sensing and controlling systems and offers a comprehensive analysis of the recent research advancements in memristor-based bionic tactile devices. These advancements incorporate artificial nociceptors and flexible electronic skin (e-skin) into the category of bio-inspired sensors equipped with capabilities for sensing, processing, and responding to stimuli, which are expected to catalyze revolutionary changes in human-computer interaction. Finally, this review discusses the challenges faced by memristor-based bionic tactile devices in terms of material selection, structural design, and sensor signal processing for the development of artificial intelligence. Additionally, it also outlines future research directions and application prospects of these devices, while proposing feasible solutions to address the identified challenges.

A Spiking Artificial Vision Architecture Based on Fully Emulating the Human Vision

Intelligent vision necessitates the deployment of detectors that are always-on and low-power, mirroring the continuous and uninterrupted responsiveness characteristic of human vision. Nonetheless, contemporary artificial vision systems attain this goal by the continuous processing of massive image frames and executing intricate algorithms, thereby expending substantial computational power and energy. In contrast, biological data processing, based on event-triggered spiking, has higher efficiency and lower energy consumption. Here, this work proposes an artificial vision architecture consisting of spiking photodetectors and artificial synapses, closely mirroring the intricacies of the human visual system. Distinct from previously reported techniques, the photodetector is self-powered and event-triggered, outputting light-modulated spiking signals directly, thereby fulfilling the imperative for always-on with low-power consumption. With the spiking signals processing through the integrated synapse units, recognition of graphics, gestures, and human action has been implemented, illustrating the potent image processing capabilities inherent within this architecture. The results prove the 90% accuracy rate in human action recognition within a mere five epochs utilizing a rudimentary artificial neural network. This novel architecture, grounded in spiking photodetectors, offers a viable alternative to the extant models of always-on low-power artificial vision system.

Memory-electroluminescence for multiple action-potentials combination in bio-inspired afferent nerves

The development of optoelectronics mimicking the functions of the biological nervous system is important to artificial intelligence. This work demonstrates an optoelectronic, artificial, afferent-nerve strategy based on memory-electroluminescence spikes, which can realize multiple action-potentials combination through a single optical channel. The memory-electroluminescence spikes have diverse morphologies due to their history-dependent characteristics and can be used to encode distributed sensor signals. As the key to successful functioning of the optoelectronic, artificial afferent nerve, a driving mode for light-emitting diodes, namely, the non-carrier injection mode, is proposed, allowing it to drive nanoscale light-emitting diodes to generate a memory-electroluminescence spikes that has multiple sub-peaks. Moreover, multiplexing of the spikes can be obtained by using optical signals with different wavelengths, allowing for a large signal bandwidth, and the multiple action-potentials transmission process in afferent nerves can be demonstrated. Finally, sensor-position recognition with the bio-inspired afferent nerve is developed and shown to have a high recognition accuracy of 98.88%. This work demonstrates a strategy for mimicking biological afferent nerves and offers insights into the construction of artificial perception systems.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

High-performance robotic vision empowers mobile and humanoid robots to detect and identify their surrounding objects efficiently, which enables them to cooperate with humans and assist human activities. For error-free execution of these robots' tasks, efficient imaging and data processing capabilities are essential, even under diverse and complex environments. However, conventional technologies fall short of meeting the high-standard requirements of robotic vision under such circumstances. Here, we discuss recent progress in artificial vision systems with high-performance imaging and data processing capabilities enabled by distinctive electrical, optical, and mechanical characteristics of nanomaterials surpassing the limitations of traditional silicon technologies. In particular, we focus on nanomaterial-based electronic eyes and in-sensor processing devices inspired by biological eyes and animal visual recognition systems, respectively. We provide perspectives on key nanomaterials, device components, and their functionalities, as well as explain the remaining challenges and future prospects of the artificial vision systems.

Towards Artificial Visual Sensory System: Organic Optoelectronic Synaptic Materials and Devices

Developing an artificial visual sensory system requires optoelectronic materials and devices that can mimic the behavior of biological synapses. Organic/polymeric semiconductors have emerged as promising candidates for optoelectronic synapses due to their tunable optoelectronic properties, mechanic flexibility, and biological compatibility. In this review, we discuss the recent progress in organic optoelectronic synaptic materials and devices, including their design principles, working mechanisms, and applications. We also highlight the challenges and opportunities in this field and provide insights into potential applications of these materials and devices in next-generation artificial visual systems. By leveraging the advances in organic optoelectronic materials and devices, we can envision its future development in artificial intelligence.

Humidity/Oxygen-Insensitive Organic Synaptic Transistors Based on Optical Radical Effect

For most organic synaptic transistors based on the charge trapping effect, different atmosphere conditions lead to significantly different device performance. Some devices even lose the synaptic responses under vacuum or inert atmosphere. The stable device performance of these organic synaptic transistors under varied working environments with different humidity and oxygen levels can be a challenge. Herein, a moisture- and oxygen-insensitive organic synaptic device based on the organic semiconductor and photoinitiator molecules is reported. Unlike the widely reported charge trapping effect, the photoinduced free radical is utilized to realize the photosynaptic performance. The resulting synaptic transistor displays typical excitatory postsynaptic current, paired-pulse facilitation, learning, and forgetting behaviors. Furthermore, the device exhibits decent and stable photosynaptic performances under high humidity and vacuum conditions. This type of organic synaptic device also demonstrates high potential in ultraviolet B perception based on its environmental stability and broad ultraviolet detection capability. Finally, the contrast-enhanced capability of the device is successfully validated by the single-layer-perceptron/double-layer network based Modified National Institute of Standards and Technology pattern recognition. This work could have important implications for the development of next-generation environment-stable organic synaptic devices and systems.

A bidirectional thermal sensory leaky integrate-and-fire (LIF) neuron model based on bipolar NbOx volatile threshold devices with ultra-low operating current

Brain-like artificial intelligence (AI) will become the main form and important platform in future computing. It will play an important and unique role in simulating brain functions, efficiently implementing AI algorithms, and improving computing power. Developing artificial neurons that can send facilitation/depression signals to artificial synapses, sense, and process temperature information is of great significance for achieving more efficient and compact brain-like computing systems. Herein, we have constructed a NbOx bipolar volatile threshold memristor, which could be operated by 1 μA ultra-low current and up to ∼104 switching ratios. By using a leaky integrate-and-fire (LIF) artificial neuron model, a bipolar LIF artificial neuron is constructed, which can realize the conventional threshold-driven firing, all-or-nothing spiking, refractory periods, and intensity-modulated frequency response bidirectionally at the positive/negative voltage stimulation, which will give the artificial synapse facilitation/depression signals. Furthermore, this bipolar LIF neuron can also explore different temperatures to output different signals, which could be constructed as a more compact thermal sensory neuron to avoid external harm to artificial robots. This study is of great significance for improving the computational efficiency of the system more effectively, achieving high integration density and low energy consumption artificial neural networks to meet the needs of brain-like neural computing.

In-Sensor Computing Realization Using Fully CMOS-Compatible TiN/HfOx-Based Neuristor Array

With the evolution of artificial intelligence, the explosive growth of data from sensory terminals gives rise to severe energy-efficiency bottleneck issues due to cumbersome data interactions among sensory, memory, and computing modules. Heterogeneous integration methods such as chiplet technology can significantly reduce unnecessary data movement; however, they fail to address the fundamental issue of the substantial time and energy overheads resulting from the physical separation of computing and sensory components. Brain-inspired in-sensor neuromorphic computing (ISNC) has plenty of room for such data-intensive applications. However, one key obstacle in developing ISNC systems is the lack of compatibility between material systems and manufacturing processes deployed in sensors and computing units. This study successfully addresses this challenge by implementing fully CMOS-compatible TiN/HfOx-based neuristor array. The developed ISNC system demonstrates several advantageous features, including multilevel analogue modulation, minimal dispersion, and no significant degradation in conductance (@125 °C). These characteristics enable stable and reproducible neuromorphic computing. Additionally, the device exhibits modulatable sensory and multi-store memory processes. Furthermore, the system achieves information recognition with a high accuracy rate of 93%, along with frequency selectivity and notable activity-dependent plasticity. This work provides a promising route to affordable and highly efficient sensory neuromorphic systems.

Rationally Designing High-Performance Versatile Organic Memristors through Molecule-Mediated Ion Movements

Organic memory has attracted tremendous attention for next-generation electronic elements for the molecules' striking ease of structural design. However, due to them being hardly controllable and their low ion transport, it is always essential and challenge to effectively control their random migration, pathway, and duration. There are very few effective strategies, and specific platforms with a view to molecules with specific coordination-groups-regulating ions have been rarely reported. In this work, as a generalized rational design strategy, the well-known tetracyanoquinodimethane (TCNQ) is introduced with multiple coordination groups and small plane structure into a stable polymers framework to modulate Ag migration and then achieve high-performance devices with ideal productivity, low operation voltage and power, stable switching cycles, and state retention. Raman mapping demonstrates that the migrated Ag can specially coordinate with the embedded TCNQ molecules. Notably, the TCNQ molecule distribution can be modulated inside the polymer framework and regulate the memristive behaviors through controlling the formed Ag conductive filaments (CFs) as demonstrated by Raman mapping, in situ conductive atomic force microscopy (C-AFM), X-ray diffraction (XRD) and depth-profiling X-ray photoelectron spectroscopy (XPS). Thus the controllable molecule-mediated Ag movements show its potential in rationally designing high-performance devices and versatile functions and is enlightening in constructing memristors with molecule-mediated ion movements.

Nearly Panoramic Neuromorphic Vision with Transparent Photosynapses

Neuromorphic vision based on photonic synapses has the ability to mimic sensitivity, adaptivity, and sophistication of bio-visual systems. Significant advances in artificial photosynapses are achieved recently. However, conventional photosyanptic devices normally employ opaque metal conductors and vertical device configuration, performing a limited hemispherical field of view. Here, a transparent planar photonic synapse (TPPS) is presented that offers dual-side photosensitive capability for nearly panoramic neuromorphic vision. The TPPS consisting of all two dimensional (2D) carbon-based derivatives exhibits ultra-broadband photodetecting (365-970 nm) and ≈360° omnidirectional viewing angle. With its intrinsic persistent photoconductivity effect, the detector possesses bio-synaptic behaviors such as short/long-term memory, experience learning, light adaptation, and a 171% pair-pulse-facilitation index, enabling the synapse array to achieve image recognition enhancement (92%) and moving object detection.

Helical Nanofiber Photoelectric Synaptic Devices for an Artificial Vision Nervous System

Inspired by the helical structure and the resultant exquisite functions of biomolecules, helical polymers have received increasing attention. Here, a series of poly(3-hexylthiophene)-block-poly(phenyl isocyanide) (P3HT-b-PPI) copolymers were prepared using a simple one-pot living polymerization method. Interestingly, the P3HT80-b-PPI30 films were found to have a helical nanofiber structure. The corresponding device has superior optoelectronic properties, such as a broadened spectral response range from the visible band to the deep ultraviolet (DUV) and an approximately 5-fold longer carrier decay time after DUV light stimulation. An energy consumption of 1.44 fJ per synaptic event was obtained, which is the lowest energy consumption achieved so far with DUV light stimulation. The encryption and decryption of images are implemented using an array of devices. Finally, a photoreceptor neural pathway was constructed to achieve early warning for the recognition of the display of harmful light. This research provides an effective strategy for the development of a novel optoelectronic synaptic device.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

Emerging Iontronic Neural Devices for Neuromorphic Sensory Computing

Living organisms have a very mysterious and powerful sensory computing system based on ion activity. Interestingly, studies on iontronic devices in the past few years have proposed a promising platform for simulating the sensing and computing functions of living organisms, because: 1) iontronic devices can generate, store, and transmit a variety of signals by adjusting the concentration and spatiotemporal distribution of ions, which analogs to how the brain performs intelligent functions by alternating ion flux and polarization; 2) through ionic-electronic coupling, iontronic devices can bridge the biosystem with electronics and offer profound implications for soft electronics; 3) with the diversity of ions, iontronic devices can be designed to recognize specific ions or molecules by customizing the charge selectivity, and the ionic conductivity and capacitance can be adjusted to respond to external stimuli for a variety of sensing schemes, which can be more difficult for electron-based devices. This review provides a comprehensive overview of emerging neuromorphic sensory computing by iontronic devices, highlighting representative concepts of both low-level and high-level sensory computing and introducing important material and device breakthroughs. Moreover, iontronic devices as a means of neuromorphic sensing and computing are discussed regarding the pending challenges and future directions.

Vertically integrated spiking cone photoreceptor arrays for color perception

The cone photoreceptors in our eyes selectively transduce the natural light into spiking representations, which endows the brain with high energy-efficiency color vision. However, the cone-like device with color-selectivity and spike-encoding capability remains challenging. Here, we propose a metal oxide-based vertically integrated spiking cone photoreceptor array, which can directly transduce persistent lights into spike trains at a certain rate according to the input wavelengths. Such spiking cone photoreceptors have an ultralow power consumption of less than 400 picowatts per spike in visible light, which is very close to biological cones. In this work, lights with three wavelengths were exploited as pseudo-three-primary colors to form 'colorful' images for recognition tasks, and the device with the ability to discriminate mixed colors shows better accuracy. Our results would enable hardware spiking neural networks with biologically plausible visual perception and provide great potential for the development of dynamic vision sensors.

Optically Readable Organic Electrochemical Synaptic Transistors for Neuromorphic Photonic Image Processing

Optically readable organic synaptic devices have great potential in both artificial intelligence and photonic neuromorphic computing. Herein, a novel optically readable organic electrochemical synaptic transistor (OR-OEST) strategy is first proposed. The electrochemical doping mechanism of the device was systematically investigated, and the basic biological synaptic behaviors that can be read by optical means are successfully achieved. Furthermore, the flexible OR-OESTs are capable of electrically switching the transparency of semiconductor channel materials in a nonvolatile manner, and thus the multilevel memory can be achieved through optical readout. Finally, the OR-OESTs are developed for the preprocessing of photonic images, such as contrast enhancement and denoising, and feeding the processed images into an artificial neural network, achieving a recognition rate of over 90%. Overall, this work provides a new strategy for the implementation of photonic neuromorphic systems.

A two-dimensional mid-infrared optoelectronic retina enabling simultaneous perception and encoding

Infrared machine vision system for object perception and recognition is becoming increasingly important in the Internet of Things era. However, the current system suffers from bulkiness and inefficiency as compared to the human retina with the intelligent and compact neural architecture. Here, we present a retina-inspired mid-infrared (MIR) optoelectronic device based on a two-dimensional (2D) heterostructure for simultaneous data perception and encoding. A single device can perceive the illumination intensity of a MIR stimulus signal, while encoding the intensity into a spike train based on a rate encoding algorithm for subsequent neuromorphic computing with the assistance of an all-optical excitation mechanism, a stochastic near-infrared (NIR) sampling terminal. The device features wide dynamic working range, high encoding precision, and flexible adaption ability to the MIR intensity. Moreover, an inference accuracy more than 96% to MIR MNIST data set encoded by the device is achieved using a trained spiking neural network (SNN).

Technology Roadmap for Flexible Sensors

Humans rely increasingly on sensors to address grand challenges and to improve quality of life in the era of digitalization and big data. For ubiquitous sensing, flexible sensors are developed to overcome the limitations of conventional rigid counterparts. Despite rapid advancement in bench-side research over the last decade, the market adoption of flexible sensors remains limited. To ease and to expedite their deployment, here, we identify bottlenecks hindering the maturation of flexible sensors and propose promising solutions. We first analyze challenges in achieving satisfactory sensing performance for real-world applications and then summarize issues in compatible sensor-biology interfaces, followed by brief discussions on powering and connecting sensor networks. Issues en route to commercialization and for sustainable growth of the sector are also analyzed, highlighting environmental concerns and emphasizing nontechnical issues such as business, regulatory, and ethical considerations. Additionally, we look at future intelligent flexible sensors. In proposing a comprehensive roadmap, we hope to steer research efforts towards common goals and to guide coordinated development strategies from disparate communities. Through such collaborative efforts, scientific breakthroughs can be made sooner and capitalized for the betterment of humanity.

Highly Reliable Threshold Switching Characteristics of Surface-Modulated Diffusive Memristors Immune to Atmospheric Changes

Active cation-based diffusive memristors featuring essentially volatile threshold switching have been proposed for novel applications, such as a selector in a one-selector-and-one-resistor structure and signal generators in neuromorphic computing. However, the high variability of the switching behavior, which results from the high electroforming voltage, external environmental conditions, and transition to the non-volatile switching mode in a high-current range, is considered a major impediment to such applications. Herein, for the first time, we developed a highly reliable threshold switching device immune to atmospheric changes based on an ultraviolet-ozone (UVO)-treated diffusive memristor consisting of Ag and SiO₂ nanorods (NRs). UVO treatment forms a stable water reservoir on the surface of SiO₂ NRs, facilitating the redox reaction and ion migration of Ag. Consequently, diffusive memristors possess reliable switching characteristics, including electroforming-free, repeatable, and consistent switching with resistance to changes in ambient conditions and compliance levels during operation. We demonstrated that our approach is suitable for various metal oxides and can be used in numerous applications.

A MoS2 Hafnium Oxide Based Ferroelectric Encoder for Temporal-Efficient Spiking Neural Network

Spiking neural network (SNN), where the information is evaluated recurrently through spikes, has manifested significant promises to minimize the energy expenditure in data-intensive machine learning and artificial intelligence. Among these applications, the artificial neural encoders are essential to convert the external stimuli to a spiking format that can be subsequently fed to the neural network. Here, a molybdenum disulfide (MoS₂ ) hafnium oxide-based ferroelectric encoder is demonstrated for temporal-efficient information processing in SNN. The fast domain switching attribute associated with the polycrystalline nature of hafnium oxide-based ferroelectric material is exploited for spike encoding, rendering it suitable for realizing biomimetic encoders. Accordingly, a high-performance ferroelectric encoder is achieved, featuring a superior switching efficiency, negligible charge trapping effect, and robust ferroelectric response, which successfully enable a broad dynamic range. Furthermore, an SNN is simulated to verify the precision of the encoded information, in which an average inference accuracy of 95.14% can be achieved, using the Modified National Insitute of Standards and Technology (MNIST) dataset for digit classification. Moreover, this ferroelectric encoder manifests prominent resilience against noise injection with an overall prediction accuracy of 94.73% under various Gaussian noise levels, showing practical promises to reduce the computational load for the neural network.

Emerging memristive neurons for neuromorphic computing and sensing

Inspired by the principles of the biological nervous system, neuromorphic engineering has brought a promising alternative approach to intelligence computing with high energy efficiency and low consumption. As pivotal components of neuromorphic system, artificial spiking neurons are powerful information processing units and can achieve highly complex nonlinear computations. By leveraging the switching dynamic characteristics of memristive device, memristive neurons show rich spiking behaviors with simple circuit. This report reviews the memristive neurons and their applications in neuromorphic sensing and computing systems. The switching mechanisms that endow memristive devices with rich dynamics and nonlinearity are highlighted, and subsequently various nonlinear spiking neuron behaviors emulated in these memristive devices are reviewed. Then, recent development is introduced on neuromorphic system with memristive neurons for sensing and computing. Finally, we discuss challenges and outlooks of the memristive neurons toward high-performance neuromorphic hardware systems and provide an insightful perspective for the development of interactive neuromorphic electronic systems.

Multiphotoconductance Levels of the Organic Semiconductor of Polyimide-Based Memristor Induced by Interface Charges

A memristor with Au/polyimide (PI)/Au structure is prepared by magnetron sputtering to investigate the multiphotoconductance resistive switching (RS) memory behavior. The PI-based memristor presents stable bipolar RS memory and is sensitive to visible light. Four discrete conductance states in both high-resistance state (HRS) and low-resistance state (LRS) are obtained when illuminating by 365, 550, 590, and 780 nm light. Electron trapping and detrapping from the defects distributed at interfaces and the PI switching layer are responsible for the observed RS memory behavior. The enhanced trapping and detrapping process by light illumination is responsible for the multiconductance states. This work provides the possibility for further development of neuromorphic vision sensors using an organic semiconductor-based memristor.

Progress of Materials and Devices for Neuromorphic Vision Sensors

The latest developments in bio-inspired neuromorphic vision sensors can be summarized in 3 keywords: smaller, faster, and smarter. (1) Smaller: Devices are becoming more compact by integrating previously separated components such as sensors, memory, and processing units. As a prime example, the transition from traditional sensory vision computing to in-sensor vision computing has shown clear benefits, such as simpler circuitry, lower power consumption, and less data redundancy. (2) Swifter: Owing to the nature of physics, smaller and more integrated devices can detect, process, and react to input more quickly. In addition, the methods for sensing and processing optical information using various materials (such as oxide semiconductors) are evolving. (3) Smarter: Owing to these two main research directions, we can expect advanced applications such as adaptive vision sensors, collision sensors, and nociceptive sensors. This review mainly focuses on the recent progress, working mechanisms, image pre-processing techniques, and advanced features of two types of neuromorphic vision sensors based on near-sensor and in-sensor vision computing methodologies.