Artificial retinas — fast, versatile image processors

Reconfigurable Homojunction Phototransistor for Near-Zero Power Consumption Artificial Biomimetic Retina Function

Semiconductor photodetectors integrating preliminary signal-processing functions play a vital role in artificial biomimetic retina systems. Herein, we propose a tungsten diselenide (WSe2) phototransistor with a dual-layer gate dielectric and an asymmetric graphene insert structure. This phototransistor exhibits a bidirectional self-powered photocurrent by controlling the gate voltage via the formation of reconfigurable p+-p and n-p homojunctions in the channel from the asymmetric graphene insert. At the same time, the nonvolatile electron and hole stored in the dual-layer gate dielectric are generated using a temporary gate voltage, which can replace the gate voltage to regulate the channel charge. Moreover, the photocurrent shows a linear relation with the temporary programming gate voltage. The phototransistor exhibits a rectification ratio of >4 orders of magnitude without the gate voltage, indicating its significant capability to operate in a fully self-powered mode with near-zero power consumption. Based on the device characteristics, we successfully simulate the biological functions of the photoreceptor layer and bipolar cell layer in the retinal receptive field. The identification of the object motion direction in the receptive field can be realized by integrating three programmable devices on the chip. Furthermore, edge enhancement of the image is achieved by independently modulating the light response of each pixel in the sensor by varying the programming gate voltage. This study will promote the developing progress of future artificial biomimetic retina systems.

A 2D Optoelectronic Logic Device with Ultralow Supply Voltage

Optoelectronic logic devices (OELDs) provide a cure for many visually impaired individuals. However, traditional OELDs have limitations, such as excessive channel resistance and complex structure, leading to high supply voltage and decreased efficiency of signal transmission. We report ultralow-voltage OELDs by seriating two 2D MoTe2 transistors with sub-10 nm channel lengths. The short channel length and atomically flat interface result in a low-resistance light-sensing unit that can operate with a low supply voltage and function well in weak-light conditions. The devices achieve an on state without light signal input and an off state with light signal input at an ultralow supply voltage of 50 mV, lower than the retinal bearing voltage of 70 mV. Additionally, MoTe2's excellent optoelectronic properties allow the device to perceive light from visible to near-infrared wavelengths with high sensitivity to weak light signals. The specific perception of visible light intensity is 0.03 mW·mm-2, and the near-infrared light intensity is 0.1 mW mm-2. The device also has a response time of 8 ms, meeting human needs. Our findings provide a promising solution for developing low-voltage artificial retinas.

Negative Photoconductivity Transistors for Visuomorphic Computing

Visuomorphic computing aims to simulate and potentially surpass the human retina by mimicking biological visual perception with an artificial retina. Despite significant progress, challenges persist in perceiving complex interactive environments. Negative photoconductivity transistors (NPTs) mimic synaptic behavior by achieving adjustable positive photoconductivity (PPC) and negative photoconductivity (NPC), simulating "excitation" and "inhibition" akin to sensory cell signals. In complex interactive environments, NPTs are desired for visuomorphic computing that can achieve a better sense of information, lower power consumption, and reduce hardware complexity. In this review, it is started by introducing the development process of NPTs, while placing a strong emphasis on the device structures, working mechanisms, and key performance parameters. The common material systems employed in NPTs based on their functions are then summarized. Moreover, it is proceeded to summarize the noteworthy applications of NPTs in optoelectronic devices, including advanced multibit nonvolatile memory, optoelectronic logic gates, optical encryption, and visual perception. Finally, the challenges and prospects that lie ahead in the ongoing development of NPTs are addressed, offering valuable insights into their applications in optoelectronics and a comprehensive understanding of their significance.

Nonvolatile and reconfigurable two-terminal electro-optic duplex memristor based on III-nitride semiconductors

With the fast development of artificial intelligence (AI), Internet of things (IOT), etc, there is an urgent need for the technology that can efficiently recognize, store and process a staggering amount of information. The AlScN material has unique advantages including immense remnant polarization, superior temperature stability and good lattice-match to other III-nitrides, making it easy to integrate with the existing advanced III-nitrides material and device technologies. However, due to the large band-gap, strong coercive field, and low photo-generated carrier generation and separation efficiency, it is difficult for AlScN itself to accumulate enough photo-generated carriers at the surface/interface to induce polarization inversion, limiting its application in in-memory sensing and computing. In this work, an electro-optic duplex memristor on a GaN/AlScN hetero-structure based Schottky diode has been realized. This two-terminal memristor shows good electrical and opto-electrical nonvolatility and reconfigurability. For both electrical and opto-electrical modes, the current on/off ratio can reach the magnitude of 104, and the resistance states can be effectively reset, written and long-termly stored. Based on this device, the "IMP" truth table and the logic "False" can be successfully reproduced, indicating the huge potential of the device in the field of in-memory sensing and computing.

Reconfigurable, non-volatile neuromorphic photovoltaics

The neural network image sensor-which mimics neurobiological functions of the human retina-has recently been demonstrated to simultaneously sense and process optical images. However, highly tunable responsivity concurrent with non-volatile storage of image data in the neural network would allow a transformative leap in compactness and function of these artificial neural networks. Here, we demonstrate a reconfigurable and non-volatile neuromorphic device based on two-dimensional semiconducting metal sulfides that is concurrently a photovoltaic detector. The device is based on a metal-semiconductor-metal (MSM) two-terminal structure with pulse-tunable sulfur vacancies at the M-S junctions. By modulating sulfur vacancy concentrations, the polarities of short-circuit photocurrent can be changed with multiple stable magnitudes. The bias-induced motion of sulfur vacancies leads to highly reconfigurable responsivities by dynamically modulating the Schottky barriers. A convolutional neuromorphic network is finally designed for image processing and object detection using the same device. The results demonstrated that neuromorphic photodetectors can be the key components of visual perception hardware.

In-Sensor Computing: Materials, Devices, and Integration Technologies

The number of sensor nodes in the Internet of Things is growing rapidly, leading to a large volume of data generated at sensory terminals. Frequent data transfer between the sensors and computing units causes severe limitations on the system performance in terms of energy efficiency, speed, and security. To efficiently process a substantial amount of sensory data, a novel computation paradigm that can integrate computing functions into sensor networks should be developed. The in-sensor computing paradigm reduces data transfer and also decreases the high computing complexity by processing data locally. Here, the hardware implementation of the in-sensor computing paradigm at the device and array levels is discussed. The physical mechanisms that lead to unique sensory response characteristics and their corresponding computing functions are illustrated. In particular, bioinspired device characteristics enable the implementation of the functionalities of neuromorphic computation. The integration technology is also discussed and the perspective on the future development of in-sensor computing is provided.

Recent progress in three-terminal artificial synapses based on 2D materials: from mechanisms to applications

Synapses are essential for the transmission of neural signals. Synaptic plasticity allows for changes in synaptic strength, enabling the brain to learn from experience. With the rapid development of neuromorphic electronics, tremendous efforts have been devoted to designing and fabricating electronic devices that can mimic synapse operating modes. This growing interest in the field will provide unprecedented opportunities for new hardware architectures for artificial intelligence. In this review, we focus on research of three-terminal artificial synapses based on two-dimensional (2D) materials regulated by electrical, optical and mechanical stimulation. In addition, we systematically summarize artificial synapse applications in various sensory systems, including bioplastic bionics, logical transformation, associative learning, image recognition, and multimodal pattern recognition. Finally, the current challenges and future perspectives involving integration, power consumption and functionality are outlined.

Emerging memristive neurons for neuromorphic computing and sensing

Inspired by the principles of the biological nervous system, neuromorphic engineering has brought a promising alternative approach to intelligence computing with high energy efficiency and low consumption. As pivotal components of neuromorphic system, artificial spiking neurons are powerful information processing units and can achieve highly complex nonlinear computations. By leveraging the switching dynamic characteristics of memristive device, memristive neurons show rich spiking behaviors with simple circuit. This report reviews the memristive neurons and their applications in neuromorphic sensing and computing systems. The switching mechanisms that endow memristive devices with rich dynamics and nonlinearity are highlighted, and subsequently various nonlinear spiking neuron behaviors emulated in these memristive devices are reviewed. Then, recent development is introduced on neuromorphic system with memristive neurons for sensing and computing. Finally, we discuss challenges and outlooks of the memristive neurons toward high-performance neuromorphic hardware systems and provide an insightful perspective for the development of interactive neuromorphic electronic systems.

Bioinspired and Low-Power 2D Machine Vision with Adaptive Machine Learning and Forgetting

Natural intelligence has many dimensions, with some of its most important manifestations being tied to learning about the environment and making behavioral changes. In primates, vision plays a critical role in learning. The underlying biological neural networks contain specialized neurons and synapses which not only sense and process visual stimuli but also learn and adapt with remarkable energy efficiency. Forgetting also plays an active role in learning. Mimicking the adaptive neurobiological mechanisms for seeing, learning, and forgetting can, therefore, accelerate the development of artificial intelligence (AI) and bridge the massive energy gap that exists between AI and biological intelligence. Here, we demonstrate a bioinspired machine vision system based on a 2D phototransistor array fabricated from large-area monolayer molybdenum disulfide (MoS₂) and integrated with an analog, nonvolatile, and programmable memory gate-stack; this architecture not only enables dynamic learning and relearning from visual stimuli but also offers learning adaptability under noisy illumination conditions at miniscule energy expenditure. In short, our demonstrated "all-in-one" hardware vision platform combines "sensing", "computing", and "storage" to not only overcome the von Neumann bottleneck of conventional complementary metal-oxide-semiconductor (CMOS) technology but also to eliminate the need for peripheral circuits and sensors.

A Bioinspired Retinomorphic Device for Spontaneous Chromatic Adaptation

Chromatic adaptation refers to the sensing and preprocessing of the spectral composition of incident light on the retina, and it is important for color-image recognition. It is challenging to apply sensing, memory, and processing functions to color images via the same physical process using the complementary metal-oxide-semiconductor technology because of redundant data detection, complicated signal conversion processes, and the requirement for additional memory modules. Inspired by the highly efficient chromatic adaptation of the human retina, a 2D oxygen-mediated platinum diselenide (PtSe₂ ) device is presented to simultaneously apply sensing, memory, and processing functions to color images. The device exhibits a wavelength-dependent bipolar photoresponse and the linear pulse-number dependence of photoconductivity, which is dominated by the photon-mediated physical adsorption and desorption of oxygen molecules on bilayer PtSe₂ . The proposed retinomorphic device shows superior image classification accuracy (over 90%) compared to an independent pseudocolor channel (less than 75%). Hence, it is promising for developing artificial vision perception systems with reduced architectural complexity.

Inorganic-Organic Hybrid Phototransistor Array with Enhanced Photogating Effect for Dynamic Near-Infrared Light Sensing and Image Preprocessing

Narrow-band-gap organic semiconductors have emerged as appealing near-infrared (NIR) sensing materials by virtue of their unique optoelectronic properties. However, their limited carrier mobility impedes the implementation of large-area, dynamic NIR sensor arrays. In this work, high-performance inorganic-organic hybrid phototransistor arrays are achieved for NIR sensing, by taking advantage of the high electron mobility of In₂O₃ and the strong NIR absorption of a BTPV-4F:PTB7-Th bulk heterojunction (BHJ) with an enhanced photogating effect. As a result, the hybrid phototransistors reach a high responsivity of 1393.0 A W^-1, a high specific detectivity of 4.8 × 10¹² jones, and a fast response of 0.72 ms to NIR light (900 nm). Meanwhile, an integrated 16 × 16 phototransistor array with a one-transistor-one-phototransistor (1T1PT) architecture is achieved. On the basis of the enhanced photogating effect, the phototransistor array can not only achieve real-time, dynamic NIR light mapping but also implement image preprocessing, which is promising for advanced NIR image sensors.

Ultrafast machine vision with artificial neural network devices based on a GaN-based micro-LED array

GaN-based micro-LED is an emerging display and communication device, which can work as well as a photodetector, enabling possible applications in machine vision. In this work, we measured the characteristics of micro-LED based photodetector experimentally and proposed a feasible simulation of a novel artificial neural network (ANN) device for the first time based on a micro-LED based photodetector array, providing ultrafast imaging (∼133 million bins per second) and a high image recognition rate. The array itself constitutes a neural network, in which the synaptic weights are tunable by the bias voltage. It has the potentials to be integrated with novel machine vision and reconfigurable computing applications, acting as a role of acceleration and similar functionality expansion. Also, the multi-functionality of micro-LED broadens its application potentials of combining ANN with display and communication.

Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception

Compared to human vision, conventional machine vision composed of an image sensor and processor suffers from high latency and large power consumption due to physically separated image sensing and processing. A neuromorphic vision system with brain-inspired visual perception provides a promising solution to the problem. Here we propose and demonstrate a prototype neuromorphic vision system by networking a retinomorphic sensor with a memristive crossbar. We fabricate the retinomorphic sensor by using WSe2/h-BN/Al2O3 van der Waals heterostructures with gate-tunable photoresponses, to closely mimic the human retinal capabilities in simultaneously sensing and processing images. We then network the sensor with a large-scale Pt/Ta/HfO2/Ta one-transistor-one-resistor (1T1R) memristive crossbar, which plays a similar role to the visual cortex in the human brain. The realized neuromorphic vision system allows for fast letter recognition and object tracking, indicating the capabilities of image sensing, processing and recognition in the full analog regime. Our work suggests that such a neuromorphic vision system may open up unprecedented opportunities in future visual perception applications.

Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor

Early processing of visual information takes place in the human retina. Mimicking neurobiological structures and functionalities of the retina provides a promising pathway to achieving vision sensor with highly efficient image processing. Here, we demonstrate a prototype vision sensor that operates via the gate-tunable positive and negative photoresponses of the van der Waals (vdW) vertical heterostructures. The sensor emulates not only the neurobiological functionalities of bipolar cells and photoreceptors but also the unique connectivity between bipolar cells and photoreceptors. By tuning gate voltage for each pixel, we achieve reconfigurable vision sensor for simultaneous image sensing and processing. Furthermore, our prototype vision sensor itself can be trained to classify the input images by updating the gate voltages applied individually to each pixel in the sensor. Our work indicates that vdW vertical heterostructures offer a promising platform for the development of neural network vision sensor.

Ultrafast machine vision with 2D material neural network image sensors

Machine vision technology has taken huge leaps in recent years, and is now becoming an integral part of various intelligent systems, including autonomous vehicles and robotics. Usually, visual information is captured by a frame-based camera, converted into a digital format and processed afterwards using a machine-learning algorithm such as an artificial neural network (ANN)¹. The large amount of (mostly redundant) data passed through the entire signal chain, however, results in low frame rates and high power consumption. Various visual data preprocessing techniques have thus been developed^2-7 to increase the efficiency of the subsequent signal processing in an ANN. Here we demonstrate that an image sensor can itself constitute an ANN that can simultaneously sense and process optical images without latency. Our device is based on a reconfigurable two-dimensional (2D) semiconductor^8,9 photodiode^10-12 array, and the synaptic weights of the network are stored in a continuously tunable photoresponsivity matrix. We demonstrate both supervised and unsupervised learning and train the sensor to classify and encode images that are optically projected onto the chip with a throughput of 20 million bins per second.

Optical image transformations for fully parallel optical analog-to-digital conversion

An optical method for a fully parallel analog-to-digital conversion is proposed. The proposed method is carried out by means of intensity transformations of an analog input image and the thresholding for the transformed images and is suitable for two-dimensional implementation based on spatial light modulators. The intensity transformations are implemented by a liquid-crystal spatial light modulator, and thresholding is simulated by computer in consideration of the optical realization.