Vertically integrated spiking cone photoreceptor arrays for color perception

Filter-Less Color-Selective Photodetector Derived from Integration of Parallel Perovskite Photoelectric Response Units

Color-selective photodetectors (PDs) play an indispensable role in spectral recognition, image sensing, and other fields. Nevertheless, complex filters and delicate optical paths in such devices significantly increase their complexity and size, which subsequently impede their integration in smart optoelectronic chips for universal applications. This work demonstrates the successful fabrication of filter-less color-selective perovskite PDs by integrating three perovskite units with different photoresponse on a single chip. The variation in photoresponse is attributed to different quantities of SnO2 nanoparticles, synthesized through controlled ultrasonic treatment on the surface of the electron transportation layer SnS2, which selectively absorb short-wavelength light, thus increasing the relative transmittance of long-wavelength light and enhancing the photoresponse of the units to long wavelengths. By integrating any two units and deriving the formula for the wavelength to the responsivity ratio, a wavelength sensor is developed which can accurately identify incident light in the range of 400-700 nm with a minimum error <3 nm. Furthermore, the device integrating three units with different photoresponse can identify red, green and blue in polychromatic light to achieve color imaging with a relative error <6%. This work provides valuable insights into wavelength identification and color imaging of perovskite PDs.

Modulation of polyaniline memristive device switching voltage by nucleotide-free analogue of vitamin B12

Memristive devices offer essential properties to become a part of the next-generation computing systems based on neuromorphic principles. Organic memristive devices exhibit a unique set of properties which makes them an indispensable choice for specific applications, such as interfacing with biological systems. While the switching rate of organic devices can be easily adjusted over a wide range through various methods, controlling the switching potential is often more challenging, as this parameter is intricately tied to the materials used. Given the limited options in the selection conductive polymers and the complexity of polymer chemical engineering, the most straightforward and accessible approach to modulate switching potentials is by introducing specific molecules into the electrolyte solution. In our study, we show polyaniline (PANI)-based device switching potential control by adding nucleotide-free analogue of vitamin B12, aquacyanocobinamide, to the electrolyte solution. The employed concentrations of this molecule, ranging from 0.2 to 2 mM, enabled organic memristive devices to achieve switching potential decrease for up to 100 mV, thus providing a way to control device properties. This effect is attributed to strong aromatic interactions between PANI phenyl groups and corrin macrocycle of the aquacyanocobinamide molecule, which was supported by ultraviolet-visible spectra analysis.

Bio-Inspired Sensory Receptors for Artificial-Intelligence Perception

In the era of artificial intelligence (AI), there is a growing interest in replicating human sensory perception. Selective and sensitive bio-inspired sensory receptors with synaptic plasticity have recently gained significant attention in developing energy-efficient AI perception. Various bio-inspired sensory receptors and their applications in AI perception are reviewed here. The critical challenges for the future development of bio-inspired sensory receptors are outlined, emphasizing the need for innovative solutions to overcome hurdles in sensor design, integration, and scalability. AI perception can revolutionize various fields, including human-machine interaction, autonomous systems, medical diagnostics, environmental monitoring, industrial optimization, and assistive technologies. As advancements in bio-inspired sensing continue to accelerate, the promise of creating more intelligent and adaptive AI systems becomes increasingly attainable, marking a significant step forward in the evolution of human-like sensory perception.

An artificial visual neuron with multiplexed rate and time-to-first-spike coding

Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.

A Spiking Artificial Vision Architecture Based on Fully Emulating the Human Vision

Intelligent vision necessitates the deployment of detectors that are always-on and low-power, mirroring the continuous and uninterrupted responsiveness characteristic of human vision. Nonetheless, contemporary artificial vision systems attain this goal by the continuous processing of massive image frames and executing intricate algorithms, thereby expending substantial computational power and energy. In contrast, biological data processing, based on event-triggered spiking, has higher efficiency and lower energy consumption. Here, this work proposes an artificial vision architecture consisting of spiking photodetectors and artificial synapses, closely mirroring the intricacies of the human visual system. Distinct from previously reported techniques, the photodetector is self-powered and event-triggered, outputting light-modulated spiking signals directly, thereby fulfilling the imperative for always-on with low-power consumption. With the spiking signals processing through the integrated synapse units, recognition of graphics, gestures, and human action has been implemented, illustrating the potent image processing capabilities inherent within this architecture. The results prove the 90% accuracy rate in human action recognition within a mere five epochs utilizing a rudimentary artificial neural network. This novel architecture, grounded in spiking photodetectors, offers a viable alternative to the extant models of always-on low-power artificial vision system.

Li Promoting Long Afterglow Organic Light-Emitting Transistor for Memory Optocoupler Module

The artificial brain is conceived as advanced intelligence technology, capable to emulate in-memory processes occurring in the human brain by integrating synaptic devices. Within this context, improving the functionality of synaptic transistors to increase information processing density in neuromorphic chips is a major challenge in this field. In this article, Li-ion migration promoting long afterglow organic light-emitting transistors, which display exceptional postsynaptic brightness of 7000 cd m-2 under low operational voltages of 10 V is presented. The postsynaptic current of 0.1 mA operating as a built-in threshold switch is implemented as a firing point in these devices. The setting-condition-triggered long afterglow is employed to drive the photoisomerization process of photochromic molecules that mimic neurotransmitter transfer in the human brain for realizing a key memory rule, that is, the transition from long-term memory to permanent memory. The combination of setting-condition-triggered long afterglow with photodiode amplifiers is also processed to emulate the human responding action after the setting-training process. Overall, the successful integration in neuromorphic computing comprising stimulus judgment, photon emission, transition, and encoding,  to emulate the complicated decision tree of the human brain is demonstrated.

Triboelectric micro-flexure-sensitive fiber electronics

Developing fiber electronics presents a practical approach for establishing multi-node distributed networks within the human body, particularly concerning triboelectric fibers. However, realizing fiber electronics for monitoring micro-physiological activities remains challenging due to the intrinsic variability and subtle amplitude of physiological signals, which differ among individuals and scenarios. Here, we propose a technical approach based on a dynamic stability model of sheath-core fibers, integrating a micro-flexure-sensitive fiber enabled by nanofiber buckling and an ion conduction mechanism. This scheme enhances the accuracy of the signal transmission process, resulting in improved sensitivity (detectable signal at ultra-low curvature of 0.1 mm-1; flexure factor >21.8% within a bending range of 10°.) and robustness of fiber under micro flexure. In addition, we also developed a scalable manufacturing process and ensured compatibility with modern weaving techniques. By combining precise micro-curvature detection, micro-flexure-sensitive fibers unlock their full potential for various subtle physiological diagnoses, particularly in monitoring fiber upper limb muscle strength for rehabilitation and training.

Toward a Brain-Neuromorphics Interface

Brain-computer interfaces (BCIs) that enable human-machine interaction have immense potential in restoring or augmenting human capabilities. Traditional BCIs are realized based on complementary metal-oxide-semiconductor (CMOS) technologies with complex, bulky, and low biocompatible circuits, and suffer with the low energy efficiency of the von Neumann architecture. The brain-neuromorphics interface (BNI) would offer a promising solution to advance the BCI technologies and shape the interactions with machineries. Neuromorphic devices and systems are able to provide substantial computation power with extremely high energy-efficiency by implementing in-materia computing such as in situ vector-matrix multiplication (VMM) and physical reservoir computing. Recent progresses on integrating neuromorphic components with sensing and/or actuating modules, give birth to the neuromorphic afferent nerve, efferent nerve, sensorimotor loop, and so on, which has advanced the technologies for future neurorobotics by achieving sophisticated sensorimotor capabilities as the biological system. With the development on the compact artificial spiking neuron and bioelectronic interfaces, the seamless communication between a BNI and a bioentity is reasonably expectable. In this review, the upcoming BNIs are profiled by introducing the brief history of neuromorphics, reviewing the recent progresses on related areas, and discussing the future advances and challenges that lie ahead.

Adapted MLP-Mixer network based on crossbar arrays of fast and multilevel switching (Co-Fe-B)x(LiNbO3)100-x nanocomposite memristors

MLP-Mixer based on multilayer perceptrons (MLPs) is a novel architecture of a neuromorphic computing system (NCS) introduced for image classification tasks without convolutional layers. Its software realization demonstrates high classification accuracy, although the number of trainable weights is relatively low. One more promising way of improving the NCS performance, especially in terms of power consumption, is its hardware realization using memristors. Therefore, in this work, we proposed an NCS with an adapted MLP-Mixer architecture and memristive weights. For this purpose, we used a passive crossbar array of (Co-Fe-B)x(LiNbO3)100-x memristors. Firstly, we studied the characteristics of such memristors, including their minimal resistive switching time, which was extrapolated to be in the picosecond range. Secondly, we created a fully hardware NCS with memristive weights that are capable of classification of simple 4-bit vectors. The system was shown to be robust to noise introduction in the input patterns. Finally, we used experimental memristive characteristics to simulate an adapted MLP-Mixer architecture that demonstrated a classification accuracy of (94.7 ± 0.3)% on the Modified National Institute of Standards and Technology (MNIST) dataset. The obtained results are the first steps toward the realization of memristive NCS with a promising MLP-Mixer architecture.

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

High-performance robotic vision empowers mobile and humanoid robots to detect and identify their surrounding objects efficiently, which enables them to cooperate with humans and assist human activities. For error-free execution of these robots' tasks, efficient imaging and data processing capabilities are essential, even under diverse and complex environments. However, conventional technologies fall short of meeting the high-standard requirements of robotic vision under such circumstances. Here, we discuss recent progress in artificial vision systems with high-performance imaging and data processing capabilities enabled by distinctive electrical, optical, and mechanical characteristics of nanomaterials surpassing the limitations of traditional silicon technologies. In particular, we focus on nanomaterial-based electronic eyes and in-sensor processing devices inspired by biological eyes and animal visual recognition systems, respectively. We provide perspectives on key nanomaterials, device components, and their functionalities, as well as explain the remaining challenges and future prospects of the artificial vision systems.

Artificial Tactile Perception System Based on Spiking Tactile Neurons and Spiking Neural Networks

The artificial tactile perception system of this work utilizes a fully connected spiking neural network (SNN) comprising two layers. Its architecture is streamlined and energy-efficient as it directly integrates spiking tactile neurons with piezoresistive sensors and Pt/NbOx/TiN memristors as input neurons. These spiking tactile neurons possess the ability to perceive and integrate pressure stimuli from multiple sensors and encode the information into rate-coded electrical spikes, closely resembling the behavior of a biological tactile neuron. The system's real-time information processing capability is demonstrated through an artificial perceptual learning system that successfully encodes and decodes the Morse code; the artificial perceptual learning system accurately recognizes and displays 26 English letters. Furthermore, the artificial tactile perception system is evaluated for the recognition of the MNIST data set, achieving a classification accuracy of 85.7% with the supervised spiking-rate-dependent plasticity learning rule. The key advantages of this artificial tactile perception system are its simple structure and high efficiency, which contributes to its practicality for various real-world applications.

Deep Learning-Assisted Intelligent Artificial Vision Platform Based on Dual-Luminescence Eu(III)-Functionalized HOF for the Diagnosis of Breast and Ovarian Cancer

Developing an advanced analytical method to detect spermine (Spm) and N-acetylneuraminic acid (NANA), the biomarkers of breast and ovarian cancers, respectively, is critical for the early diagnosis of the two cancers, which is very meaningful for women's health. Here, a deep learning-assisted artificial vision platform based on a dual-emission ratiometric fluorescence sensor is first constructed to monitor Spm and NANA. The ratiometric fluorescence sensor (Eu@TCBP-HOF, 1) can selectively detect Spm with high sensitivity based on "Turn-on" mode. After adding Spm, the new ratiometric fluorescence sensor (1-Spm, named 2) shows high sensitivity for NANA with "Turn-off" mode. Moreover, the fluorescence sensors can achieve an obvious fluorescence color response to Spm and NANA. Even in real saliva and serum samples, 1 and 2 still show high sensitivity and color responsiveness with limit of detection (LODs) of 0.5 μM for Spm and 0.96 μM for NANA. In virtue of different fluorescence responses, the DenseNet algorithm of deep learning assists the fluorescence sensors, which can simulate the human visual systems to identify fluorescence images and distinguish the concentration of Spm and NANA within 1 s with over 99% recognition accuracy. The intelligent artificial vision platform developed in this work may provide a prospective analytical method for the early diagnosis of female malignant tumors.