1
|
Wu Y, Deng W, Li K, Wang X, Liu B, Li J, Chen Z, Zhang Y. A Spiking Artificial Vision Architecture Based on Fully Emulating the Human Vision. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2312094. [PMID: 38320173 DOI: 10.1002/adma.202312094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 01/29/2024] [Indexed: 02/08/2024]
Abstract
Intelligent vision necessitates the deployment of detectors that are always-on and low-power, mirroring the continuous and uninterrupted responsiveness characteristic of human vision. Nonetheless, contemporary artificial vision systems attain this goal by the continuous processing of massive image frames and executing intricate algorithms, thereby expending substantial computational power and energy. In contrast, biological data processing, based on event-triggered spiking, has higher efficiency and lower energy consumption. Here, this work proposes an artificial vision architecture consisting of spiking photodetectors and artificial synapses, closely mirroring the intricacies of the human visual system. Distinct from previously reported techniques, the photodetector is self-powered and event-triggered, outputting light-modulated spiking signals directly, thereby fulfilling the imperative for always-on with low-power consumption. With the spiking signals processing through the integrated synapse units, recognition of graphics, gestures, and human action has been implemented, illustrating the potent image processing capabilities inherent within this architecture. The results prove the 90% accuracy rate in human action recognition within a mere five epochs utilizing a rudimentary artificial neural network. This novel architecture, grounded in spiking photodetectors, offers a viable alternative to the extant models of always-on low-power artificial vision system.
Collapse
Affiliation(s)
- Yi Wu
- Faculty of Materials and Manufacturing, Beijing University of Technology, Beijing, 100124, China
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Wenjie Deng
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Kexin Li
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Xiaoting Wang
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Bo Liu
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Jingzhen Li
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Zhijie Chen
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| | - Yongzhe Zhang
- Faculty of Materials and Manufacturing, Beijing University of Technology, Beijing, 100124, China
- Key Laboratory of Optoelectronics Technology of Education Ministry of China, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
| |
Collapse
|
2
|
Wang X, Wang Q, Zhang L, Qu Y, Yi F, Yu J, Liu Q, Xia R, Xu Z, Tong S. DCENet-based low-light image enhancement improved by spiking encoding and convLSTM. Front Neurosci 2024; 18:1297671. [PMID: 38505773 PMCID: PMC10948416 DOI: 10.3389/fnins.2024.1297671] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Accepted: 02/02/2024] [Indexed: 03/21/2024] Open
Abstract
The direct utilization of low-light images hinders downstream visual tasks. Traditional low-light image enhancement (LLIE) methods, such as Retinex-based networks, require image pairs. A spiking-coding methodology called intensity-to-latency has been used to gradually acquire the structural characteristics of an image. convLSTM has been used to connect the features. This study introduces a simplified DCENet to achieve unsupervised LLIE as well as the spiking coding mode of a spiking neural network. It also applies the comprehensive coding features of convLSTM to improve the subjective and objective effects of LLIE. In the ablation experiment for the proposed structure, the convLSTM structure was replaced by a convolutional neural network, and the classical CBAM attention was introduced for comparison. Five objective evaluation metrics were compared with nine LLIE methods that currently exhibit strong comprehensive performance, with PSNR, SSIM, MSE, UQI, and VIFP exceeding the second place at 4.4% (0.8%), 3.9% (17.2%), 0% (15%), 0.1% (0.2%), and 4.3% (0.9%) on the LOL and SCIE datasets. Further experiments of the user study in five non-reference datasets were conducted to subjectively evaluate the effects depicted in the images. These experiments verified the remarkable performance of the proposed method.
Collapse
Affiliation(s)
| | - Qiang Wang
- Equipment Management and Unmanned Aerial Vehicle Engineering School, Air Force Engineering University, Xi’an, China
| | | | | | | | | | | | | | | | | |
Collapse
|