Retina-like Imaging and Its Applications: A Brief Review

Bio-inspired foveal super-resolution method for multi-focal-length images based on local gradient constraints

Most existing super-resolution (SR) imaging systems, inspired by the bionic compound eye, utilize image registration and reconstruction algorithms to overcome the angular resolution limitations of individual imaging systems. This article introduces a multi-aperture multi-focal-length imaging system and a multi-focal-length image super-resolution algorithm, mimicking the foveal imaging of the human eye. Experimental results demonstrate that with the proposed imaging system and an SR imaging algorithm inspired by the human visual system, the proposed method can enhance the spatial resolution of the foveal region by up to 4 × compared to the original acquired image. These findings validate the effectiveness of the proposed imaging system and computational imaging algorithm in enhancing image texture and spatial resolution.

Adaptive locating foveated ghost imaging based on affine transformation

Ghost imaging (GI) has been widely used in the applications including spectral imaging, 3D imaging, and other fields due to its advantages of broad spectrum and anti-interference. Nevertheless, the restricted sampling efficiency of ghost imaging has impeded its extensive application. In this work, we propose a novel foveated pattern affine transformer method based on deep learning for efficient GI. This method enables adaptive selection of the region of interest (ROI) by combining the proposed retina affine transformer (RAT) network with minimal computational and parametric quantities with the foveated speckle pattern. For single-target and multi-target scenarios, we propose RAT and RNN-RAT (recurrent neural network), respectively. The RAT network enables an adaptive alteration of the fovea of the variable foveated patterns spot to different sizes and positions of the target by predicting the affine matrix with a minor number of parameters for efficient GI. In addition, we integrate a recurrent neural network into the proposed RAT to form an RNN-RAT model, which is capable of performing multi-target ROI detection. Simulations and experimental results show that the method can achieve ROI localization and pattern generation in 0.358 ms, which is a 1 × 105 efficiency improvement compared with the previous methods and improving the image quality of ROI by more than 4 dB. This approach not only improves its overall applicability but also enhances the reconstruction quality of ROI. This creates additional opportunities for real-time GI.

Stakes of neuromorphic foveation: a promising future for embedded event cameras

Foveation can be defined as the organic action of directing the gaze towards a visual region of interest to acquire relevant information selectively. With the recent advent of event cameras, we believe that taking advantage of this visual neuroscience mechanism would greatly improve the efficiency of event data processing. Indeed, applying foveation to event data would allow to comprehend the visual scene while significantly reducing the amount of raw data to handle. In this respect, we demonstrate the stakes of neuromorphic foveation theoretically and empirically across several computer vision tasks, namely semantic segmentation and classification. We show that foveated event data have a significantly better trade-off between quantity and quality of the information conveyed than high- or low-resolution event data. Furthermore, this compromise extends even over fragmented datasets. Our code is publicly available online at: https://github.com/amygruel/FoveationStakes_DVS .

Foveated panoramic ghost imaging

Panoramic ghost imaging (PGI) is a novel method by only using a curved mirror to enlarge the field of view (FOV) of ghost imaging (GI) to 360°, making GI a breakthrough in the applications with a wide FOV. However, high-resolution PGI with high efficiency is a serious challenge because of the large amount of data. Therefore, inspired by the variant-resolution retina structure of human eye, a foveated panoramic ghost imaging (FPGI) is proposed to achieve the coexistence of a wide FOV, high resolution and high efficiency on GI by reducing the resolution redundancy, and further to promote the practical applications of GI with a wide FOV. In FPGI system, a flexible variant-resolution annular pattern structure via log-rectilinear transformation and log-polar mapping is proposed to be used for projection, which can allocate the resolution of the region of interest (ROI) and the other region of non-interest (NROI) by setting related parameters in the radial and poloidal directions independently to meet different imaging requirements. In addition, in order to reasonably reduce the resolution redundancy and avoid the loss of the necessary resolution on NROI, the variant-resolution annular pattern structure with a real fovea is further optimized to keep the ROI at any position in the center of 360° FOV by flexibly changing the initial position of the start-stop boundary on the annular pattern structure. The experimental results of the FPGI with one fovea and multiple foveae demonstrate that, compared to the traditional PGI, the proposed FPGI not only can improve the imaging quality on the ROIs with a high resolution and flexibly remain a lower-resolution imaging on the NROI with different required resolution reduction; but also reduce the reconstruction time to improve the imaging efficiency due to the reduction of the resolution redundancy.

Retina-like Computational Ghost Imaging for an Axially Moving Target

Unlike traditional optical imaging schemes, computational ghost imaging (CGI) provides a way to reconstruct images with the spatial distribution information of illumination patterns and the light intensity collected by a single-pixel detector or bucket detector. Compared with stationary scenes, the relative motion between the target and the imaging system in a dynamic scene causes the degradation of reconstructed images. Therefore, we propose a time-variant retina-like computational ghost imaging method for axially moving targets. The illuminated patterns are specially designed with retina-like structures, and the radii of foveal region can be modified according to the axial movement of target. By using the time-variant retina-like patterns and compressive sensing algorithms, high-quality imaging results are obtained. Experimental verification has shown its effectiveness in improving the reconstruction quality of axially moving targets. The proposed method retains the inherent merits of CGI and provides a useful reference for high-quality GI reconstruction of a moving target.

Flexible foveated imaging using a single Risley-prism imaging system

Foveated imaging, which has the ability to provide overall situational awareness over a large field of view and high-resolution perception of local details, has significant advantages in many specific applications. However, existing artificially foveated imaging systems are complex, bulky, and expensive, and the flexibility of the fovea specifically has many limitations. To overcome these deficiencies, this paper proposes a method for foveated imaging by collecting multiple partially overlapping sub-fields of view. To capture the above special sub-fields of view, we propose a high-efficiency algorithm based on the characteristics of the field of view deflected by the Risley-prism and aimed at solving the prism rotation angles. In addition, we prove the reliability of the proposed algorithm by cross-validation with the particle swarm optimization algorithm. The experimental results show that the proposed method can achieve flexible foveated imaging using a single Risley-prism imaging system.