Hand gestures recognition in videos taken with a lensless camera

Dual-branch fusion model for lensless imaging

A lensless camera is an imaging system that replaces the lens with a mask to reduce thickness, weight, and cost compared to a lensed camera. The improvement of image reconstruction is an important topic in lensless imaging. Model-based approach and pure data-driven deep neural network (DNN) are regarded as two mainstream reconstruction schemes. In this paper, the advantages and disadvantages of these two methods are investigated to propose a parallel dual-branch fusion model. The model-based method and the data-driven method serve as two independent input branches, and the fusion model is used to extract features from the two branches and merge them for better reconstruction. Two types of fusion model named Merger-Fusion-Model and Separate-Fusion-Model are designed for different scenarios, where Separate-Fusion-Model is able to adaptively allocate the weights of the two branches by the attention module. Additionally, we introduce a novel network architecture named UNet-FC into the data-driven branch, which enhances reconstruction by making full use of the multiplexing property of lensless optics. The superiority of the dual-branch fusion model is verified by drawing comparison with other state-of-the-art methods on public dataset (+2.95dB peak signal-to-noise (PSNR), +0.036 structural similarity index (SSIM), -0.0172 Learned Perceptual Image Patch Similarity (LPIPS)). Finally, a lensless camera prototype is constructed to further validate the effectiveness of our method in a real lensless imaging system.

Autofocusing of Fresnel zone aperture lensless imaging for QR code recognition

Fresnel zone aperture (FZA) lensless imaging encodes the incident light into a hologram-like pattern, so that the scene image can be numerically focused at a long imaging range by the back propagation method. However, the target distance is uncertain. The inaccurate distance causes blurs and artifacts in the reconstructed images. This brings difficulties for the target recognition applications, such as quick response code scanning. We propose an autofocusing method for FZA lensless imaging. By incorporating the image sharpness metrics into the back propagation reconstruction process, the method can acquire the desired focusing distance and reconstruct noise-free high-contrast images. By combining the Tamura of the gradient metrics and nuclear norm of gradient, the relative error of estimated object distance is only 0.95% in the experiment. The proposed reconstruction method significantly improves the mean recognition rate of QR code from 4.06% to 90.00%. It paves the way for designing intelligent integrated sensors.