Photon-efficient 3D reconstruction employing a edge enhancement method

Enhancing LiDAR performance using threshold photon-number-resolving detection

Single-photon light detection and ranging (LiDAR) is widely used to reconstruct 3D scenes. Nevertheless, depth and reflectivity maps obtained by single-photon detection usually suffer from noise problems. Threshold LiDAR techniques using photon-number-resolving detectors were proposed to suppress noise by filtering low photon numbers, but these techniques renounce multiple levels of information and could not be compatible when it comes to high-noise low-signal regime. In this manuscript, we propose a detection scheme which combines the noise suppression of threshold detection with the signal amplification of photon-number-resolving detectors to further enhance LiDAR performance. The enhancement attained is compared to single-photon and threshold detection schemes under a wide range of signal and noise conditions, in terms of signal-to-noise-ratio (SNR), detection rate and false alarm rate, which are key metrics for LiDAR. Extensive simulations and real-world experiments show that the proposed scheme can reconstruct better depth and reflectivity maps. These results enable the development of high-efficient and low-noise LiDAR systems.

Reconfigurable coaxial single-photon LIDAR based on the SPAD array

The single-photon avalanche diode (SPAD) array with time-to-digital converter (TDC) circuits on each pixel is an excellent candidate detector for imaging LIDAR systems. However, the low fill-factor of the SPAD array does not allow for efficient use of laser energy when directly adopted in a LIDAR system. Here, we design a reconfigurable coaxial single-photon LIDAR based on the SPAD array and diffractive optical elements (DOEs). We use the DOE and beam expander to shape the laser beam into a laser dot matrix. The total divergence angle of the DOE spot beam is strictly matched to the total field of view (FOV) angle of the SPAD array. Meanwhile, each focused beamlet is individually matched to every active area of the SPAD array detector, which increases the use of output energy about 100 times compared to the diffusion illumination system. Besides, the system uses the active area as the minimum pixel and can support sub-pixel scanning, resulting in higher resolution images. Through this coaxial structure, two different telescope systems after transceiver switching can be reconfigured for imaging targets at different distances. Based on our single-photon LIDAR system, we achieved 3D imaging of targets at 100 m and 180 m using two different telescope configurations.

Dynamic single-photon 3D imaging with a sparsity-based neural network

Deep learning is emerging as an important tool for single-photon light detection and ranging (LiDAR) with high photon efficiency and image reconstruction quality. Nevertheless, the existing deep learning methods still suffer from high memory footprint and low inference speed, which undermine their compatibility when it comes to dynamic and long-range imaging with resource-constrained devices. By exploiting the sparsity of the data, we proposed an efficient neural network architecture which significantly reduces the storage and computation overhead by skipping the inactive sites with no photon counts. In contrast with the state-of-the-art deep learning methods, our method supports one-shot processing of data frames with high spatial resolution, and achieves over 90% acceleration in computation speed without sacrificing the reconstruction quality. In addition, the speed of our method is not sensitive to the detection distance. The experiment results on public real-world dataset and our home-built system have demonstrated the outstanding dynamic imaging capability of the algorithm, which is orders of magnitude faster than the competing methods and does not require any data pruning for hardware compatibility.

High-resolution depth imaging with a small-scale SPAD array based on the temporal-spatial filter and intensity image guidance

Currently single-photon avalanche diode (SPAD) arrays suffer from a small-scale pixel count, which makes it difficult to achieve high-resolution 3D imaging directly through themselves. We established a CCD camera-assisted SPAD array depth imaging system. Based on illumination laser lattice generated by a diffractive optical element (DOE), the registration of the low-resolution depth image gathered by SPAD and the high-resolution intensity image gathered by CCD is realized. The intensity information is used to guide the reconstruction of a resolution-enhanced depth image through a proposed method consisting of total generalized variation (TGV) regularization and temporal-spatial (T-S) filtering algorithm. Experimental results show that an increasement of 4 × 4 times for native depth image resolution is achieved and the depth imaging quality is also improved by applying the proposed method.

Deep-learning based photon-efficient 3D and reflectivity imaging with a 64 × 64 single-photon avalanche detector array

A single photon avalanche diode (SPAD) is a high sensitivity detector that can work under weak echo signal conditions (≤1 photon per pixel). The measured digital signals can be used to invert the range and reflectivity images of the target with photon-efficient imaging reconstruction algorithm. However, the existing photon-efficient imaging reconstruction algorithms are susceptible to noise, which leads to poor quality of the reconstructed range and reflectivity images of target. In this paper, a non-local sparse attention encoder (NLSA-Encoder) neural network is proposed to extract the 3D information to reconstruct both the range and reflectivity images of target. The proposed network model can effectively reduce the influence of noise in feature extraction and maintain the capability of long-range correlation feature extraction. In addition, the network is optimized for reconstruction speed to achieve faster reconstruction without performance degradation, compared with other existing deep learning photon-efficient imaging reconstruction methods. The imaging performance is verified through numerical simulation, near-field indoor and far-field outdoor experiments with a 64 × 64 SPAD array. The experimental results show that the proposed network model can achieve better results in terms of the reconstruction quality of range and reflectivity images, as well as reconstruction speed.

Single-photon 3D imaging with a multi-stage network

Active single-photon 3D imaging technology has been applied to 3D imaging of complex scenes in many frontier fields such as biomedicine, remote sensing mapping, etc. However, single-photon 3D imaging with strong background noise is still a major challenge. Several classical algorithms and machine learning methods have been proposed to solve the problem. In this paper, we propose a novel multi-stage synergistic recovery network to reconstruct an accurate depth map. In the model, we first extract multi-scale feature information using encoder and decoder architectures, then combine them with an original resolution network that retains complete spatial location information. Through this way, we can compensate the deficiencies of the original resolution network for multi-scale local feature extraction. Moreover, a self-supervised attention module (SAM) is constructed to weight local features between different stages, optimizing the feature exchange between different stages of the multi-stage architecture network. Our method currently performs the best of all the tested methods.

Robust photon-efficient imaging using a pixel-wise residual shrinkage network

Single-photon light detection and ranging (LiDAR) has been widely applied to 3D imaging in challenging scenarios. However, limited signal photon counts and high noises in the collected data have posed great challenges for predicting the depth image precisely. In this paper, we propose a pixel-wise residual shrinkage network for photon-efficient imaging from high-noise data, which adaptively generates the optimal thresholds for each pixel and denoises the intermediate features by soft thresholding. Besides, redefining the optimization target as pixel-wise classification provides a sharp advantage in producing confident and accurate depth estimation when compared with existing research. Comprehensive experiments conducted on both simulated and real-world datasets demonstrate that the proposed model outperforms the state-of-the-arts and maintains robust imaging performance under different signal-to-noise ratios including the extreme case of 1:100.