Dot-coded structured light for accurate and robust 3D reconstruction

Metasurface- and PCSEL-Based Structured Light for Monocular Depth Perception and Facial Recognition

The novel depth-sensing system presented here revolutionizes structured light (SL) technology by employing metasurfaces and photonic crystal surface-emitting lasers (PCSELs) for efficient facial recognition in monocular depth-sensing. Unlike conventional dot projectors relying on diffractive optical elements (DOEs) and collimators, our system projects approximately 45,700 infrared dots from a compact 297-μm-dimention metasurface, drastically more spots (1.43 times) and smaller (233 times) than the DOE-based dot projector in an iPhone. With a measured field-of-view (FOV) of 158° and a 0.611° dot sampling angle, the system is lens-free and lightweight and boasts lower power consumption than vertical-cavity surface-emitting laser (VCSEL) arrays, resulting in a 5-10 times reduction in power. Utilizing a GaAs-based metasurface and a simplified optical architecture, this innovation not only addresses the drawbacks of traditional SL depth-sensing but also opens avenues for compact integration into wearable devices, offering remarkable advantages in size, power efficiency, and potential for widespread adoption.

Periodic diffractive optical element for high-density and large-scale spot array structured light projection

Structured light projection has been widely used for depth sensing in computer vision. Diffractive optical elements (DOEs) play a crucial role in generating structured light projected onto objects, and spot array is a common projection pattern. However, the primary metrics of the spot array, including density and field of view, are restricted by the principle of diffraction and its calculation. In this paper, a novel, to the best of our knowledge, method is proposed to achieve high-density periodic spot array on a large scale. Further, periodic DOEs, for the first time, are optimized to increase the density of the spot array without decreasing the periods of the DOE. Simulation and experimental results of high-density and large-scale spot array structured light projection are presented, demonstrating the effectiveness of the proposed method.

Compact structured light generation based on meta-hologram PCSEL integration

Metasurfaces, a catalog of optical components, offer numerous novel functions on demand. They have been integrated with vertical cavity surface-emitting lasers (VCSELs) in previous studies. However, the performance has been limited by the features of the VCSELs such as low output power and large divergence angle. Although the solution of the module of VCSEL array could solve these issues, the practical application is limited by extra lens and large size. In this study, we experimentally demonstrate reconstruction of a holographic images using a compact integration of a photonic crystal surface-emitting laser and metasurface holograms designed for structured light generation. This research showcases the flexible design capabilities of metasurfaces, high output power (on the order of milliwatts), and the ability to produce well-uniformed images with a wide field of view without the need for a collection lens, making it suitable for 3D imaging and sensing.

High-Capacity Spatial Structured Light for Robust and Accurate Reconstruction

Spatial structured light (SL) can achieve three-dimensional measurements with a single shot. As an important branch in the field of dynamic reconstruction, its accuracy, robustness, and density are of vital importance. Currently, there is a wide performance gap of spatial SL between dense reconstruction (but less accurate, e.g., speckle-based SL) and accurate reconstruction (but often sparser, e.g., shape-coded SL). The central problem lies in the coding strategy and the designed coding features. This paper aims to improve the density and quantity of reconstructed point clouds by spatial SL whilst also maintaining a high accuracy. Firstly, a new pseudo-2D pattern generation strategy was developed, which can improve the coding capacity of shape-coded SL greatly. Then, to extract the dense feature points robustly and accurately, an end-to-end corner detection method based on deep learning was developed. Finally, the pseudo-2D pattern was decoded with the aid of the epipolar constraint. Experimental results validated the effectiveness of the proposed system.

Accurate Depth Recovery Method Based on the Fusion of Time-of-Flight and Dot-Coded Structured Light

3D vision technology has been gradually applied to intelligent terminals ever since Apple Inc. introduced structured light on iPhoneX. At present, time-of-flight (TOF) and laser speckle-based structured light (SL) are two mainstream technologies applied to intelligent terminals, both of which are widely regarded as efficient dynamic technologies, but with low accuracy. This paper explores a new approach to achieve accurate depth recovery by fusing TOF and our previous work—dot-coded SL (DCSL). TOF can obtain high-density depth information, but its results may be deformed due to multi-path interference (MPI) and reflectivity-related deviations. In contrast, DCSL can provide high-accuracy and noise-clean results, yet only a limited number of encoded points can be reconstructed. This inspired our idea to fuse them to obtain better results. In this method, the sparse result provided by DCSL can work as accurate “anchor points” to keep the correctness of the target scene’s structure, meanwhile, the dense result from TOF can guarantee full-range measurement. Experimental results show that by fusion, the MPI errors of TOF can be eliminated effectively. Dense and accurate results can be obtained successfully, which has great potential for application in the 3D vision task of intelligent terminals in the future.

Adaptive View Sampling for Efficient Synthesis of 3D View Using Calibrated Array Cameras

Recovery of three-dimensional (3D) coordinates using a set of images with texture mapping to generate a 3D mesh has been of great interest in computer graphics and 3D imaging applications. This work aims to propose an approach to adaptive view selection (AVS) that determines the optimal number of images to generate the synthesis result using the 3D mesh and textures in terms of computational complexity and image quality (peak signal-to-noise ratio (PSNR)). All 25 images were acquired by a set of cameras in a 5×5 array structure, and rectification had already been performed. To generate the mesh, depth map extraction was carried out by calculating the disparity between the matched feature points. Synthesis was performed by fully exploiting the content included in the images followed by texture mapping. Both the 2D colored images and grey-scale depth images were synthesized based on the geometric relationship between the images, and to this end, three-dimensional synthesis was performed with a smaller number of images, which was less than 25. This work determines the optimal number of images that sufficiently provides a reliable 3D extended view by generating a mesh and image textures. The optimal number of images contributes to an efficient system for 3D view generation that reduces the computational complexity while preserving the quality of the result in terms of the PSNR. To substantiate the proposed approach, experimental results are provided.