MIMONet: Structured-light 3D shape reconstruction by a multi-input multi-output network

Time-Distributed Framework for 3D Reconstruction Integrating Fringe Projection with Deep Learning

In recent years, integrating structured light with deep learning has gained considerable attention in three-dimensional (3D) shape reconstruction due to its high precision and suitability for dynamic applications. While previous techniques primarily focus on processing in the spatial domain, this paper proposes a novel time-distributed approach for temporal structured-light 3D shape reconstruction using deep learning. The proposed approach utilizes an autoencoder network and time-distributed wrapper to convert multiple temporal fringe patterns into their corresponding numerators and denominators of the arctangent functions. Fringe projection profilometry (FPP), a well-known temporal structured-light technique, is employed to prepare high-quality ground truth and depict the 3D reconstruction process. Our experimental findings show that the time-distributed 3D reconstruction technique achieves comparable outcomes with the dual-frequency dataset (p = 0.014) and higher accuracy than the triple-frequency dataset (p = 1.029 × 10-9), according to non-parametric statistical tests. Moreover, the proposed approach's straightforward implementation of a single training network for multiple converters makes it more practical for scientific research and industrial applications.

High-precision dynamic three-dimensional shape measurement of specular surfaces based on deep learning

In order to solve the difficulty of traditional phase measuring deflectometry (PMD) in considering precision and speed, an orthogonal encoding PMD method based on deep learning is presented in this paper. We demonstrate for, what we believe to be, the first time that deep learning techniques can be combined with dynamic-PMD and can be used to reconstruct high-precision 3D shapes of specular surfaces from single-frame distorted orthogonal fringe patterns, enabling high-quality dynamic measurement of specular objects. The experimental results prove that the phase and shape information measured by the proposed method has high accuracy, almost reaching the results obtained by the ten-step phase-shifting method. And the proposed method also has excellent performance in dynamic experiments, which is of great significance to the development of optical measurement and fabrication areas.

Generalized Fringe-to-Phase Framework for Single-Shot 3D Reconstruction Integrating Structured Light with Deep Learning

Three-dimensional (3D) shape acquisition of objects from a single-shot image has been highly demanded by numerous applications in many fields, such as medical imaging, robotic navigation, virtual reality, and product in-line inspection. This paper presents a robust 3D shape reconstruction approach integrating a structured-light technique with a deep learning-based artificial neural network. The proposed approach employs a single-input dual-output network capable of transforming a single structured-light image into two intermediate outputs of multiple phase-shifted fringe patterns and a coarse phase map, through which the unwrapped true phase distributions containing the depth information of the imaging target can be accurately determined for subsequent 3D reconstruction process. A conventional fringe projection technique is employed to prepare the ground-truth training labels, and part of its classic algorithm is adopted to preserve the accuracy of the 3D reconstruction. Numerous experiments have been conducted to assess the proposed technique, and its robustness makes it a promising and much-needed tool for scientific research and engineering applications.

Different structured-light patterns in single-shot 2D-to-3D image conversion using deep learning

Single-shot 3D shape reconstruction integrating structured light and deep learning has drawn considerable attention and achieved significant progress in recent years due to its wide-ranging applications in various fields. The prevailing deep-learning-based 3D reconstruction using structured light generally transforms a single fringe pattern to its corresponding depth map by an end-to-end artificial neural network. At present, it remains unclear which kind of structured-light patterns should be employed to obtain the best accuracy performance. To answer this fundamental and much-asked question, we conduct an experimental investigation of six representative structured-light patterns adopted for single-shot 2D-to-3D image conversion. The assessment results provide a valuable guideline for structured-light pattern selection in practice.