Fringe projection profilometry method with high efficiency, precision, and convenience: theoretical analysis and development

Multi-view fringe projection profilometry based on phase texture and U-Net

The separability of patterns in a light-intersected area is the fundamental property of multi-view fringe projection profilometry (FPP). The traditional method based on temporal discrete Fourier transform separation and periodic wrapped phase requires dozens of patterns for each reconstruction. To enhance projection efficiency in multi-view FPP, a phase texture technique is proposed to reduce the pattern number by encoding the wrapped phase as an aperiodic texture. The U-Net neural network is trained on virtual datasets and employed as the decoder to map the phase texture to projector coordinates. To improve the decoder's adaptability for real measurements, the virtual dataset is configured with noise and defocus, while a monotonic loss function is designed. Simulations and experiments demonstrate that the proposed patterns are separable and the encoding method achieved reconstructions with only one-fifth the number of patterns required by traditional separation methods. The experimental results prove the improved decoding performance of U-Net trained with the monotonic loss function and the enhanced dataset.

Use of Phase-Angle Model for Full-Field 3D Reconstruction under Efficient Local Calibration

Currently, 3D reconstruction methods in structured light are generally implemented in a pre-calibrated area. To realize a full-field reconstruction, the calibration plate can be moved to multiple positions in a time-consuming manner, or the whole field can be calibrated with the help of a large calibration plate, which is more costly. In this paper, we address this problem by proposing a method for obtaining a global phase-angle model under a locally calibrated region, and based on this relationship, we investigate and analyze the reconstruction inside and outside of the calibrated zone. The results show that the method can reconstruct the object outside of the calibration zone completely, and can keep the planarity error around 0.1 mm and the sphericity error below 0.06 mm. The method only requires local calibration of the projected fringes at the two calibration positions to realize the 3D reconstruction of the full-field, which makes the method more advantageous.

Quaternary Categorization Strategy for Reconstructing High-Reflectivity Surface in Structured Light Illumination

Structured light illumination is widely applied for surface defect detection due to its advantages in terms of speed, precision, and non-contact capabilities. However, the high reflectivity of metal surfaces often results in the loss of point clouds, thus reducing the measurement accuracy. In this paper, we propose a novel quaternary categorization strategy to address the high-reflectivity issue. Firstly, we classify the pixels into four types according to the phase map characteristics. Secondly, we apply tailored optimization and reconstruction strategies to each type of pixel. Finally, we fuse point clouds from multi-type pixels to accomplish precise measurements of high-reflectivity surfaces. Experimental results show that our strategy effectively reduces the high-reflectivity error when measuring metal surfaces and exhibits stronger robustness against noise compared to the conventional method.

Modeling the measurement precision of Fringe Projection Profilometry

Three-dimensional (3D) surface geometry provides elemental information in various sciences and precision engineering. Fringe Projection Profilometry (FPP) is one of the most powerful non-contact (thus non-destructive) and non-interferometric (thus less restrictive) 3D measurement techniques, featuring at its high precision. However, the measurement precision of FPP is currently evaluated experimentally, lacking a complete theoretical model for guidance. We propose the first complete FPP precision model chain including four stage models (camera intensity, fringe intensity, phase and 3D geometry) and two transfer models (from fringe intensity to phase and from phase to 3D geometry). The most significant contributions include the adoption of a non-Gaussian camera noise model, which, for the first time, establishes the connection between camera's electronics parameters (known in advance from the camera manufacturer) and the phase precision, and the formulation of the phase to geometry transfer, which makes the precision of the measured geometry representable in an explicit and concise form. As a result, we not only establish the full precision model of the 3D geometry to characterize the performance of an FPP system that has already been set up, but also explore the expression of the highest possible precision limit to guide the error distribution of an FPP system that is yet to build. Our theoretical models make FPP a more designable technique to meet the challenges from various measurement demands concerning different object sizes from macro to micro and requiring different measurement precisions from a few millimeters to a few micrometers.

Simulation-driven machine learning approach for high-speed correction of slope-dependent error in coherence scanning interferometry

Slope-dependent error often occurs in the coherence scanning interferometry (CSI) measurement of functional engineering surfaces with complex geometries. Previous studies have shown that these errors can be corrected through the characterization and phase inversion of the instrument's three-dimensional (3D) surface transfer function. However, since CSI instrument is usually not completely shift-invariant, the 3D surface transfer function characterization and correction must be repeated for different regions of the full field of view, resulting in a long computational process and a reduction of measurement efficiency. In this work, we introduce a machine learning approach based on a deep neural network that is trainable for slope-dependent error correction in CSI. Our method leverages a deep neural network to directly learn errors characteristics from simulated surface measurements provided by a previously validated physics-based virtual CSI method. The experimental results demonstrate that the trained network is capable of correcting the surface height map with 1024 × 1024 sampling points within 0.1 seconds, covering a 178 µm field of view. The accuracy is comparable to the previous phase inversion approach while the new method is two orders of magnitude faster under the same computational condition.

Scene-adaptive pattern coding-based fringe projection profilometry: diffuse surfaces identification and 3-D reconstruction in cluttered scenes

Fringe projection profilometry (FPP) is one of the most widely used optical three-dimensional (3-D) perceiving techniques. However, when applied to cluttered scenes, acquiring accurate 3-D shapes is difficult because of the influences of indirect light caused by non-diffuse surfaces. In this paper, we first theoretically analyze and model the influences of indirect light in FPP, and then propose a scene-adaptive pattern coding-based method, which can design projection patterns based on the reflective properties of the scene's surfaces, to achieve accurate 3-D perceiving in cluttered scenes. Specifically, the scene confidence analysis method is first proposed to identify the reflective properties of various surfaces and localize the camera pixels of the diffuse surface. The illumination status (i.e., "0" or "1") of each projector pixel can be determined according to the camera-projection coordinate mapping and spatial pattern coding, where only diffuse surfaces can be illuminated, thus fundamentally preventing the influences of indirect light from the point of view of the light source. The 3-D shapes of diffuse surfaces can be accurately reconstructed in cluttered scenes. Different from traditional reflective properties change or light separation solutions, the proposed method can achieve accurate 3-D perceiving of cluttered scenes without additional hardware or expensive calculation. Extensive experiments verify that the proposed method outperforms the traditional methods in terms of accuracy and robustness.

FPP-SLAM: indoor simultaneous localization and mapping based on fringe projection profilometry

Simultaneous localization and mapping (SLAM) plays an important role in autonomous driving, indoor robotics and AR/VR. Outdoor SLAM has been widely used with the assistance of LiDAR and Global Navigation Satellite System (GNSS). However, for indoor applications, the commonly used LiDAR sensor does not satisfy the accuracy requirement and the GNSS signals are blocked. Thus, an accurate and reliable 3D sensor and suited SLAM algorithms are required for indoor SLAM. One of the most promising 3D perceiving techniques, fringe projection profilometry (FPP), shows great potential but does not prevail in indoor SLAM. In this paper, we first introduce FPP to indoor SLAM, and accordingly propose suited SLAM algorithms, thus enabling a new FPP-SLAM. The proposed FPP-SLAM can achieve millimeter-level and real-time mapping and localization without any expensive equipment assistance. The performance is evaluated in both simulated controlled and real room-sized scenes. The experimental results demonstrate that our method outperforms other state-of-the-art methods in terms of efficiency and accuracy. We believe this method paves the way for FPP in indoor SLAM applications.