High-frequency background modulation fringe patterns based on a fringe-wavelength geometry-constraint model for 3D surface-shape measurement

BimodalPS: Causes and Corrections for Bimodal Multi-Path in Phase-Shifting Structured Light Scanners

Structured light illumination is an active 3D scanning technique based on projecting and capturing a set of striped patterns and measuring the warping of the patterns as they reflect off a target object's surface. As designed, each pixel in the camera sees exactly one pixel from the projector; however, there are multi-path situations where a camera pixel sees light from multiple projector positions. In the case of bimodal multi-path, the camera pixel receives light from exactly two positions, which occurs along a step edge where the edge slices through a pixel which, therefore, sees both a foreground and background surface. In this paper, we present a general mathematical model to address this bimodal multi-path issue in a phase-shifting or so-called phase-measuring-profilometry scanner to measure the constructive and destructive interference between the two light paths, and by taking advantage of this interference, separate the paths and make two decoupled depth measurements. We validate our algorithm with both simulations and a number of challenging real-world scenarios, significantly outperforming the state-of-the-art methods.

Dynamic 3D measurement based on orthogonal fringe projection and geometric constraints

Geometric constraint algorithms can solve phase ambiguity for fringe projection profilometry (FPP). However, they either require multiple cameras or suffer from a small measurement depth range. To overcome these limitations, this Letter proposes an algorithm combining orthogonal fringe projection and geometric constraints. A novel, to the best of our knowledge, scheme is developed to assess the reliabilities of the potential homologous points, which works with depth segmentation to determine the final HPs. With full consideration of lens distortions, the algorithm reconstructs two 3D results from every set of patterns. Experimental results verify that it can effectively and robustly measure discontinuous objects with complex motion over a large depth range.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

High-accuracy 3D surface measurement using hybrid multi-frequency composite-pattern temporal phase unwrapping

Multi-frequency temporal phase unwrapping (TPU) has been extensively used in phase-shifting profilometry (PSP) for the high-accuracy measurement of objects with surface discontinuities and isolated objects. However, a large number of fringe patterns are commonly required. To reduce the number of required patterns, a new hybrid multi-frequency composite-pattern TPU method was developed using fewer patterns than conventional TPU. The new method combines a unit-frequency ramp pattern with three low-frequency phase-shifted fringe patterns to form three composite patterns. These composite patterns are used together with three high-frequency phase-shifted fringe patterns to generate a high-accuracy phase map. The optimal high frequency to achieve high measurement accuracy and reliable phase unwrapping is determined by analyzing the effect of temporal intensity noise on phase error. Experimental results demonstrated that new grayscale hybrid and color hybrid multi-frequency composite-pattern TPU methods can achieve a high-accuracy measurement using only six and three images, respectively.

High-frequency color-encoded fringe-projection profilometry based on geometry constraint for large depth range

In multi-view fringe projection profilometry (FPP), a limitation of geometry-constraint based approaches is the reduced measurement depth range often used to reduce the number of candidate points and increase the corresponding point selection reliability, when high-frequency fringe patterns are used. To extend the depth range, a new method of high-frequency fringe projection profilometry was developed by color encoding the projected fringe patterns to allow reliable candidate point selection even when six candidate points are in the measurement volume. The wrapped phase is directly retrieved using the intensity component of the hue-saturation-intensity (HSI) color space and complementary-hue is introduced to identify color codes for correct corresponding point selection. Mathematical analyses of the effect of color crosstalk on phase calculation and color code identification show that the phase calculation is independent of color crosstalk and that color crosstalk has little effect on color code identification. Experiments demonstrated that the new method can achieve high accuracy in 3D measurement over a large depth range and for isolated objects, using only two high-frequency color-encoded fringe patterns.

Dual-projector structured light 3D shape measurement

Structured light illumination is an active three-dimensional scanning technique that uses a projector and camera pair to project and capture a series of stripe patterns; however, with a single camera and single projector, structured light scanning has issues associated with scan occlusions, multi-path, and weak signal reflections. To address these issues, this paper proposes dual-projector scanning using a range of projector/camera arrangements. Unlike previous attempts at dual-projector scanning, the proposed scanner drives both light engines simultaneously, using temporal-frequency multiplexing to computationally decouple the projected patterns. Besides presenting the details of how such a system is built, we also present experimental results demonstrating how multiple projectors can be used to (1) minimize occlusions; (2) achieve higher signal-to-noise ratios having twice a single projector's brightness; (3) reduce the number of component video frames required for a scan; and (4) detect multi-path interference.

Real-time motion-induced-error compensation in 3D surface-shape measurement

Object motion can introduce unknown phase shift and thus measurement error in multi-image phase-shifting methods of fringe projection profilometry. This paper presents a new method to estimate the unknown phase shifts and reduce the motion-induced error by using three phase maps computed over a multiple measurement sequence and calculating the difference between phase maps. The pixel-wise estimation of the motion-induced phase shifts permits phase-error compensation for non-homogeneous surface motion. Experiments demonstrated the ability of the method to reduce motion-induced error in real-time, for shape measurement of surfaces with high depth variation, and moving and deforming surfaces.

Motion-artifact-free dynamic 3D shape measurement with hybrid Fourier-transform phase-shifting profilometry

Fourier-transform profilometry (FTP) and phase-shifting profilometry (PSP) are two mainstream fringe projection techniques widely used for three-dimensional (3D) shape measurement. The former is well known for its single-shot nature and the latter for its higher measurement resolution and precision. However, when it comes to measuring the dynamic objects, neither approach is able to produce high-resolution, high-accuracy measurement results that are free from any depth ambiguities and motion-related artifacts. Furthermore, for scenes consisting of both static and dynamic objects, a trade-off between measurement precision and efficiency has to be made, suggesting that using a single approach can yield only suboptimal results. To this end, we propose a novel hybrid Fourier-transform phase-shifting profilometry method to integrate the advantages of both approaches. The motion vulnerability of multi-shot PSP can be overcome, or at least significantly alleviated, through the combination of single-shot FTP, while the high accuracy of PSP can also be preserved when the object is motionless. We design a phase-based, pixel-wise motion detection strategy that can accurately outline the moving object regions from their motionless counterparts. The final measurement result is obtained by fusing the determined regions where the PSP or FTP is applied correspondingly. To validate the proposed hybrid approach, we develop a real-time 3D shape measurement system for measuring multiple isolated moving objects. Experimental results demonstrate that our method achieves significantly higher precision and better robustness compared with conventional approaches where PSP or FTP is applied separately.

High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system

In this paper, we propose a high-speed 3D shape measurement technique based on the optimized composite fringe patterns and stereo-assisted structured light system. Stereo phase unwrapping, as a new-fashioned method for absolute phase retrieval based on the multi-view geometric constraints, can eliminate the phase ambiguities and obtain a continuous phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which limits the accuracy of 3D measurement. To solve this problem, we develop an optimized method for designing the composite pattern, in which the speckle pattern is embedded into the conventional 4-step phase-shifting fringe patterns without compromising the fringe modulation, and thus the phase measurement accuracy. We also present a simple and effective evaluation criterion for the correlation quality of the designed speckle pattern in order to improve the matching accuracy significantly. When the embedded speckle pattern is demodulated, the periodic ambiguities in the wrapped phase can be eliminated by combining the adaptive window image correlation with geometry constraint. Finally, some mismatched regions are further corrected based on the proposed regional diffusion compensation technique (RDC). These proposed techniques constitute a complete computational framework that allows to effectively recover an accurate, unambiguous, and distortion-free 3D point cloud with only 4 projected patterns. Experimental results verify that our method can achieve high-speed, high-accuracy, robust 3D shape measurement with dense (64-period) fringe patterns at 5000 frames per second.

High-speed real-time 3D shape measurement based on adaptive depth constraint

Stereo phase unwrapping (SPU) has been increasingly applied to high-speed real-time fringe projection profilometry (FPP) because it can retrieve the absolute phase or matching points in a stereo FPP system without projecting or acquiring additional fringe patterns. Based on a pre-defined measurement volume, artificial maximum/minimum phase maps can be created solely using geometric constraints of the FPP system, permitting phase unwrapping on a pixel-by-pixel basis. However, when high-frequency fringes are used, the phase ambiguities will increase which makes SPU unreliable. Several auxiliary techniques have been proposed to enhance the robustness of SPU, but their flexibility still needs to be improved. In this paper, we proposed an adaptive depth constraint (ADC) approach for high-speed real-time 3D shape measurement, where the measurement depth volume for geometric constraints is adaptively updated according to the current reconstructed geometry. By utilizing the spatio-temporal correlation of moving objects under measurement, a customized and tighter depth constraint can be defined, which helps enhance the robustness of SPU over a large measurement volume. Besides, two complementary techniques, including simplified left-right consistency check and feedback mechanism based on valid area, are introduced to further increase the robustness and flexibility of the ADC. Experimental results demonstrate the success of our proposed SPU approach in recovering absolute 3D geometries of both simple and complicated objects with only three phase-shifted fringe images.

High-speed and high-accuracy 3D surface measurement using a mechanical projector

This paper presents a method to achieve high-speed and high-accuracy 3D surface measurement using a custom-designed mechanical projector and two high-speed cameras. We developed a computational framework that can achieve absolute shape measurement in sub-pixel accuracy through: 1) capturing precisely phase-shifted fringe patterns by synchronizing the cameras with the projector; 2) generating a rough disparity map between two cameras by employing a standard stereo-vision method using texture images with encoded statistical patterns; and 3) utilizing the wrapped phase as a constraint to refine the disparity map. The projector can project binary patterns at a speed of up to 10,000 Hz, and the camera can capture the required number of phase-shifted fringe patterns with 1/10,000 second, and thus 3D shape measurement can be realized as high as 10,000 Hz regardless the number of phase-shifted fringe patterns required for one 3D reconstruction. Experimental results demonstrated the success of our proposed method.