Ternary Gray code-based phase unwrapping for 3D measurement using binary patterns with projector defocusing

Three-dimensional measurement method based on reusing equally spaced binary stripes

To eliminate the effect of nonlinear errors on measurement results, this paper presents a new method, to our knowledge, to overcome the nonlinear response of commercial projectors and cameras by using binary stripes for coding. The method shifts the generated equally spaced binary stripes by a fixed number of pixel points to obtain different stripe maps, followed by sequential projection of these binary stripes with a digital projector. The acquired binary stripes are reused in the 3D reconstruction combined with the phase-shift method and can be reduced to sinusoidal stripes with different phase shifts by a specific superposition method. In this paper, this method is combined with the traditional four-step phase-shift method for experiments. The results show that the accuracy of the wrapped phase obtained by the method proposed in this paper is 13.88% higher than that obtained by the traditional 16-step phase-shift method. Similarly, the accuracy of the standard ball measurement is increased by 21.05%. Additionally, the point cloud on the surface of the complex object obtained by the proposed method is smoother and more delicate than that obtained by the traditional 16-step phase-shift method.

Structured light 3-D sensing for scenes with discontinuous reflectivity: error removal based on scene reconstruction and normalization

Structured light-based 3-D sensing technique reconstructs the 3-D shape from the disparity given by pixel correspondence of two sensors. However, for scene surface containing discontinuous reflectivity (DR), the captured intensity deviates from its actual value caused by the non-ideal camera point spread function (PSF), thus generating 3-D measurement error. First, we construct the error model of fringe projection profilometry (FPP). From which, we conclude that the DR error of FPP is related to both the camera PSF and the scene reflectivity. The DR error of FPP is hard to be alleviated because of unknown scene reflectivity. Second, we introduce single-pixel imaging (SI) to reconstruct the scene reflectivity and normalize the scene with scene reflectivity "captured" by the projector. From the normalized scene reflectivity, pixel correspondence with error opposite to the original reflectivity is calculated for the DR error removal. Third, we propose an accurate 3-D reconstruction method under discontinuous reflectivity. In this method, pixel correspondence is first established by using FPP, and then refined by using SI with reflectivity normalization. Both the analysis and the measurement accuracy are verified under scenes with different reflectivity distributions in the experiments. As a result, the DR error is effectively alleviated while taking an acceptable measurement time.

Temporal phase unwrapping based on unequal phase-shifting code

In fringe projection profilometry (FPP) based on temporal phase unwrapping (TPU), reducing the number of projecting patterns has become one of the most important works in recent years. To remove the 2π ambiguity independently, this paper proposes a TPU method based on unequal phase-shifting code. Wrapped phase is still calculated from N-step conventional phase-shifting patterns with equal phase-shifting amount to guarantee the measuring accuracy. Particularly, a series of different phase-shifting amounts relative to the first phase-shifting pattern are set as codewords, and encoded to different periods to generate one coded pattern. When decoding, Fringe order with a large number can be determined from the conventional and coded wrapped phases. In addition, we develop a self-correction method to eliminate the deviation between the edge of fringe order and the 2π discontinuity. Thus, the proposed method can achieve TPU but need to only project one additional coded pattern (e. g. 3+1), which can significantly benefit dynamic 3D shape reconstruction. The theoretical and experimental analysis verify that the proposed method performs high robustness on the reflectivity of the isolated object while ensuring the measuring speed.

Three-dimensional measurement method based on a three-step phase-shifting fringe and a binary fringe

Gray-code plus phase-shifting is currently a commonly used method for structured light three-dimensional (3D) measurement that is able to measure complex surfaces. However, the Gray-code fringe patterns tend to be complicated, making the measurement process time-consuming. To solve this problem and to obtain faster speed without sacrificing accuracy, a 3D measurement method based on three-step phase-shifting and a binary fringe is proposed; the method contains three phase-shifting fringe patterns and an additional binary fringe pattern. The period of the binary fringe is designed to be the same as the three-step phase-shifting fringe. Because of the specific pattern design strategy, the three-step phase-shifting algorithm is used to obtain the wrapped phase, and the connected region labeling theorem is used to calculate the fringe order. A theoretical analysis, simulation, and experiments validate the efficiency and robustness of the proposed method. It can achieve high-precision 3D measurement, which performs almost the same as the Gray-code plus phase-shifting method. Since only one additional binary fringe pattern is required, it has the potential to achieve higher measurement speed.

Fringe-width encoded patterns for 3D surface profilometry

This paper presents a new fringe projection method for surface-shape measurement that uses novel fringe-width encoded fringe patterns. Specifically, the projection patterns are adjusted with the width of the fringe as the codeword. The wrapped phase with coding information is obtained by using the conventional wrapped phase calculation method, and the fringe order can be identified from the wrapped phase. After the fringe order is corrected based on the region growing algorithm, the fringe order and the wrapped phase can be directly used to reconstruct the surface. Static and dynamic measurements demonstrated the ability of the method to perform 3D shape measurement with only three projected patterns, single camera and projector in the least case.

Single-shot fringe projection profilometry based on deep learning and computer graphics

Multiple works have applied deep learning to fringe projection profilometry (FPP) in recent years. However, to obtain a large amount of data from actual systems for training is still a tricky problem, and moreover, the network design and optimization is still worth exploring. In this paper, we introduce graphic software to build virtual FPP systems in order to generate the desired datasets conveniently and simply. The way of constructing a virtual FPP system is described in detail firstly, and then some key factors to set the virtual FPP system much closer to reality are analyzed. With the aim of accurately estimating the depth image from only one fringe image, we also design a new loss function to enhance the overall quality and detailed information is restored. And two representative networks, U-Net and pix2pix, are compared in multiple aspects. The real experiments prove the good accuracy and generalization of the network trained by the diverse data from our virtual systems and the designed loss, providing a good guidance for real applications of deep learning methods.

Efficient intensity-based fringe projection profilometry method resistant to global illumination

Intensity-based fringe projection profilometry (IBFPP) is used widely because of its simple structure, high robustness, and noise resilience. Most IBFPP methods assume that any scene point is illuminated by direct illumination only, but global illumination effects introduce strong biases in the reconstruction result for many real-world scenes. To solve this problem, this paper describes an efficient IBFPP method for reconstructing three-dimensional geometry in the presence of global illumination. First, the average intensity of two sinusoidal patterns is used as a pixel-wise threshold to binarize the codeword patterns. The binarized template pattern is then used to convert other binarized fringe patterns into traditional Gray-code patterns. A proprietary compensation algorithm is then applied to eliminate fringe errors caused by environmental noise and lens defocusing. Finally, simple, efficient, and robust phase unwrapping can be achieved despite the effects of subsurface scattering and interreflection. Experimental results obtained in different environments show that the proposed method can obtain three-dimensional information reliably when influenced by global illumination.

Encoding technology of an asymmetric combined structured light for 3D measurement

Sinusoidal phase-shifting symmetrically combined with cyclic code is one of the most important encoding methods in the field of 3D measurement. Due to the modulation of the object surface and the influence of the noise of the image acquisition system, the periods of the cyclic code and the sinusoidal phase-shifting in the intensity image do not coincide completely, and they lead to large absolute phase decoding errors near the cycle boundaries, which are called cycle dislocation errors. In order to eliminate these errors in principle, the concept and method of region encoding for four-step sinusoidal phase-shifting are proposed, and the sinusoidal phase-shifting is combined with cyclic code asymmetrically. Under the premise that the cyclic code and the region code change at different times, the cycle dislocation error is reduced from one cycle of cyclic code to one pixel by the dual constraint of cyclic code and region code. The simulation measurement results of 3 ds max and the physical measurement results show that the asymmetric combination encoding method effectively eliminates the cycle dislocation errors; the maximum measurement error is reduced by an order of magnitude, and the root mean square measurement error is reduced by 70%.

State-of-the-art active optical techniques for three-dimensional surface metrology: a review [Invited]

This paper reviews recent developments of non-contact three-dimensional (3D) surface metrology using an active structured optical probe. We focus primarily on those active non-contact 3D surface measurement techniques that could be applicable to the manufacturing industry. We discuss principles of each technology, and its advantageous characteristics as well as limitations. Towards the end, we discuss our perspectives on the current technological challenges in designing and implementing these methods in practical applications.

Coding line structured light based on a line-scan camera and its calibration

In a conventional three-dimensional (3D) measurement technique of a line-scan camera, the projection system based on surface structured light is a compromise of traditional projection technology, which suffers from complex calibration, complex structure and low accuracy. To this end, the coding line structured light based on the coded line laser projection system is proposed to address the 3D measurement of a line-scan camera. The single-line projection and codeable characteristics of coded line laser projection system (constructed by a point laser and a micro-electro-mechanical system (MEMS) scanning galvanometer and modeled as the line projection model) are fully matched with the imaging mode of the line-scan camera. The 3D measurement model based on the height information, lateral information and absolute phase of the coding line structured light is derived. The multi-position flat display calibration method is proposed to calibrate the system parameters. In addition, in order to obtain the accurate absolute phase from the phase shift combined binary code, the periodic error correction method based on expansion-corrosion is proposed to correct the phase error. Contrary to conventional structured light methods based on a line-scan camera, the proposed method has the advantages of high measurement accuracy, high efficiency, more compactness and low cost. The experiments affirm that the coding line structured light is valid and the proposed calibration method is feasible. Experimental results also indicate that the proposed method performs well for both diffuse reflective surfaces and reflective surfaces that are difficult to measure with conventional structured light methods based on a line-scan camera.

Spatial pattern-shifting method for complete two-wavelength fringe projection profilometry

Two-wavelength fringe projection profilometry (FPP) unwraps a phase with the unambiguous phase range (UPR) of the least common multiple (LCM) of the two wavelengths. It is accurate, convenient, and robust, and thus plays an important role in shape measurement. However, when two non-coprime wavelengths are used, only a small UPR can be generated, and the unwrapping performance is compromised. In this Letter, a spatial pattern-shifting method (SPSM) is proposed to generate the maximum UPR (i.e., the product of the two wavelengths) from two non-coprime wavelengths. For the first time, to the best of our knowledge, the SPSM breaks the constraint of wavelength selection and enables a complete (i.e., either coprime or non-coprime) two-wavelength FPP. The SPSM, on the other hand, only requires spatially shift of the low-frequency pattern with the designed amounts and accordingly adjusting the fringe order determination, which is extremely convenient in implementation. Both numerical and experimental analyses verify its flexibility and correctness.

Spatial binary coding method for stripe-wise phase unwrapping

Binary coding methods have been widely used for phase unwrapping. However, traditional temporal binary coding methods require a sequence of binary patterns to encode the fringe order information. This paper presents a spatial binary coding (SBC) method that encodes the fringe order into only one binary pattern. Each stripe of the sinusoidal phase-shifting patterns is corresponding to an N-bit codeword of the binary pattern. A robust stripe-wise decoding scheme is also developed to extract the N-bit codeword, then fringe order can be determined, and stripe-wise phase unwrapping can be performed. Experiment results confirm that the SBC method can correctly recover the absolute phase of measured objects with only one additional binary pattern.

Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning

Fringe projection profilometry (FPP) has become increasingly important in dynamic 3-D shape measurement. In FPP, it is necessary to retrieve the phase of the measured object before shape profiling. However, traditional phase retrieval techniques often require a large number of fringes, which may generate motion-induced error for dynamic objects. In this paper, a novel phase retrieval technique based on deep learning is proposed, which uses an end-to-end deep convolution neural network to transform a single or two fringes into the phase retrieval required fringes. When the object's surface is located in a restricted depth, the presented network only requires a single fringe as the input, which otherwise requires two fringes in an unrestricted depth. The proposed phase retrieval technique is first theoretically analyzed, and then numerically and experimentally verified on its applicability for dynamic 3-D measurement.

Enhanced phase-coding method for three-dimensional shape measurement with half-period codeword

The phase-coding method has been widely used for 3D shape measurement, which uses sinusoidal phase-shifting patterns to recover the wrapped phase and the stair phase-coding patterns to determine the fringe order. However, due to random noises and image blurring, the fringe order is always misaligned with the wrapped phase, which will lead to fringe order errors. This paper presents an enhanced phase-coding method to address this misalignment problem by using half-period codewords, in which each codeword is aligned to the half-period of the sinusoidal patterns. Then, two complementary fringe orders with half-period dislocation can be calculated, which can effectively eliminate the fringe order errors. To extend the coding range of stair phase, this paper further develops a computational scheme based on the geometric constraint method. Simulations and experiments have been carried out, and their results confirm that the enhanced method can reliably recover the 3D shape of the measured objects.

A simple and practical jump error removal method for fringe projection profilometry based on self-alignment technique

The code-based method is one of the frequently adopted fringe projection profilometry techniques because of its robustness and high speed. However, the abnormal jump errors caused by the misalignment between the wrapped phase and the fringe order impact the phase unwrapping quality and are more serious in binary defocusing measurement with significant defocusing. This paper proposes a self-alignment technique (SAT) with high speed, no additional patterns, and no accuracy loss to eliminate such jump errors. After analyzing the relation between the grating changes and the misalignment, we designed an assist-code that can perfectly align with the wrapped phase even in significant defocusing. After that, this assist-code is used to adjust the misalignment and eliminate the jump errors. The comparison between the median filter and the proposed SAT method in simulations and experiments demonstrates that the proposed SAT method has the ability to completely remove jump errors for complex textures and step-height objects, even in significant defocusing, indicating its potential to be applied to other phase unwrapping fields, e.g., color-based methods with color coupling problems.

Dual-sensitivity profilometry with defocused projection of binary fringes

A dual-sensitivity profilometry technique based on defocused projection of binary fringes is presented. Here, two sets of fringe patterns with a sinusoidal profile are produced by applying the same analog low-pass filter (projector defocusing) to binary fringes with a high- and low-frequency spatial carrier. The high-frequency fringes have a binary square-wave profile, while the low-frequency binary fringes are produced with error-diffusion dithering. The binary nature of the binary fringes removes the need for calibration of the projector's nonlinear gamma. Working with high-frequency carrier fringes, we obtain a high-quality wrapped phase. On the other hand, working with low-frequency carrier fringes we found a lower-quality, nonwrapped phase map. The nonwrapped estimation is used as stepping stone for dual-sensitivity temporal phase unwrapping, extending the applicability of the technique to discontinuous (piecewise continuous) surfaces. We are proposing a single defocusing level for faster high- and low-frequency fringe data acquisition. The proposed technique is validated with experimental results.