Enhanced two-frequency phase-shifting method

Human Respiration Rate Measurement with High-Speed Digital Fringe Projection Technique

This paper proposes a non-contact continuous respiration monitoring method based on Fringe Projection Profilometry (FPP). This method aims to overcome the limitations of traditional intrusive techniques by providing continuous monitoring without interfering with normal breathing. The FPP sensor captures three-dimensional (3D) respiratory motion from the chest wall and abdomen, and the analysis algorithms extract respiratory parameters. The system achieved a high Signal-to-Noise Ratio (SNR) of 37 dB with an ideal sinusoidal respiration signal. Experimental results demonstrated that a mean correlation of 0.95 and a mean Root-Mean-Square Error (RMSE) of 0.11 breaths per minute (bpm) were achieved when comparing to a reference signal obtained from a spirometer.

Projected feature assisted coarse to fine point cloud registration method for large-size 3D measurement

Fringe projection profilometry has gained significant interest due to its high precision, enhanced resolution, and simplified design. Typically, the spatial and perspective measurement capability is restricted by the lenses of the camera and projector in accordance with the principles of geometric optics. Therefore, large-size object measurement requires data acquisition from multiple perspectives, followed by point cloud splicing. Current point cloud registration methods usually rely on 2D feature textures, 3D structural elements, or supplementary tools, which will increase costs or limit the scope of the application. To address large-size 3D measurement more efficiently, we propose a low-cost and feasible method that combines active projection textures, color channel multiplexing, image feature matching and coarse-to-fine point registration strategies. Using a composite structured light with red speckle patterns for larger areas and blue sinusoidal fringe patterns for smaller ones, projected onto the surface, which allows us to accomplish simultaneous 3D reconstruction and point cloud registration. Experimental results demonstrate that the proposed method is effective for the 3D measurement of large-size and weak-textured objects.

Fast fringe projection profilometry using 3 + 1 phase retrieval strategy and fringe order correction

Fringe projection profilometry (FPP) has attracted wide attention in optical 3D measurement field for its high resolution, high accuracy, and simple system construction. However, the inherent requirements of FPP's measurement speed and accuracy are contradictory. We adopt a 3+1 phase retrieval strategy combining phase shift and the Hilbert transform algorithm to achieve high-speed, high-precision simultaneous measurements with only four binary fringe patterns. Furthermore, when using temporal phase unwrapping, fringe order errors (FOEs) are inevitable. Although these FOEs do not affect neighboring pixels, they are brought into the final point cloud coordinates by the absolute phase value. We propose a fringe order correction method to eliminate FOEs. The feasibility and effectiveness of this method are verified by the experimental results.

Specular Surface Shape Measurement with Orthogonal Dual-Frequency Fourier Transform Deflectometry

Three-dimensional (3D) shape measurement for specular surfaces is becoming increasingly important in various applications. A novel orthogonal dual-frequency fringe is proposed in the specular surface shape measurement to overcome the phase jumping and discontinuities in spatial phase unwrapping. The fringe recalibrated high-accuracy phase information from its high-frequency fringe component with low-ambiguity phase information from its low-frequency fringe component. An improved Fourier transform deflectometry method based on the orthogonal dual-frequency fringe is proposed to measure 3D specular surface shapes. Simulation results showed that the orthogonal dual-frequency Fourier transform deflectometry (ODD) method could precisely reconstruct flat surfaces with an error of 2.16 nm rms, and concave surfaces with an error of 1.86 μm rms. Experimental results showed that the reconstructed shapes of both the flat mirror and the concave mirror measured by the ODD measurement system were highly comparable to those obtained by the phase-measuring deflectometry (PMD) method. This new fringe provides a distinctive approach to structured pattern construction and reduces the phase unwrapping ambiguities in specular surface shape measurement. The ODD method can achieve accurate 3D shape measurement for specular surfaces by sampling only one fringe, providing a possible basis for future real-time measurement of specular surfaces.

Fast combined-frequency phase extraction for phase shifting profilometry

Due to the nonlinearity in phase shifting profilometry (PSP) system, the captured images are often distorted with fringe harmonics, resulting in inaccurate phase map and measurement. Considering the fact that the phase error can be significantly reduced by modeling high-order fringe harmonics, this work formulates the phase extraction problem - with different frequency images and high-order fringe harmonic model - as a maximum likelihood estimation (MLE). To optimize it efficiently, we thus propose a combined-frequency phase extraction (CFPE) solution by introducing a latent phase map and incorporating the famous expectation-maximization (EM) framework. As a result, our CFPE method only needs ∼5% execution time of a high-order baseline, whilst keeps the high-order accuracy. Tested on synthetic images as well as practical measurements, our CFPE method demonstrated its performance improvement of efficiency and accuracy. In addition, our detailed implementation with experimental arrangement is also provided for interested researchers.

An Improved Synthesis Phase Unwrapping Method Based on Three-Frequency Heterodyne

An improved three-frequency heterodyne synthesis phase unwrapping method is proposed to improve the measurement accuracy through phase difference and phase sum operations. This method can reduce the effect of noise and increase the equivalent phase frequency. According to the distribution found in the phase difference calculation process, the Otsu segmentation is introduced to judge the phase threshold. The equivalent frequency obtained from the phase sum is more than those of all projected fringe patterns. In addition, the appropriate period combinations are also studied. The simulations and related experiments demonstrate the feasibility of the proposed method and the ability to improve the accuracy of the measurement results further.

Three-dimensional measurement method based on a three-step phase-shifting fringe and a binary fringe

Gray-code plus phase-shifting is currently a commonly used method for structured light three-dimensional (3D) measurement that is able to measure complex surfaces. However, the Gray-code fringe patterns tend to be complicated, making the measurement process time-consuming. To solve this problem and to obtain faster speed without sacrificing accuracy, a 3D measurement method based on three-step phase-shifting and a binary fringe is proposed; the method contains three phase-shifting fringe patterns and an additional binary fringe pattern. The period of the binary fringe is designed to be the same as the three-step phase-shifting fringe. Because of the specific pattern design strategy, the three-step phase-shifting algorithm is used to obtain the wrapped phase, and the connected region labeling theorem is used to calculate the fringe order. A theoretical analysis, simulation, and experiments validate the efficiency and robustness of the proposed method. It can achieve high-precision 3D measurement, which performs almost the same as the Gray-code plus phase-shifting method. Since only one additional binary fringe pattern is required, it has the potential to achieve higher measurement speed.

A New Pattern Quality Assessment Criterion and Defocusing Degree Determination of Laser Speckle Correlation Method

The laser speckle correlation method has found widespread application for obtaining information from vibrating objects. However, the resolution and accuracy of the laser speckle correlation method as they relate to the defocusing degree have not been analyzed sufficiently. Furthermore, the possible methods for speckle pattern quality assessment and enhancement have not been studied. In this study, the resolution and accuracy of the laser speckle correlation method are analyzed, and it is found that they are affected by the defocusing degree and speckle pattern quality, respectively. A new speckle pattern quality criterion combining the mean intensity gradient and frequency spectrum was proposed, called CMZ. The quality of the speckle pattern is higher when the CMZ is closer to zero. The proposed criterion was verified by simulated speckle patterns and real speckle patterns with different speckle sizes, densities, and gray contrasts. In the experimental setup stage, a suitable defocusing degree can be selected based on the resolution requirement and optimal speckle size, and other experimental parameters can be determined according to the CMZ criterion. Rotation and vibration experiments verified the effectiveness of the laser speckle correlation method and confirmed the reliability of the experiment preparation based on proposed CMZ criterion.

Improved spatial-shifting two-wavelength algorithm for 3D shape measurement with a look-up table

Conventional two-wavelength algorithms have been broadly used for three-dimensional shape measurement. However, the maximum unambiguous range of phase unwrapping depends on the least-common multiple of two wavelengths, and thus coprime wavelengths are commonly selected. The recently proposed spatial-shifting two-wavelength (SSTW) algorithm can achieve the maximum unambiguous range with two non-coprime wavelengths, but this algorithm tends to fail for some wavelength selections. To address this problem, this paper presents a general look-up-table-based SSTW (LUT-SSTW) algorithm with arbitrary wavelength selection. The paper also analyzes the phase unwrapping robustness in terms of phase errors and provides guidance for wavelength selection. In addition, an improved LUT-SSTW algorithm is developed to enhance the phase unwrapping robustness, and further relax wavelength selection. Some experiments have been conducted, and their results verify the efficiency of the proposed method.

Error self-correction method for phase jump in multi-frequency phase-shifting structured light

Among 3D measurement approaches, multi-frequency phase-shifting structured light has advantages such as high resolution and high sampling rate owing to its point-to-point calculation method. However, there is always phase jump in the measurement process, which greatly reduces measurement accuracy. This paper proposes an error self-correction method for phase jump based on the multi-frequency heterodyne approach. The method uses redundant measurement data to implement self-correction and does not require additional data acquisition steps. We perform both simulations and experiments using the proposed error self-correction method and the classical heterodyne approach to compare the results. The experiment results verify both the accuracy and suitability of the proposed method.

Fringe projection profilometry by conducting deep learning from its digital twin

High-accuracy and high-speed three-dimensional (3D) fringe projection profilometry (FPP) has been widely applied in many fields. Recently, researchers discovered that deep learning can significantly improve fringe analysis. However, deep learning requires numerous objects to be scanned for training data. In this paper, we propose to build the digital twin of an FPP system and perform virtual scanning using computer graphics, which can significantly save cost and labor. The proposed method extracts 3D geometry directly from a single-shot fringe image, and real-world experiments have demonstrated the success of the virtually trained model. Our virtual scanning method can automatically generate 7,200 fringe images and 800 corresponding 3D scenes within 1.5 hours.

Wide-field optical spectroscopy system integrating reflectance and spatial frequency domain imaging to measure attenuation-corrected intrinsic tissue fluorescence in radical prostatectomy specimens

The development of a multimodal optical imaging system is presented that integrates endogenous fluorescence and diffuse reflectance spectroscopy with single-wavelength spatial frequency domain imaging (SFDI) and surface profilometry. The system images specimens at visible wavelengths with a spatial resolution of 70 µm, a field of view of 25 cm² and a depth of field of ∼1.5 cm. The results of phantom experiments are presented demonstrating the system retrieves absorption and reduced scattering coefficient maps using SFDI with <6% reconstruction errors. A phase-shifting profilometry technique is implemented and the resulting 3-D surface used to compute a geometric correction ensuring optical properties reconstruction errors are maintained to <6% in curved media with height variations <20 mm. Combining SFDI-computed optical properties with data from diffuse reflectance spectra is shown to correct fluorescence using a model based on light transport in tissue theory. The system is used to image a human prostate, demonstrating its ability to distinguish prostatic tissue (anterior stroma, hyperplasia, peripheral zone) from extra-prostatic tissue (urethra, ejaculatory ducts, peri-prostatic tissue). These techniques could be integrated in robotic-assisted surgical systems to enhance information provided to surgeons and improve procedural accuracy by minimizing the risk of damage to extra-prostatic tissue during radical prostatectomy procedures and eventually detect residual cancer.

Monotonicity analysis of absolute phase unwrapping by geometric constraint in a structured light system

The monotonicity of depth in a geometric constraint based absolute phase unwrapping is analyzed and a monotonic discriminant of Δ(uc,vc) is presented in this paper. The sign of the discriminant determines the distance selection for the virtual plane to create the artificial absolute phase map for a given structured light system. As Δ(uc,vc) ≥ 0 at an arbitrary point on the CCD pixel coordinates the minimum depth distance is selected for the virtual plane, and the maximum depth distance is selected as Δ(uc,vc) ≤ 0. Two structured light systems with different signs of the monotonic discriminant are developed and the validity of the theoretical analysis is experimentally demonstrated.

Real-time high-speed three-dimensional surface imaging using band-limited illumination profilometry with a CoaXPress interface

High-speed three-dimensional (3D) surface imaging by structured-light profilometry is currently driven by numerous applications. However, the limited speeds in fringe pattern projection, image acquisition, and data transmission have strained the existing methods from reaching kilohertz-level acquisition, processing, and display of 3D information during the occurrence of dynamic events (i.e., in real time). To overcome these limitations, we have developed band-limited illumination profilometry (BLIP) with a CoaXPress interface (CI), which enables real-time high-speed 3D surface imaging. We have demonstrated the system's performance by imaging various static and fast-moving 3D objects in real time. We have also applied this system in fluid mechanics by imaging dynamics of a flag, which allowed observation of the wave propagation, gravity-induced phase mismatch, and asymmetric flapping motion. We expect CI-BLIP to find diverse scientific and industrial applications.

Fringe harmonics elimination in multi-frequency phase-shifting fringe projection profilometry

In fringe projection profilometry, the purpose of using two- or multi-frequency fringe patterns is to unwrap the measured phase maps temporally. Using the same patterns, this paper presents a least squares algorithm for, simultaneously with phase-unwrapping, eliminating the influences of fringe harmonics induced by various adverse factors. It is demonstrated that, for most of the points over the measured surface, projecting two sequences of phase-shifting fringe patterns having different frequencies enables providing sufficiently many equations for determining the coefficient of a high order fringe harmonic. As a result, solving these equations in the least squares sense results in a phase map having higher accuracy than that depending only on the fringe patterns of a single frequency. For the other few points which have special phases related to the two frequencies, this system of equations becomes under-determined. For coping with this case, this paper suggests an interpolation-based solution which has a low sensitivity to the variations of reflectivity and slope of the measured surface. Simulation and experimental results verify that the proposed method significantly suppresses the ripple-like artifacts in phase maps induced by fringe harmonics without capturing extra many fringe patterns or correcting the non-sinusoidal profiles of fringes. In addition, this method involves a quasi-pointwise operation, enabling correcting position-dependent phase errors and being helpful for protecting the edges and details of the measurement results from being blurred.

Large depth-of-field 3D shape measurement using an electrically tunable lens

The state-of-the-art 3D shape measurement system has rather shallow working volume due to the limited depth-of-field (DOF) of conventional lens. In this paper, we propose to use the electrically tunable lens to substantially enlarge the DOF. Specifically, we capture always in-focus phase-shifted fringe patterns by precisely synchronizing the tunable lens attached to the camera with the image acquisition and the pattern projection; we develop a phase unwrapping framework that fully utilizes the geometric constraint from the camera focal length setting; and we pre-calibrate the system under different focal distance to reconstruct 3D shape from unwrapped phase map. To validate the proposed idea, we developed a prototype system that can perform high-quality measurement for the depth range of approximately 1,000 mm (400 mm - 1400 mm) with the measurement error of 0.05%. Furthermore, we demonstrated that such a technique can be used for real-time 3D shape measurement by experimentally measuring moving objects.

Enhanced phase-coding method for three-dimensional shape measurement with half-period codeword

The phase-coding method has been widely used for 3D shape measurement, which uses sinusoidal phase-shifting patterns to recover the wrapped phase and the stair phase-coding patterns to determine the fringe order. However, due to random noises and image blurring, the fringe order is always misaligned with the wrapped phase, which will lead to fringe order errors. This paper presents an enhanced phase-coding method to address this misalignment problem by using half-period codewords, in which each codeword is aligned to the half-period of the sinusoidal patterns. Then, two complementary fringe orders with half-period dislocation can be calculated, which can effectively eliminate the fringe order errors. To extend the coding range of stair phase, this paper further develops a computational scheme based on the geometric constraint method. Simulations and experiments have been carried out, and their results confirm that the enhanced method can reliably recover the 3D shape of the measured objects.

Motion-induced error reduction for binary defocusing profilometry via additional temporal sampling

The recently proposed binary defocusing technique has brought speed breakthroughs for three-dimensional (3D) shape measurement with a digital fringe projection system. Despite this, motion-induced phase error is still inevitable due to the multi-shot nature of the phase-shifting algorithm. To alleviate this problem, this paper proposes a motion-induced error reduction method by taking advantage of additional temporal sampling. Particularly, each illuminated fringe pattern will be captured twice in one projection cycle, resulting in two sets of phase shifted fringe images being obtained. Due to the mechanism of binary defocusing projection, the motion-induced phase error could be effectively separated from the fixed phase shift value by evaluating the difference between the two phase maps. Based on this, an iterative compensation strategy is further applied to compensate the phase error until high-quality phase maps are generated. Meanwhile, different synchronization schemes are also proposed and tested to evaluate the error compensation effects. Both simulation and experiments demonstrated that the proposed methods can substantially reduce motion-introduced measurement errors. Since defocused 1-bit binary patterns are utilized to bypass rigid camera-projector synchronization, this method has potential for high-speed applications.

Stereo calibration with absolute phase target

Stereo cameras have been widely used for three-dimensional (3D) photogrammetry, and stereo calibration is a crucial process to estimate the intrinsic and extrinsic parameters. This paper proposes a stereo calibration method with absolute phase target by using horizontal and vertical phase-shifting fringes. The one-to-one mapping from the world points to the image points that can be recovered by referring to the absolute phase and then used to calibrate the stereo cameras. Compared with traditional methods that only use feature points within the overlapping field-of-view (FOV), the proposed method can use all feature points within the overlapping and non-overlapping FOVs. Besides, since phase is more robust against camera defocusing than intensity, the target images can be captured regardless of the depth-of-field (DOF). With the advantages of whole-field capability and defocusing tolerability, the target placement becomes very flexible. Both simulations and experiment results demonstrate the robustness and accuracy of the proposed method.

Automatic look-up table based real-time phase unwrapping for phase measuring profilometry and optimal reference frequency selection

For temporal phase unwrapping in phase measuring profilometry, it has recently been reported that two phases with co-prime frequencies can be absolutely unwrapped using a look-up table; however, frequency selection and table construction has been performed manually without optimization. In this paper, a universal phase unwrapping method is proposed to unwrap phase flexibly and automatically by using geometric analysis, and thus we can programmatically build a one-dimensional or two-dimensional look-up table for arbitrary two co-prime frequencies to correctly unwrap phases in real time. Moreover, a phase error model related to the defocus effect is derived to figure out an optimal reference frequency co-prime to the principal frequency. Experimental results verify the correctness and computational efficiency of the proposed method.

Development of an Active High-Speed 3-D Vision System

High-speed recognition of the shape of a target object is indispensable for robots to perform various kinds of dexterous tasks in real time. In this paper, we propose a high-speed 3-D sensing system with active target-tracking. The system consists of a high-speed camera head and a high-speed projector, which are mounted on a two-axis active vision system. By measuring a projected coded pattern, 3-D measurement at a rate of 500 fps was achieved. The measurement range was increased as a result of the active tracking, and the shape of the target was accurately observed even when it moved quickly. In addition, to obtain the position and orientation of the target, 500 fps real-time model matching was achieved.

Efficient phase retrieval of two-directional phase-shifting fringe patterns using geometric constraints of deflectometry

In the phase measuring deflectometry, two groups of fringe patterns in orthogonal directions are usually applied to establish the correspondences between the pixel pairs on the screen and camera. Usually, 16 phase-shifting fringe patterns with different spatial frequencies are required in order to calculate the absolute phases in the conventional temporal phase unwrapping algorithms. This requirement makes the measurement inefficient and not robust against environmental noise. In this paper, an efficient phase retrieval strategy is developed, which requires only six fringe patterns. The modulating information in one-direction is obtained by first using four fringe patterns, and then it is applied to assist the phase calculation in the other direction, so that only two extra fringe patterns are needed. Subsequently the phases are unwrapped by using the geometric constraints of the software configurable optical test system without additional image acquisition. The measurement time is saved by 5/8, compared to the conventional methods. In this way, the influence of the low-frequency disturbances can be suppressed in the workshop condition. Experiments demonstrate that the proposed method can reliably retrieve the absolute phases, and it is of significance to improve the measuring efficiency and stability of in situ deflectometry.

Absolute Phase Retrieval Using One Coded Pattern and Geometric Constraints of Fringe Projection System

Fringe projection technologies have been widely used for three-dimensional (3D) shape measurement. One of the critical issues is absolute phase recovery, especially for measuring multiple isolated objects. This paper proposes a method for absolute phase retrieval using only one coded pattern. A total of four patterns including one coded pattern and three phase-shift patterns are projected, captured, and processed. The wrapped phase, as well as average intensity and intensity modulation, are calculated from three phase-shift patterns. A code word encrypted into the coded pattern can be calculated using the average intensity and intensity modulation. Based on geometric constraints of fringe projection system, the minimum fringe order map can be created, upon which the fringe order can be calculated from the code word. Compared with the conventional method, the measurement depth range is significantly improved. Finally, the wrapped phase can be unwrapped for absolute phase map. Since only four patterns are required, the proposed method is suitable for real-time measurement. Simulations and experiments have been conducted, and their results have verified the proposed method.

Binarized dual phase-shifting method for high-quality 3D shape measurement

It has been demonstrated that using 1-bit binary patterns is better than using 8-bit sinusoidal patterns for high-speed applications. However, the phase quality generated from binary patterns is lower, especially when the projector is nearly focused. A dual phase-shifting method can effectively reduce the dominant periodical phase errors caused by high-order harmonics. Yet, such method requires an additional set of phase-shifting patterns, which thus slows down the measurement speed. To overcome this problem, this paper proposes a binarized dual phase-shifting method (BDPS) to generate a high-quality phase map using just three phase-shifted binary patterns. The basic idea is to merge the two sets of squared binary phase-shifting patterns into one set of patterns with three gray levels, which is further binarized by employing 2D area modulation. Given this, the BDPS method can realize error reduction for the binary defocusing technique while maintaining its speed advantage by neither adding the total number of patterns nor the number of bits. The effectiveness of the proposed method is verified by both a quantitative evaluation by measuring a white flat board and an additional quantitative evaluation by measuring a complex sculpture surface.

Double-pattern triangular pulse width modulation technique for high-accuracy high-speed 3D shape measurement

Using 1-bit binary patterns for three-dimensional (3D) shape measurement has been demonstrated as being advantageous over using 8-bit sinusoidal patterns in terms of achievable speeds. However, the phase quality generated by binary pattern(s) typically are not high if only a small number of phase-shifted patterns are used. This paper proposes a method to improve the phase quality by representing each pattern with the difference of two binary patterns: the first binary pattern is generated by triangular pulse width modulation (TPWM) technique, and the second being π shifted from the first pattern that is also generated by TPWM technique. The phase is retrieved by applying a three-step phase-shifting algorithm to the difference patterns. Through optimizing the modulation frequency of the triangular carrier signal, we demonstrate that a high-quality phase can be generated for a wide range of fringe periods (e.g., from 18 to 1140 pixels) with only six binary patterns. Since only 1-bit binary patterns are required for 3D shape measurement, this paper will present a real-time 3D shape measurement system that can achieve 30 Hz.

Absolute three-dimensional shape measurement with two-frequency square binary patterns

This paper presents a novel method to achieve absolute three-dimensional shape measurement solely using square binary patterns. This method uses six patterns: three low-frequency phase-shifted patterns and three phase-shifted high-frequency patterns. The phase obtained from the low-frequency phase temporally unwraps the phase obtained from high-frequency patterns. The projector is defocused such that the high-frequency patterns produce a high-quality phase, but the phase retrieved from low-frequency patterns has a large harmonic error that fails the two-frequency temporal phase unwrapping process. In this paper, we develop a computational framework to address the challenge. The proposed computational framework includes four major approaches to alleviate the harmonic error problem: (i) use more than one period of low-frequency patterns enabled by a geometric constraint-based phase unwrapping method; (ii) artificially apply a large Gaussian filter to low-frequency patterns before phase computation; (iii) create an error lookup table to compensate for harmonic error; and (iv) develop a boundary error correction method to alleviate problems associated with filtering. Both simulation and experimental results demonstrated the success of the proposed method.

Superfast high-resolution absolute 3D recovery of a stabilized flapping flight process

Scientific research of a stabilized flapping flight process (e.g. hovering) has been of great interest to a variety of fields including biology, aerodynamics, and bio-inspired robotics. Different from the current passive photogrammetry based methods, the digital fringe projection (DFP) technique has the capability of performing dense superfast (e.g. kHz) 3D topological reconstructions with the projection of defocused binary patterns, yet it is still a challenge to measure a flapping flight process with the presence of rapid flapping wings. This paper presents a novel absolute 3D reconstruction method for a stabilized flapping flight process. Essentially, the slow motion parts (e.g. body) and the fast-motion parts (e.g. wings) are segmented and separately reconstructed with phase shifting techniques and the Fourier transform, respectively. The topological relations between the wings and the body are utilized to ensure absolute 3D reconstruction. Experiments demonstrate the success of our computational framework by testing a flapping wing robot at different flapping speeds.

Three dimensional range geometry and texture data compression with space-filling curves

This paper presents a novel method to effectively store three-dimensional (3D) data and 2D texture data into a regular 24-bit image. The proposed method uses the Hilbert space-filling curve to map the normalized unwrapped phase map to two 8-bit color channels, and saves the third color channel for 2D texture storage. By further leveraging existing 2D image and video compression techniques, the proposed method can achieve high compression ratios while effectively preserving data quality. Since the encoding and decoding processes can be applied to most of the current 2D media platforms, this proposed compression method can make 3D data storage and transmission available for many electrical devices without requiring special hardware changes. Experiments demonstrate that if a lossless 2D image/video format is used, both original 3D geometry and 2D color texture can be accurately recovered; if lossy image/video compression is used, only black-and-white or grayscale texture can be properly recovered, but much higher compression ratios (e.g., 1543:1 against the ASCII OBJ format) are achieved with slight loss of 3D geometry quality.

Ternary Gray code-based phase unwrapping for 3D measurement using binary patterns with projector defocusing

The three-dimensional measurement technique using binary pattern projection with projector defocusing has become increasingly important due to its high speed and high accuracy. To obtain even faster speed without sacrificing accuracy, a ternary Gray code-based phase-unwrapping method is proposed by using even fewer binary patterns, which makes it possible to efficiently and accurately unwrap the phase. Theoretical analysis, simulations, and experiments are presented to validate the proposed method's efficiency and robustness.

Absolute three-dimensional shape measurement with a known object

This paper presents a novel method for absolute three-dimensional (3D) shape measurement that does not require conventional temporal phase unwrapping. Our proposed method uses a known object (i.e., a ping-pong ball) to provide cues for absolute phase unwrapping. During the measurement, the ping-pong ball is positioned to be close to the nearest point from the scene to the camera. We first segment ping-pong ball and spatially unwrap its phase, and then determine the integer multiple of 2π to be added such that the recovered shape matches its actual geometry. The nearest point of the ball provides zmin to generate the minimum phase Φmin that is then used to unwrap phase of the entire scene pixel by pixel. Experiments demonstrated that only three phase-shifted fringe patterns are required to measure absolute shapes of objects moving along depth z direction.

Phase-shifting profilometry combined with Gray-code patterns projection: unwrapping error removal by an adaptive median filter

Phase-shifting profilometry combined with Gray-code patterns projection has been widely used for 3D measurement. In this technique, a phase-shifting algorithm is used to calculate the wrapped phase, and a set of Gray-code binary patterns is used to determine the unwrapped phase. In the real measurement, the captured Gray-code patterns are no longer binary, resulting in phase unwrapping errors at a large number of erroneous pixels. Although this problem has been attended and well resolved by a few methods, it remains challenging when a measured object has step-heights and the captured patterns contain invalid pixels. To effectively remove unwrapping errors and simultaneously preserve step-heights, in this paper, an effective method using an adaptive median filter is proposed. Both simulations and experiments can demonstrate its effectiveness.

Multi-subzone algorithm for absolute phase retrieval in digital fringe projection profilometry

Codewords are important in encoded absolute phase retrieval techniques such as two-frequency, gray-code, and phase-coding. Each sinusoidal fringe is marked by a unique codeword so that an absolute fringe order can be determined by decoding the codeword. However, due to the limited number of unique codewords, sinusoidal fringe patterns do not contain high-frequency fringes without the use of additional patterns. A multi-subzone coding and decoding algorithm is thus proposed to overcome this limitation. Three multi-subzone coding methods based on two-frequency, gray-code, and phase-coding techniques are presented. The coding creates multiple subzones of unique codewords and the decoding enables it to use non-unique codewords to identify absolute fringe order. Specifically, the range of fringe order is estimated by the use of a wrapped phase map and the absolute fringe order is identified by a codeword. Experimental studies demonstrate the advantages of the proposed algorithm over existing coding methods. The proposed algorithm is suitable to measure objects with large step-height surface discontinuities.

Motion-induced error reduction by combining Fourier transform profilometry with phase-shifting profilometry

We propose a hybrid computational framework to reduce motion-induced measurement error by combining the Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP). The proposed method is composed of three major steps: Step 1 is to extract continuous relative phase maps for each isolated object with single-shot FTP method and spatial phase unwrapping; Step 2 is to obtain an absolute phase map of the entire scene using PSP method, albeit motion-induced errors exist on the extracted absolute phase map; and Step 3 is to shift the continuous relative phase maps from Step 1 to generate final absolute phase maps for each isolated object by referring to the absolute phase map with error from Step 2. Experiments demonstrate the success of the proposed computational framework for measuring multiple isolated rapidly moving objects.

High-resolution, real-time simultaneous 3D surface geometry and temperature measurement

This paper presents a method to simultaneously measure three-dimensional (3D) surface geometry and temperature in real time. Specifically, we developed 1) a holistic approach to calibrate both a structured light system and a thermal camera under exactly the same world coordinate system even though these two sensors do not share the same wavelength; and 2) a computational framework to determine the sub-pixel corresponding temperature for each 3D point as well as discard those occluded points. Since the thermal 2D imaging and 3D visible imaging systems do not share the same spectrum of light, they can perform sensing simultaneously in real time: we developed a hardware system that can achieve real-time 3D geometry and temperature measurement at 26 Hz with 768 × 960 points per frame.