Frequency-multiplex Fourier-transform profilometry: a single-shot three-dimensional shape measurement of objects with large height discontinuities and/or surface isolations

Real-time 3D shape measurement of dynamic scenes using fringe projection profilometry: lightweight NAS-optimized dual frequency deep learning approach

Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.

Unifying temporal phase unwrapping framework using deep learning

Temporal phase unwrapping (TPU) is significant for recovering an unambiguous phase of discontinuous surfaces or spatially isolated objects in fringe projection profilometry. Generally, temporal phase unwrapping algorithms can be classified into three groups: the multi-frequency (hierarchical) approach, the multi-wavelength (heterodyne) approach, and the number-theoretic approach. For all of them, extra fringe patterns of different spatial frequencies are required for retrieving the absolute phase. Due to the influence of image noise, people have to use many auxiliary patterns for high-accuracy phase unwrapping. Consequently, image noise limits the efficiency and the measurement speed greatly. Further, these three groups of TPU algorithms have their own theories and are usually applied in different ways. In this work, for the first time to our knowledge, we show that a generalized framework using deep learning can be developed to perform the TPU task for different groups of TPU algorithms. Experimental results show that benefiting from the assistance of deep learning the proposed framework can mitigate the impact of noise effectively and enhance the phase unwrapping reliability significantly without increasing the number of auxiliary patterns for different TPU approaches. We believe that the proposed method demonstrates great potential for developing powerful and reliable phase retrieval techniques.

In-motion 3D reconstruction of high dynamic range surfaces

Efficient and refined three-dimensional (3D) reconstruction of industrial parts has become an urgent need in the field of advanced manufacturing, and it's a great challenge when facing in-motion and online inspection requirements of high dynamic range (HDR) surfaces that have large reflectivity variations. This paper proposes a method using RGB line-scan cameras to realize in-motion multiple-shot 3D shape measurements with RGB channel fusion to increase the measurement dynamic range. First, multi-channel one-dimensional background-normalized Fourier transform profilometry (MC-1DBNFTP) is proposed as an effective in-motion HDR method. Second, for HDR surfaces with strongly overexposed areas, we propose a solution that obtains 6 results of different dynamic ranges for fusion with only 5 projected patterns, which further extends the measurement dynamic range while ensuring the small projection period. Third, we develop a fusion method based on reliability evaluation, which is more reliable than the existing methods in fringe projection systems. In addition, colored textures can be mapped to the reconstructed surfaces. Experimental results prove that the proposed method realizes accurate and reliable in-motion 3D reconstruction of HDR surfaces.

Single-Shot Multi-Frequency 3D Shape Measurement for Discontinuous Surface Object Based on Deep Learning

It is challenging to stably and rapidly achieve accurate absolute phase retrieval for isolated objects with a single-shot pattern in fringe projection profilometry (FPP). In this context, a single-shot multi-frequency absolute phase retrieval (SAPR-DL) method based on deep learning is proposed, which only needs to capture one fringe image to obtain the full-field precise absolute phase. Specifically, a low-frequency deformed fringe image is loaded into the trained one-to-two deep learning framework (DLFT) to predict unit-frequency and high-frequency deformed fringe images. Then, three fringe images with different frequencies are loaded into the trained deep learning phase retrieval framework (DLPR) to calculate the corresponding absolute phase. The experimental results prove that the proposed SAPR-DL method can obtain the three-dimensional (3D) shape measurement of multiple complex objects by collecting a single-shot fringe image, showing great prospects in advancing scientific and engineering applications.

High-speed three-dimensional shape measurement based on tripartite complementary Gray-coded light

In phase-shifting profilometry based on the Gray code, the jump error is inevitably generated and is further amplified in dynamic scenes. To tackle this problem, we propose the robust tripartite complementary Gray code method (TCG). Without projecting additional patterns, TCG uses different combinations of Gray code to calculate three complementary orders able to avoid jump error in the unwrapping process. TCG is efficient and robust, as it fully utilizes the redundant information of the Gray code. Experimental results demonstrate that TCG can realize high-efficiency and high-speed three-dimensional shape measurement at a rate of 500 fps.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

Composite fringe projection deep learning profilometry for single-shot absolute 3D shape measurement

Single-shot fringe projection profilometry (FPP) is essential for retrieving the absolute depth information of the objects in high-speed dynamic scenes. High-precision 3D reconstruction using only one single pattern has become the ultimate goal in FPP. The frequency-multiplexing (FM) method is a promising strategy for realizing single-shot absolute 3D measurement by compounding multi-frequency fringe information for phase unwrapping. In order to solve the problem of serious spectrum aliasing caused by multiplexing schemes that cannot be removed by traditional spectrum analysis algorithms, we apply deep learning to frequency multiplexing composite fringe projection and propose a composite fringe projection deep learning profilometry (CDLP). By combining physical model and data-driven approaches, we demonstrate that the model generated by training an improved deep convolutional neural network can directly perform high-precision and unambiguous phase retrieval on a single-shot spatial frequency multiplexing composite fringe image. Experiments on both static and dynamic scenes demonstrate that our method can retrieve robust and unambiguous phases information while avoiding spectrum aliasing and reconstruct high-quality absolute 3D surfaces of objects only by projecting a single composite fringe image.

Single-shot 4-step phase-shifting multispectral fringe projection profilometry

Phase-shifting profilometry (PSP) is considered to be the most accurate technique for phase retrieval with fringe projection profilometry (FPP) systems. However, PSP requires that multiple phase-shifted fringe patterns be acquired, usually sequentially, which has limited PSP to static or quasi-static imaging. In this paper, we introduce multispectral 4-step phase-shifting FPP that provides 3D imaging using a single acquisition. The method enables real-time profilometry applications. A single frame provides all four phase-shifted fringe patterns needed for the PSP phase retrieval algorithm. The multispectral nature of the system ensures that light does not leak between the spectral bands, which is a common problem in simultaneous phase-shifting with color cameras. With the use of this new concept, custom composite patterns containing multiple patterns can be acquired with a single acquisition.

Spatial-temporal phase unwrapping algorithm for fringe projection profilometry

In this paper, a generalized spatial-temporal phase unwrapping algorithm (STPUA) is proposed for extracting the absolute phase of the isolated objects with intricate surfaces. This proposed algorithm can eliminate thoroughly the order jumps of various temporal phase unwrapping algorithms (TPUAs), while inheriting the high measuring accuracy of quality-guided phase unwrapping algorithms (QGPUAs). Differing from the traditional phase unwrapping algorithms, wrapped phase is first divided into several regional wrapped phases, which can be extracted successively according to its areas and unwrapped individually by QGPUAs. Meanwhile, a series of reliable points from the fringe order map obtained from the code deformed patterns are selected to map the corresponding regional unwrapped phases into an absolute phase. The radii of selecting reliable points can provide the high measuring robustness compared with the classical point-to-point TPUAs for the complex surfaces and the motion blur, while keeping the same number of patterns. Therefore, the proposed STPUA combining SPUAs and TPUAs also can be employed in real-time three-dimensional (3D) reconstruction. Theoretical analysis and experimental results are performed to verify the effectiveness and capability of the proposed algorithm.

Single-shot detection of 8 unique monochrome fringe patterns representing 4 distinct directions via multispectral fringe projection profilometry

Spatial resolution in three-dimensional fringe projection profilometry is determined in large part by the number and spacing of fringes projected onto an object. Due to the intensity-based nature of fringe projection profilometry, fringe patterns must be generated in succession, which is time-consuming. As a result, the surface features of highly dynamic objects are difficult to measure. Here, we introduce multispectral fringe projection profilometry, a novel method that utilizes multispectral illumination to project a multispectral fringe pattern onto an object combined with a multispectral camera to detect the deformation of the fringe patterns due to the object. The multispectral camera enables the detection of 8 unique monochrome fringe patterns representing 4 distinct directions in a single snapshot. Furthermore, for each direction, the camera detects two π-phase shifted fringe patterns. Each pair of fringe patterns can be differenced to generate a differential fringe pattern that corrects for illumination offsets and mitigates the effects of glare from highly reflective surfaces. The new multispectral method solves many practical problems related to conventional fringe projection profilometry and doubles the effective spatial resolution. The method is suitable for high-quality fast 3D profilometry at video frame rates.

Single-shot 3D shape measurement of discontinuous objects based on a coaxial fringe projection system

Fringe projection profilometry has been widely used in high-speed three-dimensional (3D) shape measurement. To improve the speed without loss of accuracy, we present a novel single-shot 3D shape measuring system that utilizes a coaxial fringe projection system and a 2CCD camera. The coaxial fringe projection system, comprising a visible light (red, green, and blue) projector and an infrared (IR) light projector, can simultaneously project red, green, blue, and IR fringe patterns. The 2CCD camera, as the name suggests, has two CCD chips that can acquire visible and IR fringe patterns at the same time. Combining the two-step phase-shifting algorithm, Fourier transform profilometry, and the optimum three-frequency selection method, 3D shape measurement of complex surfaces such as large slopes or discontinuous objects can be obtained from single-shot acquisition. A virtual fringe projection measurement system has been established to generate pre-deformed fringe patterns to correct positional deviations of the coaxial fringe projection system. This method has been applied to simulations and experiments on static and dynamic objects with promising results.

A simple and practical jump error removal method for fringe projection profilometry based on self-alignment technique

The code-based method is one of the frequently adopted fringe projection profilometry techniques because of its robustness and high speed. However, the abnormal jump errors caused by the misalignment between the wrapped phase and the fringe order impact the phase unwrapping quality and are more serious in binary defocusing measurement with significant defocusing. This paper proposes a self-alignment technique (SAT) with high speed, no additional patterns, and no accuracy loss to eliminate such jump errors. After analyzing the relation between the grating changes and the misalignment, we designed an assist-code that can perfectly align with the wrapped phase even in significant defocusing. After that, this assist-code is used to adjust the misalignment and eliminate the jump errors. The comparison between the median filter and the proposed SAT method in simulations and experiments demonstrates that the proposed SAT method has the ability to completely remove jump errors for complex textures and step-height objects, even in significant defocusing, indicating its potential to be applied to other phase unwrapping fields, e.g., color-based methods with color coupling problems.

GOBO projection for 3D measurements at highest frame rates: a performance analysis

Aperiodic sinusoidal patterns that are cast by a GOBO (GOes Before Optics) projector are a powerful tool for optically measuring the surface topography of moving or deforming objects with very high speed and accuracy. We optimised the first experimental setup that we were able to measure inflating car airbags at frame rates of more than 50 kHz while achieving a 3D point standard deviation of ~500 µm. Here, we theoretically investigate the method of GOBO projection of aperiodic sinusoidal fringes. In a simulation-based performance analysis, we examine the parameters that influence the accuracy of the measurement result and identify an optimal pattern design that yields the highest measurement accuracy. We compare the results with those that were obtained via GOBO projection of phase-shifted sinusoidal fringes. Finally, we experimentally verify the theoretical findings. We show that the proposed technique has several advantages over conventional fringe projection techniques, as the easy-to-build and cost-effective GOBO projector can provide a high radiant flux, allows high frame rates, and can be used over a wide spectral range.

High-speed image-reconstruction algorithm for a spatially multiplexed image and application to digital holography

We propose a high-speed image-reconstruction algorithm for a spatially multiplexed image that is obtained by spatial frequency-division multiplexing. The algorithm utilizes smoothing and does not require any Fourier transform (FT) or iterative procedure to extract the desired information selectively from a single image. Numerical and experimental results show its validity and color holographic imaging ability. Calculation time of the proposed is less than a tenth of that of the FT method when using a central processing unit and an image sensor with four megapixels. Furthermore, throughput that is three times that of the FT method can be achieved in the case of using a graphics processing unit.

Online 3D Displacement Measurement Using Speckle Interferometer with a Single Illumination-Detection Path

Measurement systems for online nondestructive full-field three-dimensional (3D) displacement based on the single-shot and multiplexing techniques attract more and more interest, especially throughout the manufacturing industries. This paper proposes an accurate and easy-to-implement method based on an electronic speckle pattern interferometer (ESPI) with single illumination-detection path to realize the online nondestructive full-field 3D displacement measurement. The simple and compact optical system generates three different sensitivity vectors to enable the evaluation of the three orthogonal displacement components. By applying the spatial carrier phase-shifting technique, the desired information can be obtained in real time. The theoretical analysis and the measurement results have proven the feasibility of this ESPI system and quantified its relative measurement error.

One-shot profile inspection for surfaces with depth, color and reflectivity discontinuities

A one-shot technique for surfaces with depth, color, and reflectivity discontinuities is presented. It uses windowed Fourier transform to extract the fringe phases and a binary-encoded scheme to unwrap the phases. Experiments show that absolute phases could be obtained with high reliability.

Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system

In recent years, fringe projection has become an established and essential method for dynamic three-dimensional (3-D) shape measurement in different fields such as online inspection and real-time quality control. Numerous high-speed 3-D shape measurement methods have been developed by either employing high-speed hardware, minimizing the number of pattern projection, or both. However, dynamic 3-D shape measurement of arbitrarily-shaped objects with full sensor resolution without the necessity of additional pattern projections is still a big challenge. In this work, we introduce a high-speed 3-D shape measurement technique based on composite phase-shifting fringes and a multi-view system. The geometry constraint is adopted to search the corresponding points independently without additional images. Meanwhile, by analysing the 3-D position and the main wrapped phase of the corresponding point, pairs with an incorrect 3-D position or a considerable phase difference are effectively rejected. All of the qualified corresponding points are then corrected, and the unique one as well as the related period order is selected through the embedded triangular wave. Finally, considering that some points can only be captured by one of the cameras due to the occlusions, these points may have different fringe orders in the two views, so a left-right consistency check is employed to eliminate those erroneous period orders in this case. Several experiments on both static and dynamic scenes are performed, verifying that our method can achieve a speed of 120 frames per second (fps) with 25-period fringe patterns for fast, dense, and accurate 3-D measurement.

Pixel super-resolution using wavelength scanning

Undersampling and pixelation affect a number of imaging systems, limiting the resolution of the acquired images, which becomes particularly significant for wide-field microscopy applications. Various super-resolution techniques have been implemented to mitigate this resolution loss by utilizing sub-pixel displacements in the imaging system, achieved, for example, by shifting the illumination source, the sensor array and/or the sample, followed by digital synthesis of a smaller effective pixel by merging these sub-pixel-shifted low-resolution images. Herein, we introduce a new pixel super-resolution method that is based on wavelength scanning and demonstrate that as an alternative to physical shifting/displacements, wavelength diversity can be used to boost the resolution of a wide-field imaging system and significantly increase its space-bandwidth product. We confirmed the effectiveness of this new technique by improving the resolution of lens-free as well as lens-based microscopy systems and developed an iterative algorithm to generate high-resolution reconstructions of a specimen using undersampled diffraction patterns recorded at a few wavelengths covering a narrow spectrum (10-30 nm). When combined with a synthetic-aperture-based diffraction imaging technique, this wavelength-scanning super-resolution approach can achieve a half-pitch resolution of 250 nm, corresponding to a numerical aperture of ~1.0, across a large field of view (>20 mm²). We also demonstrated the effectiveness of this approach by imaging various biological samples, including blood and Papanicolaou smears. Compared with displacement-based super-resolution techniques, wavelength scanning brings uniform resolution improvement in all directions across a sensor array and requires significantly fewer measurements. This technique would broadly benefit wide-field imaging applications that demand larger space-bandwidth products.

Photorefractive moiré-like patterns for the multifringe projection method in Fourier transform profilometry

In the present paper, the method of simultaneous moiré-like fringe pattern projection for Fourier transform profilometry is described. The photorefractive holographic interferometric process produces controlled moiré-like patterns with two or more different variation directions. Each low spatial frequency fringe pattern is experimentally obtained as a result of the superposition of two high spatial frequency sinusoidal gratings, with slightly different pitches, for each fringe variation direction. These dynamic moiré-like patterns are induced due to an optical holographic beating of the sinusoidal induced gratings in the volume of the photorefractive Bi₁₂TiO₂₀ (BTO) crystal sample used as dynamic holographic medium. Two or more moiré-like fringe patterns, with at least two different variation directions, simultaneously (or not), are projected onto the object surface. Thus, this is the 2D fringe projection stage of our proposed Fourier transform procedure to determine the profile of a simple object.

Fringe order error in multifrequency fringe projection phase unwrapping: reason and correction

A multifrequency fringe projection phase unwrapping algorithm (MFPPUA) is important to fringe projection profilometry, especially when a discontinuous object is measured. However, a fringe order error (FOE) may occur when MFPPUA is adopted. An FOE will result in error to the unwrapped phase. Although this kind of phase error does not spread, it brings error to the eventual 3D measurement results. Therefore, an FOE or its adverse influence should be obviated. In this paper, reasons for the occurrence of an FOE are theoretically analyzed and experimentally explored. Methods to correct the phase error caused by an FOE are proposed. Experimental results demonstrate that the proposed methods are valid in eliminating the adverse influence of an FOE.

Three-dimensional trace measurements for fast-moving objects using binary-encoded fringe projection techniques

A fringe projection technique to trace the shape of a fast-moving object is proposed. A binary-encoded fringe pattern is illuminated by a strobe lamp and then projected onto the moving object at a sequence of time. Phases of the projected fringes obtained from the sequent measurements are extracted by the Fourier transform method. Unwrapping is then performed with reference to the binary-encoded fringe pattern. Even though the inspected object is colorful, fringe orders can be identified. A stream of profiles is therefore retrieved from the sequent unwrapped phases. This makes it possible to analyze physical properties of the dynamic objects. Advantages of the binary-encoded fringe pattern for phase unwrapping also include (1) reliable performance for colorful objects, spatially isolated objects, and surfaces with large depth discontinuities; (2) unwrapped errors only confined in a local area; and (3) low computation cost.

Fringe-projection profilometry based on two-dimensional empirical mode decomposition

In 3D shape measurement, because deformed fringes often contain low-frequency information degraded with random noise and background intensity information, a new fringe-projection profilometry is proposed based on 2D empirical mode decomposition (2D-EMD). The fringe pattern is first decomposed into numbers of intrinsic mode functions by 2D-EMD. Because the method has partial noise reduction, the background components can be removed to obtain the fundamental components needed to perform Hilbert transformation to retrieve the phase information. The 2D-EMD can effectively extract the modulation phase of a single direction fringe and an inclined fringe pattern because it is a full 2D analysis method and considers the relationship between adjacent lines of a fringe patterns. In addition, as the method does not add noise repeatedly, as does ensemble EMD, the data processing time is shortened. Computer simulations and experiments prove the feasibility of this method.

Method of excess fractions with application to absolute distance metrology: analytical solution

Multiwavelength interferometry provides a solution to a number of applications in metrology for the measurement of optical path differences longer than the source wavelength. To this day, the method of excess fractions (EF) has proved to provide very long, unambiguous measurement ranges with the highest reliability for a given set of wavelengths and level of phase noise. This is achieved because EF combines the individual phase values in an equivalent least-square problem and evaluates the correspondence for all possible solutions. However, this procedure can be slow for a number of applications. In this paper, an analytical solution for EF is presented that allows the direct calculation of the unknown integer fringe order. It is shown that this solution is consistent with the other phase unwrapping approaches as beat wavelength or Chinese remainder theorem-based solutions, but moreover, it can be understood as a unified representation and solution of the fringe order problem.

Computerized interferometric surface measurements [Invited]

The addition of electronics, computers, and software to interferometry has enabled enormous improvements in optical metrology. This paper discusses four areas in which computerized interferometric measurement improvements have been made in the measurement of surface shape and surface roughness: (a) The use of computer-generated holograms for the testing of aspheric optics, (b) phase-shifting interferometry for getting interferometric data into a computer so the data can be analyzed, (c) computerized interference microscopes, including multiple-wavelength and coherence scanning, for the precision measurement of surface microstructure, and (d) vibration-insensitive dynamic interferometers for enabling precise measurements in noncontrolled environments.

Single-shot dual-illumination phase unwrapping using a single wavelength

We propose a single-shot phase unwrapping technique using a single wavelength. In the proposed technique, an object is illuminated by two laser beams, which are emitted from the same laser, whose illumination angles and polarizations are different. Then two types of the object waves generated by the two beams are separately and simultaneously recorded by a polarization imaging camera. We conducted an experiment and a 2.5 mm height object, which was 9400 times height of the wavelength of the laser, was reconstructed without wrapping.

Color fringe-projected technique for measuring dynamic objects based on bidimensional empirical mode decomposition

A triple-frequency color fringe-projected technique is presented to measure dynamic objects. Three fringe patterns with a carrier frequency ratio of 1:3:9 are encoded in red, green, and blue channels of a color fringe pattern and projected onto an object's surface. Bidimensional empirical mode decomposition is used for decoupling the cross talk among color channels and for extracting the fundamental frequency components of the three fringe patterns. The unwrapped phase distribution of the high-frequency fringe is retrieved by a three-step phase unwrapping strategy to recover the object's height distribution. Owing to its use of only a single snapshot, the technique is suitable for measuring dynamically changing objects with large discontinuity or spatially isolated surfaces.

Single-shot color fringe projection for three-dimensional shape measurement of objects with discontinuities

A simple but effective fringe projection profilometry is proposed to measure 3D shape by using one snapshot color sinusoidal fringe pattern. One color fringe pattern encoded with a sinusoidal fringe (as red component) and one uniform intensity pattern (as blue component) is projected by a digital video projector, and the deformed fringe pattern is recorded by a color CCD camera. The captured color fringe pattern is separated into its RGB components and division operation is applied to red and blue channels to reduce the variable reflection intensity. Shape information of the tested object is decoded by applying an arcsine algorithm on the normalized fringe pattern with subpixel resolution. In the case of fringe discontinuities caused by height steps, or spatially isolated surfaces, the separated blue component is binarized and used for correcting the phase demodulation. A simple and robust method is also introduced to compensate for nonlinear intensity response of the digital video projector. The experimental results demonstrate the validity of the proposed method.

Real-time three-dimensional shape measurement of moving objects without edge errors by time-synchronized structured illumination

Structured-light illumination is a process of three-dimensional imaging where a series of time-multiplexed, striped patterns are projected onto a target scene with the corresponding captured images used to determine surface shape according to the warping of the projected patterns around the target. In a real-time system, a high-speed projector/camera pair is used such that any surface motion is small over the projected pattern sequence, but regardless of acquisition speed, there are always those pixels near the edge of a moving surface that capture the projected patterns on both fore- and background surfaces. These edge pixels then create unpredictable results that typically require expensive processing steps to remove, but in this Letter, we introduce a filtering process that identifies motion artifacts based upon the discrete Fourier transform applied to the time axis of the captured pattern sequence. The process is of very low computational complexity, and in this Letter, we demonstrate that in a real-time structured-light illumination (SLI) system, the process comes at a cost of 15 frames per second (fps), where our SLI system drops from 180 to 165 fps after deleting those edge pixels where motion was detected.

Three-dimensional profilometry using a Dammann grating

We propose three-dimensional (3D) profilometry based on a Fourier transform in which a two-dimensional (2D) Dammann grating and a cylindrical lens are used to generate structured light. The Dammann grating splits most of the illumination power into a 2D diffractive spot matrix. The cylindrical lens transforms these 2D diffractive spots into one-dimensional fringe lines that are projected on an object. The produced projection fringes have the advantages of high brightness and high contrast and compression ratios. The experiments have verified the proposed 3D profilometry. The 3D profilometry using Dammann grating should be of high interest for practical applications.

Profilometry without phase unwrapping using multi-frequency and four-step phase-shift sinusoidal fringe projection

A three-dimensional (3D) profilometry method without phase unwrapping is proposed. The key factors of the proposed profilometry are the use of composite projection of multi-frequency and four-step phase-shift sinusoidal fringes and its geometric analysis, which enable the proposed method to extract the depth information of even largely separated discontinuous objects as well as lumped continuous objects. In particular, the geometric analysis of the multi-frequency sinusoidal fringe projection identifies the shape and position of target objects in absolute coordinate system. In the paper, the depth extraction resolution of the proposed method is analyzed and experimental results are presented.

Absolute phase measurements using geometric constraints between multiple cameras and projectors

The method of phase shift with the projection of multiple cyclic patterns enables 3D measurement that is highly accurate, dense, and fast. However, this measurement is only possible for the wrapped phase value, which has ambiguities in its multiples of cycles. Two particular problems are that conventional methods require additional patterns to be projected to determine the absolute phase and that unwrapping the phase tends to fail where depth varies abruptly. Two methods are proposed: the first is to determine the absolute phase without additional patterns being projected by observing the projected pattern with multiple cameras and applying the geometric constraints between them, and the second is to prevent failure in unwrapping the phase by referring to continuities in the relative phases of multiple projected patterns. The proposed methods were achieved with a 3D scanner that can measure approximately a 180 degrees field of view within 0.5 s, with an accuracy of 0.14 mm in depth.

Acoustic grating fringe projector for high-speed and high-precision three-dimensional shape measurements

A new acoustic grating fringe projector (AGFP) was developed for high-speed and high-precision 3D measurement. A new acoustic grating fringe projection theory is also proposed to describe the optical system. The AGFP instrument can adjust the spatial phase and period of fringes with unprecedented speed and accuracy. Using rf power proportional-integral-derivative (PID) control and CCD synchronous control, we obtain fringes with fine sinusoidal characteristics and realize high-speed acquisition of image data. Using the device, we obtained a precise phase map for a 3D profile. In addition, the AGFP can work in running fringe mode, which could be applied in other measurement fields.

Three-dimensional vision from a multisensing mechanism

An approach for sensing a three-dimensional (3D) object surface with an arbitrary geometric shape is presented. Combining two different 3D sensing mechanisms, point-array encoding based on affine transformation and fringe encoding based on phase mapping, we construct a mathematic model for 3D vision in which the point-array encoding is initially applied to determine the fringe orders to create a control-vertex mesh with absolute coordinate values in 3D space. Then phase evaluation and phase unwrapping for fringe decoding is performed under the guidance of control-vertex mesh, leading to an absolute phase map and the corresponding range image of the test object. The computer simulations and experimental results are presented to demonstrate the theoretical prediction.

Three-dimensional digital imaging based on shifted point-array encoding

An approach to three-dimensional (3D) imaging based on shifted point-array encoding is presented. A kind of point-array structure light is projected sequentially onto the reference plane and onto the object surface to be tested and thus forms a pair of point-array images. A mathematical model is established to formulize the imaging process with the pair of point arrays. This formulation allows for a description of the relationship between the range image of the object surface and the lateral displacement of each point in the point-array image. Based on this model, one can reconstruct each 3D range image point by computing the lateral displacement of the corresponding point on the two point-array images. The encoded point array can be shifted digitally along both the lateral and the longitudinal directions step by step to achieve high spatial resolution. Experimental results show good agreement with the theoretical predictions. This method is applicable for implementing 3D imaging of object surfaces with complex topology or large height discontinuities.

Three-dimensional vision with dual acousto-optic deflection encoding

We report a new method of sensing a three-dimensional (3D) object surface with an arbitrary geometric shape. In this approach, the first-order beams diffracted from two acousto-optic deflectors (AODs) interfere with each other to form a spatial carrier that is used to encode the depth information from the 3D object surface. A direct digital synthesizer is utilized to control two AODs to generate sequentially spatial carriers with different spatial frequencies so that a modified temporal phase-unwrapping technique can be applied for decoding the shape information of the test surface. Preliminary experimental results are presented to demonstrate the effectiveness of this method.

Fourier fringe processing by use of an interpolated Fourier-transform technique

Recently a powerful Fourier transform technique was introduced that was able to extract the phase from only one image. However, because the method is based on the two-dimensional Fourier transform, it inherently suffers from leakage effects. A novel procedure is proposed that does not exhibit this distortion. The procedure uses localized information and estimates both the unknown frequencies and the phases of the fringe pattern (using an interpolated fast Fourier transform method). This allows us to demodulate the fringe pattern without any distortion. The proposed technique is validated on both computer simulations and the profile measurements of a tube.

Frequency-multiplexed profilometric phase coding for three-dimensional object recognition without 2pi pi phase ambiguity

A novel phase-coding technique based on Fourier-transform profilometry (FTP) is proposed for three-dimensional object recognition. Two spatially multiplexed grating patterns of different spatial frequencies are projected simultaneously onto the objects-target, and the phase changes in the distorted patterns are detected. An algebraic addition or subtraction of these phase values is utilized to code the two-dimensional plans of the objects-target with spatial harmonic modulations. The phase-coded plans of the objects and the target are cross correlated digitally to yield a high correlation peak at the target location. The 2pi ambiguity of the phase associated with FTP has been resolved in correlation results without recourse to phase unwrapping. Experimental results show an excellent discrimination capability for target recognition.

Fractional profilometry correlator for three-dimensional object recognition

A novel method for three-dimensional shape recognition is proposed. It combines the Fourier-transform-profilometry technique with a two-dimensional fractional correlation algorithm. A grating is projected onto the object surface, resulting in a distorted grating pattern that carries information about the height and the shape of the object. Three-dimensional objects are recognized by a fractional correlator by use of the transformed complex amplitude. An optoelectronic hybrid implementation is also suggested.

Absolute phase-measurement technique based on number theory in multifrequency grating projection profilometry

A simple and robust technique for absolute phase measurement based on number theory is presented. The new, to our knowledge, technique, which is compared with the Gushov-Solodkin algorithm, surmounts the shortcomings in the Gushov-Solodkin algorithm. The technique permits the three-dimensional shape measurement of objects that have discontinuous height steps and has resulted in a new and more powerful method of measuring surface absolute profile. Experimental results are presented that demonstrate the validity of the principle.

Fringe pattern correlator for three-dimensional object recognition

We propose a novel three-dimensional (3-D) object-recognition method based on a Fourier-transform profilometry technique and a two-dimensional (2-D) correlation technique. Height information on 3-D objects is transformed to phase information on 2-D complex amplitude by use of the Fourier-transform profilometry technique. 3-D objects are recognized using correlation by use of the transformed complex amplitude.

Comparison of fourier-transform and data-dependent system profilometry by use of interferometric regeneration

Fourier-transform profilometry (FTP) and data-dependent system profilometry (DDSP) are the two major phase-extraction methods that use a single interferogram. The difficulty in verifying surface profiles obtained by these methods is that the exact spot on an actual surface cannot be measured with two different instruments. An interferogram regeneration procedure is developed to solve this problem. The surface profile is then extracted from the regenerated interferogram by both FTP and DDSP. Comparisons of the actual surface profile with the extracted surface profiles show that both methods perform equally well in measuring the root mean square and the center line average, but only DDSP is able to reproduce the detailed surface profile of the reference surface.