High-speed FPGA-based phase measuring profilometry architecture

BimodalPS: Causes and Corrections for Bimodal Multi-Path in Phase-Shifting Structured Light Scanners

Structured light illumination is an active 3D scanning technique based on projecting and capturing a set of striped patterns and measuring the warping of the patterns as they reflect off a target object's surface. As designed, each pixel in the camera sees exactly one pixel from the projector; however, there are multi-path situations where a camera pixel sees light from multiple projector positions. In the case of bimodal multi-path, the camera pixel receives light from exactly two positions, which occurs along a step edge where the edge slices through a pixel which, therefore, sees both a foreground and background surface. In this paper, we present a general mathematical model to address this bimodal multi-path issue in a phase-shifting or so-called phase-measuring-profilometry scanner to measure the constructive and destructive interference between the two light paths, and by taking advantage of this interference, separate the paths and make two decoupled depth measurements. We validate our algorithm with both simulations and a number of challenging real-world scenarios, significantly outperforming the state-of-the-art methods.

High-performance phase measuring profilometry architecture based on Zynq SoC

This paper presents a novel high-performance heterogeneous computation architecture, to the best of our knowledge, for stereo structure light using the phase measuring profilometry (PMP) algorithm based on a Zynq UltraScale+ system on chip (SoC). The proposed architecture aims to achieve real-time and high-accuracy 3D shape measurement. The experiment results indicate that the calculation time of a standard four-step PMP algorithm with a resolution of 1280×1024 is 14.11 ms. It is nearly 51 times faster than the well-optimized software implementation running on a Raspberry Pi and nearly three times faster than a high-end PC, with 15 times less power consumption. Consequently, the proposed architecture is deemed suitable for real-time 3D measurements in embedded applications.

Parallel Fourier ptychographic microscopy reconstruction method based on FPGA

Fourier ptychographic microscopy (FPM) can bypass the limitation of spatial bandwidth product to get images with large field-of-view and high resolution. The complicated sequential iterative calculation in the FPM reconstruction process reduces the reconstruction efficiency of the FPM. Therefore, we propose a parallel FPM reconstruction method based on field programmable gate array (FPGA) to accelerate the FPM reconstruction process. Using this method, multiple sub-regions in the Fourier domain can be computed in parallel and we customize a dedicated high-performance computational architecture for this approach. We deploy 4 FPM reconstruct computing architectures with a parallelism of 4 in a FPGA to compute the FPM reconstruction process, achieving the speed nearly 180 times faster than traditional methods. The proposed method provides a new perspective of parallel computing for FPM reconstruction.

FMCW LiDAR with an FM nonlinear kernel function for dynamic-distance measurement

Frequency-modulated continuous-wave (FMCW) LiDAR is an absolute-distance measurement technology with the advantages of high-precision, non-cooperative target measurement capabilities and the ability to measure distance and speed simultaneously. However, the existing range extraction method for FMCW LiDAR is associated with problems, such as requiring a high sample rate and dispersion mismatch. Here, we propose and demonstrate a dynamic range extraction method based on an FM nonlinear kernel function, which improves measurement accuracy without the use of a long auxiliary interferometer (as is required for the traditional method), reduces the influence of dispersion mismatch and the Doppler effect caused by target movement and can simultaneously measure the target motion information dynamically, with a lower measurement error than that of the existing range extraction method under the same conditions.

Hybrid-quality-guided phase fusion model for high dynamic range 3D surface measurement by structured light technology

Using structured light to measure the 3D shape of a high dynamic range (HDR) surface has been always a challenging problem, and fusion of multi-group images with different exposures is recognized as an effective solution. It tends to select the phase with unsaturated and maximum gray intensity as the final phase, which has two problems: 1) the selection criteria are too simple to fully evaluate the phase quality, and 2) it is affected by the image noise, camera's nonlinear response, local reflection and other factors and the phase with the best quality among the initial phases may also have an error. Aiming to solve these issues, this paper presents a hybrid-quality-guided phase fusion (HPF) model. In this model, a hybrid-quality measure is first proposed to evaluate the phase quality more comprehensively. Then, all initial phases are weighted and fused under the guidance of the hybrid-quality measure to obtain a more accurate phase as the final one. Through this model, a more complete and accurate 3D shape of the HDR surface can be reconstructed, and its validity has been verified by several experiments.

Real-time profilometry by bicolor grating video projection

A novel real-time 2+1 three-dimensional(3D) measuring method based on bicolor grating video projection is proposed. Firstly, only two frames of bicolor gratings, in which the red channels are two sinusoidal fringes with a shifting phase of π/2 and the blue channels are the same background light equivalent to the DC component of the two sinusoidal fringes are encoded and arranged alternatively to synthesize into a repetitive bicolor grating video, While this video is projected onto the measured object, the real-time bicolor deformed pattern video can be recorded by using a color CMOS camera, and the bicolor deformed pattern sequence at different moments can be extracted by computer processing, so that the 2+1 algorithm can be used to accomplish real-time 3D measurement of moving object. Before measuring, we used the same method to design two sinusoidal fringes with a difference of π in their red channels, respectively, to calibrate the sensitivity ratio between the red and blue channels of the CMOS camera, which can effectively eliminate the chromaticity imbalance between R and B channels and reduce the color crosstalk. Experimental results and analysis confirm the feasibility and effectiveness of the proposed method. Because the proposed method needs a repetitive bicolor grating video synthesized with only two-frame bicolor gratings to be projected, the 3D measurement acquisition speed and real-time accuracy will be improved compared with the traditional 2+1 3D measuring method.

Three-dimensional measurement method of color fringe projection based on an improved three-step phase-shifting method

A three-dimensional (3D) measurement method of color fringe projection based on an improved three-step phase-shifting method is proposed. The color fringe pattern is encoded by two cosine fringe patterns with the same frequency but different shifting phase and a uniform gray flat image into three color channels R, G, and B. Although the measurement speed of the traditional three-step phase-shifting method can meet the requirements of measuring 3D objects, it makes the noise and inaccuracy of the captured images increase, and each image will cause measurement error. Therefore, we improve the three-step phase-shifting method and introduce the Hilbert transform into the three-step phase-shift method. The DC component of the fringe pattern is obtained by using the Hilbert transform principle, and the third fringe pattern in the three-step phase-shift method is replaced by the captured light intensity distribution of the DC component. The phase difference of the other two fringe patterns is fixed as π/2 by the Hilbert transform. The improved three-step phase-shifting method is used to obtain the phase information of the deformed color fringe image, and then the phase-unwrapping algorithm is used to obtain the phase distribution information of the whole field. The results show that the improved method can not only accurately calculate the phase information but also greatly improve the measurement speed and quality.

Color-encoded single-shot computer-generated Moiré profilometry

A color-encoded single-shot computer-generated Moiré profilometry (CSCGMP) is proposed. Two sinusoidal gratings with a π phase difference are encoded in red and blue channels respectively to combine a composite color grating. While this composite color grating is projected onto the measured object, the corresponding color deformed pattern can be captured. So two deformed patterns with a π phase difference are separated from its red and blue components respectively. After normalization and subtraction, the AC component of both separated deformed patterns can be extracted. If this AC component respectively multiplied by the two AC components of fringe patterns of reference plane with a π/2 phase difference prepared and saved on the computer in advance, two computer-generated Moiré fringes just respectively standing for sine and cosine of phase which is modulated by the height of the object relative to the reference plane are figured out. So the 3D shape of the measured object can be reconstructed with normal computer-generated Moiré profilometry. Both simulation and experimental results show the feasibility and validity of the proposed method. It has potential in real-time 3D measurement due to its single-shot feature.

A Single-Shot 3D Measuring Method Based on Quadrature Phase-Shifting Color Composite Grating Projection

A single-shot three-dimensional measuring method based on quadrature phase-shifting color composite grating projection is proposed. Firstly, three quadrature phase-shifting sinusoidal gratings are encoded in red (R), green (G), and blue (B) channels respectively, composed single- frame color composite grating. This color composite grating is projecting obliquely on the object by DLP. After that, the color camera which is placed in a specific location is used to capture the corresponding color deformed pattern and send it to the PC. Then, by color separation, the color deformed pattern is demodulated as the corresponding three-frame monochromatic deformed patterns with a shifted quadrature phase. Due to the existences of sensitivity differences and color crosstalk among the tricolor channels, we propose a gray imbalance correction method based on the DC component’s consistency approximation. By the established 3D reconstruction physical model, the measurement of 3D shape can be achieved. Many experimental results for static and moving objects prove the proposed method’s feasibility and practicability. Owing to the single-shot feature of the proposed method, it has a good application prospect in real-time and high-speed 3D measurement.

Geometry-invariant-based reconstruction generated from planar laser and metrical rectification with conic dual to circular points in the similarity space

3D point reconstruction is a crucial component in optical inspection. A direct reconstruction process is proposed by combining two similarity invariants in active vision. A planar reference with an isosceles-right-angle pattern and a coplanar laser are adopted to generate the laser projective point on the measured object. The first invariant is the image of the conic dual to the circular points (ICDCP), which is derived from the lines in two pairs of perpendicular directions on the reference pattern. The invariant provides the transform from the projection space to the similarity space. Then, the ratio of the line segments consisting of the laser projection points and reference points is constructed as the other similarity invariant, by which the laser projection point in the similarity space is converted to Euclidean space. The solution of the laser point is modeled by the ratio invariant of the line segments and improved by a special point selection to avoid nonlinear equations. Finally, the benchmark-camera distance, the benchmark-generator distance, the benchmark length, image noise, and the number of orthogonal lines are experimentally investigated to explore the effectiveness and reconstruction error of the method. The reconstruction error averages of 0.94, 1.22, 1.77, and 2.15 mm are observed from the experiment results with the benchmark-camera distances from 600 mm to 750 mm with a 50 mm interval. This proves the validity and practicability of the reconstruction method.

Divide and conquer: high-accuracy and real-time 3D reconstruction of static objects using multiple-phase-shifted structured light illumination

Multiple-phase-shifted structured light illumination achieves high-accuracy 3D reconstructions of static objects, while typically it can't achieve real-time phase computation. In this paper, we propose to compute modulations and phases of multiple scans in real time by using divide-and-conquer solutions. First, we categorize total N = KM images into M groups and each group contains K phase equally shifted images; second, we compute the phase of each group; and finally, we obtain the final phase by averaging all the separately computed phases. When K = 3, 4 or 6, we can use integer-valued intensities of images as inputs and build one or M look-up tables storing real-valued phases computed by using arctangent function. Thus, with addition and/or subtraction operations computing indices of the tables, we can directly access the pre-computed phases and avoid time-consuming arctangent computation. Compared with K-step phase measuring profilometry repeated for M times, the proposed is robust to nonlinear distortion of structured light systems. Experiments show that, first, the proposed is of the same accuracy level as the traditional algorithm, and secondly, with employing one core of a central processing unit, compared with the classical 12-step phase measuring profilometry algorithm, for K = 4 and M = 3, the proposed improves phase computation by a factor of 6 ×.

Dual-projector structured light 3D shape measurement

Structured light illumination is an active three-dimensional scanning technique that uses a projector and camera pair to project and capture a series of stripe patterns; however, with a single camera and single projector, structured light scanning has issues associated with scan occlusions, multi-path, and weak signal reflections. To address these issues, this paper proposes dual-projector scanning using a range of projector/camera arrangements. Unlike previous attempts at dual-projector scanning, the proposed scanner drives both light engines simultaneously, using temporal-frequency multiplexing to computationally decouple the projected patterns. Besides presenting the details of how such a system is built, we also present experimental results demonstrating how multiple projectors can be used to (1) minimize occlusions; (2) achieve higher signal-to-noise ratios having twice a single projector's brightness; (3) reduce the number of component video frames required for a scan; and (4) detect multi-path interference.

Dynamic phase measuring profilometry for rigid objects based on simulated annealing

This paper presents a dynamic phase measurement profilometry (PMP) method based on the simulated annealing algorithm. In dynamic PMP for rigid objects, pixel matching is an effective method to make one-to-one pixel correspondence in each captured pattern. However, pixel matching by the global traversing algorithm takes up most of the time in the whole reconstruction process. For the purpose of optimizing pixel matching and enhancing performance in dynamic PMP, the simulated annealing algorithm is introduced. By generating a random path based on the simulated annealing algorithm, it is sufficient to locate the approximate area of the measured object. Then the accurate position can be calculated by combining it with a partial traversing algorithm. The proposed method can reduce pixel matching time by 63% and increase reconstruction efficiency by 58%. Simulations and experiments prove feasibility and precision.

Reconstructing 3D point clouds in real time with look-up tables for structured light scanning along both horizontal and vertical directions

By scanning static, not moving, objects along both the horizontal and vertical axes instead of one, structured light illumination achieves more accurate and robust 3D surface reconstructions but with greater latency on computing 3D point clouds. If scanning is performed along only one axis, it has been reported that look-up tables, manually derived from the calibration matrices of a camera and a projector, can significantly help to speed up computation; however, it has been nearly impossible to manually derive similar look-up tables for phases scanned along two axes. In this Letter, we bridge this divide by introducing the constraint of epipolar geometry to automatically compute look-up tables and thus, significantly speed up computing 3D point clouds with only basic arithmetic operations rather than time-consuming matrix computations. Experimental results show that the proposed method, using only single-thread CPU computing, reduces process latency by an order of magnitude.

Suppression of the nonlinear phase error in phase shifting profilometry: considering non-smooth reflectivity and fractional period

Hilbert transform (HT) has been employed to compensate phase error arising from the nonlinear effect in phase shifting profilometry (PSP). However, in most common situations, pure HT may lead to a significant system error, which has a negative impact on subsequent phase error compensation. In this paper, system error from HT of non-stationary and non-continuous fringe is analyzed, and then a novel phase error suppression approach is presented. The cosine fringe without direct current (DC) component is reconstructed to eliminate the influence of non-smooth reflectivity, and the fractional periods at both ends of the reconstructed fringe are extended to generate fringe with integer number of periods. And then the HT is applied to the reconstructed and extended fringe. Finally, a revised phase-shifting algorithm is employed to calculate the phase with the fringe after HT. The proposed approach is suitable for PSP of the surface with non-smooth reflectivity (e.g. texture of complex colors), which is demonstrated in a series of experiments.