High dynamic range 3D measurement based on spectral modulation and hyperspectral imaging

Parameter selection on a multi-exposure fusion method for measuring surfaces with varying reflectivity in microscope fringe projection profilometry

As industrial and scientific advancements continue, the demand for precise measurement of three-dimensional (3D) shapes and surfaces is steadily increasing. However, accurate 3D measurement of certain surfaces, especially those with varying reflectivities, has always been a challenging issue. Multi-exposure fusion methods have shown stable, high-quality measurement results, but the selection of parameters for these methods has largely been based on experience. To address this issue, this paper has improved the multi-exposure fusion method and introduced a guided approach for parameter selection, significantly enhancing the completeness of measurement results. Additionally, a comparative model is developed to experimentally validate the specific impacts of Gaussian window variance, optimal grayscale range, and attenuation factor variance on the integrity of 3D reconstruction. The experimental results demonstrate that under the guidance of the parameter adjustment method proposed in this paper, the multi-exposure fusion for measuring the 3D topography of high-dynamic surfaces improves the restoration coverage from the original 86% (bright areas) and 50% (dark areas) to over 99%. This provides a selection strategy for parameter adjustment guidance in precise measurements based on the multi-exposure method.

Low-power laser image enhancement via deep feature recovery for HDR 3D measurement

Laser 3D measurement has gained widespread applications in industrial metrology . Still, it is usually limited by surfaces with high dynamic range (HDR) or the colorful surface texture of measured surfaces, such as metal and black industrial parts. Currently, conventional methods generally work with relatively strong-power laser intensities, which could potentially damage the sample or induce eye-safety concerns. For deep-learning-based methods, due to the different reflectivity of the measured surfaces, the HDR problem may require cumbersome adjustment of laser intensity in order to acquire enough training data. Even so, the problem of inaccurate ground truth may occur. To address these issues, this paper proposes the deep feature recovery (DFR) strategy to enhance low-light laser stripe images for achieving HDR 3D reconstruction with low cost, high robustness, and eye safety. To the best of our knowledge, this is the first attempt to tackle the challenge of high measurement costs associated with measuring HDR surfaces in laser 3D measurement. In learning the features of low-power laser images, the proposed strategy has a superior generalization ability and is insensitive to different low laser powers and variant surface reflectivity. To verify this point, we specially design the experiments by training the network merely using the diffusely reflective masks (DRM951) and testing the performance using diffusely reflective masks, metal surfaces, black industrial parts (contained in the constructed datasets DRO690, MO191, and BO107) and their hybrid scenes. Experimental results demonstrate that the proposed DFR strategy has good performances on robustness by testing different measurement scenes. For variously reflective surfaces, such as diffusely reflective surfaces, metal surfaces, and black parts surfaces, the reconstructed 3D shapes all have a similar quality to the reference method.

Physics-based supervised learning method for high dynamic range 3D measurement with high fidelity

High dynamic range (HDR) 3D measurement is a meaningful but challenging problem. Recently, many deep-learning-based methods have been proposed for the HDR problem. However, due to learning redundant fringe intensity information, their networks are difficult to converge for data with complex surface reflectivity and various illumination conditions, resulting in non-robust performance. To address this problem, we propose a physics-based supervised learning method. By introducing the physical model for phase retrieval, we design a novel, to the best of our knowledge, sinusoidal-component-to-sinusoidal-component mapping paradigm. Consequently, the scale difference of fringe intensity in various illumination scenarios can be eliminated. Compared with conventional supervised-learning methods, our method can greatly promote the convergence of the network and the generalization ability, while compared with the recently proposed unsupervised-learning method, our method can recover complex surfaces with much more details. To better evaluate our method, we specially design the experiment by training the network merely using the metal objects and testing the performance using different diffuse sculptures, metal surfaces, and their hybrid scenes. Experiments for all the testing scenarios have high-quality phase recovery with an STD error of about 0.03 rad, which reveals the superior generalization ability for complex reflectivity and various illumination conditions. Furthermore, the zoom-in 3D plots of the sculpture verify its fidelity on recovering fine details.

Enhanced Fourier-Hilbert-transform suppression for saturation-induced phase error in phase-shifting profilometry

Intensity saturation tends to induce severe errors in high dynamic range three-dimensional measurements using structured-light techniques. This paper presents an enhanced Fourier-Hilbert-transform (EFHT) method to suppress the saturation-induced phase error in phase-shifting profilometry, by considering three types of residual errors: nonuniform-reflectivity error, phase-shift error, and fringe-edge error. Background normalization is first applied to the saturated fringe patterns to suppress the effect of the nonuniform reflectivity. A self-correction method is proposed to correct the large phase-shift error in the compensated phase. The self-corrected phase error is detected to assist in locating the fringe-edge area, within which the true phase is computed based on the sub-period phase error model. Experimental results demonstrated the effectiveness of the proposed method in suppressing the saturation-induced phase error and other three types of residual errors with fewer images.

Saturation-Induced Phase Error Compensation Method Using Complementary Phase

Intensity saturation can induce phase error and, thus, measurement error in fringe projection profilometry. To reduce saturation-induced phase errors, a compensation method is developed. The mathematical model of saturation-induced phase errors is analyzed for N-step phase-shifting profilometry, and the phase error is approximately N-folder of the frequency of the projected fringe. Additional N-step phase-shifting fringe patterns with initial phase-shift π/N are projected for generating a complementary phase map. The final phase map is obtained by averaging the original phase map extracted from the original fringe patterns and the complementary phase map, and then the phase error can be canceled out. Both simulations and experiments demonstrated that the proposed method can substantially reduce the saturation-induced phase error and realize accurate measurements for a highly dynamic range of scenarios.

Generic saturation-induced phase error correction for structured light 3D shape measurement

Intensity saturation is a challenging problem in structured light 3D shape measurement. Most of the existing methods achieve high dynamic range (HDR) measurement by sacrificing measurement speed, making them limited in high-speed dynamic applications. This Letter proposes a generic efficient saturation-induced phase error correction method for HDR measurement without increasing any fringe patterns. We first theoretically analyze the saturated signal model and deduce the periodic characteristic of saturation-induced phase error. Based on this, we specially design a saturation-induced phase error correction method by joint Fourier analysis and Hilbert transform. Furthermore, the relationship among phase error, saturation degree, and number of phase-shifting steps is established by numerical simulation. Since the proposed method requires no extra captured images or complicated intensity calibration, it is extremely convenient in implementation and is applicable to performing high-speed 3D shape measurements. Simulations and experiments verify the feasibility of the proposed method.

Adaptive Binocular Fringe Dynamic Projection Method for High Dynamic Range Measurement

Three-dimensional measurement with fringe projection sensor has been commonly researched. However, the measurement accuracy and efficiency of most fringe projection sensors are still seriously affected by image saturation and the non-linear effects of the projector. In order to solve the challenge, in conjunction with the advantages of stereo vision technology and fringe projection technology, an adaptive binocular fringe dynamic projection method is proposed. The proposed method can avoid image saturation by adaptively adjusting the projection intensity. Firstly, the flowchart of the proposed method is explained. Then, an adaptive optimal projection intensity method based on multi-threshold segmentation is introduced to adjust the projection illumination. Finally, the mapping relationship of binocular saturation point and projection point is established by binocular transformation and left camera-projector mapping. Experiments demonstrate that the proposed method can achieve higher accuracy for high dynamic range measurement.

High-resolution few-pattern method for 3D optical measurement

Accurately and quickly obtaining the three-dimensional (3D) shape information of objects has become increasingly important in various scientific fields. However, simultaneously achieving the high-resolution and high-speed 3D shape measurement of unknown objects remains challenging in practice. In this Letter, we propose a novel variant shifting-phase method for 3D optical measurement. Based on a digital fringe projection system, the method performs a point-wise high-resolution measurement of objects by projecting only four intensity-coded patterns. We can retrieve the wrapped phases and their corresponding fringe orders simultaneously from these four patterns, and do not require any pre-acquired information of the object. The experimental results successfully demonstrate the effectiveness of the easy-to-operate method.

Multiple Laser Stripe Scanning Profilometry Based on Microelectromechanical Systems Scanning Mirror Projection

In traditional laser-based 3D measurement technology, the width of the laser stripe is uncontrollable and uneven. In addition, speckle noise in the image and the noise caused by mechanical movement may reduce the accuracy of the scanning results. This work proposes a new multiple laser stripe scanning profilometry (MLSSP) based on microelectromechanical systems (MEMS) scanning mirror which can project high quality movable laser stripe. It can implement full-field scanning in a short time and does not need to move the measured object or camera. Compared with the traditional laser stripe, the brightness, width and position of the new multiple laser stripes projected by MEMS scanning mirror can be controlled by programming. In addition, the new laser strip can generate high-quality images and the noise caused by mechanical movement is completely eliminated. The experimental results show that the speckle noise is less and the light intensity distribution is more even. Furthermore, the number of pictures needed to be captured is significantly reduced to 1 / N ( N is the number of multiple laser stripes projected by MEMS scanning mirror) and the measurement efficiency is increased by N times, improving the efficiency and accuracy of 3D measurement.