Single-shot 3D shape measurement of discontinuous objects based on a coaxial fringe projection system

Phase calculation of smooth surface with multi-reflectivity based on phase measurement deflectometry

With the continuous advancement of precision machining technology and the growing demand for products, increasingly complex objects with high reflectivity are becoming more prevalent in production and daily life. phase measurement deflectometry (PMD) is a technique that utilizes a surface light source to project structured light for comprehensive detection of highly reflective surfaces. It offers advantages such as high accuracy, fast speed, low cost, and non-contact operation. However, when the surface of the object being measured has varying levels of reflectivity, this method may produce errors due to significant differences in fringe contrast between different reflective areas. In order to enable the fringe deflection system to simultaneously detect multiple reflective objects without sacrificing accuracy, this paper proposes an adaptive method for fringe generation detection. This method can adaptively adjust the intensity based on the reflectivity of the measured surface and compensate for the light at the reflectivity boundary, ultimately achieving phase calculation for multiple reflective surfaces.

Modeling the measurement precision of Fringe Projection Profilometry

Three-dimensional (3D) surface geometry provides elemental information in various sciences and precision engineering. Fringe Projection Profilometry (FPP) is one of the most powerful non-contact (thus non-destructive) and non-interferometric (thus less restrictive) 3D measurement techniques, featuring at its high precision. However, the measurement precision of FPP is currently evaluated experimentally, lacking a complete theoretical model for guidance. We propose the first complete FPP precision model chain including four stage models (camera intensity, fringe intensity, phase and 3D geometry) and two transfer models (from fringe intensity to phase and from phase to 3D geometry). The most significant contributions include the adoption of a non-Gaussian camera noise model, which, for the first time, establishes the connection between camera's electronics parameters (known in advance from the camera manufacturer) and the phase precision, and the formulation of the phase to geometry transfer, which makes the precision of the measured geometry representable in an explicit and concise form. As a result, we not only establish the full precision model of the 3D geometry to characterize the performance of an FPP system that has already been set up, but also explore the expression of the highest possible precision limit to guide the error distribution of an FPP system that is yet to build. Our theoretical models make FPP a more designable technique to meet the challenges from various measurement demands concerning different object sizes from macro to micro and requiring different measurement precisions from a few millimeters to a few micrometers.

Experimental analysis of high-temperature creep in FV566 steel based on digital image correlation

Digital image correlation (DIC) is an optical measurement method of material strain/displacement based on visible light illumination, which can be used for the measurement of long-term mechanical behavior. In this paper, an experimental method for analyzing high-temperature creep in FV566 steel material based on DIC was independently designed. Aiming at the problems of glass observation window medium refraction and thermal airflow disturbance in high-temperature testing, the corresponding correction methods were proposed to improve the measurement accuracy. Based on the above methods, high-temperature creep tests were carried out on three specimens with different shapes, and the strain concentration area at 600°C was calculated. Then, the influences of shape and other properties on material creep failure, stress distribution, and actual strain were investigated. Finally, the DIC calculation results were analyzed and compared with results of finite element analysis and the final fracture position of the specimen. The three results had a high degree of consistency, which verified that the proposed method can accurately measure and analyze the creep behavior of materials.

An Improved Circular Fringe Fourier Transform Profilometry

Circular fringe projection profilometry (CFPP), as a branch of carrier fringe projection profilometry, has attracted research interest in recent years. Circular fringe Fourier transform profilometry (CFFTP) has been used to measure out-of-plane objects quickly because the absolute phase can be obtained by employing fewer fringes. However, the existing CFFTP method needs to solve a quadratic equation to calculate the pixel displacement amount related to the height of the object, in which the root-seeking process may get into trouble due to the phase error and the non-uniform period of reference fringe. In this paper, an improved CFFTP method based on a non-telecentric model is presented. The calculation of displacement amount is performed by solving a linear equation instead of a quadratic equation after introducing an extra projection of circular fringe with circular center translation. In addition, Gerchberg iteration is employed to eliminate phase error of the region close to the circular center, and the plane calibration technique is used to eliminate system error by establishing a displacement-to-height look-up table. The mathematical model and theoretical analysis are presented. Simulations and experiments have demonstrated the effectiveness of the proposed method.

Deep learning-based method for non-uniform motion-induced error reduction in dynamic microscopic 3D shape measurement

The non-uniform motion-induced error reduction in dynamic fringe projection profilometry is complex and challenging. Recently, deep learning (DL) has been successfully applied to many complex optical problems with strong nonlinearity and exhibits excellent performance. Inspired by this, a deep learning-based method is developed for non-uniform motion-induced error reduction by taking advantage of the powerful ability of nonlinear fitting. First, a specially designed dataset of motion-induced error reduction is generated for network training by incorporating complex nonlinearity. Then, the corresponding DL-based architecture is proposed and it contains two parts: in the first part, a fringe compensation module is developed as network pre-processing to reduce the phase error caused by fringe discontinuity; in the second part, a deep neural network is employed to extract the high-level features of error distribution and establish a pixel-wise hidden nonlinear mapping between the phase with motion-induced error and the ideal one. Both simulations and real experiments demonstrate the feasibility of the proposed method in dynamic macroscopic measurement.

Application of Moire Profilometry in Three-Dimensional Profile Reconstruction of Key Parts in Railway

Moire profilometry (MP) is one of the three-dimensional (3D) topography measurement methods of structured light, which has the advantages of single frame reconstruction, high speed, no contact and high precision, and is suitable for dynamic measurement scenes. In this article, the digital MP is applied to the wheel tread measurement, the virtual grating is generated by computer to project to the object surface, the moire fringe pattern of the object is obtained by filtering, and finally the continuous phase pattern is obtained by phase unwrapping. The 3D shape reconstruction of the wheel tread is realized, and a new method of wheel tread detection is provided. At the same time, in this paper, the results of using different filters are compared, and the significance of the frequency domain filtering to MP is proved. It is necessary to choose a suitable filtering method according to different environmental conditions. At present, digital MP can be used in industrial static detection, and it can be extended to the dynamic detection of rolling wheels in the future, so as to improve the detection efficiency and realize the automatic detection of trains.

Advances and Prospects of Vision-Based 3D Shape Measurement Methods

Vision-based three-dimensional (3D) shape measurement techniques have been widely applied over the past decades in numerous applications due to their characteristics of high precision, high efficiency and non-contact. Recently, great advances in computing devices and artificial intelligence have facilitated the development of vision-based measurement technology. This paper mainly focuses on state-of-the-art vision-based methods that can perform 3D shape measurement with high precision and high resolution. Specifically, the basic principles and typical techniques of triangulation-based measurement methods as well as their advantages and limitations are elaborated, and the learning-based techniques used for 3D vision measurement are enumerated. Finally, the advances of, and the prospects for, further improvement of vision-based 3D shape measurement techniques are proposed.

3D shape measurement method for high-reflection surface based on fringe projection

3D measurement methods based on fringe projection have attracted extensive research. However, it is a challenge to deal with overshooting on a high-reflection or specular surface. To eliminate the saturated pixels caused by overshooting, we propose a projection intensity adaptive adjustment method. First, we project three uniform gray-level images and estimate the projection intensity of the measured surface through the captured uniform gray-level images. Then we can obtain the optimal projection fringes in the camera coordinate system. Second, a set of horizontal and vertical gray-coded patterns are used to establish a coordinate matching relationship between the projected image and the captured image. To check the decoding result of the gray-coded patterns, a set of horizontal and vertical sinusoidal fringes are used to calculate the high-reflection mapping area (HRMA) in the projector coordinate system. Through the distribution of HRMA, we can check whether the decoding is reliable or not. Finally, we project the optimal intensity fringes and obtain the measurement results. We develop a measurement system to verify the validity of the proposed method. Experimental results show that the proposed method can effectively avoid overshooting and obtain measurement results with a minimum rms error.

Determining Surface Shape of Translucent Objects with the Combination of Laser-Beam-Based Structured Light and Polarization Technique

In this study, we focus on the 3D surface measurement and reconstruction of translucent objects. The proposed approach of surface-shape determination of translucent objects is based on the combination of the projected laser-beam-based sinusoidal structured light and the polarization technique. The theoretical analyses are rigorously completed in this work, including the formation, propagation, and physical features of the generated sinusoidal signal by the designed optical system, the reflection and transmission of the projected monochromatic fringe pattern on the surface of the translucent object, and the formation and the separation of the direct-reflection and the global components of the surface radiance of the observed object. The results of experimental investigation designed in accordance with our theoretical analyses have confirmed that accurate reconstructions can be obtained using the one-shot measurement based on the proposed approach of this study and Fourier transform profilometry, while the monochromaticity and the linearly-polarized characteristic of the projected sinusoidal signal can be utilized by using a polarizer and an optical filter simultaneously for removing the global component, i.e., the noised signal contributed by multiply-scattered photons and the background illuminance in the frame of our approach. Moreover, this study has also revealed that the developed method is capable of getting accurate measurements and reconstructions of translucent objects when the background illumination exists, which has been considered as a challenging issue for 3D surface measurement and reconstruction of translucent objects.

3D reconstruction of structured light fields based on point cloud adaptive repair for highly reflective surfaces

In this paper, a novel method, to the best of our knowledge, of structured light fields based on point cloud adaptive repair is proposed to realize 3D reconstruction for highly reflective surfaces. We have designed and built a focused light field camera whose spatial and angular resolution can be flexibly adjusted as required. Then the subaperture image extraction algorithm based on image mosaic is deduced and presented to obtain multidirectional images. After that, the 3D reconstruction of structured light field imaging based on point cloud adaptive repair is presented to accurately reconstruct for highly reflective surfaces. In addition, a method based on smoothness and repair rate is also proposed to objectively evaluate the performance of the 3D reconstruction. Experimental results demonstrate the validity of the proposed method to perform high-quality depth reconstruction for highly reflective surfaces. Generally, our method takes advantage of the multidirectional imaging of the light field camera and can ensure good modulation effect of structured light while avoiding hardware complexity, which makes it application more convenient.

3D Measurement of Structured Specular Surfaces Using Stereo Direct Phase Measurement Deflectometry

With the rapid development of modern manufacturing processes, ultra-precision structured freeform surfaces are being widely explored for components with special surface functioning. Measurement of the 3D surface form of structured specular objects remains a challenge because of the complexity of the surface form. Benefiting from a high dynamic range and large measuring area, phase measurement deflectometry (PMD) exhibits great potential in the inspection of the specular surfaces. However, the PMD is insensitive to object height, which leads to the PMD only being used for smooth specular surface measurement. Direct phase measurement deflectometry (DPMD) has been introduced to measure structured specular surfaces, but the surface form measurement resolution and accuracy are limited. This paper presents a method named stereo-DPMD for measuring structured specular objects by introducing a stereo deflectometor into DPMD, so that it combines the advantages of slope integration of the stereo deflectometry and discontinuous height measurement from DPMD. The measured object is separated into individual continuous regions, so the surface form of each region can be recovered precisely by slope integration. Then, the relative positions between different regions are evaluated by DPMD system to reconstruct the final 3D shape of the object. Experimental results show that the structured specular surfaces can be measured accurately by the proposed stereo-DPMD method.

Separation of interreflections based on parallel single-pixel imaging

Interreflections introduced by points in a scene are not only illuminated by the light source used but also by other points in the scene. Interreflections cause inaccuracy and the failure of 3D recovery and optical measurements. In this research, a novel method for separating interreflections through parallel single-pixel imaging (PSI) is proposed, which can decompose interreflections into 1st bounce light, 2nd bounce light, and a higher order light component. PSI is used in obtaining the light transport coefficients of each camera pixel, and light transport coefficients are used in decomposing the intensity distribution of a projector and the component of interreflections. Results show that the proposed method can separate the interreflections of a real static scene in a concave surface.

Fourier-Transform-Based Surface Measurement and Reconstruction of Human Face Using the Projection of Monochromatic Structured Light

This work presents a new approach of surface measurement of human face via the combination of the projection of monochromatic structured light, the optical filtering technique, the polarization technique and the Fourier-transform-based image-processing algorithm. The theoretical analyses and experimental results carried out in this study showed that the monochromatic feature of projected fringe pattern generated using our designed laser-beam-based optical system ensures the use of optical filtering technique for removing the effect of background illumination; the linearly-polarized characteristic makes it possible to employ a polarizer for eliminating the noised signal contributed by multiply-scattered photons; and the high-contrast sinusoidal fringes of the projected structured light provide the condition for accurate reconstruction using one-shot measurement based on Fourier transform profilometry. The proposed method with the portable and stable optical setup may have potential applications of indoor medical scan of human face and outdoor facial recognition without strict requirements of a dark environment and a stable object being observed.

3D shape measurement in the presence of strong interreflections by using single-pixel imaging in a camera-projector system

Optical 3D shape measurements, such as fringe projection profilometry (FPP), are popular methods for recovering the surfaces of an object. However, traditional FPP cannot be applied to measure regions that contain strong interreflections, resulting in failure in 3D shape measurement. In this study, a method based on single-pixel imaging (SI) is proposed to measure 3D shapes in the presence of interreflections. SI is utilized to separate direct illumination from indirect illumination. Then, the corresponding points between the pixels of a camera and a projector can be obtained through the direct illumination. The 3D shapes of regions with strong interreflections can be reconstructed with the obtained corresponding points based on triangulation. Experimental results demonstrate that the proposed method can be used to separate direct and indirect illumination and measure 3D objects with interreflections.

Accuracy improvement of time-of-flight depth measurement by combination of a high-resolution color camera

Time-of-flight (ToF) cameras can acquire the distance between the sensor and objects with high frame rates, offering bright prospects for ToF cameras in many applications. Low-resolution and depth errors limit the accuracy of ToF cameras, however. In this paper, we present a flexible accuracy improvement method for depth compensation and feature points position correction of ToF cameras. First, a distance-error model of each pixel in the depth image is established to model sinusoidal waves of ToF cameras and compensate for the measured depth data. Second, a more accurate feature point position is estimated with the aid of a high-resolution camera. Experiments evaluate the proposed method, and the result shows the root mean square error is reduced from 4.38 mm to 3.57 mm.

Optical-topological concepts in isomorphisms projecting bi-Ronchi masks to obtain 3D profiles from objects in 2D images

We introduce a technique for obtaining three-dimensional (3D) profiles of objects captured in two-dimensional (2D) flat images (FI). This technique performs a numerical approximation of the object's topography from their image gray tones, analyzing topological concepts such as algebraic bases construction, metric functions in terms of such bases, as well as modeling and development of an isomorphism to project masks (fringe patterns) in the FI, allowing us to use the optimal 3D profilometry techniques. Among these techniques, phase shifting (four steps) is applied in the pattern shifts of the projected masks, but with a 2D shift from left to right and from top to bottom, simultaneously. Moreover, the fringe patterns in the masks are binary and with superposed periods in x, y (bi-Ronchi). We show the results of the construction of the masks, as well as their projection into the FIs. We also show the 3D profilometry of the objects after the projection and phase-shifting application and a simple segmentation to observe a particular object. Subsequently, we perform the analysis of a single Mg crystal's micrography.

3D shape measurement of translucent objects based on Fourier single-pixel imaging in projector-camera system

3D shape measurement by structured light is a popular technique for recovering object surfaces. However, structured light technique assumes that scene points are directly illuminated by the light source(s). Consequently, global illumination effects, such as subsurface scattering in translucent objects, may cause measurement errors in recovered 3D shapes. In this research, we propose a 3D shape measurement method of translucent objects based on Fourier single-pixel imaging (FSI) technique. The 3D shapes of the translucent objects are reconstructed through stereo matching of direct illumination light, which is separated from the surface. Experimental results show that the proposed method can separate the direct illumination light and the subsurface scattering light. The feasibility and accuracy of the method are analyzed, and the qualitative and quantitative results of the method are provided.

Measurement of the Three-Dimensional Shape of Discontinuous Specular Objects Using Infrared Phase-Measuring Deflectometry

Phase-measuring deflectometry (PMD)-based methods have been widely used in the measurement of the three-dimensional (3D) shape of specular objects, and the existing PMD methods utilize visible light. However, specular surfaces are sensitive to ambient light. As a result, the reconstructed 3D shape is affected by the external environment in actual measurements. To overcome this problem, an infrared PMD (IR-PMD) method is proposed to measure specular objects by directly establishing the relationship between absolute phase and depth data for the first time. Moreover, the proposed method can measure discontinuous surfaces. In addition, a new geometric calibration method is proposed by combining fringe projection and fringe reflection. The proposed IR-PMD method uses a projector to project IR sinusoidal fringe patterns onto a ground glass, which can be regarded as an IR digital screen. The IR fringe patterns are reflected by the measured specular surfaces, and the deformed fringe patterns are captured by an IR camera. A multiple-step phase-shifting algorithm and the optimum three-fringe number selection method are applied to the deformed fringe patterns to obtain wrapped and unwrapped phase data, respectively. Then, 3D shape data can be directly calculated by the unwrapped phase data on the screen located in two positions. The results here presented validate the effectiveness and accuracy of the proposed method. It can be used to measure specular components in the application fields of advanced manufacturing, automobile industry, and aerospace industry.

Real-time subtraction-based calibration methods for deformation measurement using structured light techniques

Calibration of the optical trigonometric relationships in structured light is essential for the accuracy of out-of-plane displacement measurement. A simple calibration mechanism based on real-time image subtraction is proposed. Details of calibrating the height-to-phase ratio of the digital fringe projection method, the relationship between out-of-plane and in-plane displacements of the digital projection-speckle correlation method, and the scale factor of the optical system are described. The calibration methods are applied to the deformation measurement of a clamped Plexiglas plate, which demonstrates the effectiveness of the calibration methods. The real-time subtraction-based calibration methods provide alternatives for calibrating structured light techniques used for out-of-plane displacement measurement.

A Calibration Method for System Parameters in Direct Phase Measuring Deflectometry

Phase measuring deflectometry has been widely studied as a way of obtaining the three-dimensional shape of specular objects. Recently, a new direct phase measuring deflectometry technique has been developed to measure the three-dimensional shape of specular objects that have discontinuous and/or isolated surfaces. However, accurate calibration of the system parameters is an important step in direct phase measuring deflectometry. This paper proposes a new calibration method that uses phase information to obtain the system parameters. Phase data are used to accurately calibrate the relative orientation of two liquid crystal display screens in a camera coordinate system, by generating and displaying horizontal and vertical sinusoidal fringe patterns on the two screens. The results of the experiments with an artificial specular step and a concave mirror showed that the proposed calibration method can build a highly accurate relationship between the absolute phase map and the depth data.