State-of-the-art active optical techniques for three-dimensional surface metrology: a review [Invited]

Accurate 3D Measurement of Complex Texture Objects by Height Compensation Using a Dual-Projector Structure

Fringe projection profilometry is a widely used technique for 3D measurement due to its high accuracy and speed. However, the accuracy significantly decreases when measuring complex texture objects, especially in the junction of different colors. This paper analyzes the causes of errors resulting from complex textures and proposes a height compensation method to revise the error by employing a dual-projector structure. Moreover, the dual-projector is capable of acquiring a pair of errors with opposite signs, which can be utilized to calculate the accurate 3D information after determining the ratio of this pair of errors. Experiments provide significant improvement in measuring complex texture objects, demonstrating the proposed method's ability.

Nanoscale surface metrology with a liquid crystal-based phase-shifting angular shearing interferometer

In this Letter, a phase-shifting angular shearing interferometer has been proposed for the application in optical surface metrology (SM) by using a combination of a wedge-shaped liquid crystal (LC) cell and a polarization phase shifter. The demonstration of this angular shearing interferometer for step height measurement is accomplished with the help of a phase-shifting technique. Four phase-shifted interferograms produced by a geometrical phase shifter are subjected to a simplified Wiener deconvolution method, which resembles a simple analysis technique for shearing interferograms in comparison to alternative approaches. A simulation study has been conducted to validate the proposed technique. The experimental results show an accuracy of 5.56% for determining the step height, which also agrees with the results obtained by atomic force microscopy. Owing to the tunability of birefringence, the proposed LC-based angular shearing interferometry technique will be useful to control spatial resolution in optical metrology.

Fast topographic optical imaging using encoded search focal scan

A central quest in optics is to rapidly extract quantitative information from a sample. Existing topographical imaging tools allow non-contact and three-dimensional measurements at the micro and nanoscales and are essential in applications including precision engineering and optical quality control. However, these techniques involve acquiring a focal stack of images, a time-consuming process that prevents measurement of moving samples. Here, we propose a method for increasing the speed of topographic imaging by orders of magnitude. Our approach involves collecting a reduced set of images, each integrated during the full focal scan, whilst the illumination is synchronously modulated during exposure. By properly designing the modulation sequence for each image, unambiguous reconstruction of the object height map is achieved using far fewer images than conventional methods. We describe the theoretical foundations of our technique, characterise its performance, and demonstrate sub-micrometric topographic imaging over 100 µm range of static and dynamic systems at rates as high as 67 topographies per second, limited by the camera frame rate. The high speed of the technique and its ease of implementation could enable a paradigm shift in optical metrology, allowing the real-time characterisation of large or rapidly moving samples.

Color crosstalk compensation method for color phase-shifting fringe projection profilometry based on the phase correction matrix

Color phase-shifting fringe projection profilometry is one of the single-shot three-dimensional shape measurement techniques. The color crosstalk of the projector-camera system yields undesired phase errors when using phase-shifting method. In this paper, a color crosstalk compensation method based on phase correction matrix is proposed. In this method, the phase correction matrix is established to compensate the deviations between the actual phase-shift values in the acquired fringes and the standard ones in the ideal fringes. Only two fringe patterns are utilized to obtain the phase correction matrix. The quadratic equations for calculating the actual phase-shift values of the fringes in the three color channels are derived. The actual phase-shift values and the corresponding standard ones are employed to form the equilibrium equations for computing the phase correction coefficients in the matrix. Experimental results demonstrate the feasibility of the proposed method and it can effectively reduce the induced overall phase error caused by the color crosstalk.

Dual-wavelength Fourier ptychographic microscopy for topographic measurement

Topographic measurements of micro- or nanostructures are essential in cutting-edge scientific disciplines such as optical communications, metrology, and structural biology. Despite the advances in surface metrology, measuring micron-scale steps with wide field of view (FOV) and high-resolution remains difficult. This study demonstrates a dual-wavelength Fourier ptychographic microscopy for high-resolution topographic measurement across a wide FOV using an aperture scanning structure. This structure enables the capture of a three-dimensional (3D) sample's scattered field with two different wavelength lasers, thus allowing the axial measurement range growing from nano- to micro-scale with enhanced lateral resolution. To suppress the unavoidable noises and artifacts caused by temporal coherence, system vibration, etc., a total variation (TV) regularization algorithm is introduced for phase retrieval. A blazed grating with micron-scale steps is used as the sample to validate the performance of our method. The agreement between the high-resolution reconstructed topography with our method and that with atomic force microscopy verified the effectiveness. Meanwhile, numerical simulations suggest that the method has the potential to characterize samples with high aspect-ratio steps.

Optimization of parameters for a fringe projection measurement system by use of an improved differential evolution method

Fringe projection 3D measurement is widely used for object surface reconstruction. While improving measurement accuracy is a crucial task. Measurement accuracy is profoundly affected by various optical structural parameters. However, the current practice of system construction lacks theoretical guidelines and often relies on the experience of the operator, inevitably leading to unpredictable error. This paper investigates a theoretical optimization model and proposes an automatic optimization method for qualitatively determining the multiple optimal optical structural parameters in fringe projection measurement system. The aim is to enhance measurement accuracy conducting a rational comprehensive optimal structural parameters design prior to the system construction. Firstly, the mathematical model of the measurement system is established based on the principle of optical triangulation, and the phase sensitivity criterion is defined as the optimization norm. Within the full measurement range, the optimization merit function is formulated by combing three positions: the center position, the left and right boundary of the CCD. The imaging effectiveness criteria and sensor geometric dimensions are taken into account as the constraint boundaries. Subsequently, a combined improved differential evolution and Levy flight optimization algorithm is applied to search for the optimal parameters. The optimal structural parameters of the system were designed based on the optimization process. Experimental results validated the improvement in measurement accuracy achieved by the optimized structural parameters.

High-precision surface profilometry on a micron-groove based on dual-comb electronically controlled optical sampling

We demonstrate an optical method for 3D profilometry of micro-nano devices with large step structures. The measurement principle is based on a dual-comb direct time-of-flight detection. An electronically controlled optical sampling (ECOPS) approach is used to improve the acquisition rate. In a proof-of-principle distance measurement experiment, the measurement precision reaches 15 nm at 4000-times averages. The method has been used to characterize the profile of a large aspect-ratio rectangular micron-groove with 10 µm width and 62.3 µm depth. By point-by-point scanning, a 3D point cloud image is obtained, and the 3D profile of the micro-structure is quantitatively reconstructed with sub-micrometer precision. The proposed high-precision, high-speed surface 3D profile measurement technology could be applied to profilometry and inspection of complex microelectronics devices in the future.

Robust method to process nonuniform intensity holograms in digital holographic microscopy for nanoscale surface metrology

In this work, we propose a method based on nonlinear optimization to process holograms corrupted with nonuniform intensity fluctuations in digital holographic microscopy. Our method focuses on formulating an objective function from the recorded signal and subsequently minimizing it using a second-order optimization algorithm. We demonstrate the effectiveness of our method for phase extraction in the presence of severe noise and rapid intensity variations through extensive numerical simulations. Further, we validate the practical applicability of our method for nanoscale surface topography of standard test samples in digital holographic microscopy.

Interferometric metrology probe for highly sloped freeform optics

A wide variety of systems are employed to measure the surface profile of aspheric and freeform optical surfaces. Freeform metrology systems must accurately characterize the full surface under test, which can be difficult with steep surface slopes. Here we present an interferometric surface metrology probe for highly sloped aspheric and freeform optical surfaces. The optical design of this probe allows the measurement of surface slopes up to 50 deg without tilting the probe, which simplifies stage design and increases the accuracy of the system. A spectrally controlled light source is used to create a virtual ball in front of the probe tip to measure the surface distance and angle. This system produces a cloud of points, to which Zernike polynomials are fit and used to reconstruct the surface. We will show sensitivity tests and accuracy results.

Dynamic 3D phase-shifting profilometry based on a corner optical flow algorithm

Real-time 3D reconstruction has been applied in many fields, calling for many ongoing efforts to improve the speed and accuracy of the used algorithms. Phase shifting profilometry based on the Lucas-Kanade optical flow method is a fast and highly precise method to construct and display the three-dimensional shape of objects. However, in this method, a dense optical flow calculation is required for the modulation image corresponding to the acquired deformed fringe pattern, which consumes a lot of time and affects the real-time performance of 3D reconstruction and display. Therefore, this paper proposes a dynamic 3D phase shifting profilometry based on a corner optical flow algorithm to mitigate this issue. Therein, the Harris corner algorithm is utilized to locate the feature points of the measured object, so that the optical flow needs to calculate for only the feature points which, greatly reduces the amount of calculation time. Both our experiments and simulations show that our method improves the efficiency of pixel matching by four times and 3D reconstruction by two times.

DGE-CNN: 2D-to-3D holographic display based on a depth gradient extracting module and ZCNN network

Holography is a crucial technique for the ultimate three-dimensional (3D) display, because it renders all optical cues from the human visual system. However, the shortage of 3D contents strictly restricts the extensive application of holographic 3D displays. In this paper, a 2D-to-3D-display system by deep learning-based monocular depth estimation is proposed. By feeding a single RGB image of a 3D scene into our designed DGE-CNN network, a corresponding display-oriented 3D depth map can be accurately generated for layer-based computer-generated holography. With simple parameter adjustment, our system can adapt the distance range of holographic display according to specific requirements. The high-quality and flexible holographic 3D display can be achieved based on a single RGB image without 3D rendering devices, permitting potential human-display interactive applications such as remote education, navigation, and medical treatment.

Structured light 3-D sensing for scenes with discontinuous reflectivity: error removal based on scene reconstruction and normalization

Structured light-based 3-D sensing technique reconstructs the 3-D shape from the disparity given by pixel correspondence of two sensors. However, for scene surface containing discontinuous reflectivity (DR), the captured intensity deviates from its actual value caused by the non-ideal camera point spread function (PSF), thus generating 3-D measurement error. First, we construct the error model of fringe projection profilometry (FPP). From which, we conclude that the DR error of FPP is related to both the camera PSF and the scene reflectivity. The DR error of FPP is hard to be alleviated because of unknown scene reflectivity. Second, we introduce single-pixel imaging (SI) to reconstruct the scene reflectivity and normalize the scene with scene reflectivity "captured" by the projector. From the normalized scene reflectivity, pixel correspondence with error opposite to the original reflectivity is calculated for the DR error removal. Third, we propose an accurate 3-D reconstruction method under discontinuous reflectivity. In this method, pixel correspondence is first established by using FPP, and then refined by using SI with reflectivity normalization. Both the analysis and the measurement accuracy are verified under scenes with different reflectivity distributions in the experiments. As a result, the DR error is effectively alleviated while taking an acceptable measurement time.

A Spectroscopic Reflectance-Based Low-Cost Thickness Measurement System for Thin Films: Development and Testing

The requirement for alternatives in roll-to-roll (R2R) processing to expand thin film inspection in wider substrates at lower costs and reduced dimensions, and the need to enable newer control feedback options for these types of processes, represents an opportunity to explore the applicability of newer reduced-size spectrometers sensors. This paper presents the hardware and software development of a novel low-cost spectroscopic reflectance system using two state-of-the-art sensors for thin film thickness measurements. The parameters to enable the thin film measurements using the proposed system are the light intensity for two LEDs, the microprocessor integration time for both sensors and the distance from the thin film standard to the device light channel slit for reflectance calculations. The proposed system can deliver better-fit errors compared with a HAL/DEUT light source using two methods: curve fitting and interference interval. By enabling the curve fitting method, the lowest root mean squared error (RMSE) obtained for the best combination of components was 0.022 and the lowest normalised mean squared error (MSE) was 0.054. The interference interval method showed an error of 0.09 when comparing the measured with the expected modelled value. The proof of concept in this research work enables the expansion of multi-sensor arrays for thin film thickness measurements and the potential application in moving environments.

Pixel-wise rational model for a structured light system

This Letter presents a novel structured light system model that effectively considers local lens distortion by pixel-wise rational functions. We leverage the stereo method for initial calibration and then estimate the rational model for each pixel. Our proposed model can achieve high measurement accuracy within and outside the calibration volume, demonstrating its robustness and accuracy.

Flexible structured light system calibration method with all digital features

We propose an innovative method for single-camera and single-projector structured light system calibration in that it eliminates the need for calibration targets with physical features. Instead, a digital display such as a liquid crystal display (LCD) screen is used to present a digital feature pattern for camera intrinsic calibration, while a flat surface such as a mirror is used for projector intrinsic and extrinsic calibration. To carry out this calibration, a secondary camera is required to facilitate the entire process. Because no specially made calibration targets with real physical features are required for the entire calibration process, our method offers greater flexibility and simplicity in achieving accurate calibration for structured light systems. Experimental results have demonstrated the success of this proposed method.

A Method for Spatially Registered Microprofilometry Combining Intensity-Height Datasets from Interferometric Sensors

A recognized problem in profilometry applied to artworks is the spatial referencing of the surface topography at micrometer scale due to the lack of references in the height data with respect to the "visually readable" surface. We demonstrate a novel workflow for spatially referenced microprofilometry based on conoscopic holography sensors for scanning in situ heterogeneous artworks. The method combines the raw intensity signal collected by the single-point sensor and the (interferometric) height dataset, which are mutually registered. This dual dataset provides a surface topography registered to the artwork features up to the precision that is given by the acquisition scanning system (mainly, scan step and laser spot). The advantages are: (1) the raw signal map provides additional information about materials texture, e.g., color changes or artist marks, for spatial registration and data fusion tasks; (2) and microtexture information can be reliably processed for precision diagnostic tasks, e.g., surface metrology in specific sub-domains and multi-temporal monitoring. Proof of concept is given with exemplary applications: book heritage, 3D artifacts, surface treatments. The potential of the method is clear for both quantitative surface metrology and qualitative inspection of the morphology, and it is expected to open future applications for microprofilometry in heritage science.

Fourier ptychographic topography

Topography measurement is essential for surface characterization, semiconductor metrology, and inspection applications. To date, performing high-throughput and accurate topography remains challenging due to the trade-off between field-of-view (FOV) and spatial resolution. Here we demonstrate a novel topography technique based on the reflection-mode Fourier ptychographic microscopy, termed Fourier ptychograhpic topography (FPT). We show that FPT provides both a wide FOV and high resolution, and achieves nanoscale height reconstruction accuracy. Our FPT prototype is based on a custom-built computational microscope consisting of programmable brightfield and darkfield LED arrays. The topography reconstruction is performed by a sequential Gauss-Newton-based Fourier ptychographic phase retrieval algorithm augmented with total variation regularization. We achieve a synthetic numerical aperture (NA) of 0.84 and a diffraction-limited resolution of 750 nm, increasing the native objective NA (0.28) by 3×, across a 1.2 × 1.2 mm² FOV. We experimentally demonstrate the FPT on a variety of reflective samples with different patterned structures. The reconstructed resolution is validated on both amplitude and phase resolution test features. The accuracy of the reconstructed surface profile is benchmarked against high-resolution optical profilometry measurements. In addition, we show that the FPT provides robust surface profile reconstructions even on complex patterns with fine features that cannot be reliably measured by the standard optical profilometer. The spatial and temporal noise of our FPT system is characterized to be 0.529 nm and 0.027 nm, respectively.

Development of an Optoelectronic Integrated Sensor for a MEMS Mirror-Based Active Structured Light System

Micro-electro-mechanical system (MEMS) scanning micromirrors are playing an increasingly important role in active structured light systems. However, the initial phase error of the structured light generated by a scanning micromirror seriously affects the accuracy of the corresponding system. This paper reports an optoelectronic integrated sensor with high irradiance responsivity and high linearity that can be used to correct the phase error of the micromirror. The optoelectronic integrated sensor consists of a large-area photodetector (PD) and a receiving circuit, including a post amplifier, an operational amplifier, a bandgap reference, and a reference current circuit. The optoelectronic sensor chip is fabricated in a 180 nm CMOS process. Experimental results show that with a 5 V power supply, the optoelectronic sensor has an irradiance responsivity of 100 mV/(μW/cm²) and a -3 dB bandwidth of 2 kHz. The minimal detectable light power is about 19.4 nW, which satisfies the requirements of many active structured light systems. Through testing, the application of the chip effectively reduces the phase error of the micromirror to 2.5%.

Dynamic pressure surface deformation measurement based on a chromatic confocal sensor

To enable the real-time measurement of pressure deformations in sealed cavities, a high-precision method of detecting the deformation of a material surface is proposed. By combining a chromatic confocal displacement sensor with a pressure sensor, we can acquire dynamic online strain measurements that consider the effects of the deformed material and internal environmental conditions. A 90m m×90m m static mechanical cylindrical cavity is simulated using finite element software. The interior of the cylindrical cavity is continuously pressurized at up to 400 kPa with a material deformation of 300 µm. We experimentally obtain the spectral peak wavelengths corresponding to the surface deformation experiment, record the spectral data at 20 kPa intervals, and use the Voigt fitting algorithm to reduce sensor errors. The results show that the experimental results differ from the simulated results by 1.43 µm, with a relative error of 0.083% after sequential pressurization and depressurization using a pressure calibrator, and the uncertainty error of pressure deformation measurement is 1.495 µm. Thus, the proposed method is robust against external disturbances and is suitable for micrometer-level surface deformation monitoring, which has numerous applications in the high-precision inspection industry.

From picture to 3D hologram: end-to-end learning of real-time 3D photorealistic hologram generation from 2D image input

In this Letter, we demonstrate a deep-learning-based method capable of synthesizing a photorealistic 3D hologram in real-time directly from the input of a single 2D image. We design a fully automatic pipeline to create large-scale datasets by converting any collection of real-life images into pairs of 2D images and corresponding 3D holograms and train our convolutional neural network (CNN) end-to-end in a supervised way. Our method is extremely computation-efficient and memory-efficient for 3D hologram generation merely from the knowledge of on-hand 2D image content. We experimentally demonstrate speckle-free and photorealistic holographic 3D displays from a variety of scene images, opening up a way of creating real-time 3D holography from everyday pictures.

A Novel Method for Pose and Position Calibration of Laser Displacement Sensors

Laser displacement sensors are widely used in the aviation industry for the purpose of surface normal measurements. The measurement of a surface normal depends on prior knowledge of the poses and positions of the sensors, which are obtained through calibration. This paper introduces a new parameter to the traditional calibration procedure, to reduce the calibration error, and explores the factors affecting calibration using the Monte Carlo method. In the experiment, the normal measurement error of the probe consisted of four sensors after calibration was less than 0.1∘, which satisfied the established requirements. This paper indicates the boundary conditions for a successful calibration and validates the proposed method, which provides a new method for the pose and position calibration of laser displacement sensors and other similar sensors.

Prediction and Optimization Algorithm for Intersection Point of Spatial Multi-Lines Based on Photogrammetry

The basic theory of photogrammetry is mature and widely used in engineering. The environment in engineering is very complex, resulting in the corners or multi-line intersections being blocked and unable to be measured directly. In order to solve this problem, a prediction and optimization algorithm for intersection point of spatial multi-lines based on photogrammetry is proposed. The coordinates of points on space lines are calculated by photogrammetry algorithm. Due to the influence of image point distortion and point selection error, many lines do not strictly intersect at one point. The equations of many space lines are used to fit their initial value of intersection point. The initial intersection point is projected onto each image, and the distances between the projection point and each line on the image plane are used to weight the calculated spatial lines in combination with the information entropy. Then the intersection point coordinates are re-fitted, and the intersection point is repeatedly projected and recalculate until the error is less than the threshold value or reached the set number of iterations. Three different scenarios are selected for experiments. The experimental results show that the proposed algorithm significantly improves the prediction accuracy of the intersection point.

An Improved Synthesis Phase Unwrapping Method Based on Three-Frequency Heterodyne

An improved three-frequency heterodyne synthesis phase unwrapping method is proposed to improve the measurement accuracy through phase difference and phase sum operations. This method can reduce the effect of noise and increase the equivalent phase frequency. According to the distribution found in the phase difference calculation process, the Otsu segmentation is introduced to judge the phase threshold. The equivalent frequency obtained from the phase sum is more than those of all projected fringe patterns. In addition, the appropriate period combinations are also studied. The simulations and related experiments demonstrate the feasibility of the proposed method and the ability to improve the accuracy of the measurement results further.

Optical three-dimensional shape measurement based on structured light and a binocular vision system

Three-dimensional shape measurement based on structured light is affected by two factors: the number of fringe patterns and the phase unwrapping process. Although one-shot technology can get the wrapped phase, it is not suitable for measuring complex surface. Moreover, phase unwrapping also affects measurement speed and accuracy. To overcome these problems, a two-dimensional wavelet transform with binocular vision system is proposed. Wavelet transform is used to get the wrapped phase based on the Morlet wavelet. In order to get a three-dimensional shape without phase unwrapping, a binocular vision system is used. The increase matching accuracy, the preliminary disparity, and the sub-pixel optimization are calculated, respectively. Based on the calibration parameters, three-dimensional information can be obtained directly from the wrapped phase. In addition, the average phase is calculated based on ambient pixels to confirm wrapped phase boundary. Experimental results demonstrate the feasibility and advantage of the proposed method. Compared with traditional methods, both measurement accuracy and measurement speed can be increased.

3D Metrology Using One Camera with Rotating Anamorphic Lenses

In this paper, a novel 3D metrology method using one camera with rotating anamorphic lenses is presented based on the characteristics of double optical centers for anamorphic imaging. When the anamorphic lens rotates -90° around its optical axis, the 3D data of the measured object can be reconstructed from the two anamorphic images captured before and after the anamorphic rotation. The anamorphic lens imaging model and a polynomial anamorphic distortion model are firstly proposed. Then, a 3D reconstruction model using one camera with rotating anamorphic lenses is presented. Experiments were carried out to validate the proposed method and evaluate its measurement accuracy. Compared with stereo vision, the main advantage of the proposed 3D metrology approach is the simplicity of point matching, which makes it suitable for developing compact sensors for fast 3D measurements, such as car navigation applications.

Pixel-wise structured light calibration method with a color calibration target

We propose to use a calibration target with a narrow spectral color range for the background (e.g., from blue) and broader spectral color range for the feature points (e.g., blue + red circles), and fringe patterns matching the background color for accurate phase extraction. Since the captured fringe patterns are not affected by the high contrast of the calibration target, phase information can be accurately extracted without edging artifacts. Those feature points can be clearly "seen" by the camera if the ambient light matches the feature color or without the background color. We extract each calibration pose for three-dimensional coordinate determination for each pixel, and then establish pixel-wise relationship between each coordinate and phase. Comparing with our previously published method, this method significantly fundamentally simplifies and improves the algorithm by eliminating the computational framework estimate smooth phase near high-contrast feature edges. Experimental results demonstrated the success of our proposed calibration method.

Fringe projection profilometry method with high efficiency, precision, and convenience: theoretical analysis and development

Fringe projector profilometry (FPP) is an important three-dimensional (3D) measurement technique, especially when high precision and speed are required. Thus, theoretical interrogation is critical to provide deep understanding and possible improvement of FPP. By dividing an FPP measurement process into four steps (system calibration, phase measurement, pixel correspondence, and 3D reconstruction), we give theoretical analysis on the entire process except for the extensively studied calibration step. Our study indeed reveals a series of important system properties, to the best of our knowledge, for the first time: (i) in phase measurement, the optimal and worst fringe angles are proven perpendicular and parallel to epipolar line, respectively, and can be considered as system parameters and can be directly made available during traditional calibration, highlighting the significance of the epipolar line; (ii) in correspondence, when two sets of fringes with different fringe orientations are projected, the highest correspondence precision can be achieved with arbitrary orientations as long as these two orientations are perpendicular to each other; (iii) in reconstruction, a higher reconstruction precision is given by the 4-equation methods, while we notice that the 3-equation methods are almost dominatingly used in literature. Based on these theoretical results, we propose a novel FPP measurement method which (i) only projects one set of fringes with optimal fringe angle to explicitly work together with the epipolar line for precise pixel correspondence; (ii) for the first time, the optimal fringe angle is determined directly from the calibration parameters, instead of being measured; (iii) uses 4 equations for precise 3D reconstruction but we can remove one equation which is equivalent to an epipolar line, making it the first algorithm that can use 3-equation solution to achieve 4-equation precision. Our method is efficient (only one set of fringe patterns is required in projection and the speed is doubled in reconstruction), precise (in both pixel correspondence and 3D reconstruction), and convenient (the computable optimal fringe angle and a closed-form 3-equation solution). We also believe that our work is insightful in revealing fundamental FPP properties, provides a more reasonable measurement for practice, and thus is beneficial to further FPP studies.

Surface Metrology Based on Scanning Conoscopic Holography for In Situ and In-Process Monitoring of Microtexture in Paintings

In the field of engineering, surface metrology is a valuable tool codified by international standards that enables the quantitative study of small-scale surface features. However, it is not recognized as a resource in the field of cultural heritage. Motivated by this fact, in this work, we demonstrate the use and the usefulness of surface metrology based on scanning conoscopic holography for monitoring treatments on the Venetian masterpiece by Tintoretto St. Martial in Glory with the Saints Peter and Paul. We carried out in situ and in-process monitoring of the painting microtexture during an experimental, innovative laser-chemical treatment, and we performed a statistical analysis based on ISO areal field parameters. A wide and in-band roughness analysis through the complementary use of amplitude, spatial, and hybrid parameters confirmed the noninvasive nature of the whole treatment on the painting surface topography, giving us the chance to review and critically discuss the use of these parameters in a real case in heritage science.

Time-overlapping structured-light projection: high performance on 3D shape measurement for complex dynamic scenes

High-speed three-dimensional (3D) shape measurement has been continuously researched due to the demand for analyzing dynamic behavior in transient scenes. In this work, a time-overlapping structured-light 3D shape measuring technique is proposed to realize high-speed and high-performance measurement on complex dynamic scenes. Time-overlapping structured-light projection is presented to maximumly reduce the information redundancy in temporal sequences and improve the measuring efficiency; generalized tripartite phase unwrapping (Tri-PU) is used to ensure the measuring robustness; fringe period extension is achieved by improving overlapping rate to further double the encoding fringe periods for higher measuring accuracy. Based on the proposed measuring technique, one new pixel-to-pixel and unambiguous 3D reconstruction result can be updated with three newly required patterns at a reconstruction rate of 3174 fps. Three transient scenes including collapsing wood blocks struck by a flying arrow, free-falling foam snowflakes and flying water balloon towards metal grids were measured to verify the high performance of the proposed method in various complex dynamic scenes.

Quality Assessment of a Novel Camera-Based Measurement System for Roughness Determination of Concrete Surfaces-Accuracy Evaluation and Validation

The roughness of a surface is a decisive parameter of a material. In rehabilitation of concrete structures, for example, it significantly affects the adhesion between the coating material and the base concrete. However, the standard measurement procedure in construction suffers from considerable disadvantages, which leads to the demand for more sophisticated methods. In a research project, we, therefore, developed a novel camera-based measurement system, which is customized to meet the prevailing requirements for practical use on construction sites. In this article, we provide an overview of the measurement system and present comprehensive examinations to evaluate the accuracy and to provide evidence of validity. First, we examined the accuracy of the system by empirically assessing both trueness and precision of measurements using three concrete specimens. Trueness was determined by comparing the surface measurements to those of a highly accurate microscope system, revealing RMSE values of around 40–50 µm. Precision, on the other hand, was assessed considering the scattering of the roughness measurements under repeat conditions, which led to standard deviations of less than 6 µm. Furthermore, to proof validity, a comparative study was conducted based on sixteen concrete specimens, which includes the sand patch method and laser triangulation as established roughness measurement methods in practice. The empirically determined correlation coefficients between all three methods were greater than 0.99, indicating extraordinarily high linear relationships. Among them, the greatest correlation was between the camera-based system and laser triangulation.

Linear Laser Scanning Measurement Method Tracking by a Binocular Vision

The 3D scanning of a freeform structure relies on the laser probe and the localization system. The localization system, determining the effect of the point cloud reconstruction, will generate positioning errors when the laser probe works in complex paths with a fast speed. To reduce the errors, in this paper, a linear laser scanning measurement method is proposed based on binocular vision calibration. A simple and effective eight-point positioning marker attached to the scanner is proposed to complete the positioning and tracking procedure. Based on this, the method of marked point detection based on image moment and the principle of global coordinate system calibration are introduced in detail. According to the invariance principle of space distance, the corresponding points matching method between different coordinate systems is designed. The experimental results show that the binocular vision system can complete localization under different light intensities and complex environments, and that the repeated translation error of the binocular vision system is less than 0.22 mm, while the rotation error is less than 0.15°. The repeated error of the measurement system is less than 0.36 mm, which can meet the requirements of the 3D shape measurement of the complex workpiece.

Three-dimensional computer holography enabled from a single 2D image

To compute a high-quality computer-generated hologram (CGH) for true 3D real scenes, a huge amount of 3D data must be physically acquired and provided depending on specific devices or 3D rendering techniques. Here, we propose a computational framework for generating a CGH from a single image based on the idea of 2D-to-3D wavefront conversion. We devise a deep view synthesis neural network to synthesize light-field contents from a single image and convert the light-field data to the diffractive wavefront of the hologram using a ray-wave algorithm. The method is able to achieve extremely straightforward 3D CGH generation from hand-accessible 2D image content and outperforms existing real-world-based CGH computation, which inevitably relies on a high-cost depth camera and cumbersome 3D data rendering. We experimentally demonstrate 3D reconstructions of indoor and outdoor scenes from a single image enabled phase-only CGH.

Calibration method for parabolic reflector measurement by using reverse Hartmann test

Careful calibration of system geometric position and camera errors is crucial to achieve high testing accuracy for large-scale aspherical surfaces. We propose a geometrical optical calibration method for increasing the overall accuracy of a large-scale reverse Hartmann test system to realize the parabolic reflector measurement. By using a flat crystal, a mask with uniform holes and an external pinhole aperture in front of the camera, camera lens distortions and keystone distortion, as well as system geometry correspondence between LCD screen, camera, and reflecting point on the test surface, are determined accurately. We experimentally demonstrated that the proposed calibration method achieves high slope detection accuracy for a large-scale parabolic reflector: tens of microradians within 1620(H)×850(V)mm².

Dynamic shape measurement for objects with patterns by Fourier fringe projection profilometry based on variational decomposition and multi-scale Retinex

In practical measurement, we often need to measure the shape of objects with patterns or letters. As far as we know, no paper has ever reported the shape measurement for objects with patterns or letters by Fourier fringe projection profilometry (FPP). In this paper, we propose a method based on the variational decomposition TV-Hilbert-L² model and multi-scale Retinex (MSR) to measure the shape of objects with patterns and letters by Fourier FPP. In this method, we first use the TV-Hilbert-L² model to obtain the fringe part, then perform MSR enhancement on the fringe part, and finally decompose the enhanced fringe part with TV-Hilbert-L² again. We evaluate the performance of this method via application to one computer-simulated noisy fringe projection pattern and two experimental fringe projection patterns with different types of patterns or letters, and comparison with the Fourier transform method, the variational image decomposition TV-Hilbert-L² model. Furthermore, we apply the proposed method to the dynamic three-dimensional shape measurement of hand posture with pattern. The experimental results show that our method can effectively measure the dynamic shape of objects with patterns or letters from a single-frame fringe projection pattern.

High-efficiency 3D reconstruction with a uniaxial MEMS-based fringe projection profilometry

Micro-Electro-Mechanical System (MEMS) scanning is increasingly popular in 3D surface measurement with the merits of the compact structure and high frame-rate. In this paper, we achieve real-time fringe structured 3D reconstruction by using a uniaxial MEMS-based projector. To overcome the limitations on uniaxial MEMS-based projector of lensless structure and unidirectional fringe projection, a novel isophase plane model is proposed, in which the laser line from MEMS-based projector is regarded as an isophase plane. Our model directly establishes the mapping relationship between phase and spatial 3D coordinates through the intersection point of camera back-projection light ray and isophase plane. Furthermore, a flexible calibration strategy to obtain 3D mapping coefficients is introduced with a specially designed planar target. Experiments demonstrated that our method can achieve high-accuracy and real-time 3D reconstruction.

4D line-scan hyperspectral imaging

This paper proposes a 4D line-scan hyperspectral imager that combines 3D geometrical measurement and spectral detection with high spectral resolution and spatial accuracy. We investigated the geometrical optical model of a camera attaching with a spectrograph, theoretically explored the mathematical model for line-scan fringe projection profilometry, and established the 3D reconstruction and calibration methods under this proposed line-scan high-dimensional imaging system. The spectral resolution of the system is 2.8 nm, and the spatial root-mean-square-error is 0.0895 mm when measuring a standard sphere with a diameter of 40.234 mm. We measure a colored statue to showcase the intensity change along the dimension of wavelength. In addition, the quality and defect of the spinach leaves are inspected based on spectral data and depth data, which demonstrates the potential application of the system in the food industry.

A super-grayscale and real-time computer-generated Moiré profilometry using video grating projection

By using the time-division multiplexing characteristics of the projector and the integral exposure characteristics of the charge coupled device (CCD) camera, a super-grayscale and real-time computer-generated Moiré profilometry based on video grating projection is proposed. The traditional digital static grating is of 256-grayscale at most. If an expected super-grayscale grating with a maximum grayscale of 766 is designed and divided into three 256-grayscale fringe patterns with balanced grayscale as far as possible, they can be synthesized into a repeated playing video grating instead of the traditional static grating. When the video grating is projected onto the measured object, as long as the exposure time is set to three times the refresh cycle of the video grating, the super-grayscale deformed patterns in the 766-grayscale can be captured with a 10-bit CCD camera, so that the deformed patterns are realistic. The digital error in computer-generated Moiré profilometry is effectively reduced. In addition, this method can expand the linear range of the deformed pattern by 20% in computer Moiré profilometry. Therefore, the proposed method has the perspectives of high accuracy and real-time measurement. Theoretical analysis and experimental results demonstrate the validity and capability of the proposed method.

Paraxial 3D shape measurement using parallel single-pixel imaging

Three-dimensional (3D) shape measurement with fringe projection technique and vertical scanning setup can alleviate the problem of shadow and occlusion. However, the shape-from-defocus based method suffers from limited sensitivity and low signal-to-noise ratio (SNR), whereas the projection-triangulation based is sensitive to the zero-phase detection. In this paper, we propose paraxial 3D shape measurement using parallel single-pixel imaging (PSI). The depth is encoded in the radial distance to the projector optical center, which is determined by the projection of light transport coefficients (LTCs). The third-order polynomial fitting is used for depth mapping and calibration. Experiments on 5 objects with different materials and textures are conducted, and standards are measured to test the accuracy. The results verified that the proposed method can achieve robust, dense reconstruction with depth accuracy at 20 μm while the root-mean-square error (RMSE) of plane fitting up to 43 μm.

In-Line Measurement of the Surface Texture of Rolls Using Long Slender Piezoresistive Microprobes

Long slender piezoresistive silicon microprobes are a new type of sensor for measurement of surface roughness. Their advantage is the ability to measure at speeds of up to 15 mm/s, which is much faster than conventional stylus probes. The drawbacks are their small measurement range and tendency to break easily when deflected by more than the allowed range of 1 mm. In this article, previously developed microprobes were tested in the laboratory to evaluate their metrological properties, then tested under industrial conditions. There are several industrial measurement applications in which microprobes are useful. Measurement of the roughness of paper machine rolls was selected for testing in this study. The integration of a microprobe into an existing roll measurement device is presented together with the measurement results. The results are promising, indicating that measurements using a microprobe can give useful data on the grinding process.

3D Measurement of Structured Specular Surfaces Using Stereo Direct Phase Measurement Deflectometry

With the rapid development of modern manufacturing processes, ultra-precision structured freeform surfaces are being widely explored for components with special surface functioning. Measurement of the 3D surface form of structured specular objects remains a challenge because of the complexity of the surface form. Benefiting from a high dynamic range and large measuring area, phase measurement deflectometry (PMD) exhibits great potential in the inspection of the specular surfaces. However, the PMD is insensitive to object height, which leads to the PMD only being used for smooth specular surface measurement. Direct phase measurement deflectometry (DPMD) has been introduced to measure structured specular surfaces, but the surface form measurement resolution and accuracy are limited. This paper presents a method named stereo-DPMD for measuring structured specular objects by introducing a stereo deflectometor into DPMD, so that it combines the advantages of slope integration of the stereo deflectometry and discontinuous height measurement from DPMD. The measured object is separated into individual continuous regions, so the surface form of each region can be recovered precisely by slope integration. Then, the relative positions between different regions are evaluated by DPMD system to reconstruct the final 3D shape of the object. Experimental results show that the structured specular surfaces can be measured accurately by the proposed stereo-DPMD method.

Infrared phase measuring deflectometry by using defocused binary fringe

Three-dimensional surface information acquisition of specular objects plays an important role in the fields of automobile industry, aerospace, cultural relic protection, intelligent robotics, equipment manufacturing, and so on. Most of the existing specular surface measurement methods are based on focused sinusoidal fringe patterns, so there are certain requirements for the range of the depth of field (DOF) of the camera on the focus position. However, for many specular surfaces with a large gradient, the tested objects may not always be in the DOF of the camera, so sinusoidal fringe patterns are defocused to be vulnerable to the noise. In this Letter, a new infrared phase measuring deflectometry (PMD) based on defocused binary fringe is proposed that combines a binary fringe defocusing technique and direct PMD. The measurement principle and the corresponding system calibration method are described. The feasibility and measurement accuracy of fringe defocus in specular measurement are studied in principle. The experimental results on several specular objects show that the proposed method can effectively measure specular surfaces out of the DOF of the camera.

Phase Demodulation Method for Fringe Projection Measurement Based on Improved Variable-Frequency Coded Patterns

The phase-to-height imaging model, as a three-dimensional (3D) measurement technology, has been commonly applied in fringe projection to assist surface profile measurement, where the efficient and accurate calculation of phase plays a critical role in precise imaging. To deal with multiple extra coded patterns and 2π jump error caused to the existing absolute phase demodulation methods, a novel method of phase demodulation is proposed based on dual variable-frequency (VF) coded patterns. In this paper, the frequency of coded fringe is defined as the number of coded fringes within a single sinusoidal fringe period. First, the effective wrapped phase (EWP) as calculated using the four-step phase shifting method was split into the wrapped phase region with complete period and the wrapped phase region without complete period. Second, the fringe orders in wrapped phase region with complete period were decoded according to the frequency of the VF coded fringes and the continuous characteristic of the fringe order. Notably, the sampling frequency of fast Fourier transform (FFT) was determined by the length of the decoding interval and can be adjusted automatically with the variation in height of the object. Third, the fringe orders in wrapped phase region without complete period were decoded depending on the consistency of fringe orders in the connected region of wrapped phase. Last, phase demodulation was performed. The experimental results were obtained to confirm the effectiveness of the proposed method in the phase demodulation of both discontinuous objects and highly abrupt objects.

Improved spatial-shifting two-wavelength algorithm for 3D shape measurement with a look-up table

Conventional two-wavelength algorithms have been broadly used for three-dimensional shape measurement. However, the maximum unambiguous range of phase unwrapping depends on the least-common multiple of two wavelengths, and thus coprime wavelengths are commonly selected. The recently proposed spatial-shifting two-wavelength (SSTW) algorithm can achieve the maximum unambiguous range with two non-coprime wavelengths, but this algorithm tends to fail for some wavelength selections. To address this problem, this paper presents a general look-up-table-based SSTW (LUT-SSTW) algorithm with arbitrary wavelength selection. The paper also analyzes the phase unwrapping robustness in terms of phase errors and provides guidance for wavelength selection. In addition, an improved LUT-SSTW algorithm is developed to enhance the phase unwrapping robustness, and further relax wavelength selection. Some experiments have been conducted, and their results verify the efficiency of the proposed method.

Method for large-scale structured-light system calibration

We propose a multi-stage calibration method for increasing the overall accuracy of a large-scale structured light system by leveraging the conventional stereo calibration approach using a pinhole model. We first calibrate the intrinsic parameters at a near distance and then the extrinsic parameters with a low-cost large-calibration target at the designed measurement distance. Finally, we estimate pixel-wise errors from standard stereo 3D reconstructions and determine the pixel-wise phase-to-coordinate relationships using low-order polynomials. The calibrated pixel-wise polynomial functions can be used for 3D reconstruction for a given pixel phase value. We experimentally demonstrated that our proposed method achieves high accuracy for a large volume: sub-millimeter within 1200(H) × 800 (V) × 1000(D) mm³.

Polarization-Dependent All-Dielectric Metasurface for Single-Shot Quantitative Phase Imaging

Phase retrieval is a noninterferometric quantitative phase imaging technique that has become an essential tool in optical metrology and label-free microscopy. Phase retrieval techniques require multiple intensity measurements traditionally recorded by camera or sample translation, which limits their applicability mostly to static objects. In this work, we propose the use of a single polarization-dependent all-dielectric metasurface to facilitate the simultaneous recording of two images, which are utilized in phase calculation based on the transport-of-intensity equation. The metasurface acts as a multifunctional device that splits two orthogonal polarization components and adds a propagation phase shift onto one of them. As a proof-of-principle, we demonstrate the technique in the wavefront sensing of technical samples using a standard imaging setup. Our metasurface-based approach fosters a fast and compact configuration that can be integrated into commercial imaging systems.

Adapted Fringe Projection Sequences for Changing Illumination Conditions on the Example of Measuring a Wrought-Hot Object Influenced by Forced Cooling

Optical 3D geometry reconstruction, or more specific, fringe projection profilometry, is a state-of-the-art technique for the measurement of the shape of objects in confined spaces or under rough environmental conditions, e.g., while inspecting a wrought-hot specimen after a forging operation. While the contact-less method enables the measurement of such an object, the results are influenced by the light deflection effect occurring due to the inhomogeneous refractive index field induced by the hot air around the measurand. However, the developed active compensation methods to fight this issue exhibits a major drawback, namely an additional cooling of the object and a subsequent transient illumination component. In this paper, we investigate the cooling and its effect on temporal phase reconstruction algorithms and take a theoretical approach to its compensation. The simulated compensation measures are transferred to a fringe projection profilometry setup and are evaluated using established and newly developed methods. The results show a significant improvement when measuring specimens under a transient illumination and are easily transferable to any kind of multi-frequency phase-shift measurement.

Dataset License: CC-BY

Data-Driven Intelligent 3D Surface Measurement in Smart Manufacturing: Review and Outlook

High-fidelity characterization and effective monitoring of spatial and spatiotemporal processes are crucial for high-performance quality control of many manufacturing processes and systems in the era of smart manufacturing. Although the recent development in measurement technologies has made it possible to acquire high-resolution three-dimensional (3D) surface measurement data, it is generally expensive and time-consuming to use such technologies in real-world production settings. Data-driven approaches that stem from statistics and machine learning can potentially enable intelligent, cost-effective surface measurement and thus allow manufacturers to use high-resolution surface data for better decision-making without introducing substantial production cost induced by data acquisition. Among these methods, spatial and spatiotemporal interpolation techniques can draw inferences about unmeasured locations on a surface using the measurement of other locations, thus decreasing the measurement cost and time. However, interpolation methods are very sensitive to the availability of measurement data, and their performances largely depend on the measurement scheme or the sampling design, i.e., how to allocate measurement efforts. As such, sampling design is considered to be another important field that enables intelligent surface measurement. This paper reviews and summarizes the state-of-the-art research in interpolation and sampling design for surface measurement in varied manufacturing applications. Research gaps and future research directions are also identified and can serve as a fundamental guideline to industrial practitioners and researchers for future studies in these areas.

High-Accuracy 3-D Sensor for Rivet Inspection Using Fringe Projection Profilometry with Texture Constraint

Riveted workpieces are widely used in manufacturing; however, current inspection sensors are mainly limited in nondestructive testing and obtaining the high-accuracy dimension automatically is difficult. We developed a 3-D sensor for rivet inspection using fringe projection profilometry (FPP) with texture constraint. We used multi-intensity high dynamic range (HDR) FPP method to address the varying reflectance of the metal surface then utilized an additional constraint calculated from the fused HDR texture to compensate for the artifacts caused by phase mixture around the stepwise edge. By combining the 2-D contours and 3-D FPP data, rivets can be easily segmented, and the edge points can be further refined for diameter measurement. We tested the performance on a sample of riveted aluminum frame and evaluated the accuracy using standard objects. Experiments show that denser 3-D data of a riveted metal workpiece can be acquired with high accuracy. Compared with the traditional FPP method, the diameter measurement accuracy can be improved by 50%.

Calibration method for the electrically tunable lens based on shape-changing polymer

In this paper, a calibration method for the camera system with electrically tunable lens (ETL) based on shape-changing polymer (SCP) is proposed to improve the accuracy, robustness and practicality of the system. The camera model of the ETL based on SCP is proposed based on the analyses of its optical properties. The calibration strategy, including initial estimation of camera parameters and bundle adjustment is presented. To eliminate the influence of temperature on ETL in machine vision applications, a real-time temperature compensation method is proposed. The proposed method makes use of the existing calibration hardware without adding new components to the system. Both simulations and experiments are conducted to evaluate the effectiveness and accuracy of the proposed camera model and calibration method. The measurement error with the proposed calibration method is below 20 microns at high magnification, whose measurement accuracy is improved by five times than the existing method at high magnification. With the proposed calibration method for the camera system with ETL based on SCP, the calibration workload is reduced and accurate calibration at high magnification is achieved. It also benefits the development of autofocusing 3D measurement technology.