Novel 3D measurement system based on speckle and fringe pattern projection

Periodic diffractive optical element for high-density and large-scale spot array structured light projection

Structured light projection has been widely used for depth sensing in computer vision. Diffractive optical elements (DOEs) play a crucial role in generating structured light projected onto objects, and spot array is a common projection pattern. However, the primary metrics of the spot array, including density and field of view, are restricted by the principle of diffraction and its calculation. In this paper, a novel, to the best of our knowledge, method is proposed to achieve high-density periodic spot array on a large scale. Further, periodic DOEs, for the first time, are optimized to increase the density of the spot array without decreasing the periods of the DOE. Simulation and experimental results of high-density and large-scale spot array structured light projection are presented, demonstrating the effectiveness of the proposed method.

Semi-Global Matching Assisted Absolute Phase Unwrapping

Measuring speed is a critical factor to reduce motion artifacts for dynamic scene capture. Phase-shifting methods have the advantage of providing high-accuracy and dense 3D point clouds, but the phase unwrapping process affects the measurement speed. This paper presents an absolute phase unwrapping method capable of using only three speckle-embedded phase-shifted patterns for high-speed three-dimensional (3D) shape measurement on a single-camera, single-projector structured light system. The proposed method obtains the wrapped phase of the object from the speckle-embedded three-step phase-shifted patterns. Next, it utilizes the Semi-Global Matching (SGM) algorithm to establish the coarse correspondence between the image of the object with the embedded speckle pattern and the pre-obtained image of a flat surface with the same embedded speckle pattern. Then, a computational framework uses the coarse correspondence information to determine the fringe order pixel by pixel. The experimental results demonstrated that the proposed method can achieve high-speed and high-quality 3D measurements of complex scenes.

Digital image correlation assisted absolute phase unwrapping

This paper presents an absolute phase unwrapping method for high-speed three-dimensional (3D) shape measurement. This method uses three phase-shifted patterns and one binary random pattern on a single-camera, single-projector structured light system. We calculate the wrapped phase from phase-shifted images and determine the coarse correspondence through the digital image correlation (DIC) between the captured binary random pattern of the object and the pre-captured binary random pattern of a flat surface. We then developed a computational framework to determine fringe order number pixel by pixel using the coarse correspondence information. Since only one additional pattern is used, the proposed method can be used for high-speed 3D shape measurement. Experimental results successfully demonstrated that the proposed method can achieve high-speed and high-quality measurement of complex scenes.

Advances and Prospects of Vision-Based 3D Shape Measurement Methods

Vision-based three-dimensional (3D) shape measurement techniques have been widely applied over the past decades in numerous applications due to their characteristics of high precision, high efficiency and non-contact. Recently, great advances in computing devices and artificial intelligence have facilitated the development of vision-based measurement technology. This paper mainly focuses on state-of-the-art vision-based methods that can perform 3D shape measurement with high precision and high resolution. Specifically, the basic principles and typical techniques of triangulation-based measurement methods as well as their advantages and limitations are elaborated, and the learning-based techniques used for 3D vision measurement are enumerated. Finally, the advances of, and the prospects for, further improvement of vision-based 3D shape measurement techniques are proposed.

Surface Reconstruction of Microscale Objects Based on Grid-Patterned Structured-Light Measurements

A structured-light projection system was designed for microscale objects with surface heights that ranged from tens to hundreds of microns. The system was composed of a universal projector and microscope system that supported editing the attributes of structured-light patterns in real-time and was capable of projecting microscale patterns. On this basis, reconstructing the metal surfaces of microscale objects based on grid patterns of structured light was investigated, the internal and external parameters of microscope vision and projection systems were calibrated, and an image algorithm for grid-node detection was designed. The results indicated that the proposed method successfully reconstructed the three-dimensional (3D) surface of microscale objects, and the reconstruction results were consistent with the original surfaces. With 95% confidence, the reconstruction precision in the X- and Y-directions was approximately ±4.0 μm and in the Z-direction was approximately ±7.5 μm. The designed system and the proposed method were suitable for 3D-shape measurement of microstructures in microscopic fields and can be adapted to meet a broader range of applications, as compared to current methods.

MIMONet: Structured-light 3D shape reconstruction by a multi-input multi-output network

Reconstructing 3D geometric representation of objects with deep learning frameworks has recently gained a great deal of interest in numerous fields. The existing deep-learning-based 3D shape reconstruction techniques generally use a single red-green-blue (RGB) image, and the depth reconstruction accuracy is often highly limited due to a variety of reasons. We present a 3D shape reconstruction technique with an accuracy enhancement strategy by integrating the structured-light scheme with deep convolutional neural networks (CNNs). The key idea is to transform multiple (typically two) grayscale images consisting of fringe and/or speckle patterns into a 3D depth map using an end-to-end artificial neural network. Distinct from the existing autoencoder-based networks, the proposed technique reconstructs the 3D shape of target using a refinement approach that fuses multiple feature maps to obtain multiple outputs with an accuracy-enhanced final output. A few experiments have been conducted to verify the robustness and capabilities of the proposed technique. The findings suggest that the proposed network approach can be a promising 3D reconstruction technique for future academic research and industrial applications.

Dot-coded structured light for accurate and robust 3D reconstruction

Speckle dots have the advantage of easy projection, which makes them good candidate features of structured light (SL) cameras, such as Kinect v1. However, they generally yield poor accuracy due to block matching. To improve their accuracy, this paper proposes a dot-coded SL, the coding information of which is added into dot distribution. Some of the dots are arranged regularly to provide easy-to-locate corner features, while others are specially designed to form different shapes of unique identification. A Gaussian-cross module and a simplified ResNet have been proposed to conduct robust decoding. Various experiments are performed to verify the accuracy and robustness of our framework.

Microscopic Three-Dimensional Measurement Based on Telecentric Stereo and Speckle Projection Methods

Three-dimensional (3D) measurement of microstructures has become increasingly important, and many microscopic measurement methods have been developed. For the dimension in several millimeters together with the accuracy at sub-pixel or sub-micron level, there is almost no effective measurement method now. Here we present a method combining the microscopic stereo measurement with the digital speckle projection. A microscopy experimental setup mainly composed of two telecentric cameras and an industrial projection module is established and a telecentric binocular stereo reconstruction procedure is carried out. The measurement accuracy has firstly been verified by performing 3D measurements of grid arrays at different locations and cylinder arrays with different height differences. Then two Mitutoyo step masters have been used for further verification. The experimental results show that the proposed method can obtain 3D information of the microstructure with a sub-pixel and even sub-micron measuring accuracy in millimeter scale.

3D shape measurement in the presence of strong interreflections by epipolar imaging and regional fringe projection

A 3D shape measurement method in the presence of strong interreflections is presented. Traditional optical 3D shape measurement methods such as fringe projection profilometry (FPP) cannot measure regions that contain strong interreflections, which result in 3D shape measurement failure. In the proposed method, epipolar imaging with speckle patterns is utilized to eliminate the effects of interreflections and obtain the initial 3D shape measurement result. Regional fringe projection based on the initial measurement result is further applied to achieve high-accuracy measurement. Experimental results show that the proposed method can measure the regions that contain strong interreflections at a high accuracy.

High-speed and high-accuracy 3D surface measurement using a mechanical projector

This paper presents a method to achieve high-speed and high-accuracy 3D surface measurement using a custom-designed mechanical projector and two high-speed cameras. We developed a computational framework that can achieve absolute shape measurement in sub-pixel accuracy through: 1) capturing precisely phase-shifted fringe patterns by synchronizing the cameras with the projector; 2) generating a rough disparity map between two cameras by employing a standard stereo-vision method using texture images with encoded statistical patterns; and 3) utilizing the wrapped phase as a constraint to refine the disparity map. The projector can project binary patterns at a speed of up to 10,000 Hz, and the camera can capture the required number of phase-shifted fringe patterns with 1/10,000 second, and thus 3D shape measurement can be realized as high as 10,000 Hz regardless the number of phase-shifted fringe patterns required for one 3D reconstruction. Experimental results demonstrated the success of our proposed method.

A Novel Camera Calibration Method Based on Polar Coordinate

A novel calibration method based on polar coordinate is proposed. The world coordinates are expressed in the form of polar coordinates, which are converted to world coordinates in the calibration process. In the beginning, the calibration points are obtained in polar coordinates. By transformation between polar coordinates and rectangular coordinates, the points turn into form of rectangular coordinates. Then, the points are matched with the corresponding image coordinates. At last, the parameters are obtained by objective function optimization. By the proposed method, the relationships between objects and cameras are expressed in polar coordinates easily. It is suitable for multi-camera calibration. Cameras can be calibrated with fewer points. The calibration images can be positioned according to the location of cameras. The experiment results demonstrate that the proposed method is an efficient calibration method. By the method, cameras are calibrated conveniently with high accuracy.