1
|
Sun J, Yang Z, Li F, Hao Q, Zhang S. Projected feature assisted coarse to fine point cloud registration method for large-size 3D measurement. OPTICS EXPRESS 2023; 31:18379-18398. [PMID: 37381550 DOI: 10.1364/oe.492045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Accepted: 04/29/2023] [Indexed: 06/30/2023]
Abstract
Fringe projection profilometry has gained significant interest due to its high precision, enhanced resolution, and simplified design. Typically, the spatial and perspective measurement capability is restricted by the lenses of the camera and projector in accordance with the principles of geometric optics. Therefore, large-size object measurement requires data acquisition from multiple perspectives, followed by point cloud splicing. Current point cloud registration methods usually rely on 2D feature textures, 3D structural elements, or supplementary tools, which will increase costs or limit the scope of the application. To address large-size 3D measurement more efficiently, we propose a low-cost and feasible method that combines active projection textures, color channel multiplexing, image feature matching and coarse-to-fine point registration strategies. Using a composite structured light with red speckle patterns for larger areas and blue sinusoidal fringe patterns for smaller ones, projected onto the surface, which allows us to accomplish simultaneous 3D reconstruction and point cloud registration. Experimental results demonstrate that the proposed method is effective for the 3D measurement of large-size and weak-textured objects.
Collapse
|
2
|
Liao YH, Zhang S. Semi-Global Matching Assisted Absolute Phase Unwrapping. SENSORS (BASEL, SWITZERLAND) 2022; 23:411. [PMID: 36617015 PMCID: PMC9824817 DOI: 10.3390/s23010411] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 12/20/2022] [Accepted: 12/23/2022] [Indexed: 06/17/2023]
Abstract
Measuring speed is a critical factor to reduce motion artifacts for dynamic scene capture. Phase-shifting methods have the advantage of providing high-accuracy and dense 3D point clouds, but the phase unwrapping process affects the measurement speed. This paper presents an absolute phase unwrapping method capable of using only three speckle-embedded phase-shifted patterns for high-speed three-dimensional (3D) shape measurement on a single-camera, single-projector structured light system. The proposed method obtains the wrapped phase of the object from the speckle-embedded three-step phase-shifted patterns. Next, it utilizes the Semi-Global Matching (SGM) algorithm to establish the coarse correspondence between the image of the object with the embedded speckle pattern and the pre-obtained image of a flat surface with the same embedded speckle pattern. Then, a computational framework uses the coarse correspondence information to determine the fringe order pixel by pixel. The experimental results demonstrated that the proposed method can achieve high-speed and high-quality 3D measurements of complex scenes.
Collapse
|
3
|
Liao YH, Xu M, Zhang S. Digital image correlation assisted absolute phase unwrapping. OPTICS EXPRESS 2022; 30:33022-33034. [PMID: 36242352 DOI: 10.1364/oe.470704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 08/16/2022] [Indexed: 06/16/2023]
Abstract
This paper presents an absolute phase unwrapping method for high-speed three-dimensional (3D) shape measurement. This method uses three phase-shifted patterns and one binary random pattern on a single-camera, single-projector structured light system. We calculate the wrapped phase from phase-shifted images and determine the coarse correspondence through the digital image correlation (DIC) between the captured binary random pattern of the object and the pre-captured binary random pattern of a flat surface. We then developed a computational framework to determine fringe order number pixel by pixel using the coarse correspondence information. Since only one additional pattern is used, the proposed method can be used for high-speed 3D shape measurement. Experimental results successfully demonstrated that the proposed method can achieve high-speed and high-quality measurement of complex scenes.
Collapse
|
4
|
Juarez-Salazar R, Rios-Orellana OI, Diaz-Ramirez VH. Stereo-phase rectification for metric profilometry with two calibrated cameras and one uncalibrated projector. APPLIED OPTICS 2022; 61:6097-6109. [PMID: 36256221 DOI: 10.1364/ao.461168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 06/28/2022] [Indexed: 06/16/2023]
Abstract
Fringe projection profilometry requires calibrating both cameras and projectors for metric measurements. Cameras are relatively simple to calibrate, but projectors require more sophisticated procedures. In this paper, a fringe projection profilometer with two calibrated cameras and one uncalibrated projector is developed for metric measurements. A phase rectification method, which is crucial for stereo matching, is designed by minimizing the perspective distortion. Also, a simple method for point matching using stereo rectified phase maps is proposed. The principles of metric profilometry using the proposed rectification method are introduced. The developed system is evaluated experimentally by the metric measurement of three-dimensional objects. The obtained results confirm a high accuracy of metric measurement and versatility in the design of fringe projection profilometers with uncalibrated projectors.
Collapse
|
5
|
Sui C, He K, Lyu C, Liu YH. Accurate 3D Reconstruction of Dynamic Objects by Spatial-Temporal Multiplexing and Motion-Induced Error Elimination. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:2106-2121. [PMID: 35167454 DOI: 10.1109/tip.2022.3150297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Three-dimensional (3D) reconstruction of dynamic objects has broad applications, including object recognition and robotic manipulation. However, achieving high-accuracy reconstruction and robustness to motion simultaneously is a challenging task. In this paper, we present a novel method for 3D reconstruction of dynamic objectS, whose main features are as follows. Firstly, a structured-light multiplexing method is developed that only requires 3 patterns to achieve high-accuracy encoding. Fewer projected patterns require shorter image acquisition time, thus, the object motion is reduced in each reconstruction cycle. The three patterns, i.e. spatial-temporally encoded patterns, are generated by embedding a specifically designed spatial-coded texture map into the temporal-encoded three-step phase-shifting fringes. A temporal codeword and three spatial codewords are extracted from the composite patterns using a proposed extraction algorithm. The two types of codewords are utilized separately in stereo matching: the temporal codeword ensures the high accuracy, while the spatial codewords are responsible for removing phase ambiguity. Secondly, we aim to eliminate the reconstruction error induced by motion between frames abbreviated as motion induced error (MiE). Instead of assuming the object to be static when acquiring the 3 images, we derive the motion of projection pixels among frames. Using the extracted spatial codewords, correspondences between different frames are found, i.e. pixels with the same codewords are traceable in the image sequences. Therefore, we can obtain the phase map at each image-acquisition moment without being affected by the object motion. Then the object surfaces corresponding to all the images can be recovered. Experimental results validate the high reconstruction accuracy and precision of the proposed method for dynamic objects with different motion speeds. Comparative experiments show that the presented method demonstrates superior performance with various types of motion, including translation in different directions and deformation.
Collapse
|
6
|
A Topology-Based Stereo Matching Method for One Shot 3D Measurement Using Coded Spot-Array Structured Light. SENSORS 2021; 21:s21196444. [PMID: 34640765 PMCID: PMC8512960 DOI: 10.3390/s21196444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Revised: 09/21/2021] [Accepted: 09/22/2021] [Indexed: 11/16/2022]
Abstract
In this paper, a topology-based stereo matching method for 3D measurement using a single pattern of coded spot-array structured light is proposed. The pattern of spot array is designed with a central reference ring spot, and each spot in the pattern can be uniquely coded with the row and column indexes according to the predefined topological search path. A method using rectangle templates to find the encoded spots in the captured images is proposed in the case where coding spots are missing, and an interpolation method is also proposed for rebuilding the missing spots. Experimental results demonstrate that the proposed technique could exactly and uniquely decode each spot and establish the stereo matching relation successfully, which can be used to obtain three-dimensional (3D) reconstruction with a single-shot method.
Collapse
|
7
|
Wang J, Yang Y, Zhou Y. 3-D shape reconstruction of non-uniform reflectance surface based on pixel intensity, pixel color and camera exposure time adaptive adjustment. Sci Rep 2021; 11:4700. [PMID: 33633127 PMCID: PMC7907344 DOI: 10.1038/s41598-021-83779-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Accepted: 02/04/2021] [Indexed: 11/09/2022] Open
Abstract
High dynamic range 3-D shape measurement is a challenge. In this work, we propose a novel method to solve the 3-D shape reconstruction of high-reflection and colored surfaces. First, we propose a method to establish a fast pixel-level mapping between the projected image and the captured image. Secondly, we propose a color texture extraction method using a black-and-white (B/W) camera and a pixel-level projection color adjustment method. Third, we give an optimal projection fringe modulation/background intensity ratio. Fourth, we propose a method for estimating the reflectivity of the object surface and ambient light interference, and a method for adjusting the projection intensity at the pixel level and a method for estimating the optimal exposure time. Experiments show that, compared with the existing methods, the proposed method not only can obtain high-quality captured images, but also has higher measurement efficiency and wider application range.
Collapse
Affiliation(s)
- Jianhua Wang
- School of Information and Control Engineering, Qingdao University of Technology, Qingdao, 266520, China.
| | - Yanxi Yang
- School of Automation and Information Engineering, Xi'an University of Technology, Xi'an, 710048, China
| | - Yuguo Zhou
- School of Information and Control Engineering, Qingdao University of Technology, Qingdao, 266520, China
| |
Collapse
|
8
|
Yin W, Zhong J, Feng S, Tao T, Han J, Huang L, Chen Q, Zuo C. Composite deep learning framework for absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation. JPHYS PHOTONICS 2020. [DOI: 10.1088/2515-7647/abbcd9] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022] Open
|
9
|
Wan Y, Cao Y, Liu X, Tao T, Kofman J. High-frequency color-encoded fringe-projection profilometry based on geometry constraint for large depth range. OPTICS EXPRESS 2020; 28:13043-13058. [PMID: 32403786 DOI: 10.1364/oe.388579] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/20/2020] [Accepted: 04/07/2020] [Indexed: 06/11/2023]
Abstract
In multi-view fringe projection profilometry (FPP), a limitation of geometry-constraint based approaches is the reduced measurement depth range often used to reduce the number of candidate points and increase the corresponding point selection reliability, when high-frequency fringe patterns are used. To extend the depth range, a new method of high-frequency fringe projection profilometry was developed by color encoding the projected fringe patterns to allow reliable candidate point selection even when six candidate points are in the measurement volume. The wrapped phase is directly retrieved using the intensity component of the hue-saturation-intensity (HSI) color space and complementary-hue is introduced to identify color codes for correct corresponding point selection. Mathematical analyses of the effect of color crosstalk on phase calculation and color code identification show that the phase calculation is independent of color crosstalk and that color crosstalk has little effect on color code identification. Experiments demonstrated that the new method can achieve high accuracy in 3D measurement over a large depth range and for isolated objects, using only two high-frequency color-encoded fringe patterns.
Collapse
|
10
|
Hyun JS, Zhang S. Influence of projector pixel shape on ultrahigh-resolution 3D shape measurement. OPTICS EXPRESS 2020; 28:9510-9520. [PMID: 32225557 DOI: 10.1364/oe.389331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Accepted: 03/10/2020] [Indexed: 06/10/2023]
Abstract
The state-of-art three-dimensional (3D) shape measurement with digital fringe projection (DFP) techniques assume that the influence of projector pixel shape is negligible. However, our research reveals that when the camera pixel size is much smaller than the projector pixel size in object space (e.g., 1/5), the shape of projector pixel can play a critical role on ultimate measurement quality. This paper evaluates the performance of two shapes of projector pixels: rectangular and diamond shaped. Both simulation and experimental results demonstrated that when the camera pixel size is significantly smaller than the projector pixel size, it is advantageous for ultrahigh resolution 3D shape measurement system to use a projector with rectangular-shaped pixels than a projector with diamond-shaped pixels.
Collapse
|
11
|
Zhang S, Wu M, Wu Y, Wu G, Liu F. Fixed window aggregation AD-census algorithm for phase-based stereo matching. APPLIED OPTICS 2019; 58:8950-8958. [PMID: 31873677 DOI: 10.1364/ao.58.008950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/12/2019] [Accepted: 10/14/2019] [Indexed: 06/10/2023]
Abstract
Phase-based stereo matching (PSM) is a vital step in binocular structured light. PSM is hard to strike a balance between efficiency and accuracy, especially because of the absolute phase matrix's (APM) double data type. It means that PSM needs more run time and memory than conventional intensity images' stereo matching. In this paper, we propose a modified absolute difference (AD)-Census algorithm called the fixed window aggregation AD-Census (FWA-AD-Census) to balance the contradiction between efficiency and accuracy in PSM. The FWA-AD-Census aggregates matching cost in a fixed support window instead of an adaptive support window in AD-Census. We analyze the reason why PSM is more suitable to aggregate the matching cost in a fixed support window. Simulations and experiments are conducted to verify the FWA-AD-Census's performances by comparing the FWA-AD-Census with two other local stereo matching algorithms. One is the AD-Census, which is more accurate but less efficient. Its matching cost is aggregated in an adaptive support window. Another is the sum of absolute difference (SAD), which is more efficient but less accurate, and its matching cost is aggregated in a fixed support window. Theoretical analysis and experimental results both indicate that the proposed algorithm can achieve similar accuracy to the AD-Census with less run time.
Collapse
|
12
|
Liu X, He D, Hu H, Liu L. Fast 3D Surface Measurement with Wrapped Phase and Pseudorandom Image. SENSORS 2019; 19:s19194185. [PMID: 31561597 PMCID: PMC6806322 DOI: 10.3390/s19194185] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/19/2019] [Revised: 09/23/2019] [Accepted: 09/24/2019] [Indexed: 11/16/2022]
Abstract
Balancing the accuracy and speed of three-dimensional (3D) surface measurement of objects is crucial in many important applications. In this paper, we present a wrapped phase and pseudorandom image method and develop an experimental system aiming to avoid the process of phase unwrapping. Our approach can reduce the length of image sequences and improve the speed of pattern projection and image acquisition and can be used as a good candidate for high-speed 3D measurement. The most critical step in our new methodology is using the wrapped phase and the epipolar constraint between one camera and a projector, which can obtain several candidate 3D points within the measurement volume (MV). The false points from the obtained candidate 3D points can be eliminated by the pseudorandom images. A systematic accuracy with MV better than 0.01 mm is achievable. 3D human body measurement results are given to confirm the fast speed of image acquisition capability.
Collapse
Affiliation(s)
- Xing Liu
- School of Marine Science and Technology, Northwestern Polytechnical University, Xi'an 710072, China.
| | - Dong He
- Shenzhen ESUN Display Co., Ltd, Shenzhen 518000, China.
| | - Hao Hu
- Qingdao Research Institute, Northwestern Polytechnical University, Qingdao 266200, China.
| | - Lixin Liu
- School of Physics and Optoelectronic Engineering, Xidian University, Xi'an 710071, China.
| |
Collapse
|
13
|
Namiki A, Shimada K, Kin Y, Ishii I. Development of an Active High-Speed 3-D Vision System. SENSORS 2019; 19:s19071572. [PMID: 30939792 PMCID: PMC6480288 DOI: 10.3390/s19071572] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/01/2019] [Revised: 03/15/2019] [Accepted: 03/22/2019] [Indexed: 11/20/2022]
Abstract
High-speed recognition of the shape of a target object is indispensable for robots to perform various kinds of dexterous tasks in real time. In this paper, we propose a high-speed 3-D sensing system with active target-tracking. The system consists of a high-speed camera head and a high-speed projector, which are mounted on a two-axis active vision system. By measuring a projected coded pattern, 3-D measurement at a rate of 500 fps was achieved. The measurement range was increased as a result of the active tracking, and the shape of the target was accurately observed even when it moved quickly. In addition, to obtain the position and orientation of the target, 500 fps real-time model matching was achieved.
Collapse
Affiliation(s)
- Akio Namiki
- Department of Mechanical Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba 263-8522, Japan.
| | - Keitaro Shimada
- Department of Mechanical Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba 263-8522, Japan.
| | - Yusuke Kin
- Department of Mechanical Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba 263-8522, Japan.
| | - Idaku Ishii
- Department of System Cybernetics, Hiroshima University, 1-4-1 Kagamiyama, Higashi-Hiroshima, Hiroshima 739-8527, Japan.
| |
Collapse
|
14
|
Niu Z, Xu X, Zhang X, Wang W, Zhu Y, Ye J, Xu M, Jiang X. Efficient phase retrieval of two-directional phase-shifting fringe patterns using geometric constraints of deflectometry. OPTICS EXPRESS 2019; 27:8195-8207. [PMID: 31052642 DOI: 10.1364/oe.27.008195] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/25/2019] [Accepted: 02/27/2019] [Indexed: 06/09/2023]
Abstract
In the phase measuring deflectometry, two groups of fringe patterns in orthogonal directions are usually applied to establish the correspondences between the pixel pairs on the screen and camera. Usually, 16 phase-shifting fringe patterns with different spatial frequencies are required in order to calculate the absolute phases in the conventional temporal phase unwrapping algorithms. This requirement makes the measurement inefficient and not robust against environmental noise. In this paper, an efficient phase retrieval strategy is developed, which requires only six fringe patterns. The modulating information in one-direction is obtained by first using four fringe patterns, and then it is applied to assist the phase calculation in the other direction, so that only two extra fringe patterns are needed. Subsequently the phases are unwrapped by using the geometric constraints of the software configurable optical test system without additional image acquisition. The measurement time is saved by 5/8, compared to the conventional methods. In this way, the influence of the low-frequency disturbances can be suppressed in the workshop condition. Experiments demonstrate that the proposed method can reliably retrieve the absolute phases, and it is of significance to improve the measuring efficiency and stability of in situ deflectometry.
Collapse
|
15
|
Cai Z, Liu X, Chen Z, Tang Q, Gao BZ, Pedrini G, Osten W, Peng X. Light-field-based absolute phase unwrapping. OPTICS LETTERS 2018; 43:5717-5720. [PMID: 30499976 DOI: 10.1364/ol.43.005717] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/09/2018] [Accepted: 10/24/2018] [Indexed: 06/09/2023]
Abstract
Ambiguity caused by a wrapped phase is an intrinsic problem in fringe projection-based 3D shape measurement. Among traditional methods for avoiding phase ambiguity, spatial phase unwrapping is sensitive to sensor noise and depth discontinuity, and temporal phase unwrapping requires additional encoding information that leads to an increase of image sequence acquisition time or a reduction of fringe contrast. Here, to the best of our knowledge, we report a novel method of absolute phase unwrapping based on light field imaging. In a recorded light field under structured illumination, i.e., a structured light field, a wrapped phase-encoded field can be retrieved and resampled in diverse image planes associated with several possible fringe orders in a measurement volume. Then, by leveraging phase consistency constraint in the resampled wrapped phase-encoded field, correct fringe orders can be determined to unwrap the wrapped phase without any additional encoding information. Experimental results demonstrated that the proposed method was suitable for accurate and robust absolute phase unwrapping.
Collapse
|
16
|
Bao W, Xiao X, Xu Y, Zhang X. Reference image based phase unwrapping framework for a structured light system. OPTICS EXPRESS 2018; 26:29588-29599. [PMID: 30470119 DOI: 10.1364/oe.26.029588] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2018] [Accepted: 10/14/2018] [Indexed: 06/09/2023]
Abstract
A novel real-time full-field phase unwrapping framework is proposed for the one-projector and one-camera structured light system. In this framework, only four patterns (including three fringe patterns and a binary speckle pattern) are required to measure the absolute 3D shape of the targets. We use the structured light system to capture four images of a nearly planar target (e.g. wall), of which the speckle image is taken as the reference image, and the corresponding absolute phase map is computed and stored, before measuring. So each pixel in the reference image can be mapped to an absolute phase value. In this way, if we can create the correspondences between the current and the reference speckle images in the process of measurement by using a matching algorithm, we can directly map the absolute values for the pixels of the current image. The mapped absolute phases can be used to determine the period of the relative phases. The experimental results verified the effectiveness and efficiency of the proposed framework. On a consumer-grade GPU (Nvidia GTX1060), our method can run at 187 fps.
Collapse
|
17
|
Heist S, Dietrich P, Landmann M, Kühmstedt P, Notni G, Tünnermann A. GOBO projection for 3D measurements at highest frame rates: a performance analysis. LIGHT, SCIENCE & APPLICATIONS 2018; 7:71. [PMID: 30302242 PMCID: PMC6168569 DOI: 10.1038/s41377-018-0072-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/23/2018] [Revised: 09/07/2018] [Accepted: 09/07/2018] [Indexed: 06/08/2023]
Abstract
Aperiodic sinusoidal patterns that are cast by a GOBO (GOes Before Optics) projector are a powerful tool for optically measuring the surface topography of moving or deforming objects with very high speed and accuracy. We optimised the first experimental setup that we were able to measure inflating car airbags at frame rates of more than 50 kHz while achieving a 3D point standard deviation of ~500 µm. Here, we theoretically investigate the method of GOBO projection of aperiodic sinusoidal fringes. In a simulation-based performance analysis, we examine the parameters that influence the accuracy of the measurement result and identify an optimal pattern design that yields the highest measurement accuracy. We compare the results with those that were obtained via GOBO projection of phase-shifted sinusoidal fringes. Finally, we experimentally verify the theoretical findings. We show that the proposed technique has several advantages over conventional fringe projection techniques, as the easy-to-build and cost-effective GOBO projector can provide a high radiant flux, allows high frame rates, and can be used over a wide spectral range.
Collapse
Affiliation(s)
- Stefan Heist
- Institute of Applied Physics, Abbe Center of Photonics, Friedrich Schiller University, 07745 Jena, Germany
- Fraunhofer Institute for Applied Optics and Precision Engineering IOF, 07745 Jena, Germany
| | - Patrick Dietrich
- Institute of Applied Physics, Abbe Center of Photonics, Friedrich Schiller University, 07745 Jena, Germany
- Fraunhofer Institute for Applied Optics and Precision Engineering IOF, 07745 Jena, Germany
| | - Martin Landmann
- Institute of Applied Physics, Abbe Center of Photonics, Friedrich Schiller University, 07745 Jena, Germany
- Fraunhofer Institute for Applied Optics and Precision Engineering IOF, 07745 Jena, Germany
| | - Peter Kühmstedt
- Fraunhofer Institute for Applied Optics and Precision Engineering IOF, 07745 Jena, Germany
| | - Gunther Notni
- Fraunhofer Institute for Applied Optics and Precision Engineering IOF, 07745 Jena, Germany
- Department of Mechanical Engineering, Ilmenau University of Technology, 98693 Ilmenau, Germany
| | - Andreas Tünnermann
- Institute of Applied Physics, Abbe Center of Photonics, Friedrich Schiller University, 07745 Jena, Germany
- Fraunhofer Institute for Applied Optics and Precision Engineering IOF, 07745 Jena, Germany
| |
Collapse
|
18
|
Shamata A, Thompson T. Documentation and analysis of traumatic injuries in clinical forensic medicine involving structured light three-dimensional surface scanning versus photography. J Forensic Leg Med 2018; 58:93-100. [DOI: 10.1016/j.jflm.2018.05.004] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2018] [Revised: 04/01/2018] [Accepted: 05/06/2018] [Indexed: 10/16/2022]
|
19
|
Jiang C, Lim B, Zhang S. Three-dimensional shape measurement using a structured light system with dual projectors. APPLIED OPTICS 2018; 57:3983-3990. [PMID: 29791369 DOI: 10.1364/ao.57.003983] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2018] [Accepted: 04/16/2018] [Indexed: 06/08/2023]
Abstract
This paper introduces a structured light system with two projectors and one camera for three-dimensional (3D) shape measurement to alleviate problems created by a single projector such as the shadow problem. In particular, we developed (1) a system calibration framework that can accurately calibrate each such camera-projector system; (2) a residual error correction method based on the system error function; and (3) a data fusion method utilizing the angle between the projection direction and surface normal. Experimental results demonstrate that the proposed dual-projector structured light system improves the measurement accuracy besides extending the measurement range of a single projector system.
Collapse
|
20
|
Shamata A, Thompson T. Using structured light three-dimensional surface scanning on living individuals: Key considerations and best practice for forensic medicine. J Forensic Leg Med 2018; 55:58-64. [PMID: 29471248 DOI: 10.1016/j.jflm.2018.02.017] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2017] [Revised: 09/21/2017] [Accepted: 02/12/2018] [Indexed: 11/26/2022]
Abstract
Non-contact three-dimensional (3D) surface scanning methods have been applied to forensic medicine to record injuries and to mitigate ordinary photography shortcoming. However, there are no literature concerning practical guidance for 3D surface scanning of live victims. This paper aimed to investigate key 3D scanning issues of the live body to develop a series of scanning principles for future use on injured victims. The Pico Scan 3D surface scanner was used on live test subjects. The work focused on analysing the following concerns: (1) an appropriate 3D scanning technique to scan different body areas, (2) the ideal number of scans, (3) scanning approaches to access various areas of the body and (4) elimination of environmental background noise in the acquired data. Results showed that scanning only a required surface of the body area in the stable manner was more efficient when compared to complete 360°-scanning; therefore, it used as a standard 3D scanning technique. More than three scans were sufficient when trying to obtain an optimal wireframe mode presentation of the result. Three different approaches were suggested to provide access to the various areas of the body. Undertaking scanning using a black background eliminated the background noise. The work demonstrated that the scanner will be promising to reconstruct injuries from different body areas, although the 3D scanning of the live subjects faced some challenges.
Collapse
Affiliation(s)
- Awatif Shamata
- School of Science and Engineering, Teesside University, Middlesbrough, TS1 3BA, UK.
| | - Tim Thompson
- School of Science and Engineering, Teesside University, Middlesbrough, TS1 3BA, UK.
| |
Collapse
|
21
|
Hyun JS, Chiu GTC, Zhang S. High-speed and high-accuracy 3D surface measurement using a mechanical projector. OPTICS EXPRESS 2018; 26:1474-1487. [PMID: 29402021 DOI: 10.1364/oe.26.001474] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/06/2017] [Accepted: 01/11/2018] [Indexed: 06/07/2023]
Abstract
This paper presents a method to achieve high-speed and high-accuracy 3D surface measurement using a custom-designed mechanical projector and two high-speed cameras. We developed a computational framework that can achieve absolute shape measurement in sub-pixel accuracy through: 1) capturing precisely phase-shifted fringe patterns by synchronizing the cameras with the projector; 2) generating a rough disparity map between two cameras by employing a standard stereo-vision method using texture images with encoded statistical patterns; and 3) utilizing the wrapped phase as a constraint to refine the disparity map. The projector can project binary patterns at a speed of up to 10,000 Hz, and the camera can capture the required number of phase-shifted fringe patterns with 1/10,000 second, and thus 3D shape measurement can be realized as high as 10,000 Hz regardless the number of phase-shifted fringe patterns required for one 3D reconstruction. Experimental results demonstrated the success of our proposed method.
Collapse
|
22
|
Single-Shot Dense Depth Sensing with Color Sequence Coded Fringe Pattern. SENSORS 2017; 17:s17112558. [PMID: 29113125 PMCID: PMC5712869 DOI: 10.3390/s17112558] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/19/2017] [Revised: 10/28/2017] [Accepted: 10/31/2017] [Indexed: 11/17/2022]
Abstract
A single-shot structured light method is widely used to acquire dense and accurate depth maps for dynamic scenes. In this paper, we propose a color sequence coded fringe depth sensing method. To overcome the phase unwrapping problem encountered in phase-based methods, the color-coded sequence information is embedded into the phase information. We adopt the color-encoded De Bruijn sequence to denote the period of the phase information and assign the sequence into two channels of the pattern, while the third channel is used to code the phase information. Benefiting from this coding strategy, the phase information distributed in multiple channels can improve the quality of the phase intensity by channel overlay, which results in precise phase estimation. Meanwhile, the wrapped phase period assists the sequence decoding to obtain a precise period order. To evaluate the performance of the proposed method, an experimental platform is established. Quantitative and qualitative experiments demonstrate that the proposed method generates a higher precision depth, as compared to a Kinect and larger resolution ToF (Time of Flight) camera.
Collapse
|
23
|
Willomitzer F, Häusler G. Single-shot 3D motion picture camera with a dense point cloud. OPTICS EXPRESS 2017; 25:23451-23464. [PMID: 29041645 DOI: 10.1364/oe.25.023451] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2017] [Accepted: 08/25/2017] [Indexed: 06/07/2023]
Abstract
We discuss physical and information theoretical limits of optical 3D metrology. Based on these principal considerations we introduce a novel single-shot 3D movie camera that almost reaches these limits. The camera is designed for the 3D acquisition of macroscopic live scenes. Like a hologram, each movie-frame encompasses the full 3D information about the object surface and the observation perspective can be varied while watching the 3D movie. The camera combines single-shot ability with a point cloud density close to the theoretical limit. No space-bandwidth is wasted by pattern codification. With 1-megapixel sensors, the 3D camera delivers nearly 300,000 independent 3D points within each frame. The 3D data display a lateral resolution and a depth precision only limited by physics. The approach is based on multi-line triangulation. The requisite low-cost technology is simple. Only two properly positioned synchronized cameras solve the profound ambiguity problem omnipresent in 3D metrology.
Collapse
|
24
|
Liu X, Kofman J. High-frequency background modulation fringe patterns based on a fringe-wavelength geometry-constraint model for 3D surface-shape measurement. OPTICS EXPRESS 2017; 25:16618-16628. [PMID: 28789163 DOI: 10.1364/oe.25.016618] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/09/2017] [Accepted: 06/30/2017] [Indexed: 06/07/2023]
Abstract
A new fringe projection method for surface-shape measurement was developed using four high-frequency phase-shifted background modulation fringe patterns. The pattern frequency is determined using a new fringe-wavelength geometry-constraint model that allows only two corresponding-point candidates in the measurement volume. The correct corresponding point is selected with high reliability using a binary pattern computed from intensity background encoded in the fringe patterns. Equations of geometry-constraint parameters permit parameter calculation prior to measurement, thus reducing measurement computational cost. Experiments demonstrated the ability of the method to perform 3D shape measurement for a surface with geometric discontinuity, and for spatially isolated objects.
Collapse
|
25
|
Dai J, An Y, Zhang S. Absolute three-dimensional shape measurement with a known object. OPTICS EXPRESS 2017; 25:10384-10396. [PMID: 28468410 DOI: 10.1364/oe.25.010384] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
This paper presents a novel method for absolute three-dimensional (3D) shape measurement that does not require conventional temporal phase unwrapping. Our proposed method uses a known object (i.e., a ping-pong ball) to provide cues for absolute phase unwrapping. During the measurement, the ping-pong ball is positioned to be close to the nearest point from the scene to the camera. We first segment ping-pong ball and spatially unwrap its phase, and then determine the integer multiple of 2π to be added such that the recovered shape matches its actual geometry. The nearest point of the ball provides zmin to generate the minimum phase Φmin that is then used to unwrap phase of the entire scene pixel by pixel. Experiments demonstrated that only three phase-shifted fringe patterns are required to measure absolute shapes of objects moving along depth z direction.
Collapse
|
26
|
Zhou Q, Qiao X, Ni K, Li X, Wang X. Depth detection in interactive projection system based on one-shot black-and-white stripe pattern. OPTICS EXPRESS 2017; 25:5341-5351. [PMID: 28380796 DOI: 10.1364/oe.25.005341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
A novel method enabling estimation of not only the screen surface as the conventional one, but the depth information from two-dimensional coordinates in an interactive projection system was proposed in this research. In this method, a one-shot black-and-white stripe pattern from a projector is projected on a screen plane, where the deformed pattern is captured by a charge-coupled device camera. An algorithm based on object/shadow simultaneous detection is proposed for fulfillment of the correspondence. The depth information of the object is then calculated using the triangulation principle. This technology provides a more direct feeling of virtual interaction in three dimensions without using auxiliary equipment or a special screen as interaction proxies. Simulation and experiments are carried out and the results verified the effectiveness of this method in depth detection.
Collapse
|
27
|
Li B, Liu Z, Zhang S. Motion-induced error reduction by combining Fourier transform profilometry with phase-shifting profilometry. OPTICS EXPRESS 2016; 24:23289-23303. [PMID: 27828393 DOI: 10.1364/oe.24.023289] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
We propose a hybrid computational framework to reduce motion-induced measurement error by combining the Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP). The proposed method is composed of three major steps: Step 1 is to extract continuous relative phase maps for each isolated object with single-shot FTP method and spatial phase unwrapping; Step 2 is to obtain an absolute phase map of the entire scene using PSP method, albeit motion-induced errors exist on the extracted absolute phase map; and Step 3 is to shift the continuous relative phase maps from Step 1 to generate final absolute phase maps for each isolated object by referring to the absolute phase map with error from Step 2. Experiments demonstrate the success of the proposed computational framework for measuring multiple isolated rapidly moving objects.
Collapse
|
28
|
Peng J, Liu X, Deng D, Guo H, Cai Z, Peng X. Suppression of projector distortion in phase-measuring profilometry by projecting adaptive fringe patterns. OPTICS EXPRESS 2016; 24:21846-21860. [PMID: 27661920 DOI: 10.1364/oe.24.021846] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
In phase-measuring profilometry, the lens distortion of commercial projectors may introduce additional bending carrier phase and thus lead to measurement errors. To address this problem, this paper presents an adaptive fringe projection technique in which the carrier phase in the projected fringe patterns is modified according to the projector distortion. After projecting these adaptive fringe patterns, the bending carrier phase induced by the projector distortion is eliminated. Experimental results demonstrate this method to be effective and efficient in suppressing the projector distortion for phase-measuring profilometry. More importantly, this method does not need to calibrate the projector and system parameters, such as the distortion coefficients of the projector and the angle between the optical axes of projector and camera lenses. Hence, it has low computational complexity and enables us to improve the measurement precision for an arbitrary phase-measuring profilometry system.
Collapse
|
29
|
An Y, Hyun JS, Zhang S. Pixel-wise absolute phase unwrapping using geometric constraints of structured light system. OPTICS EXPRESS 2016; 24:18445-18459. [PMID: 27505808 DOI: 10.1364/oe.24.018445] [Citation(s) in RCA: 71] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
This paper presents a method to unwrap phase pixel by pixel by solely using geometric constraints of the structured light system without requiring additional image acquisition or another camera. Specifically, an artificial absolute phase map, Φmin, at a given virtual depth plane z = zmin, is created from geometric constraints of the calibrated structured light system; the wrapped phase is pixel-by-pixel unwrapped by referring to Φmin. Since Φmin is defined in the projector space, the unwrapped phase obtained from this method is absolute for each pixel. Experimental results demonstrate the success of this proposed novel absolute phase unwrapping method.
Collapse
|
30
|
Gai S, Da F, Dai X. Novel 3D measurement system based on speckle and fringe pattern projection. OPTICS EXPRESS 2016; 24:17686-17697. [PMID: 27505737 DOI: 10.1364/oe.24.017686] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
An efficient three-dimensional shape measurement system is proposed, which is based on the combining projection of single digital speckle pattern and phase-shifting fringe patterns. At the beginning, the initial corresponding point for each pixel is obtained by a novel speckle-phase combination method. The initial information can be calculated by the single speckle pattern in a short time, while the phase information is used to ensure the results. Unlike the conventional methods, it is not necessary to obtain the unwrapped phase, therefore the number of projected patterns is reduced greatly. Then accurate corresponding coordinates are obtained according to the wrapped phase. Three cases are analyzed while adjusting the initial corresponding coordinates locally. Thus accuracy coordinates are obtained without missing or incorrect points. Experiments demonstrate that we can achieve accurate reconstruction results with reduced measurement time by the proposed method.
Collapse
|
31
|
Wu Q, Zhang B, Huang J, Wu Z, Zeng Z. Flexible 3D reconstruction method based on phase-matching in multi-sensor system. OPTICS EXPRESS 2016; 24:7299-7318. [PMID: 27137020 DOI: 10.1364/oe.24.007299] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Considering the measuring range limitation of a single sensor system, multi-sensor system has become essential in obtaining complete image information of the object in the field of 3D image reconstruction. However, for the traditional multi-sensors worked independently in its system, there was some point in calibrating each sensor system separately. And the calibration between all single sensor systems was complicated and required a long time. In this paper, we present a flexible 3D reconstruction method based on phase-matching in multi-sensor system. While calibrating each sensor, it realizes the data registration of multi-sensor system in a unified coordinate system simultaneously. After all sensors are calibrated, the whole 3D image data directly exist in the unified coordinate system, and there is no need to calibrate the positions between sensors any more. Experimental results prove that the method is simple in operation, accurate in measurement, and fast in 3D image reconstruction.
Collapse
|
32
|
Jiang H, Zhao H, Li X, Quan C. Hyper thin 3D edge measurement of honeycomb core structures based on the triangular camera-projector layout & phase-based stereo matching. OPTICS EXPRESS 2016; 24:5502-5513. [PMID: 29092373 DOI: 10.1364/oe.24.005502] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
We propose a novel hyper thin 3D edge measurement technique to measure the profile of 3D outer envelope of honeycomb core structures. The width of the edges of the honeycomb core is less than 0.1 mm. We introduce a triangular layout design consisting of two cameras and one projector to measure hyper thin 3D edges and eliminate data interference from the walls. A phase-shifting algorithm and the multi-frequency heterodyne phase-unwrapping principle are applied for phase retrievals on edges. A new stereo matching method based on phase mapping and epipolar constraint is presented to solve correspondence searching on the edges and remove false matches resulting in 3D outliers. Experimental results demonstrate the effectiveness of the proposed method for measuring the 3D profile of honeycomb core structures.
Collapse
|
33
|
Heist S, Kühmstedt P, Tünnermann A, Notni G. Theoretical considerations on aperiodic sinusoidal fringes in comparison to phase-shifted sinusoidal fringes for high-speed three-dimensional shape measurement. APPLIED OPTICS 2015; 54:10541-10551. [PMID: 26836883 DOI: 10.1364/ao.54.010541] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
The demand for optically reconstructing the three-dimensional (3D) surface shape of moving objects or deformation processes makes the development of high-speed projectors necessary. Our 3D sensor containing an array projector can achieve frame rates of several tens of kilohertz and is based on the projection of aperiodic sinusoidal fringes. This approach is compared with phase-shifting fringe projection as probably the most widely used technique. Theoretical considerations as well as extensive simulations are conducted to derive criteria for the design of optimal sequences of aperiodic sinusoidal fringes and to compare the number of patterns of both approaches necessary for comparable accuracies.
Collapse
|
34
|
Lohry W, Zhang S. High-speed absolute three-dimensional shape measurement using three binary dithered patterns. OPTICS EXPRESS 2014; 22:26752-26762. [PMID: 25401823 DOI: 10.1364/oe.22.026752] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
This paper describes a method to reconstruct high-speed absolute three-dimensional (3D) geometry using only three encoded 1-bit binary dithered patterns. Because of the use of 1-bit binary patterns, high-speed 3D shape measurement could also be achieved. By matching the right camera image pixel to the left camera pixel in the object space rather than image space, robust correspondence can be established. Experiments demonstrate the robustness of the proposed algorithm and the potential to achieve high-speed 3D shape measurements.
Collapse
|