Vision processing for realtime 3-D data acquisition based on coded structured light

BimodalPS: Causes and Corrections for Bimodal Multi-Path in Phase-Shifting Structured Light Scanners

Structured light illumination is an active 3D scanning technique based on projecting and capturing a set of striped patterns and measuring the warping of the patterns as they reflect off a target object's surface. As designed, each pixel in the camera sees exactly one pixel from the projector; however, there are multi-path situations where a camera pixel sees light from multiple projector positions. In the case of bimodal multi-path, the camera pixel receives light from exactly two positions, which occurs along a step edge where the edge slices through a pixel which, therefore, sees both a foreground and background surface. In this paper, we present a general mathematical model to address this bimodal multi-path issue in a phase-shifting or so-called phase-measuring-profilometry scanner to measure the constructive and destructive interference between the two light paths, and by taking advantage of this interference, separate the paths and make two decoupled depth measurements. We validate our algorithm with both simulations and a number of challenging real-world scenarios, significantly outperforming the state-of-the-art methods.

Orthogonal Spatial Binary Coding Method for High-Speed 3D Measurement

Temporal phase unwrapping based on single auxiliary binary coded pattern has been proven to be effective for high-speed 3D measurement. However, in traditional spatial binary coding, it often leads to an imbalance between the number of periodic divisions and codewords. To meet this challenge, a large codewords orthogonal spatial binary coding method is proposed in this paper. By expanding spatial multiplexing from 1D to 2D orthogonal direction, it goes beyond the traditional 8 codewords to 27 codewords at three-level periodic division. In addition, a novel full-period connected domain segmentation technique based on local localization is proposed to avoid the time-consuming global iterative erosion and complex anomaly detection in traditional methods. For the decoding process, a purely spatial codewords recognition and a spatial-temporal hybrid codewords recognition methods are established to better suppress the percentage offset caused by static defocusing and dynamic motion, respectively. Obviating the need for intricate symbol recognition, the decoding process in our proposed method encompasses a straightforward analysis of statistical distribution. Building upon the development of special spatial binary coding, we have achieved a well-balance between low periodic division and large codewords for the first time. The experimental results verify the feasibility and validity of our proposed whole image processing method in both static and dynamic measurements.

In-motion 3D reconstruction of high dynamic range surfaces

Efficient and refined three-dimensional (3D) reconstruction of industrial parts has become an urgent need in the field of advanced manufacturing, and it's a great challenge when facing in-motion and online inspection requirements of high dynamic range (HDR) surfaces that have large reflectivity variations. This paper proposes a method using RGB line-scan cameras to realize in-motion multiple-shot 3D shape measurements with RGB channel fusion to increase the measurement dynamic range. First, multi-channel one-dimensional background-normalized Fourier transform profilometry (MC-1DBNFTP) is proposed as an effective in-motion HDR method. Second, for HDR surfaces with strongly overexposed areas, we propose a solution that obtains 6 results of different dynamic ranges for fusion with only 5 projected patterns, which further extends the measurement dynamic range while ensuring the small projection period. Third, we develop a fusion method based on reliability evaluation, which is more reliable than the existing methods in fringe projection systems. In addition, colored textures can be mapped to the reconstructed surfaces. Experimental results prove that the proposed method realizes accurate and reliable in-motion 3D reconstruction of HDR surfaces.

Low-cost structured light imaging of regional volume changes for use in assessing mechanical ventilation

BACKGROUND

Optimal setting of mechanical ventilators is critical for improving outcomes. Accurate, predictive lung mechanics models are effective in optimizing MV settings, but only at a global level as they cannot estimate regional lung volume ventilation to assess the potential of local distension or under-ventilation. This study presents a low-cost structured light system for non-contact high resolution chest motion measurement to estimate regional lung volume changes.

METHODS

The system consists of a structured light projector and two cameras. A new pattern is designed to extract motion from sub-regions of the chest surface, and an efficient feature is proposed to provide a fast and accurate correspondence matching between two views. Reconstruction of 3D surface points is based on the matched points and stereo method. Asymmetric distribution of tidal volume into left and right lungs is estimated based on reconstructed regional chest expansion. A proof-of-concept experiment using a dummy model and two test lungs connected to a ventilator to provide differential chest expansion is conducted under tidal volumes of 400 ml, 500 ml and 600 ml, with results compared to the widely-used SURF and ORB methods.

RESULTS

Compared to the SURF and ORB methods, the proposed method is more computationally efficient with ∼40% less computational time cost, and higher accuracy for dense point correspondence. Finally, the proposed method estimated the region lung volumes with the maximum error of 8 ml under 600 ml tidal volume, indicating a good accuracy.

CONCLUSIONS

Surface reconstruction results in a proof-of-concept experiment with differential chest expansion show good performance for the proposed pattern and method in extracting the key information for regional chest expansion. The proposed method is generalizable, with potential for use in other applications.

3D Reconstruction with Single-Shot Structured Light RGB Line Pattern

Single-shot 3D reconstruction technique is very important for measuring moving and deforming objects. After many decades of study, a great number of interesting single-shot techniques have been proposed, yet the problem remains open. In this paper, a new approach is proposed to reconstruct deforming and moving objects with the structured light RGB line pattern. The structured light RGB line pattern is coded using parallel red, green, and blue lines with equal intervals to facilitate line segmentation and line indexing. A slope difference distribution (SDD)-based image segmentation method is proposed to segment the lines robustly in the HSV color space. A method of exclusion is proposed to index the red lines, the green lines, and the blue lines respectively and robustly. The indexed lines in different colors are fused to obtain a phase map for 3D depth calculation. The quantitative accuracies of measuring a calibration grid and a ball achieved by the proposed approach are 0.46 and 0.24 mm, respectively, which are significantly lower than those achieved by the compared state-of-the-art single-shot techniques.

A Residual Network and FPGA Based Real-Time Depth Map Enhancement System

Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 fps@ (1280 × 960 + 328 × 248 × 3).

Complete grid pattern decoding method for a one-shot structured light system

Structured light 3D reconstruction methods using a De Bruijn sequence-based color grid pattern have an impressive advantage of fast and accurate decoding, which leads to fast 3D reconstruction. They are especially suitable for capturing moving objects. However, the drawback of these methods is their high false decoding rate while dealing with feature points at the object's boundaries, and objects can be prone to becoming deformed by the uneven structure of the dynamic scene. To solve this problem, we present an efficient opened-grid-point detector and a complete grid pattern decoding method. Specifically, a new, to the best of our knowledge, color grid pattern is designed to reduce the influence of color noise and increase the density of 3D cloud points. In addition, a LCD screen projected with the proposed pattern is utilized to calibrate the camera-projector system. The experiments, conducted in a laboratory without a light curtain, demonstrate that the proposed method can fully satisfy the requirements of real applications.

Dual-projector structured light 3D shape measurement

Structured light illumination is an active three-dimensional scanning technique that uses a projector and camera pair to project and capture a series of stripe patterns; however, with a single camera and single projector, structured light scanning has issues associated with scan occlusions, multi-path, and weak signal reflections. To address these issues, this paper proposes dual-projector scanning using a range of projector/camera arrangements. Unlike previous attempts at dual-projector scanning, the proposed scanner drives both light engines simultaneously, using temporal-frequency multiplexing to computationally decouple the projected patterns. Besides presenting the details of how such a system is built, we also present experimental results demonstrating how multiple projectors can be used to (1) minimize occlusions; (2) achieve higher signal-to-noise ratios having twice a single projector's brightness; (3) reduce the number of component video frames required for a scan; and (4) detect multi-path interference.

Indirect Time-of-Flight Depth Sensor with Two-Step Comparison Scheme for Depth Frame Difference Detection

A depth sensor with integrated frame difference detection is proposed. Instead of frame difference detection using light intensity, which is vulnerable to ambient light, the difference in depth between successive frames can be acquired. Because the conventional time-of-flight depth sensor requires two frames of depth-image acquisition with four-phase modulation, it has large power consumption, as well as a large area for external frame memories. Therefore, we propose a simple two-step comparison scheme for generating the depth frame difference in a single frame. With the proposed scheme, only a single frame is needed to obtain the frame difference, with less than half of the power consumption of the conventional depth sensor. Because the frame difference is simply generated by column-parallel circuits, no access of the external frame memory is involved, nor is a digital signal processor. In addition, we used an over-pixel metal-insulator-metal capacitor to store temporary signals for enhancing the area efficiency. A prototype chip was fabricated using a 90 nm backside illumination complementary metal-oxide-semiconductor (CMOS) image sensor process. We measured the depth frame difference in the range of 1-2.5 m. With a 10 MHz modulation frequency, a depth frame difference of >10 cm was successfully detected even for objects with different reflectivity. The maximum relative error from the difference of the reflectivity (white and wooden targets) was <3%.

Absolute Phase Retrieval Using One Coded Pattern and Geometric Constraints of Fringe Projection System

Fringe projection technologies have been widely used for three-dimensional (3D) shape measurement. One of the critical issues is absolute phase recovery, especially for measuring multiple isolated objects. This paper proposes a method for absolute phase retrieval using only one coded pattern. A total of four patterns including one coded pattern and three phase-shift patterns are projected, captured, and processed. The wrapped phase, as well as average intensity and intensity modulation, are calculated from three phase-shift patterns. A code word encrypted into the coded pattern can be calculated using the average intensity and intensity modulation. Based on geometric constraints of fringe projection system, the minimum fringe order map can be created, upon which the fringe order can be calculated from the code word. Compared with the conventional method, the measurement depth range is significantly improved. Finally, the wrapped phase can be unwrapped for absolute phase map. Since only four patterns are required, the proposed method is suitable for real-time measurement. Simulations and experiments have been conducted, and their results have verified the proposed method.

Accurate three dimensional body scanning system based on structured light

Three dimensional (3D) body scanning has been of great interest to many fields, yet it is still a challenge to generate accurate human body models in a convenient manner. In this paper, we present an accurate 3D body scanning system based on structured light technology. A four-step phase shifting combined with Gray-code method is applied to match pixels in camera and projector planes. The calculation of 3D point coordinates are also derived. The main contribution of this paper is twofold. First, an improved registration algorithm is proposed to align point clouds reconstructed from different views. Second, we propose a graph optimization algorithm to further minimize registration errors. Experimental results demonstrate that our system can produce accurate 3D body models conveniently.

Modified Gray-Level Coding Method for Absolute Phase Retrieval

Fringe projection systems have been widely applied in three-dimensional (3D) shape measurements. One of the important issues is how to retrieve the absolute phase. This paper presents a modified gray-level coding method for absolute phase retrieval. Specifically, two groups of fringe patterns are projected onto the measured objects, including three phase-shift patterns for the wrapped phase, and three n-ary gray-level (nGL) patterns for the fringe order. Compared with the binary gray-level (bGL) method which just uses two intensity values, the nGL method can generate many more unique codewords with multiple intensity values. With assistance from the average intensity and modulation of phase-shift patterns, the intensities of nGL patterns are normalized to deal with ambient light and surface contrast. To reduce the codeword detection errors caused by camera/projector defocus, nGL patterns are designed as n-ary gray-code (nGC) patterns to ensure that at most, one code changes at each point. Experiments verify the robustness and effectiveness of the proposed method to measure isolated objects with complex surfaces.

Maximum a posteriori-based depth sensing with a single-shot maze pattern

This study addressed the general problem of correspondence retrieval for single-shot depth sensing where the coded features cannot be detected perfectly. The traditional correspondence retrieval technique can be regarded as maximum likelihood estimation with a uniform distribution prior assumption, which may lead to mismatches for two types of insignificant features: 1) incomplete features that cannot be detected completely because of edges, tiny objects, and many depth variations, etc.; and 2) distorted features disturbed by environmental noise. To overcome the drawback of the uniform distribution assumption, we propose a maximum a posteriori estimation-based correspondence retrieval method that uses the significant features as priors to estimate the weak or missing features. We also propose a novel monochromatic maze-like pattern, which is more robust to ambient illumination and the colors in scenes than the traditional patterns. Our experimental results demonstrate that the proposed system performs better than the popular RGB-D cameras and traditional single-shot techniques in terms of accuracy and robustness, especially with challenging scenes.

The Osteometry of Equine Third Phalanx by the Use of Three-Dimensional Scanning: New Measurement Possibilities

This study consisted in analyzing the asymmetry between bilateral third phalanges (coffin bones) in cold-blood horses based on the angle range of the plantar margin of the bone. The study employed a scanner projecting a hybrid set of images, consisting of sinusoidal stripes preceded by a Gray code sequence. As it turned out, three-dimensional scanning can be used to effectively determine the angle range for a selected portion of the studied bone. This provides broad possibilities for osteometric studies, as it enables the determination of angle distribution in a given fragment. The results obtained indicate a weak correlation between age and bilateral third-phalanx asymmetry in terms of the angle range of the plantar margins and no correlation between body weight and the asymmetry described.

Beyond the interference problem: hierarchical patterns for multiple-projector structured light system

Three-dimensional reconstruction of a dynamic object based on the structured light (SL) technique has attracted intensive research. Since a single projector only covers a limited area of the scene, multiple projectors should be employed simultaneously to extend the imaging area for a 3D object. However, patterns projected by different projectors superpose each other in a light-intersected area, which makes it difficult to recognize the original patterns and obtain the correct depth maps. To solve such a problem, we propose a method to design hierarchical patterns that can be separated from each other. In the proposed patterns, each pixel in binary patterns based on the de Bruijn sequence is replaced by a different bin with limited size. Then the proposed patterns can be separated by identifying distributions of colors in each bin in superposed patterns, and depth maps are obtained by decoding the separated patterns. To verify the performance of the proposed method, we design two hierarchical patterns and conduct several experiments in different scenes. Experimental results demonstrate that the proposed patterns can be separated in a multiple-projector SL system to obtain accurate depth maps, and they are robust for different conditions.

Random-profiles-based 3D face recognition system

In this paper, a noble nonintrusive three-dimensional (3D) face modeling system for random-profile-based 3D face recognition is presented. Although recent two-dimensional (2D) face recognition systems can achieve a reliable recognition rate under certain conditions, their performance is limited by internal and external changes, such as illumination and pose variation. To address these issues, 3D face recognition, which uses 3D face data, has recently received much attention. However, the performance of 3D face recognition highly depends on the precision of acquired 3D face data, while also requiring more computational power and storage capacity than 2D face recognition systems. In this paper, we present a developed nonintrusive 3D face modeling system composed of a stereo vision system and an invisible near-infrared line laser, which can be directly applied to profile-based 3D face recognition. We further propose a novel random-profile-based 3D face recognition method that is memory-efficient and pose-invariant. The experimental results demonstrate that the reconstructed 3D face data consists of more than 50 k 3D point clouds and a reliable recognition rate against pose variation.

Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization

Time-of-flight (ToF) cameras calculate depth maps by reconstructing phase shifts of amplitude-modulated signals. For broad illumination of transparent objects, reflections from multiple scene points can illuminate a given pixel, giving rise to an erroneous depth map. We report here a sparsity-regularized solution that separates K interfering components using multiple modulation frequency measurements. The method maps ToF imaging to the general framework of spectral estimation theory and has applications in improving depth profiles and exploiting multiple scattering.

Real-time scalable depth sensing with hybrid structured light illumination

Time multiplexing (TM) and spatial neighborhood (SN) are two mainstream structured light techniques widely used for depth sensing. The former is well known for its high accuracy and the latter for its low delay. In this paper, we explore a new paradigm of scalable depth sensing to integrate the advantages of both the TM and SN methods. Our contribution is twofold. First, we design a set of hybrid structured light patterns composed of phase-shifted fringe and pseudo-random speckle. Under the illumination of the hybrid patterns, depth can be decently reconstructed either from a few consecutive frames with the TM principle for static scenes or from a single frame with the SN principle for dynamic scenes. Second, we propose a scene-adaptive depth sensing framework based on which a global or region-wise optimal depth map can be generated through motion detection. To validate the proposed scalable paradigm, we develop a real-time (20 fps) depth sensing system. Experimental results demonstrate that our method achieves an efficient balance between accuracy and speed during depth sensing that has rarely been exploited before.

Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects

This Letter presents a multiview phase shifting (MPS) framework for full-resolution and high-speed reconstruction of arbitrary shape dynamic objects. Unlike conventional methods, this framework can directly find the corresponding points from the wrapped phase-maps. Therefore, only a minimum number of images are required for phase shifting to measure arbitrary shape objects, including discontinuous surfaces. Benefit from phase shifting MPS can achieve full spatial resolution and high, accurate 3D reconstruction. Benefit from multiview constraint MPS is also robust to discontinuities. Experimental results are presented to verify the performance of the proposed technique.

Structured-light-based highly dense and robust 3D reconstruction

In this paper, a structured-light-based highly dense and robust 3D reconstruction method is proposed by combining a Gray code and region-shifting pattern. The region-shifting pattern is transformed to the trapezoidal and triangle wave shifting pattern by combining all frames of the region-shifting pattern, and then the boundary of the trapezoidal wave shifting pattern and the peak and phase of the triangle wave shifting pattern are estimated. Through this technique, the spatial resolution is increased about three times. Consequently, the 3D points are reconstructed with a resolution much higher than a camera image resolution. Moreover, as the proposed method measures the boundary and the peak with all frames, it increases the signal-to-noise ratio and is more robust than the conventional methods that use only one or two frames to detect them.

Self-adaptive image reconstruction inspired by insect compound eye mechanism

Inspired by the mechanism of imaging and adaptation to luminosity in insect compound eyes (ICE), we propose an ICE-based adaptive reconstruction method (ARM-ICE), which can adjust the sampling vision field of image according to the environment light intensity. The target scene can be compressive, sampled independently with multichannel through ARM-ICE. Meanwhile, ARM-ICE can regulate the visual field of sampling to control imaging according to the environment light intensity. Based on the compressed sensing joint sparse model (JSM-1), we establish an information processing system of ARM-ICE. The simulation of a four-channel ARM-ICE system shows that the new method improves the peak signal-to-noise ratio (PSNR) and resolution of the reconstructed target scene under two different cases of light intensity. Furthermore, there is no distinct block effect in the result, and the edge of the reconstructed image is smoother than that obtained by the other two reconstruction methods in this work.

Color code identification in coded structured light

Color code is widely employed in coded structured light to reconstruct the three-dimensional shape of objects. Before determining the correspondence, a very important step is to identify the color code. Until now, the lack of an effective evaluation standard has hindered the progress in this unsupervised classification. In this paper, we propose a framework based on the benchmark to explore the new frontier. Two basic facets of the color code identification are discussed, including color feature selection and clustering algorithm design. First, we adopt analysis methods to evaluate the performance of different color features, and the order of these color features in the discriminating power is concluded after a large number of experiments. Second, in order to overcome the drawback of K-means, a decision-directed method is introduced to find the initial centroids. Quantitative comparisons affirm that our method is robust with high accuracy, and it can find or closely approach the global peak.

Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.

Motion analysis of live objects by super-resolution fluorescence microscopy

Motion analysis plays an important role in studing activities or behaviors of live objects in medicine, biotechnology, chemistry, physics, spectroscopy, nanotechnology, enzymology, and biological engineering. This paper briefly reviews the developments in this area mostly in the recent three years, especially for cellular analysis in fluorescence microscopy. The topic has received much attention with the increasing demands in biomedical applications. The tasks of motion analysis include detection and tracking of objects, as well as analysis of motion behavior, living activity, events, motion statistics, and so forth. In the last decades, hundreds of papers have been published in this research topic. They cover a wide area, such as investigation of cell, cancer, virus, sperm, microbe, karyogram, and so forth. These contributions are summarized in this review. Developed methods and practical examples are also introduced. The review is useful to people in the related field for easy referral of the state of the art.

Active vision in robotic systems: A survey of recent developments

In this paper we provide a broad survey of developments in active vision in robotic applications over the last 15 years. With increasing demand for robotic automation, research in this area has received much attention. Among the many factors that can be attributed to a high-performance robotic system, the planned sensing or acquisition of perceptions on the operating environment is a crucial component. The aim of sensor planning is to determine the pose and settings of vision sensors for undertaking a vision-based task that usually requires obtaining multiple views of the object to be manipulated. Planning for robot vision is a complex problem for an active system due to its sensing uncertainty and environmental uncertainty. This paper describes such problems arising from many applications, e.g. object recognition and modeling, site reconstruction and inspection, surveillance, tracking and search, as well as robotic manipulation and assembly, localization and mapping, navigation and exploration. A bundle of solutions and methods have been proposed to solve these problems in the past. They are summarized in this review while enabling readers to easily refer solution methods for practical applications. Representative contributions, their evaluations, analyses, and future research trends are also addressed in an abstract level.

Asymptotic identity in min-plus algebra: a report on CPNS

Network calculus is a theory initiated primarily in computer communication networks, especially in the aspect of real-time communications, where min-plus algebra plays a role. Cyber-physical networking systems (CPNSs) are recently developing fast and models in data flows as well as systems in CPNS are, accordingly, greatly desired. Though min-plus algebra may be a promising tool to linearize any node in CPNS as can be seen from its applications to the Internet computing, there are tough problems remaining unsolved in this regard. The identity in min-plus algebra is one problem we shall address. We shall point out the confusions about the conventional identity in the min-plus algebra and present an analytical expression of the asymptotic identity that may not cause confusions.

Reliability Fusion of Time-of-Flight Depth and Stereo Geometry for High Quality Depth Maps

Time-of-flight range sensors have error characteristics, which are complementary to passive stereo. They provide real-time depth estimates in conditions where passive stereo does not work well, such as on white walls. In contrast, these sensors are noisy and often perform poorly on the textured scenes where stereo excels. We explore their complementary characteristics and introduce a method for combining the results from both methods that achieve better accuracy than either alone. In our fusion framework, the depth probability distribution functions from each of these sensor modalities are formulated and optimized. Robust and adaptive fusion is built on a pixel-wise reliability weighting function calculated for each method. In addition, since time-of-flight devices have primarily been used as individual sensors, they are typically poorly calibrated. We introduce a method that substantially improves upon the manufacturer's calibration. We demonstrate that our proposed techniques lead to improved accuracy and robustness on an extensive set of experimental results.

Determining both surface position and orientation in structured-light-based sensing

Position and orientation profiles are two principal descriptions of shape in space. We describe how a structured light system, coupled with the illumination of a pseudorandom pattern and a suitable choice of feature points, can allow not only the position but also the orientation of individual surface elements to be determined independently. Unlike traditional designs which use the centroids of the illuminated pattern elements as the feature points, the proposed design uses the grid points between the pattern elements instead. The grid points have the essences that their positions in the image data are inert to the effect of perspective distortion, their individual extractions are not directly dependent on one another, and the grid points possess strong symmetry that can be exploited for their precise localization in the image data. Most importantly, the grid lines of the illuminated pattern that form the grid points can aid in determining surface normals. In this paper, we describe how each of the grid points can be labeled with a unique color code, what symmetry they possess and how the symmetry can be exploited for their precise localization at subpixel accuracy in the image data, and how 3D orientation in addition to 3D position can be determined at each of them. Both the position and orientation profiles can be determined with only a single pattern illumination and a single image capture.

Maximum SNR pattern strategy for phase shifting methods in structured light illumination

Structured light illumination by means of phase shifting patterns is a widely employed method for three-dimensional (3-D) image acquisition that is robust to ambient light and object albedo but may be especially susceptible to sensor and environment noise. In this paper, we study the specific technique of phase measuring profilometry (PMP) and the maximization of a pattern's signal to noise ratio (SNR). By treating the design of an N-pattern PMP process as placing points in an N-dimensional coding space, we define a pattern's SNR in terms of a pattern set's computational length and the number of coded phase periods in the projected patterns. Then, without introducing phase ambiguities, we propose a so-called edge pattern strategy that maximizes the computational length and number of periods. Theoretically, the edge pattern technique improves the SNR by 1.2381 times when using three component patterns and by 15.5421 times when using five patterns. Experimental results further demonstrate the improved SNR of the proposed edge pattern technique such that more accurate 3-D results are achieved using fewer component patterns.

Real-time three-dimensional shape measurement of moving objects without edge errors by time-synchronized structured illumination

Structured-light illumination is a process of three-dimensional imaging where a series of time-multiplexed, striped patterns are projected onto a target scene with the corresponding captured images used to determine surface shape according to the warping of the projected patterns around the target. In a real-time system, a high-speed projector/camera pair is used such that any surface motion is small over the projected pattern sequence, but regardless of acquisition speed, there are always those pixels near the edge of a moving surface that capture the projected patterns on both fore- and background surfaces. These edge pixels then create unpredictable results that typically require expensive processing steps to remove, but in this Letter, we introduce a filtering process that identifies motion artifacts based upon the discrete Fourier transform applied to the time axis of the captured pattern sequence. The process is of very low computational complexity, and in this Letter, we demonstrate that in a real-time structured-light illumination (SLI) system, the process comes at a cost of 15 frames per second (fps), where our SLI system drops from 180 to 165 fps after deleting those edge pixels where motion was detected.

Dual-frequency pattern scheme for high-speed 3-D shape measurement

A novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information, and the unit-frequency component is used to reduce phase unwrapping ambiguities. With our proposed pattern scheme, phase unwrapping can overcome the major shortcomings of conventional spatial phase unwrapping: phase jumping and discontinuities. Compared with conventional temporal phase unwrapped approaches, the proposed pattern scheme can achieve higher quality phase data using a less number of patterns. To process data in real time, we also propose and develop look-up table based fast and accurate algorithms for phase generation and 3-D reconstruction. Those fast algorithms can be applied to our pattern scheme as well as traditional phase measuring profilometry. For a 640 x 480 video stream, we can generate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 8.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies.