3D Face Profilometry Based on Galvanometer Scanner with Infrared Fringe Projection in High Speed

SE-FSCNet: full-scale connection network for single-shot phase demodulation

The accuracy of phase demodulation has significant impact on the accuracy of fringe projection 3D measurement. Currently, researches based on deep learning methods for extracting wrapped phase mostly use U-Net as the subject of network. The connection method between its hierarchies has certain shortcomings in global information transmission, which hinders the improvement of wrapped phase prediction accuracy. We propose a single-shot phase demodulation method for fringe projection based on a novel full-scale connection network SE-FSCNet. The encoder and decoder of the SE-FSCNet have the same number of hierarchies but are not completely symmetrical. At the decoder a full-scale connection method and feature fusion module are designed so that SE-FSCNet has better abilities of feature transmission and utilization compared with U-Net. A channel attention module based on squeeze and excitation is also introduced to assign appropriate weights to features with different scales, which has been proved by the ablation study. The experiments conducted on the test set have demonstrated that the SE-FSCNet can achieve higher precision than the traditional Fourier transform method and the U-Net in phase demodulation.

Ultra-small, low-cost, and simple-to-control PSP projector based on SLCD technology

Demand for ultra-small, inexpensive, and high-accurate 3D shape measurement devices is growing rapidly, especially in the industrial and consumer electronics sectors. Phase shifting profilometry (PSP) is a powerful candidate due to its advantages of high accuracy, great resolution, and insensitivity to ambient light. As a key component in PSP, the projector used to generate the phase-shifting sinusoidal fringes must be ultra-small (several millimeters), low-cost, and simple to control. However, existing projection methods make it difficult to meet these requirements simultaneously. In this paper, we present a modern technique that can be used to fabricate the desired projector. A specifically designed device based on segmented liquid crystal display (SLCD) technology is used to display the projected patterns, and a cylindrical lens is used as the projection lens. The SLCD device can display four sets of specific filled binary patterns, each yielding a sinusoidal fringe, and all four sinusoidal fringes satisfy the four-step phase shift relation. 3D shape measurement experiments verify the performance of the projector. Considering that the size of SLCD devices can be reduced to a few millimeters, the proposed technique can be easily used to manufacture ultra-small, low-cost, and simple-to-control PSP projectors.

The analysis of infrared high-speed motion capture system on motion aesthetics of aerobics athletes under biomechanics analysis

This paper uses an infrared high-speed motion capture system based on deep learning to analyze difficult movements, which helps aerobics athletes master difficult movements more accurately. Firstly, changes in joint angle, speed of movement, and ground pressure are used to analyze the impact and role of motion fluency and completion based on a biomechanical perspective. Moreover, based on the existing infrared high-speed motion capture systems, the Restricted Boltzmann Machine (RBM) model is introduced to construct an unsupervised similarity framework model. Next, the motion data is reorganized based on three-dimensional information to adapt to the model's input. Then, the framework performs similar frame matching to obtain a set of candidate frames that can be used as motion graph nodes. After the infrared high-speed motion capture system and inertial sensors are simultaneously applied to subjects, the multi-correlation coefficients (CMC) values of the hip, knee, and ankle angles are 0.94 ± 0.06, 0.98 ± 0.01, and 0.87 ± 0.09, respectively. The two systems show a high degree of correlation in the measurement results, and the knee joint is the most significant correlation. Finally, a motion graph is constructed to control its trajectory and adjust its motion pattern. The infrared high-speed motion capture system optimized for deep learning can extract features from human bone data and capture motion more accurately, helping trainers to fully understand difficult movements.

High-flexibility and high-accuracy phase delay calibration method for MEMS-based fringe projection systems

Microelectromechanical system (MEMS) mirror based laser beam scanning (LBS) projectors for fringe projection profilometry (FPP) are becoming increasingly popular attributing to their small size and low cost. However, the initial phase of the scanning MEMS mirror employed in an LBS projector may vary over time, resulting in unstable and distorted fringe patterns. The distorted fringe patterns will largely decrease the accuracy of the three-dimensional (3D) topographic reconstruction. In this paper, an efficient phase delay calibration method based on a unique fringe projection sequence and a corresponding image processing algorithm is proposed. The proposed method can compensate the phase uncertainty and variation with no need to add any extra components. One LBS projector has been constructed using a uniaxial electrostatic MEMS mirror that has a mirror size of 2.5 mm × 2.5 mm and a scanning field of view of 60 ^∘ at its resonance of 1523 Hz. 3D reconstruction experiments are conducted to study how the 3D reconstruction results are affected by the phase delay. The standard deviation of a sphere reconstruction is improved from 2.05 mm to 0.20 mm after the positive phase delay deviation of 5 μs is compensated using this new calibration method.

Phase Target-Based Calibration of Projector Radial Chromatic Aberration for Color Fringe 3D Measurement Systems

The camera and projector are indispensable hardware parts of a color fringe projection 3D measurement system. Chromatic aberration between different color channels of the projector and camera has an impact on the measurement accuracy of the color fringe projection 3D profile measurement. There are many studies on camera calibration, but the chromatic aberration of the projector remains a question deserving of further investigation. In view of the complex system architecture and theoretical derivation of the traditional projector radial chromatic aberration method, a phase target based on projector radial chromatic aberration measurement and the correction method are proposed in this paper. This method uses a liquid crystal display with a holographic projection film as the phase target. The liquid crystal display sequentially displays red, green, and blue horizontal and vertical sinusoidal fringe images. The projector projects red, green, and blue horizontal and vertical sinusoidal fringe images to the phase target in turn, and calculates the absolute phases of the display fringes and reflection fringes, respectively. Taking the green channel as the reference channel, a phase coordinate system is established based on the phases of the vertical and horizontal directions displayed on the display screen, using the phase of the reflection fringes on the display screen as the ideal phase value of the phase point. Then, the phase coordinate system of the red and blue channels is transferred to the green phase coordinate system to calculate the chromatic aberration of the red-green channels and the blue-green channels, and pre-compensation is conducted. Experimental results prove that this method can measure and calibrate the radial chromatic aberration of the projector without being affected by the image quality of the camera. The correction effect of this method is that the maximum chromatic aberration of the red-green channel decreases from 1.9591/pixel to 0.5759/pixel, and the average chromatic aberration decreases from 0.2555/pixel to 0.1865/pixel. In addition, blue-green channel maximum chromatic aberration decreased from 1.8906/pixel to 0.5938/pixel, and the average chromatic aberration decreased from 0.2347/pixel to 0.1907/pixel. This method can improve the projection quality for fringe projection 3D profile measurement technology.

Spatiotemporal Correlation-Based Accurate 3D Face Imaging Using Speckle Projection and Real-Time Improvement

The reconstruction of 3D face data is widely used in the fields of biometric recognition and virtual reality. However, the rapid acquisition of 3D data is plagued by reconstruction accuracy, slow speed, excessive scenes and contemporary reconstruction-technology. To solve this problem, an accurate 3D face-imaging implementation framework based on coarse-to-fine spatiotemporal correlation is designed, improving the spatiotemporal correlation stereo matching process and accelerating the processing using a spatiotemporal box filter. The reliability of the reconstruction parameters is further verified in order to resolve the contention between the measurement accuracy and time cost. A binocular 3D data acquisition device with a rotary speckle projector is used to continuously and synchronously acquire an infrared speckle stereo image sequence for reconstructing an accurate 3D face model. Based on the face mask data obtained by the high-precision industrial 3D scanner, the relationship between the number of projected speckle patterns, the matching window size, the reconstruction accuracy and the time cost is quantitatively analysed. An optimal combination of parameters is used to achieve a balance between reconstruction speed and accuracy. Thus, to overcome the problem of a long acquisition time caused by the switching of the rotary speckle pattern, a compact 3D face acquisition device using a fixed three-speckle projector is designed. Using the optimal combination parameters of the three speckles, the parallel pipeline strategy is adopted in each core processing unit to maximise system resource utilisation and data throughput. The most time-consuming spatiotemporal correlation stereo matching activity was accelerated by the graphical processing unit. The results show that the system achieves real-time image acquisition, as well as 3D face reconstruction, while maintaining acceptable systematic precision.

3D face imaging with the spatial-temporal correlation method using a rotary speckle projector

In this paper, a compact, cost-effective, and fast rotary speckle projector (RSP) is designed and manufactured for high-precision three-dimensional (3D) face data acquisition. Compared with the common speckle projectors, RSP uses a simple speckle pattern design method and has a good performance in high-speed projection and compact structure, which allows a flexible balance between measurement accuracy and time cost in a real acquisition task. Using a carefully designed rotation angle of the speckle mask, temporally and spatially non-correlative speckle patterns in the measurement volume can be generated. The rotation angle of the speckle mask is carefully checked and optimally selected via detailed theoretical analysis, simulation, and experiments to ensure 3D reconstruction accuracy across the reconstruction area. Subsequently, a binocular 3D face imaging system composed of the RSP and two cameras is constructed. With captured stereo speckle image pairs, we adopted our previously well-established spatial-temporal correlation method to determine the disparity. The accuracy of the 3D face imaging system was verified by using a real face mask, which is standardized by a certified, high-precision industrial 3D scanner. The real face data collection under various expressions has demonstrated that the proposed system also has a good performance for 3D face imaging in dynamic scenes.

Three-Dimensional Morphology and Size Measurement of High-Temperature Metal Components Based on Machine Vision Technology: A Review

The three-dimensional (3D) size and morphology of high-temperature metal components need to be measured in real time during manufacturing processes, such as forging and rolling. Since the surface temperature of a metal component is very high during the forming and manufacturing process, manually measuring the size of a metal component at a close distance is difficult; hence, a non-contact measurement technology is required to complete the measurement. Recently, machine vision technology has been developed, which is a non-contact measurement technology that only needs to capture multiple images of a measured object to obtain the 3D size and morphology information, and this technology can be used in some extreme conditions. Machine vision technology has been widely used in industrial, agricultural, military and other fields, especially fields involving various high-temperature metal components. This paper provides a comprehensive review of the application of machine vision technology in measuring the 3D size and morphology of high-temperature metal components. Furthermore, according to the principle and method of measuring equipment structures, this review highlights two aspects in detail: laser scanning measurement and multi-view stereo vision technology. Special attention is paid to each method through comparisons and analyses to provide essential technical references for subsequent researchers.

Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning

Fringe projection profilometry (FPP) has become increasingly important in dynamic 3-D shape measurement. In FPP, it is necessary to retrieve the phase of the measured object before shape profiling. However, traditional phase retrieval techniques often require a large number of fringes, which may generate motion-induced error for dynamic objects. In this paper, a novel phase retrieval technique based on deep learning is proposed, which uses an end-to-end deep convolution neural network to transform a single or two fringes into the phase retrieval required fringes. When the object's surface is located in a restricted depth, the presented network only requires a single fringe as the input, which otherwise requires two fringes in an unrestricted depth. The proposed phase retrieval technique is first theoretically analyzed, and then numerically and experimentally verified on its applicability for dynamic 3-D measurement.

Adaptive Binocular Fringe Dynamic Projection Method for High Dynamic Range Measurement

Three-dimensional measurement with fringe projection sensor has been commonly researched. However, the measurement accuracy and efficiency of most fringe projection sensors are still seriously affected by image saturation and the non-linear effects of the projector. In order to solve the challenge, in conjunction with the advantages of stereo vision technology and fringe projection technology, an adaptive binocular fringe dynamic projection method is proposed. The proposed method can avoid image saturation by adaptively adjusting the projection intensity. Firstly, the flowchart of the proposed method is explained. Then, an adaptive optimal projection intensity method based on multi-threshold segmentation is introduced to adjust the projection illumination. Finally, the mapping relationship of binocular saturation point and projection point is established by binocular transformation and left camera-projector mapping. Experiments demonstrate that the proposed method can achieve higher accuracy for high dynamic range measurement.

Structured Light Three-Dimensional Measurement Based on Machine Learning

The three-dimensional measurement of structured light is commonly used and has widespread applications in many industries. In this study, machine learning is used for structured light 3D measurement to recover the phase distribution of the measured object by employing two machine learning models. Without phase shift, the measurement operational complexity and computation time decline renders real-time measurement possible. Finally, a grating-based structured light measurement system is constructed, and machine learning is used to recover the phase. The calculated phase of distribution is wrapped in only one dimension and not in two dimensions, as in other methods. The measurement error is observed to be under 1%.