Automatic and rapid whole-body 3D shape measurement based on multinode 3D sensing and speckle projection

DSCNet: lightweight and efficient self-supervised network via depthwise separable cross convolution blocks for speckle image matching

Speckle structured light has become a research hotspot due to its ability to acquire target three-dimensional information with single image projection in recent years. To address the challenges of a low number of extracted speckle feature points, high mismatch rate and poor real-time performance in traditional algorithms, as well as the obstacle of requiring expensive annotation data in deep learning-based methods, a lightweight and efficient self-supervised convolutional neural network (CNN) is proposed to achieve high-precision and rapid matching of speckle images. First, to efficiently utilize the speckle projection information, a feature extraction backbone based on the depthwise separable cross convolution blocks is proposed. Second, in the feature detection module, a softargmax detection head is designed to refine the coordinates of speckle feature points to sub-pixel accuracy. In the feature description module, a coarse-to-fine module is presented to further refine matching accuracy. Third, we adopt strategies of transfer learning and self-supervised learning to improve the generalization and feature representation capabilities of the model. Data augmentation and real-time training techniques are used to improve the robustness of the model. The experimental results show that the proposed method achieves a mean matching accuracy of 91.62% for speckle feature points on the pilot's helmet, with mere 0.95% mismatch rate. The full model runs at 42ms for a speckle image pair on an RTX 3060.

Panoramic three-dimensional optical digitization system assisted by a bi-mirror

The digitization of objects' full surfaces finds widespread applications in fields such as virtual reality, art and design, and medical and biological sciences. For the realization of three-dimensional full-surface digitization of objects within complex sceneries, we propose a straightforward, efficient, and robust panoramic three-dimensional optical digitization system. This system contains a laser-based optical three-dimensional measurement system and a bi-mirror. By integrating mirrors into the system, we enable the illumination of the object from all angles using the projected laser beam in a single scanning process. Moreover, the main camera employed in the system can acquire three-dimensional information of the object from several different viewpoints. The rotational scanning method enhances the efficiency and applicability of the three-dimensional scanning process, enabling the acquisition of surface information of large-scale objects. After obtaining the three-dimensional data of the sample from different viewpoints using laser triangulation, mirror reflection transformation was employed to obtain the full-surface three-dimensional data of the object in the global coordinate system. The proposed method has been subjected to precision and validity experiments using samples with different surface characteristics and sizes, resulting in the demonstration of its capability for achieving correct three-dimensional digitization of the entire surface in diverse complex sceneries.

Surface Reconstruction of Microscale Objects Based on Grid-Patterned Structured-Light Measurements

A structured-light projection system was designed for microscale objects with surface heights that ranged from tens to hundreds of microns. The system was composed of a universal projector and microscope system that supported editing the attributes of structured-light patterns in real-time and was capable of projecting microscale patterns. On this basis, reconstructing the metal surfaces of microscale objects based on grid patterns of structured light was investigated, the internal and external parameters of microscope vision and projection systems were calibrated, and an image algorithm for grid-node detection was designed. The results indicated that the proposed method successfully reconstructed the three-dimensional (3D) surface of microscale objects, and the reconstruction results were consistent with the original surfaces. With 95% confidence, the reconstruction precision in the X- and Y-directions was approximately ±4.0 μm and in the Z-direction was approximately ±7.5 μm. The designed system and the proposed method were suitable for 3D-shape measurement of microstructures in microscopic fields and can be adapted to meet a broader range of applications, as compared to current methods.

Spatiotemporal Correlation-Based Accurate 3D Face Imaging Using Speckle Projection and Real-Time Improvement

The reconstruction of 3D face data is widely used in the fields of biometric recognition and virtual reality. However, the rapid acquisition of 3D data is plagued by reconstruction accuracy, slow speed, excessive scenes and contemporary reconstruction-technology. To solve this problem, an accurate 3D face-imaging implementation framework based on coarse-to-fine spatiotemporal correlation is designed, improving the spatiotemporal correlation stereo matching process and accelerating the processing using a spatiotemporal box filter. The reliability of the reconstruction parameters is further verified in order to resolve the contention between the measurement accuracy and time cost. A binocular 3D data acquisition device with a rotary speckle projector is used to continuously and synchronously acquire an infrared speckle stereo image sequence for reconstructing an accurate 3D face model. Based on the face mask data obtained by the high-precision industrial 3D scanner, the relationship between the number of projected speckle patterns, the matching window size, the reconstruction accuracy and the time cost is quantitatively analysed. An optimal combination of parameters is used to achieve a balance between reconstruction speed and accuracy. Thus, to overcome the problem of a long acquisition time caused by the switching of the rotary speckle pattern, a compact 3D face acquisition device using a fixed three-speckle projector is designed. Using the optimal combination parameters of the three speckles, the parallel pipeline strategy is adopted in each core processing unit to maximise system resource utilisation and data throughput. The most time-consuming spatiotemporal correlation stereo matching activity was accelerated by the graphical processing unit. The results show that the system achieves real-time image acquisition, as well as 3D face reconstruction, while maintaining acceptable systematic precision.

3D face imaging with the spatial-temporal correlation method using a rotary speckle projector

In this paper, a compact, cost-effective, and fast rotary speckle projector (RSP) is designed and manufactured for high-precision three-dimensional (3D) face data acquisition. Compared with the common speckle projectors, RSP uses a simple speckle pattern design method and has a good performance in high-speed projection and compact structure, which allows a flexible balance between measurement accuracy and time cost in a real acquisition task. Using a carefully designed rotation angle of the speckle mask, temporally and spatially non-correlative speckle patterns in the measurement volume can be generated. The rotation angle of the speckle mask is carefully checked and optimally selected via detailed theoretical analysis, simulation, and experiments to ensure 3D reconstruction accuracy across the reconstruction area. Subsequently, a binocular 3D face imaging system composed of the RSP and two cameras is constructed. With captured stereo speckle image pairs, we adopted our previously well-established spatial-temporal correlation method to determine the disparity. The accuracy of the 3D face imaging system was verified by using a real face mask, which is standardized by a certified, high-precision industrial 3D scanner. The real face data collection under various expressions has demonstrated that the proposed system also has a good performance for 3D face imaging in dynamic scenes.

Laser-speckle-projection-based handheld anthropometric measurement system with synchronous redundancy reduction

Human body measurement is essential in modern rehabilitation medicine, which can be effectively combined with the technology of additive manufacturing. Digital image correlation based on laser speckle projection is a single-shot, accurate, and robust technique for human body measurement. In this paper, we present a handheld anthropometric measurement system based on laser speckle projection. A flexible retroreflective marker target is designed for multi-view data registration. Meanwhile, a synchronous redundancy-reduction algorithm based on a re-projected global disparity map is proposed. Experiment results validate that the proposed system is effective and accurate for different human body part measurements. Comparative experiments show that the proposed redundancy-reduction algorithm has high efficiency and can effectively preserve the features of complex shapes. The comprehensive performance of the algorithm is better than the other two tested methods.

3-D face registration solution with speckle encoding based spatial-temporal logical correlation algorithm

3-D information acquisition (registration) of whole face plays a significant role in 3-D human face recognition application. In this paper, we develop a prototype of 3-D system consisting of two binocular measurement units that allows a full 3-D reconstruction by utilizing the advantages of a novel correlation algorithm. In this system, we use optical modulation to produce temporally and spatially varying high-density binary speckle patterns to encode the tested face, then propose a spatial-temporal logical correlation (STLC) stereo matching algorithm to fast determine the accurate disparity with a coarse and refined strategy. Finally the 3-D information of whole face from left- and right ear (~180°) can be obtainable by fusing the data from two measurement units. Comparative researches are performed to test a plastic model and a real human face by simulating real application situations. The results verify the feasibility and good performances of our computational frameworks and experimental configuration in terms of accuracy and time cost, which show a good application prospect in our future 3-D human face recognition research.

Microscopic Three-Dimensional Measurement Based on Telecentric Stereo and Speckle Projection Methods

Three-dimensional (3D) measurement of microstructures has become increasingly important, and many microscopic measurement methods have been developed. For the dimension in several millimeters together with the accuracy at sub-pixel or sub-micron level, there is almost no effective measurement method now. Here we present a method combining the microscopic stereo measurement with the digital speckle projection. A microscopy experimental setup mainly composed of two telecentric cameras and an industrial projection module is established and a telecentric binocular stereo reconstruction procedure is carried out. The measurement accuracy has firstly been verified by performing 3D measurements of grid arrays at different locations and cylinder arrays with different height differences. Then two Mitutoyo step masters have been used for further verification. The experimental results show that the proposed method can obtain 3D information of the microstructure with a sub-pixel and even sub-micron measuring accuracy in millimeter scale.

Accurate three dimensional body scanning system based on structured light

Three dimensional (3D) body scanning has been of great interest to many fields, yet it is still a challenge to generate accurate human body models in a convenient manner. In this paper, we present an accurate 3D body scanning system based on structured light technology. A four-step phase shifting combined with Gray-code method is applied to match pixels in camera and projector planes. The calculation of 3D point coordinates are also derived. The main contribution of this paper is twofold. First, an improved registration algorithm is proposed to align point clouds reconstructed from different views. Second, we propose a graph optimization algorithm to further minimize registration errors. Experimental results demonstrate that our system can produce accurate 3D body models conveniently.