Fast Descriptors and Correspondence Propagation for Robust Global Point Cloud Registration

Point cloud completion in challenging indoor scenarios with human motion

Combining and completing point cloud data from two or more sensors with arbitrarily relative perspectives in a dynamic, cluttered, and complex environment is challenging, especially when the two sensors have significant perspective differences while the large overlap ratio and feature-rich scene cannot be guaranteed. We create a novel approach targeting this challenging scenario by registering two camera captures in a time series with unknown perspectives and human movements to easily use our system in a real-life scene. In our approach, we first reduce the six unknowns of 3D point cloud completion to three by aligning the ground planes found by our previous perspective-independent 3D ground plane estimation algorithm. Subsequently, we use a histogram-based approach to identify and extract all the humans from each frame generating a three-dimensional (3D) human walking sequence in a time series. To enhance accuracy and performance, we convert 3D human walking sequences to lines by calculating the center of mass (CoM) point of each human body and connecting them. Finally, we match the walking paths in different data trials by minimizing the Fréchet distance between two walking paths and using 2D iterative closest point (ICP) to find the remaining three unknowns in the overall transformation matrix for the final alignment. Using this approach, we can successfully register the corresponding walking path of the human between the two cameras' captures and estimate the transformation matrix between the two sensors.

Collaborative 3D Scene Reconstruction in Large Outdoor Environments Using a Fleet of Mobile Ground Robots

Teams of mobile robots can be employed in many outdoor applications, such as precision agriculture, search and rescue, and industrial inspection, allowing an efficient and robust exploration of large areas and enhancing the operators' situational awareness. In this context, this paper describes an active and decentralized framework for the collaborative 3D mapping of large outdoor areas using a team of mobile ground robots under limited communication range and bandwidth. A real-time method is proposed that allows the sharing and registration of individual local maps, obtained from 3D LiDAR measurements, to build a global representation of the environment. A conditional peer-to-peer communication strategy is used to share information over long-range and short-range distances while considering the bandwidth constraints. Results from both real-world and simulated experiments, executed in an actual solar power plant and in its digital twin representation, demonstrate the reliability and efficiency of the proposed decentralized framework for such large outdoor operations.

High-Accuracy Relative Pose Measurement of Noncooperative Objects Based on Double-Constrained Intersurface Mutual Projections

Relative pose measurement for noncooperative objects is an important part of 3D shape recognition and motion tracking. The methods based on scanning point clouds have better environmental adaptability and stability than image-based methods. However, the discrete points obtained from a continuous surface are sparse, which leads to point-to-point dislocations in the overlapping area and seriously reduces the accuracy. Therefore, this paper proposed a relative-pose-measurement algorithm based on double-constrained intersurface mutual projections. First, the initial corresponding set was constructed using mutual projections between the areas with similar feature descriptors, and then the final corresponding set was determined through the rigid-transformation-consistency constraint to improve the accuracy of the matchings and achieve a high-accuracy relative pose measurement. In the Stanford dataset, the rotation error and translation error were reduced by 19.3% and 13.4%, respectively. Furthermore, based on the proposed evaluation method, which separated the error of the pose-measurement algorithm from that of the instrument, the experiments were carried out with a self-made swept-frequency interferometer. The rotation error was reduced by 39.8%, and the surface deviation was reduced by 4.9%, which further proved the advancement of the method.

Composite Ski-Resort Registration Method Based on Laser Point Cloud Information

The environment of ski resorts is usually complex and changeable, and there are few characteristic objects in the background, which creates many difficulties for the registration of ski-resort point cloud datasets. However, in the traditional iterative closest point (ICP) algorithm, two points need to have good initial positions, otherwise it is easy to get caught up in local optimizations in registration. Aiming at this problem, according to the topographic features of ski resorts, this paper put forward a ski-resort coarse registration method based on extraction, and matching between feature points is proposed to adjust the initial position of two point clouds. Firstly, the feature points of the common part of the point cloud datasets are extracted based on the SIFT algorithm; secondly, the Euclidean distance between the feature normal vectors is used as the pairing condition to complete the pairing between the feature points in the point cloud datasets; then, the feature point pair is purified by using the included angle of the normal vector; finally, in the process of coarse registration, the rotation matrix and translation vector between point clouds are solved by the unit quaternion method. Experiments demonstrate that the proposed coarse registration method based on the normal vector of feature points is helpful to the smooth completion of the subsequent fine registration process, avoids the phenomenon of falling into local optimization, and effectively completes the ski-resort point cloud registration.

Robust affine registration method using line/surface normals and correntropy criterion

The problem of matching point clouds is an efficient way of registration, which is significant for many research fields including computer vision, machine learning, and robotics. There may be linear or non-linear transformation between point clouds, but determining the affine relation is more challenging among linear cases. Various methods have been presented to overcome this problem in the literature and one of them is the affine variant of the iterative closest point (ICP) algorithm. However, traditional affine ICP variants are highly sensitive to effects such as noises, deformations, and outliers; the least-square metric is substituted with the correntropy criterion to increase the robustness of ICPs to such effects. Correntropy-based robust affine ICPs available in the literature use point-to-point metric to estimate transformation between point clouds. Conversely, in this study, a line/surface normal that examines point-to-curve or point-to-plane distances is employed together with the correntropy criterion for affine point cloud registration problems. First, the maximum correntropy criterion measure is built for line/surface normal conditions. Then, the closed-form solution that maximizes the similarity between point sets is achieved for 2D registration and extended for 3D registration. Finally, the application procedure of the developed robust affine ICP method is given and its registration performance is examined through extensive experiments on 2D and 3D point sets. The results achieved highlight that our method can align point clouds more robustly and precisely than the state-of-the-art methods in the literature, while the registration time of the process remains at reasonable levels.

Novel CE-CBCE feature extraction method for object classification using a low-density LiDAR point cloud

Low-end LiDAR sensor provides an alternative for depth measurement and object recognition for lightweight devices. However due to low computing capacity, complicated algorithms are incompatible to be performed on the device, with sparse information further limits the feature available for extraction. Therefore, a classification method which could receive sparse input, while providing ample leverage for the classification process to accurately differentiate objects within limited computing capability is required. To achieve reliable feature extraction from a sparse LiDAR point cloud, this paper proposes a novel Clustered Extraction and Centroid Based Clustered Extraction Method (CE-CBCE) method for feature extraction followed by a convolutional neural network (CNN) object classifier. The integration of the CE-CBCE and CNN methods enable us to utilize lightweight actuated LiDAR input and provides low computing means of classification while maintaining accurate detection. Based on genuine LiDAR data, the final result shows reliable accuracy of 97% through the method proposed.

Super Edge 4-Points Congruent Sets-Based Point Cloud Global Registration

With the acceleration in three-dimensional (3D) high-frame-rate sensing technologies, dense point clouds collected from multiple standpoints pose a great challenge for the accuracy and efficiency of registration. The combination of coarse registration and fine registration has been extensively promoted. Unlike the requirement of small movements between scan pairs in fine registration, coarse registration can match scans with arbitrary initial poses. The state-of-the-art coarse methods, Super 4-Points Congruent Sets algorithm based on the 4-Points Congruent Sets, improves the speed of registration to a linear order via smart indexing. However, the lack of reduction in the scale of original point clouds limits the application. Besides, the coplanarity of registration bases prevents further reduction of search space. This paper proposes a novel registration method called the Super Edge 4-Points Congruent Sets to address the above problems. The proposed algorithm follows a three-step procedure, including boundary segmentation, overlapping regions extraction, and bases selection. Firstly, an improved method based on vector angle is used to segment the original point clouds aiming to thin out the scale of the initial point clouds. Furthermore, overlapping regions extraction is executed to find out the overlapping regions on the contour. Finally, the proposed method selects registration bases conforming to the distance constraints from the candidate set without consideration about coplanarity. Experiments on various datasets with different characteristics have demonstrated that the average time complexity of the proposed algorithm is improved by 89.76%, and the accuracy is improved by 5 mm on average than the Super 4-Points Congruent Sets algorithm. More encouragingly, the experimental results show that the proposed algorithm can be applied to various restrictive cases, such as few overlapping regions and massive noise. Therefore, the algorithm proposed in this paper is a faster and more robust method than Super 4-Points Congruent Sets under the guarantee of the promised quality.

Outline viewpoint feature histogram: An improved point cloud descriptor for recognition and grasping of workpieces

We propose an improved point cloud global descriptor for recognition and grasping of similar workpieces. In the industry, different types of workpieces need to be recognized precisely in some intelligent systems. Deep learning requires a lot of preparation work, and it is difficult to adapt to the variety of workpieces. Furthermore, traditional descriptors based on point pairs cannot meet the requirements of identification. To solve this problem, the Outline Viewpoint Feature Histogram (Outline-VFH) descriptor remains part of the recognition ability of the Viewpoint Feature Histogram (VFH) descriptor and contains an extra outline description, which is established based on the oriented bounding box theory. To validate the effectiveness of the proposed descriptor, experiments were conducted on public dataset and some physical workpieces. The results show that the Outline-VFH is much better than VFH and some other descriptors on recognition and has great potential in vision-based robot grasping applications.

Automatic Registration Between Dental Cone-Beam CT and Scanned Surface via Deep Pose Regression Neural Networks and Clustered Similarities

Computerized registration between maxillofacial cone-beam computed tomography (CT) images and a scanned dental model is an essential prerequisite for surgical planning for dental implants or orthognathic surgery. We propose a novel method that performs fully automatic registration between a cone-beam CT image and an optically scanned model. To build a robust and automatic initial registration method, deep pose regression neural networks are applied in a reduced domain (i.e., two-dimensional image). Subsequently, fine registration is performed using optimal clusters. A majority voting system achieves globally optimal transformations while each cluster attempts to optimize local transformation parameters. The coherency of clusters determines their candidacy for the optimal cluster set. The outlying regions in the iso-surface are effectively removed based on the consensus among the optimal clusters. The accuracy of registration is evaluated based on the Euclidean distance of 10 landmarks on a scanned model, which have been annotated by experts in the field. The experiments show that the registration accuracy of the proposed method, measured based on the landmark distance, outperforms the best performing existing method by 33.09%. In addition to achieving high accuracy, our proposed method neither requires human interactions nor priors (e.g., iso-surface extraction). The primary significance of our study is twofold: 1) the employment of lightweight neural networks, which indicates the applicability of neural networks in extracting pose cues that can be easily obtained and 2) the introduction of an optimal cluster-based registration method that can avoid metal artifacts during the matching procedures.

Registration of multi-view point sets under the perspective of expectation-maximization

Multi-view registration plays a critical role in 3D model reconstruction. To solve this problem, most previous methods align point sets by either partially exploring available information or blindly utilizing unnecessary information, which may lead to undesired results or extra computation complexity. Accordingly, we propose a novel solution for the multi-view registration under the perspective of Expectation-Maximization (EM). The proposed method assumes that each data point is generated from one unique Gaussian Mixture Model (GMM), where its corresponding points in other point sets are regarded as Gaussian centroids with equal covariance and membership probabilities. As it is difficult to obtain real corresponding points in the registration problem, they are approximated by the nearest neighbor in each other aligned point sets. Based on this assumption, it is reasonable to define the likelihood function including all rigid transformations, which require to be estimated for multi-view registration. Subsequently, the EM algorithm is derived to estimate rigid transformations with one Gaussian covariance by maximizing the likelihood function. Since the GMM component number is automatically determined by the number of point sets, there is no trade-off between registration accuracy and efficiency in the proposed method. Finally, the proposed method is tested on several benchmark data sets and compared with state-of-the-art algorithms. Experimental results demonstrate its superior performance on the accuracy, efficiency, and robustness for multi-view registration.

Calibration of Multiple Depth Sensor Network Using Reflective Pattern on Spheres: Theory and Experiments

The usage of depth data from time-of-flight or any equivalent devices overcomes visual challenges presented under low illumination. For integrating such data from multiple sources, the paper proposes a novel tool for the calibration of sensors. The proposed tool consists of retro-reflective stripped spheres. To correctly estimate these spheres, the paper investigates the performance of spherical estimations from either infrared or depth data. The relationship between sensors is determined by calculating the poses between the calibration tool and each sensor. The paper evaluates and compares the proposed approach against other state-of-the-art approaches in terms of shape reconstruction and spatial consistency.

Automatic point cloud registration algorithm based on the feature histogram of local surface

In this paper, we present an efficient algorithm for point cloud registration in presence of low overlap rate and high noise. The proposed registration method mainly includes four parts: the loop voxel filtering, the curvature-based key point selection, the robust geometric descriptor, and the determining and optimization of correspondences based on key point spatial relationship. The loop voxel filtering filters point clouds to a specified resolution. We propose a key point selection algorithm which has a better anti-noise and fast ability. The feature descriptor of key points is highly exclusive which is based on the geometric relationship between the neighborhood points and the center of gravity of the neighborhood. The correspondences in the pair of two point clouds are determined according to the combined features of key points. Finally, the singular value decomposition and ICP algorithm are applied to align two point clouds. The proposed registration method can accurately and quickly register point clouds of different resolutions in noisy situations. We validate our proposal by presenting a quantitative experimental comparison with state-of-the-art methods. Experimental results show that the proposed point cloud registration algorithm has faster calculation speed, higher registration accuracy, and better anti-noise performance.

Global 3D Non-Rigid Registration of Deformable Objects Using a Single RGB-D Camera

We present a novel global non-rigid registration method for dynamic 3D objects. Our method allows objects to undergo large non-rigid deformations and achieves high-quality results even with substantial pose change or camera motion between views. In addition, our method does not require a template prior and uses less raw data than tracking-based methods since only a sparse set of scans is needed. We simultaneously compute the deformations of all the scans by optimizing a global alignment problem to avoid the well-known loop closure problem and use an as-rigid-as-possible constraint to eliminate the shrinkage problem of the deformed shapes, especially near open boundaries of scans. To cope with large-scale problems, we design a coarse-to-fine multi-resolution scheme, which also avoids the optimization being trapped into local minima. The proposed method is evaluated on public datasets and real datasets captured by an RGB-D sensor. The experimental results demonstrate that the proposed method obtains better results than several state-of-the-art methods.

A Building Extraction Approach Based on the Fusion of LiDAR Point Cloud and Elevation Map Texture Features

Building extraction is an important way to obtain information in urban planning, land management, and other fields. As remote sensing has various advantages such as large coverage and real-time capability, it becomes an essential approach for building extraction. Among various remote sensing technologies, the capability of providing 3D features makes the LiDAR point cloud become a crucial means for building extraction. However, the LiDAR point cloud has difficulty distinguishing objects with similar heights, in which case texture features are able to extract different objects in a 2D image. In this paper, a building extraction method based on the fusion of point cloud and texture features is proposed, and the texture features are extracted by using an elevation map that expresses the height of each point. The experimental results show that the proposed method obtains better extraction results than that of other texture feature extraction methods and ENVI software in all experimental areas, and the extraction accuracy is always higher than 87%, which is satisfactory for some practical work.

A Review of Point Set Registration: From Pairwise Registration to Groupwise Registration

This paper presents a comprehensive literature review on point set registration. The state-of-the-art modeling methods and algorithms for point set registration are discussed and summarized. Special attention is paid to methods for pairwise registration and groupwise registration. Some of the most prominent representative methods are selected to conduct qualitative and quantitative experiments. From the experiments we have conducted on 2D and 3D data, CPD-GL pairwise registration algorithm and JRMPC groupwise registration algorithm seem to outperform their rivals both in accuracy and computational complexity. Furthermore, future research directions and avenues in the area are identified.

Multi-view registration of unordered range scans by fast correspondence propagation of multi-scale descriptors

This paper proposes a global approach for the multi-view registration of unordered range scans. Our method starts with the pair-wise registration, where multi-scale descriptor is selected for feature point and the propagation of feature correspondence is accordingly accelerated. Subsequently, we design an effective rule to judge the reliability of these pair-wise registration results. According to the judgment of reliability, we propose a model fusion method, which can utilize reliable results of pair-wise registration to augment the model shape. Finally, multi-view registration can be achieved by operating the pair-wise registration, reliability judgment, and model fusion alternately. The proposed approach can be applied to scene reconstruction and robot mapping. Experimental results conducted on public datasets show that the proposed approach can automatically achieve multi-view registration of unordered range scans. Compared with other related approaches, the proposed approach has superior performances in accuracy and effectiveness.

Toward the Repeatability and Robustness of the Local Reference Frame for 3D Shape Matching: An Evaluation

The local reference frame (LRF), as an independent coordinate system constructed on the local 3D surface, is broadly employed in 3D local feature descriptors. The benefits of the LRF include rotational invariance and full 3D spatial information, thereby greatly boosting the distinctiveness of a 3D feature descriptor. There are numerous LRF methods in the literature; however, no comprehensive study comparing their repeatability and robustness performance under different application scenarios and nuisances has been conducted. This paper evaluates eight state-of-the-art LRF proposals on six benchmarks with different data modalities (e.g., LiDAR, Kinect, and Space Time) and application contexts (e.g., shape retrieval, 3D registration, and 3D object recognition). In addition, the robustness of each LRF to a variety of nuisances, including varying support radii, Gaussian noise, outliers (shot noise), mesh resolution variation, distance to boundary, keypoint localization error, clutter, occlusion, and partial overlap, is assessed. The experimental study also measures the performance under different keypoint detectors, descriptor matching performance when using different LRFs and feature representation combinations, as well as computational efficiency. Considering the evaluation outcomes, we summarize the traits, advantages, and current limitations of the tested LRF methods.