Registration of 3D point clouds and meshes: a survey from rigid to nonrigid

Point Cloud Completion: A Survey

Point cloud completion is the task of producing a complete 3D shape given an input of a partial point cloud. It has become a vital process in 3D computer graphics, vision and applications such as autonomous driving, robotics, and augmented reality. These applications often rely on the presence of a complete 3D representation of the environment. Over the past few years, many completion algorithms have been proposed and a substantial amount of research has been carried out. However, there are not many in-depth surveys that summarise the research progress in such a way that allows users to make an informed choice of what algorithms to employ given the type of data they have, the end result they want, the challenges they may face and the possible strategies they could use. In this study, we present a comprehensive survey and classification of articles on point cloud completion untill August 2023 based on the strategies, techniques, inputs, outputs, and network architectures. We will also cover datasets, evaluation methods, and application areas in point cloud completion. Finally, we discuss challenges faced by the research community and future research directions.

Vessel-based CTA-image to spatial anatomy registration using tracked catheter position data: preclinical evaluation of in vivo accuracy

Electromagnetic tracking of endovascular instruments has the potential to substantially decrease radiation exposure of patients and personnel. In this study, we evaluated the in vivo accuracy of a vessel-based method to register preoperative computed tomography angiography (CTA) images to physical coordinates using an electromagnetically tracked guidewire. Centerlines of the aortoiliac arteries were extracted from preoperative CTA acquired from five swine. Intravascular positions were obtained from an electromagnetically tracked guidewire. An iterative-closest-point algorithm registered the position data to the preoperative image centerlines. To evaluate the registration accuracy, a guidewire was placed inside the superior mesenteric, left and right renal arteries under fluoroscopic guidance. Position data was acquired with electromagnetic tracking as the guidewire was pulled into the aorta. The resulting measured positions were compared to the corresponding ostia manually identified in the CTA images after applying the registration. The three-dimensional (3D) Euclidean distances were calculated between each corresponding ostial point, and the root mean square (RMS) was calculated for each registration. The median 3D RMS for all registrations was 4.82 mm, with an interquartile range of 3.53-6.14 mm. A vessel-based registration of CTA images to vascular anatomy is possible with acceptable accuracy and encourages further clinical testing. RELEVANCE STATEMENT: This study shows that the centerline algorithm can be used to register preoperative CTA images to vascular anatomy, with the potential to further reduce ionizing radiation exposure during vascular procedures. KEY POINTS: Preoperative images can be used to guide the procedure without ionizing intraoperative imaging. Preoperative imaging can be the only imaging modality used for guidance of vascular procedures. No need to use external fiducial markers to register/match images and spatial anatomy. Acceptable accuracy can be achieved for navigation in a preclinical setting.

Accurate Registration of Cross-Modality Geometry via Consistent Clustering

The registration of unitary-modality geometric data has been successfully explored over past decades. However, existing approaches typically struggle to handle cross-modality data due to the intrinsic difference between different models. To address this problem, in this article, we formulate the cross-modality registration problem as a consistent clustering process. First, we study the structure similarity between different modalities based on an adaptive fuzzy shape clustering, from which a coarse alignment is successfully operated. Then, we optimize the result using fuzzy clustering consistently, in which the source and target models are formulated as clustering memberships and centroids, respectively. This optimization casts new insight into point set registration, and substantially improves the robustness against outliers. Additionally, we investigate the effect of fuzzier in fuzzy clustering on the cross-modality registration problem, from which we theoretically prove that the classical Iterative Closest Point (ICP) algorithm is a special case of our newly defined objective function. Comprehensive experiments and analysis are conducted on both synthetic and real-world cross-modality datasets. Qualitative and quantitative results demonstrate that our method outperforms state-of-the-art approaches with higher accuracy and robustness. Our code is publicly available at https://github.com/zikai1/CrossModReg.

Robust Point Cloud Registration for Aircraft Engine Pipeline Systems

Aircraft engine systems are composed of numerous pipelines. It is crucial to regularly inspect these pipelines to detect any damages or failures that could potentially lead to serious accidents. The inspection process typically involves capturing complete 3D point clouds of the pipelines using 3D scanning techniques from multiple viewpoints. To obtain a complete and accurate representation of the aircraft pipeline system, it is necessary to register and align the individual point clouds acquired from different views. However, the structures of aircraft pipelines often appear similar from different viewpoints, and the scanning process is prone to occlusions, resulting in incomplete point cloud data. The occlusions pose a challenge for existing registration methods, as they can lead to missing or wrong correspondences. To this end, we present a novel registration framework specifically designed for aircraft pipeline scenes. The proposed framework consists of two main steps. First, we extract the point feature structure of the pipeline axis by leveraging the cylindrical characteristics observed between adjacent blocks. Then, we design a new 3D descriptor called PL-PPFs (Point Line-Point Pair Features), which combines information from both the pipeline features and the engine assembly line features within the aircraft pipeline point cloud. By incorporating these relevant features, our descriptor enables accurate identification of the structure of the engine's piping system. Experimental results demonstrate the effectiveness of our approach on aircraft engine pipeline point cloud data.

Effect of triangular mesh resolution on the geometrical trueness of segmented CBCT maxillofacial data into STL format

OBJECTIVES

To determine the optimal level of mesh reduction that would maintain acceptable levels of geometrical trueness while also minimizing the impact on other parameters such as file size and processing time.

METHODS

Intraoral and extraoral maxillofacial defects were created on 8 cadaver heads and scanned by using a CBCT scanner (NewTom 3D Imaging, Verona). DICOM data were segmented to produce head (n=8) and skull models (n=8) saved as standard tessellation language (STL) files. A further processing of head models was preformed to produce face (n=8) and ear models (n=8). A mesh reduction process was performed for each STL model (reference, R0) by generating 50% (R1), 75% (R2), and 90% (R3) reductions. The 3 datasets were compared to the R0 file using 3D evaluation software (GOM Inspect) using a global best-fit algorithm, to calculate the root mean square (RMS) deviations. Statistical analyses were performed at a level of significance of α=0.05.

RESULTS

There was no 3D deviation after the 50% triangular mesh reduction in the 4 datasets. Minor 3D deviations were observed after 75% reduction, in all groups. After 90% reduction, higher 3D deviations were observed, and especially in head and skull. Statistically significant increase in 3D deviations was observed with higher degrees of mesh reduction (p < 0.001).

CONCLUSION

The resolution of CBCT-based maxillofacial defect models can be reduced up to 50%, with neglectable concern to inaccuracy.

CLINICAL SIGNIFICANCE

Accurate maxillofacial models can be obtained from CBCT DICOM files after segmentation and export as STL files, even when the mesh resolution is reduced up to 50%. This information can be valuable for practitioners and researchers working with 3D models of maxillofacial defects.

Virtual and Augmented Reality in Interventional Radiology: Current Applications, Challenges, and Future Directions

Virtual reality (VR) and augmented Reality (AR) are emerging technologies with the potential to revolutionize Interventional radiology (IR). These innovations offer advantages in patient care, interventional planning, and educational training by improving the visualization and navigation of medical images. Despite progress, several challenges hinder their widespread adoption, including limitations in navigation systems, cost, clinical acceptance, and technical constraints of AR/VR equipment. However, ongoing research holds promise with recent advancements such as shape-sensing needles and improved organ deformation modeling. The development of deep learning techniques, particularly for medical imaging segmentation, presents a promising avenue to address existing accuracy and precision issues. Future applications of AR/VR in IR include simulation-based training, preprocedural planning, intraprocedural guidance, and increased patient engagement. As these technologies advance, they are expected to facilitate telemedicine, enhance operational efficiency, and improve patient outcomes, marking a new frontier in interventional radiology.

A Comprehensive Study of 3-D Vision-Based Robot Manipulation

Robot manipulation, for example, pick-and-place manipulation, is broadly used for intelligent manufacturing with industrial robots, ocean engineering with underwater robots, service robots, or even healthcare with medical robots. Most traditional robot manipulations adopt 2-D vision systems with plane hypotheses and can only generate 3-DOF (degrees of freedom) pose accordingly. To mimic human intelligence and endow the robot with more flexible working capabilities, 3-D vision-based robot manipulation has been studied. However, this task is still challenging in the open world especially for general object recognition and pose estimation with occlusion in cluttered backgrounds and human-like flexible manipulation. In this article, we propose a comprehensive analysis of recent progress about the 3-D vision for robot manipulation, including 3-D data acquisition and representation, robot-vision calibration, 3-D object detection/recognition, 6-DOF pose estimation, grasping estimation, and motion planning. We then present some public datasets, evaluation criteria, comparisons, and challenges. Finally, the related application domains of robot manipulation are given, and some future directions and open problems are studied as well.

Self-Supervised Learning for Non-Rigid Registration Between Near-Isometric 3D Surfaces in Medical Imaging

Non-rigid registration between 3D surfaces is an important but notorious problem in medical imaging, because finding correspondences between non-isometric instances is mathematically non-trivial. We propose a novel self-supervised method to learn shape correspondences directly from a group of bone surfaces segmented from CT scans, without any supervision from time-consuming and error-prone manual annotations. Relying on a Siamese architecture, DiffusionNet as the feature extractor is jointly trained with a pair of randomly rotated and scaled copies of the same shape. The learned embeddings are aligned in spectral domain using eigenfunctions of the Laplace-Beltrami Operator. Additional normalization and regularization losses are incorporated to guide the learned embeddings towards a similar uniform representation over spectrum, which promotes the embeddings to encode multiscale features and advocates sparsity and diagonality of the inferred functional maps. Our method achieves state-of-the-art results among the unsupervised methods on several benchmarks, and presents greater robustness and efficacy in registering moderately deformed shapes. A hybrid refinement strategy is proposed to retrieve smooth and close-to-conformal point-to-point correspondences from the inferred functional map. Our method is orientation and discretization-invariant. Given a pair of near-isometric surfaces, our method automatically computes registration in high accuracy, and outputs anatomically meaningful correspondences. In this study, we show that it is possible to use neural networks to learn general embeddings from 3D shapes in a self-supervised way. The learned features are multiscale, informative, and discriminative, which might potentially benefit almost all types of morphology-related downstream tasks, such as diagnostics, data screening and statistical shape analysis in future.

Generating High-Resolution 3D Faces and Bodies Using VQ-VAE-2 with PixelSNAIL Networks on 2D Representations

Modeling and representing 3D shapes of the human body and face is a prominent field due to its applications in the healthcare, clothes, and movie industry. In our work, we tackled the problem of 3D face and body synthesis by reducing 3D meshes to 2D image representations. We show that the face can naturally be modeled on a 2D grid. At the same time, for more challenging 3D body geometries, we proposed a novel non-bijective 3D-2D conversion method representing the 3D body mesh as a plurality of rendered projections on the 2D grid. Then, we trained a state-of-the-art vector-quantized variational autoencoder (VQ-VAE-2) to learn a latent representation of 2D images and fit a PixelSNAIL autoregressive model to sample novel synthetic meshes. We evaluated our method versus a classical one based on principal component analysis (PCA) by sampling from the empirical cumulative distribution of the PCA scores. We used the empirical distributions of two commonly used metrics, specificity and diversity, to quantitatively demonstrate that the synthetic faces generated with our method are statistically closer to real faces when compared with the PCA ones. Our experiment on the 3D body geometry requires further research to match the test set statistics but shows promising results.

A method based on 3D affine alignment for the quantification of palatal expansion

INTRODUCTION

The current methodologies to quantify the palatal expansion are based on a preliminary rigid superimposition of 3D digital models representing the status of a given patient at different times. A new method based on affine alignment is proposed and compared to the gold standard, leading to the automatic analysis of 3-dimensional structural changes and to a simple numeric quantification of overall expansion vector and a better alignment of the digital models.

MATERIALS AND METHODS

40 digital models (timing span delta 25.8 ± 12.5 months) from young patients (mean age 10.7 ± 2.6) treated with two different palatal expansion techniques (20 subjects with RME-Rapid Maxillary Expander, and 20 subjects with NiTiSE, NiTi self-expander) were superimposed with the new affine alignment technique implemented as an extension package of the open-source MeshLab, from a golden standard starting point of rigid alignment. The results were then compared.

RESULTS

The new measurement function indicates a mean expansion expressed in a single numeric value of 9.3%, 10.3% for the RME group and 8.4% for the NiTiSE group respectively. The comparison with the golden standard showed a decrease to the average error from 0.91 mm to 0.58 mm.

CONCLUSIONS

Affine alignment improves the current perspective of structural change quantification in the specific group of growing patients treated with palatal expanders giving the clinician useful information on the 3-dimensional morphological changes.

Consistent Two-Flow Network for Tele-Registration of Point Clouds

Rigid registration of partial observations is a fundamental problem in various applied fields. In computer graphics, special attention has been given to the registration between two partial point clouds generated by scanning devices. State-of-the-art registration techniques still struggle when the overlap region between the two point clouds is small, and completely fail if there is no overlap between the scan pairs. In this article, we present a learning-based technique that alleviates this problem, and allows registration between point clouds, presented in arbitrary poses, and having little or even no overlap, a setting that has been referred to as tele-registration. Our technique is based on a novel neural network design that learns a prior of a class of shapes and can complete a partial shape. The key idea is combining the registration and completion tasks in a way that reinforces each other. In particular, we simultaneously train the registration network and completion network using two coupled flows, one that register-and-complete, and one that complete-and-register, and encourage the two flows to produce a consistent result. We show that, compared with each separate flow, this two-flow training leads to robust and reliable tele-registration, and hence to a better point cloud prediction that completes the registered scans. It is also worth mentioning that each of the components in our neural network outperforms state-of-the-art methods in both completion and registration. We further analyze our network with several ablation studies and demonstrate its performance on a large number of partial point clouds, both synthetic and real-world, that have only small or no overlap.

A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces

The 3D Morphable Model (3DMM) is a powerful statistical tool for representing 3D face shapes. To build a 3DMM, a training set of face scans in full point-to-point correspondence is required, and its modeling capabilities directly depend on the variability contained in the training data. Thus, to increase the descriptive power of the 3DMM, establishing a dense correspondence across heterogeneous scans with sufficient diversity in terms of identities, ethnicities, or expressions becomes essential. In this manuscript, we present a fully automatic approach that leverages a 3DMM to transfer its dense semantic annotation across raw 3D faces, establishing a dense correspondence between them. We propose a novel formulation to learn a set of sparse deformation components with local support on the face that, together with an original non-rigid deformation algorithm, allow the 3DMM to precisely fit unseen faces and transfer its semantic annotation. We extensively experimented our approach, showing it can effectively generalize to highly diverse samples and accurately establish a dense correspondence even in presence of complex facial expressions. The accuracy of the dense registration is demonstrated by building a heterogeneous, large-scale 3DMM from more than 9,000 fully registered scans obtained by joining three large datasets together.

Collaborative Measurement System of Dual Mobile Robots That Integrates Visual Tracking and 3D Measurement

The manufacturing accuracy of large-scale complex components determines the performance and quality of aircraft, ships, high-speed rail, and other equipment. High-precision 3D measurement plays a crucial role in ensuring manufacturing accuracy. At present, the existing measurement methods rely heavily on manual labor, which cannot satisfy the requirements of industry quality and efficiency. This paper introduces an integrated mobile robotic measurement system for the accurate and automatic 3D measurement of large-scale components with complex curved surfaces. The system consists of the mobile optical scanning measurement device, visual tracking device, and software platform, which can realize comprehensive and accurate data acquisition and stitching of large-scale complex components. The combination of visual tracking and 3D measurement based on the coordinated motion of the dual robot achieved the automatic data acquisition of large-scale complex components without sticking coded targets. Additionally, this paper also introduces a DeepMerge algorithm that combines local and global features of the point cloud, which effectively corrects the initial stitching error of the visual tracking system. The validity of the measurement system and method was shown by the measurement and stitching experiments on the surface of the vehicle nose, ensuring the accurate measurement of the robot’s wide range of motion.

Fast and Robust Iterative Closest Point

The iterative closest point (ICP) algorithm and its variants are a fundamental technique for rigid registration between two point sets, with wide applications in different areas from robotics to 3D reconstruction. The main drawbacks for ICP are its slow convergence, as well as its sensitivity to outliers, missing data, and partial overlaps. Recent work such as Sparse ICP achieves robustness via sparsity optimization at the cost of computational speed. In this paper, we propose a new method for robust registration with fast convergence. First, we show that the classical point-to-point ICP can be treated as a majorization-minimization (MM) algorithm, and propose an Anderson acceleration approach to speed up its convergence. In addition, we introduce a robust error metric based on the Welsch's function, which is minimized efficiently using the MM algorithm with Anderson acceleration. On challenging datasets with noises and partial overlaps, we achieve similar or better accuracy than Sparse ICP while being at least an order of magnitude faster. Finally, we extend the robust formulation to point-to-plane ICP, and solve the resulting problem using a similar Anderson-accelerated MM strategy. Our robust ICP methods improve the registration accuracy on benchmark datasets while being competitive in computational time.

A 3D Image Registration Method for Laparoscopic Liver Surgery Navigation

At present, laparoscopic augmented reality (AR) navigation has been applied to minimally invasive abdominal surgery, which can help doctors to see the location of blood vessels and tumors in organs, so as to perform precise surgery operations. Image registration is the process of optimally mapping one or more images to the target image, and it is also the core of laparoscopic AR navigation. The key is how to shorten the registration time and optimize the registration accuracy. We have studied the three-dimensional (3D) image registration technology in laparoscopic liver surgery navigation and proposed a new registration method combining rough registration and fine registration. First, the adaptive fireworks algorithm (AFWA) is applied to rough registration, and then the optimized iterative closest point (ICP) algorithm is applied to fine registration. We proposed a method that is validated by the computed tomography (CT) dataset 3D-IRCADb-01. Experimental results show that our method is superior to other registration methods based on stochastic optimization algorithms in terms of registration time and accuracy.

FractureNet: A 3D Convolutional Neural Network Based on the Architecture of m-Ary Tree for Fracture Type Identification

To address the problem of automatic identification of fine-grained fracture types, in this paper, we propose a novel framework using 3D convolutional neural network (CNN) to learn fracture features from voxelized bone models which are obtained by establishing isomorphic mapping from fractured bones to a voxelized template. The network, which is named FractureNet, consists of four discriminators forming a multi-stage hierarchy. Each discriminator includes multiple sub-classifiers. These sub-classifiers are chained by two kinds of feature chains (feature map chain and classification feature chain) in the form of a full m-ary tree to perform multi-stage classification tasks. The features learned and classification results obtained at previous stages serve as prior knowledge for current learning and classification. All sub-classifiers are jointly learned in an end-to-end network via a multi-stage loss function integrating losses of the four discriminators. To make our FractureNet more robust and accurate, a data augmentation strategy termed r-combination with constraints is further proposed on the basis of an adjacency relation and a continuity relation between voxels to create a large-scale fracture dataset of voxel models. Extensive experiments show that the proposed method can recognize various fracture types in patients accurately and effectively, and enables significant improvements over the state-of-the-arts on a variety of fracture recognition tasks. Moreover, ancillary experiments on the CIFAR-10 and the PadChest datasets at large scales further support the superior performance of the proposed FractureNet.

Scanning Technologies to Building Information Modelling: A Review

Building information modelling (BIM) is evolving significantly in the architecture, engineering and construction industries. BIM involves various remote-sensing tools, procedures and standards that are useful for collating the semantic information required to produce 3D models. This is thanks to LiDAR technology, which has become one of the key elements in BIM, useful to capture a semantically rich geometric representation of 3D models in terms of 3D point clouds. This review paper explains the ‘Scan to BIM’ methodology in detail. The paper starts by summarising the 3D point clouds of LiDAR and photogrammetry. LiDAR systems based on different platforms, such as mobile, terrestrial, spaceborne and airborne, are outlined and compared. In addition, the importance of integrating multisource data is briefly discussed. Various methodologies involved in point-cloud processing such as sampling, registration and semantic segmentation are explained in detail. Furthermore, different open BIM standards are summarised and compared. Finally, current limitations and future directions are highlighted to provide useful solutions for efficient BIM models.

Fast Registration of Terrestrial LiDAR Point Clouds Based on Gaussian-Weighting Projected Image Matching

Terrestrial point cloud registration plays an important role in 3D reconstruction, heritage restoration and topographic mapping, etc. Unfortunately, current research studies heavily rely on matching the 3D features of overlapped areas between point clouds, which is error-prone and time-consuming. To this end, we propose an automatic point cloud registration method based on Gaussian-weighting projected image matching, which can quickly and robustly register multi-station terrestrial point clouds. Firstly, the point cloud is regularized into a 2D grid, and the point density of each cell in the grid is normalized by our Gaussian-weighting function. A grayscale image is subsequently generated by shifting and scaling the x-y coordinates of the grid to the image coordinates. Secondly, the scale-invariant features (SIFT) algorithm is used to perform image matching, and a line segment endpoint verification method is proposed to filter out negative matches. Thirdly, the transformation matrix between point clouds from two adjacent stations is calculated based on reliable image matching. Finally, a global least-square optimization is conducted to align multi-station point clouds and then obtain a complete model. To test the performance of our framework, we carry out the experiment on six datasets. Compared to previous work, our method achieves the state-of-the-art performance on both efficiency and accuracy. In terms of efficiency, our method is comparable to an existing projection-based methods and 4 times faster on the indoor datasets and 10 times faster on the outdoor datasets than 4PCS-based methods. In terms of accuracy, our framework is ~2 times better than the existing projection-based method and 6 times better than 4PCS-based methods.

IMU-Aided Registration of MLS Point Clouds Using Inertial Trajectory Error Model and Least Squares Optimization

Mobile laser scanning (MLS) point cloud registration plays a critical role in mobile 3D mapping and inspection, but conventional point cloud registration methods for terrain LiDAR scanning (TLS) are not suitable for MLS. To cope with this challenge, we use inertial measurement unit (IMU) to assist registration and propose an MLS point cloud registration method based on an inertial trajectory error model. First, we propose an error model of inertial trajectory over a short time period to construct the constraints between trajectory points at different times. On this basis, a relationship between the point cloud registration error and the inertial trajectory error is established, then trajectory error parameters are estimated by minimizing the point cloud registration error using the least squares optimization. Finally, a reliable and concise inertial-assisted MLS registration algorithm is realized. We carried out experiments in three different scenarios: indoor, outdoor and integrated indoor–outdoor. We evaluated the overall performance, accuracy and efficiency of the proposed method. Compared with the ICP method, the accuracy and speed of the proposed method were improved by 2 and 2.8 times, respectively, which verified the effectiveness and reliability of the proposed method. Furthermore, experimental results show the significance of our method in constructing a reliable and scalable mobile 3D mapping system suitable for complex scenes.

A Robust Rigid Registration Framework of 3D Indoor Scene Point Clouds Based on RGB-D Information

Rigid registration of 3D indoor scenes is a fundamental yet vital task in various fields that include remote sensing (e.g., 3D reconstruction of indoor scenes), photogrammetry measurement, geometry modeling, etc. Nevertheless, state-of-the-art registration approaches still have defects when dealing with low-quality indoor scene point clouds derived from consumer-grade RGB-D sensors. The major challenge is accurately extracting correspondences between a pair of low-quality point clouds when they contain considerable noise, outliers, or weak texture features. To solve the problem, we present a point cloud registration framework in view of RGB-D information. First, we propose a point normal filter for effectively removing noise and simultaneously maintaining sharp geometric features and smooth transition regions. Second, we design a correspondence extraction scheme based on a novel descriptor encoding textural and geometry information, which can robustly establish dense correspondences between a pair of low-quality point clouds. Finally, we propose a point-to-plane registration technology via a nonconvex regularizer, which can further diminish the influence of those false correspondences and produce an exact rigid transformation between a pair of point clouds. Compared to existing state-of-the-art techniques, intensive experimental results demonstrate that our registration framework is excellent visually and numerically, especially for dealing with low-quality indoor scenes.

Deep Learning-Based Computer-Aided Detection System for Automated Treatment Response Assessment of Brain Metastases on 3D MRI

BACKGROUND

Although accurate treatment response assessment for brain metastases (BMs) is crucial, it is highly labor intensive. This retrospective study aimed to develop a computer-aided detection (CAD) system for automated BM detection and treatment response evaluation using deep learning.

METHODS

We included 214 consecutive MRI examinations of 147 patients with BM obtained between January 2015 and August 2016. These were divided into the training (174 MR images from 127 patients) and test datasets according to temporal separation (temporal test set #1; 40 MR images from 20 patients). For external validation, 24 patients with BM and 11 patients without BM from other institutions were included (geographic test set). In addition, we included 12 MRIs from BM patients obtained between August 2017 and March 2020 (temporal test set #2). Detection sensitivity, dice similarity coefficient (DSC) for segmentation, and agreements in one-dimensional and volumetric Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) criteria between CAD and radiologists were assessed.

RESULTS

In the temporal test set #1, the sensitivity was 75.1% (95% confidence interval [CI]: 69.6%, 79.9%), mean DSC was 0.69 ± 0.22, and false-positive (FP) rate per scan was 0.8 for BM ≥ 5 mm. Agreements in the RANO-BM criteria were moderate (κ, 0.52) and substantial (κ, 0.68) for one-dimensional and volumetric, respectively. In the geographic test set, sensitivity was 87.7% (95% CI: 77.2%, 94.5%), mean DSC was 0.68 ± 0.20, and FP rate per scan was 1.9 for BM ≥ 5 mm. In the temporal test set #2, sensitivity was 94.7% (95% CI: 74.0%, 99.9%), mean DSC was 0.82 ± 0.20, and FP per scan was 0.5 (6/12) for BM ≥ 5 mm.

CONCLUSIONS

Our CAD showed potential for automated treatment response assessment of BM ≥ 5 mm.

A Fast and Robust Rotation Search and Point Cloud Registration Method for 2D Stitching and 3D Object Localization

Rotation search and point cloud registration are two fundamental problems in robotics, geometric vision, and remote sensing, which aim to estimate the rotation and transformation between the 3D vector sets and point clouds, respectively. Due to the presence of outliers (probably in very large numbers) among the putative vector or point correspondences in real-world applications, robust estimation is of great importance. In this paper, we present Inlier searching using COmpatible Structures (ICOS), a novel, efficient, and highly robust solver for both the correspondence-based rotation search and point cloud registration problems. Specifically, we (i) propose and construct a series of compatible structures for the two problems, based on which various invariants can be established, and (ii) design time-efficient frameworks to filter out outliers and seek inliers from the invariant-constrained random sampling based on the compatible structures proposed. In this manner, even with extreme outlier ratios, inliers can be effectively sifted out and collected for solving the optimal rotation and transformation, leading to our robust solver ICOS. Through plentiful experiments over standard datasets, we demonstrated that: (i) our solver ICOS is fast, accurate, and robust against over 95% outliers with nearly a 100% recall ratio of inliers for rotation search and both known-scale and unknown-scale registration, outperforming other state-of-the-art methods, and (ii) ICOS is practical for use in real-world application problems including 2D image stitching and 3D object localization.

Point Set Registration for 3D Range Scans Using Fuzzy Cluster-Based Metric and Efficient Global Optimization

This study presents a new point set registration method to align 3D range scans. In our method, fuzzy clusters are utilized to represent a scan, and the registration of two given scans is realized by minimizing a fuzzy weighted sum of the distances between their fuzzy cluster centers. This fuzzy cluster-based metric has a broad basin of convergence and is robust to noise. Moreover, this metric provides analytic gradients, allowing standard gradient-based algorithms to be applied for optimization. Based on this metric, the outlier issues are addressed. In addition, for the first time in rigid point set registration, a registration quality assessment in the absence of ground truth is provided. Furthermore, given specified rotation and translation spaces, we derive the upper and lower bounds of the fuzzy cluster-based metric and develop a branch-and-bound (BnB)-based optimization scheme, which can globally minimize the metric regardless of the initialization. This optimization scheme is performed in an efficient coarse-to-fine fashion: First, fuzzy clustering is applied to describe each of the two given scans by a small number of fuzzy clusters. Then, a global search, which integrates BnB and gradient-based algorithms, is implemented to achieve a coarse alignment for the two scans. During the global search, the registration quality assessment offers a beneficial stop criterion to detect whether a good result is obtained. Afterwards, a relatively large number of points of the two scans are directly taken as the fuzzy cluster centers, and then, the coarse solution is refined to be an exact alignment using the gradient-based local convergence. Compared to existing counterparts, this optimization scheme makes a large improvement in terms of robustness and efficiency by virtue of the fuzzy cluster-based metric and the registration quality assessment. In the experiments, the registration results of several 3D range scan pairs demonstrate the accuracy and effectiveness of the proposed method, as well as its superiority to state-of-the-art registration approaches.

Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation

Standard registration algorithms need to be independently applied to each surface to register, following careful pre-processing and hand-tuning. Recently, learning-based approaches have emerged that reduce the registration of new scans to running inference with a previously-trained model. The potential benefits are multifold: inference is typically orders of magnitude faster than solving a new instance of a difficult optimization problem, deep learning models can be made robust to noise and corruption, and the trained model may be re-used for other tasks, e.g. through transfer learning. In this paper, we cast the registration task as a surface-to-surface translation problem, and design a model to reliably capture the latent geometric information directly from raw 3D face scans. We introduce Shape-My-Face (SMF), a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model that we smoothly integrate with the mesh convolutions. Compared to the previous state-of-the-art learning algorithms for non-rigid registration of face scans, SMF only requires the raw data to be rigidly aligned (with scaling) with a pre-defined face template. Additionally, our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets. We extensively evaluate the quality of our registrations on diverse data. We demonstrate the robustness and generalizability of our model with in-the-wild face scans across different modalities, sensor types, and resolutions. Finally, we show that, by learning to register scans, SMF produces a hybrid linear and non-linear morphable model. Manipulation of the latent space of SMF allows for shape generation, and morphing applications such as expression transfer in-the-wild. We train SMF on a dataset of human faces comprising 9 large-scale databases on commodity hardware.

A method for finding high accuracy surface zones on 3D printed bone models

The use of three-dimensional (3D) printing for surgical applications is steadily increasing. Errors in the printed models can lead to complications, especially when the model is used for surgery planning or diagnostics. In patient care, the validation of printed models should therefore be performed routinely. However, there currently is no standard method to determine whether the printed model meets the necessary quality requirements. In this work, we present a method that not only finds surface deviations of a printed model, but also shows high accuracy zones of a potentially corrupted model, that are safe to be used for surgery planning. Our method was tested on printed patient bone models with acetabular fractures and was compared to two common methods in orthopedics, simple landmark registration as well as landmark plus subsequent iterative closest point registration. In order to find suitable parameters and to evaluate the performance of our method, 15 digital acetabular bone models were artificially deformed, imitating four typical 3D printing errors. A sensitivity of over 95% and a specificity of over 99% was observed in finding these surface deformations. Then, the method was applied to 32 printed models that had been re-digitized using a computed tomography scanner. It was found that only 25% of these printed models were free of significant deformations. However, focussing on two common implant locations, our method revealed that 72% of the models were within the acceptable error tolerance. In comparison, simple landmark registration resulted in a 9% acceptance rate and landmark registration followed by iterative closest point registration resulted in a 41% acceptance rate. This outcome shows that our method, named Similarity Subgroups Registration, allows clinicians to safely use partially corrupted 3D printed models for surgery planning. This improves efficiency and reduces time to treatment by avoiding reprints. The similarity subgroups registration is applicable in further clinical domains as well as non-medical applications that share the requirement of local high accuracy zones on the surface of a 3D model.

RegARD: Symmetry-Based Coarse Registration of Smartphone’s Colorful Point Clouds with CAD Drawings for Low-Cost Digital Twin Buildings

Coarse registration of 3D point clouds plays an indispensable role for parametric, semantically rich, and realistic digital twin buildings (DTBs) in the practice of GIScience, manufacturing, robotics, architecture, engineering, and construction. However, the existing methods have prominently been challenged by (i) the high cost of data collection for numerous existing buildings and (ii) the computational complexity from self-similar layout patterns. This paper studies the registration of two low-cost data sets, i.e., colorful 3D point clouds captured by smartphones and 2D CAD drawings, for resolving the first challenge. We propose a novel method named ‘Registration based on Architectural Reflection Detection’ (RegARD) for transforming the self-symmetries in the second challenge from a barrier of coarse registration to a facilitator. First, RegARD detects the innate architectural reflection symmetries to constrain the rotations and reduce degrees of freedom. Then, a nonlinear optimization formulation together with advanced optimization algorithms can overcome the second challenge. As a result, high-quality coarse registration and subsequent low-cost DTBs can be created with semantic components and realistic appearances. Experiments showed that the proposed method outperformed existing methods considerably in both effectiveness and efficiency, i.e., 49.88% less error and 73.13% less time, on average. The RegARD presented in this paper first contributes to coarse registration theories and exploitation of symmetries and textures in 3D point clouds and 2D CAD drawings. For practitioners in the industries, RegARD offers a new automatic solution to utilize ubiquitous smartphone sensors for massive low-cost DTBs.

A Strip Adjustment Method of UAV-Borne LiDAR Point Cloud Based on DEM Features for Mountainous Area

Due to the trajectory error of the low-precision position and orientation system (POS) used in unmanned aerial laser scanning (ULS), discrepancies usually exist between adjacent LiDAR (Light Detection and Ranging) strips. Strip adjustment is an effective way to eliminate these discrepancies. However, it is difficult to apply existing strip adjustment methods in mountainous areas with few artificial objects. Thus, digital elevation model-iterative closest point (DEM-ICP), a pair-wise registration method that takes topography features into account, is proposed in this paper. First, DEM-ICP filters the point clouds to remove the non-ground points. Second, the ground points are interpolated to generate continuous DEMs. Finally, a point-to-plane ICP algorithm is performed to register the adjacent DEMs with the overlapping area. A graph-based optimization is utilized following DEM-ICP to estimate the correction parameters and achieve global consistency between all strips. Experiments were carried out using eight strips collected by ULS in mountainous areas to evaluate the proposed method. The average root-mean-square error (RMSE) of all data was less than 0.4 m after the proposed strip adjustment, which was only 0.015 m higher than the result of manual registration (ground truth). In addition, the plane fitting accuracy of lateral point clouds was improved 4.2-fold, from 1.565 to 0.375 m, demonstrating the robustness and accuracy of the proposed method.

OICP: An Online Fast Registration Algorithm Based on Rigid Translation Applied to Wire Arc Additive Manufacturing of Mold Repair

As promising technology with low requirements and high depositing efficiency, Wire Arc Additive Manufacturing (WAAM) can significantly reduce the repair cost and improve the formation quality of molds. To further improve the accuracy of WAAM in repairing molds, the point cloud model that expresses the spatial distribution and surface characteristics of the mold is proposed. Since the mold has a large size, it is necessary to be scanned multiple times, resulting in multiple point cloud models. The point cloud registration, such as the Iterative Closest Point (ICP) algorithm, then plays the role of merging multiple point cloud models to reconstruct a complete data model. However, using the ICP algorithm to merge large point clouds with a low-overlap area is inefficient, time-consuming, and unsatisfactory. Therefore, this paper provides the improved Offset Iterative Closest Point (OICP) algorithm, which is an online fast registration algorithm suitable for intelligent WAAM mold repair technology. The practicality and reliability of the algorithm are illustrated by the comparison results with the standard ICP algorithm and the three-coordinate measuring instrument in the Experimental Setup Section. The results are that the OICP algorithm is feasible for registrations with low overlap rates. For an overlap rate lower than 60% in our experiments, the traditional ICP algorithm failed, while the Root Mean Square (RMS) error reached 0.1 mm, and the rotation error was within 0.5 degrees, indicating the improvement of the proposed OICP algorithm.

Bronchoscope motion tracking using centerline-guided Gaussian mixture model in navigated bronchoscopy

Electromagnetic-based navigation bronchoscopy requires accurate and robust estimation of the bronchoscope position inside the bronchial tree. However, respiratory motion, coughing, patient movement, and airway deformation inflicted by bronchoscope significantly hinder the accuracy of intraoperative bronchoscopic localization. In this study, a real-time and automatic registration procedure was proposed to superimpose the current location of the bronchoscope to corresponding locations on a centerline extracted from bronchial computed tomography (CT) images. A centerline-guided Gaussian mixture model (CG-GMM) was introduced to register a bronchoscope's position concerning extracted centerlines. A GMM was fitted to bronchoscope positions where the orientation likelihood was chosen to assign the membership probabilities of the mixture model, which led to preserving the global and local structures. The problem was formulated and solved under the expectation maximization framework, where the feature correspondence and spatial transformation are estimated iteratively. Validation was performed on a dynamic phantom with four different respiratory motions and four human real bronchoscopy (RB) datasets. Results of the experiments conducted on the bronchial phantom showed that the average positional tracking error using the proposed approach was equal to 1.98 [Formula: see text] 0.98 mm that was reduced in comparison with independent electromagnetic tracking (EMT), iterative closest point (ICP), and coherent point drift (CPD) methods by 64%, 58%, and 53%, respectively. In the patient assessment part of the study, the average positional tracking error was 4.73 [Formula: see text] 4.76 mm and compared to ICP, and CPD methods showed 31.4% improvement of successfully tracked frames. Our approach introduces a novel method for real-time respiratory motion compensation that provides reliable guidance during bronchoscopic interventions and, thus could increase the diagnostic yield of transbronchial biopsy.

Automatic Registration Between Dental Cone-Beam CT and Scanned Surface via Deep Pose Regression Neural Networks and Clustered Similarities

Computerized registration between maxillofacial cone-beam computed tomography (CT) images and a scanned dental model is an essential prerequisite for surgical planning for dental implants or orthognathic surgery. We propose a novel method that performs fully automatic registration between a cone-beam CT image and an optically scanned model. To build a robust and automatic initial registration method, deep pose regression neural networks are applied in a reduced domain (i.e., two-dimensional image). Subsequently, fine registration is performed using optimal clusters. A majority voting system achieves globally optimal transformations while each cluster attempts to optimize local transformation parameters. The coherency of clusters determines their candidacy for the optimal cluster set. The outlying regions in the iso-surface are effectively removed based on the consensus among the optimal clusters. The accuracy of registration is evaluated based on the Euclidean distance of 10 landmarks on a scanned model, which have been annotated by experts in the field. The experiments show that the registration accuracy of the proposed method, measured based on the landmark distance, outperforms the best performing existing method by 33.09%. In addition to achieving high accuracy, our proposed method neither requires human interactions nor priors (e.g., iso-surface extraction). The primary significance of our study is twofold: 1) the employment of lightweight neural networks, which indicates the applicability of neural networks in extracting pose cues that can be easily obtained and 2) the introduction of an optimal cluster-based registration method that can avoid metal artifacts during the matching procedures.

When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)

This paper reviews recent deep learning-based registration methods. Registration is the process that computes the transformation that aligns datasets, and the accuracy of the result depends on multiple factors. The most significant factors are the size of input data; the presence of noise, outliers and occlusions; the quality of the extracted features; real-time requirements; and the type of transformation, especially those defined by multiple parameters, such as non-rigid deformations. Deep Registration Networks (DRNs) are those architectures trying to solve the alignment task using a learning algorithm. In this review, we classify these methods according to a proposed framework based on the traditional registration pipeline. This pipeline consists of four steps: target selection, feature extraction, feature matching, and transform computation for the alignment. This new paradigm introduces a higher-level understanding of registration, which makes explicit the challenging problems of traditional approaches. The main contribution of this work is to provide a comprehensive starting point to address registration problems from a learning-based perspective and to understand the new range of possibilities.

Assessment of manual adjustment performed in clinical practice following deep learning contouring for head and neck organs at risk in radiotherapy

BACKGROUND AND PURPOSE

Auto-contouring performance has been widely studied in development and commissioning studies in radiotherapy, and its impact on clinical workflow assessed in that context. This study aimed to evaluate the manual adjustment of auto-contouring in routine clinical practice and to identify improvements regarding the auto-contouring model and clinical user interaction, to improve the efficiency of auto-contouring.

MATERIALS AND METHODS

A total of 103 clinical head and neck cancer cases, contoured using a commercial deep-learning contouring system and subsequently checked and edited for clinical use were retrospectively taken from clinical data over a twelve-month period (April 2019-April 2020). The amount of adjustment performed was calculated, and all cases were registered to a common reference frame for assessment purposes. The median, 10th and 90th percentile of adjustment were calculated and displayed using 3D renderings of structures to visually assess systematic and random adjustment. Results were also compared to inter-observer variation reported previously. Assessment was performed for both the whole structures and for regional sub-structures, and according to the radiation therapy technologist (RTT) who edited the contour.

RESULTS

The median amount of adjustment was low for all structures (<2 mm), although large local adjustment was observed for some structures. The median was systematically greater or equal to zero, indicating that the auto-contouring tends to under-segment the desired contour.

CONCLUSION

Auto-contouring performance assessment in routine clinical practice has identified systematic improvements required technically, but also highlighted the need for continued RTT training to ensure adherence to guidelines.

Multilevel Optimization for Registration of Deformable Point Clouds

Handling deformation is one of the biggest challenges associated with point cloud registration. When deformation happens due to the motion of an animated object which actively changes its location and general shape, registration of two instances of the same object turns out to be a challenging task. The focus of this work is to address the problem by leveraging the complementary attributes of local and global geometric structures of the point clouds. We define an energy function which consists of local and global terms, as well as a semi-local term to model the intermediate level geometry of the point cloud. The local energy estimates the transformation parameters at the lowest level by assuming a reduced deformation model. The parameters are estimated in a closed form solution, which are then used to assign the initial probability of a stochastic model working at the intermediate level. The global energy term estimates the overall transformation parameters by minimizing a nonlinear least square function via Gauss-Newton optimization framework. The total energy is optimized in a block coordinate descent fashion, updating one term at a time while keeping others constant. Experiments on three publicly available datasets show that the method performs significantly better than several state-of-the-art algorithms in registering pairwise point cloud data.

Efficient Similarity Point Set Registration by Transformation Decomposition

Point set registration is one of the basic problems in computer vision. When the overlap ratio between point sets is small or the relative transformation is large, local methods cannot guarantee the accuracy. However, the time complexity of the branch and bound (BnB) optimization used in most existing global methods is exponential in the dimensionality of parameter space. Therefore, seven-Degrees of Freedom (7-DoF) similarity transformation is a big challenge for BnB. In this paper, a novel rotation and scale invariant feature is introduced to decouple the optimization of translation, rotation, and scale in similarity point set registration, so that BnB optimization can be done in two lower dimensional spaces. With the transformation decomposition, the translation is first estimated and then the rotation is optimized by maximizing a robust objective function defined on consensus set. Finally, the scale is estimated according to the potential correspondences in the obtained consensus set. Experiments on synthetic data and clinical data show that our method is approximately two orders of magnitude faster than the state-of-the-art global method and more accurate than a typical local method. When the outlier ratio with respect to the inliers is up to 1.0, our method still achieves accurate registration.

Deep Global Features for Point Cloud Alignment

Point cloud registration is a key problem in computer vision applications and involves finding a rigid transform from a point cloud into another such that they align together. The iterative closest point (ICP) method is a simple and effective solution that converges to a local optimum. However, despite the fact that point cloud registration or alignment is addressed in learning-based methods, such as PointNetLK, they do not offer good generalizability for point clouds. In this stud, we proposed a learning-based approach that addressed existing problems, such as finding local optima for ICP and achieving minimum generalizability. The proposed model consisted of three main parts: an encoding network, an auxiliary module that weighed the contribution of each input point cloud, and feature alignment to achieve the final transform. The proposed architecture offered greater generalization among the categories. Experiments were performed on ModelNet40 with different configurations and the results indicated that the proposed approach significantly outperformed the state-of-the-art point cloud alignment methods.

Assessment and application of the coherent point drift algorithm to augmented reality surgical navigation for laparoscopic partial nephrectomy

PURPOSE

The surface-based registration approach to laparoscopic augmented reality (AR) has clear advantages. Nonrigid point-set registration paves the way for surface-based registration. Among current non-rigid point set registration methods, the coherent point drift (CPD) algorithm is rarely used because of two challenges: (1) volumetric deformation is difficult to predict, and (2) registration from intraoperative visible tissue surface to whole anatomical preoperative model is a "part-to-whole" registration that CPD cannot be applied directly to. We preliminarily applied CPD on surgical navigation for laparoscopic partial nephrectomy (LPN). However, it introduces normalization errors and lacks navigation robustness. This paper presents important advances for more effectively applying CPD to LPN surgical navigation while attempting to quantitatively evaluate the accuracy of CPD-based surgical navigation.

METHODS

First, an optimized volumetric deformation (Op-VD) algorithm is proposed to achieve accurate prediction of volume deformation. Then, a projection-based partial selection method is presented to conveniently and robustly apply the CPD to LPN surgical navigation. Finally, kidneys with different deformations in vitro, phantom and in vivo experiments are performed to evaluate the accuracy and effectiveness of our approach.

RESULTS

The average root-mean-square error of volume deformation was refined to 0.84 mm. The mean target registration error (TRE) of the surface and inside markers in the in vitro experiments decreased to 1.51 mm and 1.29 mm, respectively. The robustness and precision of CPD-based navigation were validated in phantom and in vivo experiments, and the mean navigation TRE of the phantom experiments was found to be [Formula: see text] mm.

CONCLUSION

Accurate volumetric deformation and robust navigation results can be achieved in AR navigation of LPN by using surface-based registration with CPD. Evaluation results demonstrate the effectiveness of our proposed methods while showing the clinical application potential of CPD. This work has important guiding significance for the application of the CPD in laparoscopic AR.

A robust and semi-automatic quantitative measurement of patellofemoral instability based on four dimensional computed tomography

Patellofemoral instability is a motion related disease, featured as the patella dislocating from the trochlear groove. Four dimensional computed tomography (4DCT) enables full assessment of the patellofemoral movement. Nevertheless, the quantitative measurements of patellofemoral instability are still under research and currently of limited practical use. The aim of this study is to develop a robust and semi-automatic workflow to quantitatively describe the patellofemoral movement in a patient group of eight suffering from patellofemoral instability. The initial results show agreement with manual observations of the tibial tubercle - trochlear groove (TT-TG) distance in routine practice, and the possibility to evaluate both TT-TG distance and patellar centre - trochlear groove (PC-TG) distance dynamically during active flexion-extension-flexion movement of the knee.

Augmented and Mixed Reality: Technologies for Enhancing the Future of IR

Augmented and mixed reality are emerging interactive and display technologies. These technologies are able to merge virtual objects, in either 2 or 3 dimensions, with the real world. Image guidance is the cornerstone of interventional radiology. With augmented or mixed reality, medical imaging can be more readily accessible or displayed in actual 3-dimensional space during procedures to enhance guidance, at times when this information is most needed. In this review, the current state of these technologies is addressed followed by a fundamental overview of their inner workings and challenges with 3-dimensional visualization. Finally, current and potential future applications in interventional radiology are highlighted.

A Grassmannian Graph Approach to Affine Invariant Feature Matching

In this work, we present a novel, theoretical approach to address one of the longstanding problems in computer vision: 2D and 3D affine invariant feature matching. Our proposed Grassmannian Graph (GrassGraph) framework employs a two stage procedure that is capable of robustly recovering correspondences between two unorganized, affinely related feature (point) sets. In the ideal case, the first stage maps the feature sets to an affine invariant Grassmannian representation, where the features are mapped into the same subspace. It turns out that coordinate representations extracted from the Grassmannian differ by an arbitrary orthonormal matrix. In the second stage, by approximating the Laplace-Beltrami operator (LBO) on these coordinates, this extra orthonormal factor is nullified, providing true affine invariant coordinates which we then utilize to recover correspondences via simple mutual nearest neighbor relations. Our validation benchmarks use large number of experimental trials performed on 2D and 3D datasets. Experimental results show that the proposed GrassGraph method successfully recovers large affine transformations.

Evaluating Local Geometric Feature Representations for 3D Rigid Data Matching

Local geometric descriptors act as an essential component for 3D rigid data matching. A rotational invariant local geometric descriptor usually consists of two components: local reference frame (LRF) and feature representation. However, existing evaluation efforts have mainly been paid on the LRF or the overall descriptor and the quantitative comparison of feature representations remains unexplored. This paper fills the gap by comprehensively evaluating nine state-of-the-art local geometric feature representations. In particular, our evaluation assesses feature representations based on ground-truth LRFs such that the ranking of tested methods is more convincing as compared with existing studies. The experiments are deployed on six standard datasets with various application scenarios (shape retrieval, point cloud registration, and object recognition) and data modalities (LiDAR, Kinect, and Space Time) as well as perturbations including Gaussian noise, shot noise, data decimation, clutter, occlusion, and limited overlap. The evaluated terms cover the major concerns for a feature representation, e.g., distinctiveness, robustness, compactness, and efficiency. The outcomes present interesting findings that may shed new light on this community and provide complementary perspectives to existing evaluations on the topic of local geometric feature description. A summary of evaluated methods regarding their peculiarities is finally presented to guide real-world applications and new descriptor crafting.