Robust Point Set Registration Using Gaussian Mixture Models

Research on Student's T-Distribution Point Cloud Registration Algorithm Based on Local Features

LiDAR offers a wide range of uses in autonomous driving, remote sensing, urban planning, and other areas. The laser 3D point cloud acquired by LiDAR typically encounters issues during registration, including laser speckle noise, Gaussian noise, data loss, and data disorder. This work suggests a novel Student's t-distribution point cloud registration algorithm based on the local features of point clouds to address these issues. The approach uses Student's t-distribution mixture model (SMM) to generate the probability distribution of point cloud registration, which can accurately describe the data distribution, in order to tackle the problem of the missing laser 3D point cloud data and data disorder. Owing to the disparity in the point cloud registration task, a full-rank covariance matrix is built based on the local features of the point cloud during the objective function design process. The combined penalty of point-to-point and point-to-plane distance is then added to the objective function adaptively. Simultaneously, by analyzing the imaging characteristics of LiDAR, according to the influence of the laser waveform and detector on the LiDAR imaging, the composite weight coefficient is added to improve the pertinence of the algorithm. Based on the public dataset and the laser 3D point cloud dataset acquired in the laboratory, the experimental findings demonstrate that the proposed algorithm has high practicability and dependability and outperforms the five comparison algorithms in terms of accuracy and robustness.

Accurate Registration of Cross-Modality Geometry via Consistent Clustering

The registration of unitary-modality geometric data has been successfully explored over past decades. However, existing approaches typically struggle to handle cross-modality data due to the intrinsic difference between different models. To address this problem, in this article, we formulate the cross-modality registration problem as a consistent clustering process. First, we study the structure similarity between different modalities based on an adaptive fuzzy shape clustering, from which a coarse alignment is successfully operated. Then, we optimize the result using fuzzy clustering consistently, in which the source and target models are formulated as clustering memberships and centroids, respectively. This optimization casts new insight into point set registration, and substantially improves the robustness against outliers. Additionally, we investigate the effect of fuzzier in fuzzy clustering on the cross-modality registration problem, from which we theoretically prove that the classical Iterative Closest Point (ICP) algorithm is a special case of our newly defined objective function. Comprehensive experiments and analysis are conducted on both synthetic and real-world cross-modality datasets. Qualitative and quantitative results demonstrate that our method outperforms state-of-the-art approaches with higher accuracy and robustness. Our code is publicly available at https://github.com/zikai1/CrossModReg.

Anisotropic Scale-Invariant Ellipse Detection

Detecting ellipses poses a challenging low-level task indispensable to many image analysis applications. Existing ellipse detection methods commonly encounter two fundamental issues. First, inferior detection accuracy could be incurred on a small ellipse than that on a large one; this introduces the scale issue. Second, inferior detection accuracy could be yielded along the minor axis than along the major one of the same ellipse; this leads to the anisotropy issue. To address these issues simultaneously, a novel anisotropic scale-invariant (ASI) ellipse detection methodology is proposed. Our basic idea is to perform ellipse detection in a transformed image space referred to as the ellipse normalization (EN) space, in which the desired ellipse from the original image is 'normalized' to the unit circle. With the establishment of the EN-space, an analytical ellipse fitting scheme and a set of distance measures are developed. Theoretical justifications are then derived to prove that both our ellipse fitting scheme and distance measures are invariant to anisotropic scaling, and thus each ellipse can be detected with the same accuracy regardless of its size and ellipticity. By incorporating these components into two recent state-of-the-art algorithms, two ASI ellipse detectors are finally developed and exploited to verify the effectiveness of our proposed methodology.

Embedding expertise knowledge into inverse treatment planning for low-dose-rate brachytherapy of hepatic malignancies

BACKGROUND

Leveraging the precision of its radiation dose distribution and the minimization of postoperative complications, low-dose-rate (LDR) permanent seed brachytherapy is progressively adopted in addressing hepatic malignancies.

PURPOSE

The present study endeavors to devise a sophisticated treatment planning system (TPS) to optimize LDR brachytherapy for hepatic lesions.

METHODS

Our TPS encompasses four integral modules: multi-organ segmentation, seed distribution initialization, puncture pathway selection, and inverse dose planning. By amalgamating an array of deep learning models, the segmentation module proficiently labels 17 discrete abdominal targets within the images. We introduce a knowledge-based seed distribution initialization methodology that discerns the most analogous tumor shape in the reference treatment plan from the knowledge base. Subsequently, the seed distribution from the reference plan is transmuted to the current case, thus establishing seed distribution initialization. Furthermore, we parameterize the puncture needles and seeds, while concurrently constraining the puncture needle angle through the employment of a virtual puncture panel to augment planning algorithm efficiency. We also presented a user interface that includes a range of interactive features, seamlessly integrated with the treatment planning generation function.

RESULTS

The multi-organ segmentation module, which is trained by 50 cases of in-house CT scans and 694 cases of publicly available CT scans, achieved average Dice of 0.80 and Hausdorff distance of 5.2 mm in testing datasets. The results demonstrate that knowledge-based initialization exhibits a marked enhancement in expediting the convergence rate. Our TPS also demonstrates a dominant advantage in dose-volume-histogram criteria and execution time in comparison to commercial TPS.

CONCLUSION

The study proposes an innovative treatment planning system for low-dose-rate permanent seed brachytherapy for hepatic malignancies. We show that the generated treatment plans meet clinical requirement.

Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation

Reading out neuronal activity from three-dimensional (3D) functional imaging requires segmenting and tracking individual neurons. This is challenging in behaving animals if the brain moves and deforms. The traditional approach is to train a convolutional neural network with ground-truth (GT) annotations of images representing different brain postures. For 3D images, this is very labor intensive. We introduce 'targeted augmentation', a method to automatically synthesize artificial annotations from a few manual annotations. Our method ('Targettrack') learns the internal deformations of the brain to synthesize annotations for new postures by deforming GT annotations. This reduces the need for manual annotation and proofreading. A graphical user interface allows the application of the method end-to-end. We demonstrate Targettrack on recordings where neurons are labeled as key points or 3D volumes. Analyzing freely moving animals exposed to odor pulses, we uncover rich patterns in interneuron dynamics, including switching neuronal entrainment on and off.

Learning Canonical Embeddings for Unsupervised Shape Correspondence With Locally Linear Transformations

We present a new approach to unsupervised shape correspondence learning between pairs of point clouds. We make the first attempt to adapt the classical locally linear embedding algorithm (LLE)-originally designed for nonlinear dimensionality reduction-for shape correspondence. The key idea is to find dense correspondences between shapes by first obtaining high-dimensional neighborhood-preserving embeddings of low-dimensional point clouds and subsequently aligning the source and target embeddings using locally linear transformations. We demonstrate that learning the embedding using a new LLE-inspired point cloud reconstruction objective results in accurate shape correspondences. More specifically, the approach comprises an end-to-end learnable framework of extracting high-dimensional neighborhood-preserving embeddings, estimating locally linear transformations in the embedding space, and reconstructing shapes via divergence measure-based alignment of probability density functions built over reconstructed and target shapes. Our approach enforces embeddings of shapes in correspondence to lie in the same universal/canonical embedding space, which eventually helps regularize the learning process and leads to a simple nearest neighbors approach between shape embeddings for finding reliable correspondences. Comprehensive experiments show that the new method makes noticeable improvements over state-of-the-art approaches on standard shape correspondence benchmark datasets covering both human and nonhuman shapes.

Detecting continuous structural heterogeneity in single-molecule localization microscopy data

Fusion of multiple chemically identical complexes, so-called particles, in localization microscopy, can improve the signal-to-noise ratio and overcome under-labeling. To this end, structural homogeneity of the data must be assumed. Biological heterogeneity, however, could be present in the data originating from distinct conformational variations or (continuous) variations in particle shapes. We present a prior-knowledge-free method for detecting continuous structural variations with localization microscopy. Detecting this heterogeneity leads to more faithful fusions and reconstructions of the localization microscopy data as their heterogeneity is taken into account. In experimental datasets, we show the continuous variation of the height of DNA origami tetrahedrons imaged with 3D PAINT and of the radius of Nuclear Pore Complexes imaged in 2D with STORM. In simulation, we study the impact on the heterogeneity detection pipeline of Degree Of Labeling and of structural variations in the form of two independent modes.

Data-Weighted Multivariate Generalized Gaussian Mixture Model: Application to Point Cloud Robust Registration

In this paper, a weighted multivariate generalized Gaussian mixture model combined with stochastic optimization is proposed for point cloud registration. The mixture model parameters of the target scene and the scene to be registered are updated iteratively by the fixed point method under the framework of the EM algorithm, and the number of components is determined based on the minimum message length criterion (MML). The KL divergence between these two mixture models is utilized as the loss function for stochastic optimization to find the optimal parameters of the transformation model. The self-built point clouds are used to evaluate the performance of the proposed algorithm on rigid registration. Experiments demonstrate that the algorithm dramatically reduces the impact of noise and outliers and effectively extracts the key features of the data-intensive regions.

Simulation of a new respiratory phase sorting method for 4D-imaging using optical surface information towards precision radiotherapy

BACKGROUND

Respiratory signal detection is critical for 4-dimensional (4D) imaging. This study proposes and evaluates a novel phase sorting method using optical surface imaging (OSI), aiming to improve the precision of radiotherapy.

METHOD

Based on 4D Extended Cardiac-Torso (XCAT) digital phantom, OSI in point cloud format was generated from the body segmentation, and image projections were simulated using the geometries of Varian 4D kV cone-beam-CT (CBCT). Respiratory signals were extracted respectively from the segmented diaphragm image (reference method) and OSI respectively, where Gaussian Mixture Model and Principal Component Analysis (PCA) were used for image registration and dimension reduction respectively. Breathing frequencies were compared using Fast-Fourier-Transform. Consistency of 4DCBCT images reconstructed using Maximum Likelihood Expectation Maximization algorithm was also evaluated quantitatively, where high consistency can be suggested by lower Root-Mean-Square-Error (RMSE), Structural-Similarity-Index (SSIM) value closer to 1, and larger Peak-Signal-To-Noise-Ratio (PSNR) respectively.

RESULTS

High consistency of breathing frequencies was observed between the diaphragm-based (0.232 Hz) and OSI-based (0.251 Hz) signals, with a slight discrepancy of 0.019Hz. Using end of expiration (EOE) and end of inspiration (EOI) phases as examples, the mean±1SD values of the 80 transverse, 100 coronal and 120 sagittal planes were 0.967, 0,972, 0.974 (SSIM); 1.657 ± 0.368, 1.464 ± 0.104, 1.479 ± 0.297 (RMSE); and 40.501 ± 1.737, 41.532 ± 1.464, 41.553 ± 1.910 (PSNR) for the EOE; and 0.969, 0.973, 0.973 (SSIM); 1.686 ± 0.278, 1.422 ± 0.089, 1.489 ± 0.238 (RMSE); and 40.535 ± 1.539, 41.605 ± 0.534, 41.401 ± 1.496 (PSNR) for EOI respectively.

CONCLUSIONS

This work proposed and evaluated a novel respiratory phase sorting approach for 4D imaging using optical surface signals, which can potentially be applied to precision radiotherapy. Its potential advantages were non-ionizing, non-invasive, non-contact, and more compatible with various anatomic regions and treatment/imaging systems.

How to make sense of 3D representations for plant phenotyping: a compendium of processing and analysis techniques

Computer vision technology is moving more and more towards a three-dimensional approach, and plant phenotyping is following this trend. However, despite its potential, the complexity of the analysis of 3D representations has been the main bottleneck hindering the wider deployment of 3D plant phenotyping. In this review we provide an overview of typical steps for the processing and analysis of 3D representations of plants, to offer potential users of 3D phenotyping a first gateway into its application, and to stimulate its further development. We focus on plant phenotyping applications where the goal is to measure characteristics of single plants or crop canopies on a small scale in research settings, as opposed to large scale crop monitoring in the field.

Robust Point Cloud Registration Framework Based on Deep Graph Matching

3D point cloud registration is a fundamental problem in computer vision and robotics. Recently, learning-based point cloud registration methods have made great progress. However, these methods are sensitive to outliers, which lead to more incorrect correspondences. In this paper, we propose a novel deep graph matching-based framework for point cloud registration. Specifically, we first transform point clouds into graphs and extract deep features for each point. Then, we develop a module based on deep graph matching to calculate a soft correspondence matrix. By using graph matching, not only the local geometry of each point but also its structure and topology in a larger range are considered in establishing correspondences, so that more correct correspondences are found. We train the network with a loss directly defined on the correspondences, and in the test stage the soft correspondences are transformed into hard one-to-one correspondences so that registration can be performed by a correspondence-based solver. Furthermore, we introduce a transformer-based method to generate edges for graph construction, which further improves the quality of the correspondences. Extensive experiments on object-level and scene-level benchmark datasets show that the proposed method achieves state-of-the-art performance.

Geodesic-Based Bayesian Coherent Point Drift

Coherent point drift is a well-known algorithm for non-rigid registration, i.e., a procedure for deforming a shape to match another shape. Despite its prevalence, the algorithm has a major drawback that remains unsolved: It unnaturally deforms the different parts of a shape, e.g., human legs, when they are neighboring each other. The inappropriate deformations originate from a proximity-based deformation constraint, called motion coherence. This study proposes a non-rigid registration method that addresses the drawback. The key to solving the problem is to redefine the motion coherence using a geodesic, i.e., the shortest route between points on a shape's surface. We also propose the accelerated variant of the registration method. In numerical studies, we demonstrate that the algorithms can circumvent the drawback of coherent point drift. We also show that the accelerated algorithm can be applied to shapes comprising several millions of points.

Fuzzy Cluster-based Group-wise Point Set Registration with Quality Assessment

This article studies group-wise point set registration and makes the following contributions: "FuzzyGReg", which is a new fuzzy cluster-based method to register multiple point sets jointly, and "FuzzyQA", which is the associated quality assessment to check registration accuracy automatically. Given a group of point sets, FuzzyGReg creates a model of fuzzy clusters and equally treats all the point sets as the elements of the fuzzy clusters. Then, the group-wise registration is turned into a fuzzy clustering problem. To resolve this problem, FuzzyGReg applies a fuzzy clustering algorithm to identify the parameters of the fuzzy clusters while jointly transforming all the point sets to achieve an alignment. Next, based on the identified fuzzy clusters, FuzzyQA calculates the spatial properties of the transformed point sets and then checks the alignment accuracy by comparing the similarity degrees of the spatial properties of the point sets. When a local misalignment is detected, a local re-alignment is performed to improve accuracy. The proposed method is cost-efficient and convenient to be implemented. In addition, it provides reliable quality assessments in the absence of ground truth and user intervention. In the experiments, different point sets are used to test the proposed method and make comparisons with state-of-the-art registration techniques. The experimental results demonstrate the effectiveness of our method. The code is available at https://gitsvn-nt.oru.se/qianfang.liao/FuzzyGRegWithQA.

GraphReg: Dynamical Point Cloud Registration With Geometry-Aware Graph Signal Processing

This study presents a high-accuracy, efficient, and physically induced method for 3D point cloud registration, which is the core of many important 3D vision problems. In contrast to existing physics-based methods that merely consider spatial point information and ignore surface geometry, we explore geometry aware rigid-body dynamics to regulate the particle (point) motion, which results in more precise and robust registration. Our proposed method consists of four major modules. First, we leverage the graph signal processing (GSP) framework to define a new signature, i.e., point response intensity for each point, by which we succeed in describing the local surface variation, resampling keypoints, and distinguishing different particles. Then, to address the shortcomings of current physics-based approaches that are sensitive to outliers, we accommodate the defined point response intensity to median absolute deviation (MAD) in robust statistics and adopt the X84 principle for adaptive outlier depression, ensuring a robust and stable registration. Subsequently, we propose a novel geometric invariant under rigid transformations to incorporate higher-order features of point clouds, which is further embedded for force modeling to guide the correspondence between pairwise scans credibly. Finally, we introduce an adaptive simulated annealing (ASA) method to search for the global optimum and substantially accelerate the registration process. We perform comprehensive experiments to evaluate the proposed method on various datasets captured from range scanners to LiDAR. Results demonstrate that our proposed method outperforms representative state-of-the-art approaches in terms of accuracy and is more suitable for registering large-scale point clouds. Furthermore, it is considerably faster and more robust than most competitors. Our implementation is publicly available at https://github.com/zikai1/GraphReg.

Global optimization point-set registration based on translation/rotation decoupling for image-guided surgery applications

PURPOSE

In image-guided surgery systems, image-to-patient spatial registration is to get the spatial transformation between the image space and the actual operating space. Although the image-to-patient spatial registration methods using paired point or surface matching are used in some image-guided neurosurgery systems, the key problem is that the global optimization registration result cannot be achieved. Therefore, this paper proposes a new rotation invariant feature for decoupling rotation and translation space, based on which global optimization point set registration method is proposed.

METHODS

The new rotation invariant features, constructed based on the edges and the angles, are the rotation invariant, which has high feature resolution. Some of them are not only the rotation invariant, but also the translation invariant. To obtain the global optimal solution, branch and bound search strategy is used to search the parameter space of the translation and the computational cost is reduced simultaneously. The registration accuracy of the spatial registration method is analyzed by comparing the difference between the estimated transform and the standard transform to calculate the registration error.

RESULTS

To validate the performance of the spatial registration method proposed, the registration performance was analyzed by comparing the experimental results with the results of the two mainstream registration methods (the iterative closest point [ICP] registration method and the coherent point drift method). In the experiments, the comparison was based on the registration accuracy and the execution time. We show our registration method can obtain higher accuracy in a shorter time in most cases. At the same time, when using ICP to further refine our results, the ICP method can converge in a very short time, which also shows that our method provides a good initial pose for the ICP method and can help the ICP converge to the global optimal solution faster. Our method can achieve an average rotation error of 0.124 degrees and an average translation error of 0.38 mm on 10 clinical data.

CONCLUSIONS

The results reveal that the surface registration method based on translation rotation decoupling can achieve superior performance regarding both the registration accuracy and the time efficiency in the image-to-patient spatial registration.

A Damage Detection Approach for Axially Loaded Beam-like Structures Based on Gaussian Mixture Model

Axially loaded beam-like structures represent a challenging case study for unsupervised learning vibration-based damage detection. Under real environmental and operational conditions, changes in axial load cause changes in the characteristics of the dynamic response that are significantly greater than those due to damage at an early stage. In previous works, the authors proposed the adoption of a multivariate damage feature composed of eigenfrequencies of multiple vibration modes. Successful results were obtained by framing the problem of damage detection as that of unsupervised outlier detection, adopting the well-known Mahalanobis squared distance (MSD) to define an effective damage index. Starting from these promising results, a novel approach based on unsupervised learning data clustering is proposed in this work, which increases the sensitivity to damage and significantly reduces the uncertainty associated with the results, allowing for earlier damage detection. The novel approach, which is based on Gaussian mixture model, is compared with the benchmark one based on the MSD, under the effects of an uncontrolled environment and, most importantly, in the presence of real damage due to corrosion.

CAR-Net: A Deep Learning-Based Deformation Model for 3D/2D Coronary Artery Registration

Percutaneous coronary intervention is widely applied for the treatment of coronary artery disease under the guidance of X-ray coronary angiography (XCA) image. However, the projective nature of XCA causes the loss of 3D structural information, which hinders the intervention. This issue can be addressed by the deformable 3D/2D coronary artery registration technique, which fuses the pre-operative computed tomography angiography volume with the intra-operative XCA image. In this study, we propose a deep learning-based neural network for this task. The registration is conducted in a segment-by-segment manner. For each vessel segment pair, the centerlines that preserve topological information are decomposed into an origin tensor and a spherical coordinate shape tensor as network input through independent branches. Features of different modalities are fused and processed for predicting angular deflections, which is a special type of deformation field implying motion and length preservation constraints for vessel segments. The proposed method achieves an average error of 1.13 mm on the clinical dataset, which shows the potential to be applied in clinical practice.

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback–Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions.

Bayesian iterative closest point for mobile robot localization

Accurate localization of a robot in a known environment is a fundamental capability for successfully performing path planning, manipulation, and grasping tasks. Particle filters, also known as Monte Carlo localization (MCL), are a commonly used method to determine the robot’s pose within its environment. For ground robots, noisy wheel odometry readings are typically used as a motion model to predict the vehicle’s location. Such a motion model requires tuning of various parameters based on terrain and robot type. However, such an ego-motion estimation is not always available for all platforms. Scan matching using the iterative closest point (ICP) algorithm is a popular alternative approach, providing ego-motion estimates for localization. Iterative closest point computes a point estimate of the transformation between two poses given point clouds captured at these locations. Being a point estimate method, ICP does not deal with the uncertainties in the scan alignment process, which may arise due to sensor noise, partial overlap, or the existence of multiple solutions. Another challenge for ICP is the high computational cost required to align two large point clouds, limiting its applicability to less dynamic problems. In this paper, we address these challenges by leveraging recent advances in probabilistic inference. Specifically, we first address the run-time issue and propose SGD-ICP, which employs stochastic gradient descent (SGD) to solve the optimization problem of ICP. Next, we leverage SGD-ICP to obtain a distribution over transformations and propose a Markov Chain Monte Carlo method using stochastic gradient Langevin dynamics (SGLD) updates. Our ICP variant, termed Bayesian-ICP, is a full Bayesian solution to the problem. To demonstrate the benefits of Bayesian-ICP for mobile robotic applications, we propose an adaptive motion model employing Bayesian-ICP to produce proposal distributions for Monte Carlo Localization. Experiments using both Kinect and 3D LiDAR data show that our proposed SGD-ICP method achieves the same solution quality as standard ICP while being significantly more efficient. We then demonstrate empirically that Bayesian-ICP can produce accurate distributions over pose transformations and is fast enough for online applications. Finally, using Bayesian-ICP as a motion model alleviates the need to tune the motion model parameters from odometry, resulting in better-calibrated localization uncertainty.

Fast and Robust Iterative Closest Point

The iterative closest point (ICP) algorithm and its variants are a fundamental technique for rigid registration between two point sets, with wide applications in different areas from robotics to 3D reconstruction. The main drawbacks for ICP are its slow convergence, as well as its sensitivity to outliers, missing data, and partial overlaps. Recent work such as Sparse ICP achieves robustness via sparsity optimization at the cost of computational speed. In this paper, we propose a new method for robust registration with fast convergence. First, we show that the classical point-to-point ICP can be treated as a majorization-minimization (MM) algorithm, and propose an Anderson acceleration approach to speed up its convergence. In addition, we introduce a robust error metric based on the Welsch's function, which is minimized efficiently using the MM algorithm with Anderson acceleration. On challenging datasets with noises and partial overlaps, we achieve similar or better accuracy than Sparse ICP while being at least an order of magnitude faster. Finally, we extend the robust formulation to point-to-plane ICP, and solve the resulting problem using a similar Anderson-accelerated MM strategy. Our robust ICP methods improve the registration accuracy on benchmark datasets while being competitive in computational time.

Fast and Accurate Registration of Terrestrial Point Clouds Using a Planar Approximation of Roof Features

3D reconstructed models are becoming more diffused daily, especially in the Cultural Heritage field. These geometric models are typically obtained from elaborating a 3D point cloud. A significant limit in using these methods is the realignment of different point clouds acquired from different acquisitions, particularly for those whose dimensions are millions of points. Although several methodologies have tried to propose a solution for this necessity, none of these seems to solve definitively the problems related to the realignment of large point clouds. This paper presents a new and innovative procedure for the fine registration of large point clouds. The method performs an alignment by using planar approximations of roof features, taking the roof’s extension into account. It looks particularly suitable for the alignment of large point clouds acquired in urban and archaeological environments. The proposed methodology is compared in terms of accuracy and time with a standard photogrammetric reconstruction based on Ground Control Points (GCPs) and other ones, aligned by the Iterative Closest Point method (ICP) and markers. The results evidence the excellent performance of the methodology, which could represent an alternative for aligning extensive photogrammetric reconstructions without the use of GCPs.

A Global Fundamental Matrix Estimation Method of Planar Motion Based on Inlier Updating

A fundamental matrix estimation based on matching points is a critical problem in epipolar geometry. In this paper, a global fundamental matrix estimation method based on inlier updating is proposed. Firstly, the coplanar constraint was incorporated into the solution of the fundamental matrix to reduce the number of parameters to be solved. Subsequently, an inlier updating matrix was introduced according to the threshold of the epipolar geometry distance to eliminate the potential outliers and obtain a reliable initial value of the fundamental matrix. On this basis, we employed a four-point iterative method to estimate the fundamental matrix and make it satisfy the rank constraint at the same time. Finally, the epipolar geometry in binocular vision was extended to triple-view, and the fundamental matrix obtained in the previous step was globally optimized by minimizing the coordinate deviation between the intersection point and feature point in each group of images. The experiments show that the proposed fundamental matrix estimation method is robust to noise and outliers. In the attitude measurement, the maximum static error was 0.104° and dynamic measurement error was superior to 0.273°, which improved the reconstruction accuracy of feature points. Indoor images were further used to test the method, and the mean rotation angle error was 0.362°. The results demonstrate that the estimation method proposed in this paper has a good practical application prospect in multi-view 3D reconstruction and visual localization.

Online Hard Patch Mining using Shape Models and Bandit Algorithm for Multi-organ Segmentation

Hard sample selection can effectively improve model convergence by extracting the most representative samples from a training set. However, due to the large capacity of medical images, existing sampling strategies suffer from insufficient exploitation for hard samples or high time cost for sample selection when adopted by 3D patch-based models in the field of multi-organ segmentation. In this paper, we present a novel and effective online hard patch mining (OHPM) algorithm. In our method, an average shape model that can be mapped with all training images is constructed to guide the exploration of hard patches and aggregate feedback from predicted patches. The process of hard mining is formalized as a multi-armed bandit problem and solved with bandit algorithms. With the shape model, OHPM requires negligible time consumption and can intuitively locate difficult anatomical areas during training. The employment of bandit algorithms ensures online and sufficient hard mining. We integrate OHPM with advanced segmentation networks and evaluate them on two datasets containing different anatomical structures. Comparative experiments with other sampling strategies demonstrate the superiority of OHPM in boosting segmentation performance and improving model convergence. The results in each dataset with each network suggest that OHPM significantly outperforms other sampling strategies by nearly 2% average Dice score.

Real-time multimodal image registration with partial intraoperative point-set data

We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial.

An Unsupervised Non-rigid Registration Network for Fast Medical Shape Alignment

Medical shapes alignment can provide doctors with abundant structure information of the organs. As for a pair of the given related medical shapes, the traditional registration methods often depend on geometric transformations required for iterative search to align two shapes. To achieve the accurate and fast alignment of 3D medical shapes, we propose an unsupervised and nonrigid registration network. Different from the existing iterative registration methods, our method estimates the point drift for shape alignment directly by learning the displacement field function, which can omit additional iterative optimization process. In addition, the nonrigid registration network can also adapt to the geometric shape transformations of different complexity. The experiments on two types of 3D medical shapes (liver and heart) at different-level deformations verify the impressive performance of our unsupervised and nonrigid registration network.Clinical Relevance-This paper achieves the real-time medical shape alignment with high accuracy, which can help doctors to understand the pathological conditions of organs better.

Topology Constrained Shape Correspondence

To better address the deformation and structural variation challenges inherently present in 3D shapes, researchers have shifted their focus from designing handcrafted point descriptors to learning point descriptors and their correspondences in a data-driven manner. Recent studies have developed deep neural networks for robust point descriptor and shape correspondence learning in consideration of local structural information. In this article, we developed a novel shape correspondence learning network, called TC-NET, which further enhances performance by encouraging the topological consistency between the embedding feature space and the input shape space. Specifically, in this article, we first calculate the topology-associated edge weights to represent the topological structure of each point. Then, in order to preserve this topological structure in high-dimensional feature space, a structural regularization term is defined to minimize the topology-consistent feature reconstruction loss (Topo-Loss) during the correspondence learning process. Our proposed method achieved state-of-the-art performance on three shape correspondence benchmark datasets. In addition, the proposed topology preservation concept can be easily generalized to other learning-based shape analysis tasks to regularize the topological structure of high-dimensional feature spaces.

Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds

Shape reconstruction from sparse point clouds/images is a challenging and relevant task required for a variety of applications in computer vision and medical image analysis (e.g. surgical navigation, cardiac motion analysis, augmented/virtual reality systems). A subset of such methods, viz. 3D shape reconstruction from 2D contours, is especially relevant for computer-aided diagnosis and intervention applications involving meshes derived from multiple 2D image slices, views or projections. We propose a deep learning architecture, coined Mesh Reconstruction Network (MR-Net), which tackles this problem. MR-Net enables accurate 3D mesh reconstruction in real-time despite missing data and with sparse annotations. Using 3D cardiac shape reconstruction from 2D contours defined on short-axis cardiac magnetic resonance image slices as an exemplar, we demonstrate that our approach consistently outperforms state-of-the-art techniques for shape reconstruction from unstructured point clouds. Our approach can reconstruct 3D cardiac meshes to within 2.5-mm point-to-point error, concerning the ground-truth data (the original image spatial resolution is ∼1.8×1.8×10mm³). We further evaluate the robustness of the proposed approach to incomplete data, and contours estimated using an automatic segmentation algorithm. MR-Net is generic and could reconstruct shapes of other organs, making it compelling as a tool for various applications in medical image analysis.

Point Set Registration for 3D Range Scans Using Fuzzy Cluster-Based Metric and Efficient Global Optimization

This study presents a new point set registration method to align 3D range scans. In our method, fuzzy clusters are utilized to represent a scan, and the registration of two given scans is realized by minimizing a fuzzy weighted sum of the distances between their fuzzy cluster centers. This fuzzy cluster-based metric has a broad basin of convergence and is robust to noise. Moreover, this metric provides analytic gradients, allowing standard gradient-based algorithms to be applied for optimization. Based on this metric, the outlier issues are addressed. In addition, for the first time in rigid point set registration, a registration quality assessment in the absence of ground truth is provided. Furthermore, given specified rotation and translation spaces, we derive the upper and lower bounds of the fuzzy cluster-based metric and develop a branch-and-bound (BnB)-based optimization scheme, which can globally minimize the metric regardless of the initialization. This optimization scheme is performed in an efficient coarse-to-fine fashion: First, fuzzy clustering is applied to describe each of the two given scans by a small number of fuzzy clusters. Then, a global search, which integrates BnB and gradient-based algorithms, is implemented to achieve a coarse alignment for the two scans. During the global search, the registration quality assessment offers a beneficial stop criterion to detect whether a good result is obtained. Afterwards, a relatively large number of points of the two scans are directly taken as the fuzzy cluster centers, and then, the coarse solution is refined to be an exact alignment using the gradient-based local convergence. Compared to existing counterparts, this optimization scheme makes a large improvement in terms of robustness and efficiency by virtue of the fuzzy cluster-based metric and the registration quality assessment. In the experiments, the registration results of several 3D range scan pairs demonstrate the accuracy and effectiveness of the proposed method, as well as its superiority to state-of-the-art registration approaches.

Automatic Point Cloud Registration for Large Outdoor Scenes Using a Priori Semantic Information

As an important and fundamental step in 3D reconstruction, point cloud registration aims to find rigid transformations that register two point sets. The major challenge in point cloud registration techniques is finding correct correspondences in the scenes that may contain many repetitive structures and noise. This paper is primarily concerned with improving registration using a priori semantic information in the search for correspondences. In particular, we present a new point cloud registration pipeline for large, outdoor scenes that takes advantage of semantic segmentation. Our method consisted of extracting semantic segments from point clouds using an efficient deep neural network, then detecting the key points of the point cloud and using a feature descriptor to get the initial correspondence set, and, finally, applying a Random Sample Consensus (RANSAC) strategy to estimate the transformations that align segments with the same labels. Instead of using all points to estimate a global alignment, our method aligned two point clouds using transformations calculated by each segment with the highest inlier ratio. We evaluated our method on the publicly available Whu-TLS registration data set. These experiments demonstrate how a priori semantic information improves registration in terms of precision and speed.

Acceleration of Non-Rigid Point Set Registration With Downsampling and Gaussian Process Regression

Non-rigid point set registration is the process of transforming a shape represented as a point set into a shape matching another shape. In this paper, we propose an acceleration method for solving non-rigid point set registration problems. We accelerate non-rigid registration by dividing it into three steps: i) downsampling of point sets; ii) non-rigid registration of downsampled point sets; and iii) interpolation of shape deformation vectors corresponding to points removed during downsampling. To register downsampled point sets, we use a registration algorithm based on a prior distribution, called motion coherence prior. Using the same prior, we derive an interpolation method interpreted as Gaussian process regression. Through numerical experiments, we demonstrate that our algorithm registers point sets containing over ten million points. We also show that our algorithm reduces computing time more radically than a state-of-the-art acceleration algorithm.

Fast deep neural correspondence for tracking and identifying neurons in C. elegans using semi-synthetic training

We present an automated method to track and identify neurons in C. elegans, called 'fast Deep Neural Correspondence' or fDNC, based on the transformer network architecture. The model is trained once on empirically derived semi-synthetic data and then predicts neural correspondence across held-out real animals. The same pre-trained model both tracks neurons across time and identifies corresponding neurons across individuals. Performance is evaluated against hand-annotated datasets, including NeuroPAL (Yemini et al., 2021). Using only position information, the method achieves 79.1% accuracy at tracking neurons within an individual and 64.1% accuracy at identifying neurons across individuals. Accuracy at identifying neurons across individuals is even higher (78.2%) when the model is applied to a dataset published by another group (Chaudhary et al., 2021). Accuracy reaches 74.7% on our dataset when using color information from NeuroPAL. Unlike previous methods, fDNC does not require straightening or transforming the animal into a canonical coordinate system. The method is fast and predicts correspondence in 10 ms making it suitable for future real-time applications.

A Bayesian Formulation of Coherent Point Drift

Coherent point drift is a well-known algorithm for solving point set registration problems, i.e., finding corresponding points between shapes represented as point sets. Despite its advantages over other state-of-the-art algorithms, theoretical and practical issues remain. Among theoretical issues, (1) it is unknown whether the algorithm always converges, and (2) the meaning of the parameters concerning motion coherence is unclear. Among practical issues, (3) the algorithm is relatively sensitive to target shape rotation, and (4) acceleration of the algorithm is restricted to the use of the Gaussian kernel. To overcome these issues and provide a different and more general perspective to the algorithm, we formulate coherent point drift in a Bayesian setting. The formulation brings the following consequences and advances to the field: convergence of the algorithm is guaranteed by variational Bayesian inference; the definition of motion coherence as a prior distribution provides a basis for interpretation of the parameters; rigid and non-rigid registration can be performed in a single algorithm, enhancing robustness against target rotation. We also propose an acceleration scheme for the algorithm that can be applied to non-Gaussian kernels and that provides greater efficiency than coherent point drift.

A hybrid feature-based patient-to-image registration method for robot-assisted long bone osteotomy

PURPOSE

The purpose of this study is to provide a simple, feasible and effective patient-to-image registration method for robot-assisted long bone osteotomy, which has rarely been systematically reported. The practical requirement is to meet the accuracy of 1 mm or even higher without bone-implanted markers.

METHODS

A hybrid feature-based registration method termed CR-RAMSICP is proposed. Point-based coarse registration (CR) is accomplished relying on the optical retro-reflective markers attached to the tracked rigid body fixed out of the bone. In surface-based fine registration, an improved iterative closest point (ICP) algorithm based on the range-adaptive matching strategy (termed RAMSICP) is presented to cope with the robust precise matching between the asymmetric patient and image point clouds, which avoids converging to a local minimum.

RESULTS

A series of registration experiments based on the isolated porcine iliums are carried out. The results illustrate that CR-RAMSICP not only significantly outperforms CR and CR-ICP in the accuracy and reproducibility, but also exhibits better robustness to the CR errors and less sensitiveness to the distribution and number of fiducial points located in the patient point cloud than CR-ICP.

CONCLUSION

The proposed registration method CR-RAMSICP can stably satisfy the desired registration accuracy without the use of bone-implanted markers like fiducial screws. Besides, the RAMSICP algorithm used in fine registration is convenient for programming because any complex metrics or models are not involved.

Detecting structural heterogeneity in single-molecule localization microscopy data

Particle fusion for single molecule localization microscopy improves signal-to-noise ratio and overcomes underlabeling, but ignores structural heterogeneity or conformational variability. We present a-priori knowledge-free unsupervised classification of structurally different particles employing the Bhattacharya cost function as dissimilarity metric. We achieve 96% classification accuracy on mixtures of up to four different DNA-origami structures, detect rare classes of origami occuring at 2% rate, and capture variation in ellipticity of nuclear pore complexes.

AD Course Map charts Alzheimer's disease progression

Alzheimer's disease (AD) is characterized by the progressive alterations seen in brain images which give rise to the onset of various sets of symptoms. The variability in the dynamics of changes in both brain images and cognitive impairments remains poorly understood. This paper introduces AD Course Map a spatiotemporal atlas of Alzheimer's disease progression. It summarizes the variability in the progression of a series of neuropsychological assessments, the propagation of hypometabolism and cortical thinning across brain regions and the deformation of the shape of the hippocampus. The analysis of these variations highlights strong genetic determinants for the progression, like possible compensatory mechanisms at play during disease progression. AD Course Map also predicts the patient's cognitive decline with a better accuracy than the 56 methods benchmarked in the open challenge TADPOLE. Finally, AD Course Map is used to simulate cohorts of virtual patients developing Alzheimer's disease. AD Course Map offers therefore new tools for exploring the progression of AD and personalizing patients care.