Non-Iterative Rigid 2D/3D Point-Set Registration Using Semidefinite Programming

Scan-Free and Fully Automatic Tracking of Native Knee Anatomy from Dynamic Stereo-Radiography with Statistical Shape and Intensity Models

Kinematic tracking of native anatomy from stereo-radiography provides a quantitative basis for evaluating human movement. Conventional tracking procedures require significant manual effort and call for acquisition and annotation of subject-specific volumetric medical images. The current work introduces a framework for fully automatic tracking of native knee anatomy from dynamic stereo-radiography which forgoes reliance on volumetric scans. The method consists of three computational steps. First, captured radiographs are annotated with segmentation maps and anatomic landmarks using a convolutional neural network. Next, a non-convex polynomial optimization problem formulated from annotated landmarks is solved to acquire preliminary anatomy and pose estimates. Finally, a global optimization routine is performed for concurrent refinement of anatomy and pose. An objective function is maximized which quantifies similarities between masked radiographs and digitally reconstructed radiographs produced from statistical shape and intensity models. The proposed framework was evaluated against manually tracked trials comprising dynamic activities, and additional frames capturing a static knee phantom. Experiments revealed anatomic surface errors routinely below 1.0 mm in both evaluation cohorts. Median absolute errors of individual bone pose estimates were below 1.0∘ or mm for 15 out of 18 degrees of freedom in both evaluation cohorts. Results indicate that accurate pose estimation of native anatomy from stereo-radiography may be performed with significantly reduced manual effort, and without reliance on volumetric scans.

Fully automatic tracking of native glenohumeral kinematics from stereo-radiography

The current work introduces a system for fully automatic tracking of native glenohumeral kinematics in stereo-radiography sequences. The proposed method first applies convolutional neural networks to obtain segmentation and semantic key point predictions in biplanar radiograph frames. Preliminary bone pose estimates are computed by solving a non-convex optimization problem with semidefinite relaxations to register digitized bone landmarks to semantic key points. Initial poses are then refined by registering computed tomography-based digitally reconstructed radiographs to captured scenes, which are masked by segmentation maps to isolate the shoulder joint. A particular neural net architecture which exploits subject-specific geometry is also introduced to improve segmentation predictions and increase robustness of subsequent pose estimates. The method is evaluated by comparing predicted glenohumeral kinematics to manually tracked values from 17 trials capturing 4 dynamic activities. Median orientation differences between predicted and ground truth poses were 1.7∘ and 8.6∘ for the scapula and humerus, respectively. Joint-level kinematics differences were less than 2∘ in 65%, 13%, and 63% of frames for XYZ orientation DoFs based on Euler angle decompositions. Automation of kinematic tracking can increase scalability of tracking workflows in research, clinical, or surgical applications.

Enhancing the depth perception of DSA images with 2D-3D registration

Objective

Today, cerebrovascular disease has become an important health hazard. Therefore, it is necessary to perform a more accurate and less time-consuming registration of preoperative three-dimensional (3D) images and intraoperative two-dimensional (2D) projection images which is very important for conducting cerebrovascular disease interventions. The 2D-3D registration method proposed in this study is designed to solve the problems of long registration time and large registration errors in 3D computed tomography angiography (CTA) images and 2D digital subtraction angiography (DSA) images.

Methods

To make a more comprehensive and active diagnosis, treatment and surgery plan for patients with cerebrovascular diseases, we propose a weighted similarity measure function, the normalized mutual information-gradient difference (NMG), which can evaluate the 2D-3D registration results. Then, using a multi-resolution fusion optimization strategy, the multi-resolution fused regular step gradient descent optimization (MR-RSGD) method is presented to attain the optimal value of the registration results in the process of the optimization algorithm.

Result

In this study, we adopt two datasets of the brain vessels to validate and obtain similarity metric values which are 0.0037 and 0.0003, respectively. Using the registration method proposed in this study, the time taken for the experiment was calculated to be 56.55s and 50.8070s, respectively, for the two sets of data. The results show that the registration methods proposed in this study are both better than the Normalized Mutual (NM) and Normalized Mutual Information (NMI).

Conclusion

The experimental results in this study show that in the 2D-3D registration process, to evaluate the registration results more accurately, we can use the similarity metric function containing the image gray information and spatial information. To improve the efficiency of the registration process, we can choose the algorithm with gradient optimization strategy. Our method has great potential to be applied in practical interventional treatment for intuitive 3D navigation.

Deep Learning Segmentation in 2D X-ray Images and Non-Rigid Registration in Multi-Modality Images of Coronary Arteries

X-ray angiography is commonly used in the diagnosis and treatment of coronary artery disease with the advantage of visualization of the inside of blood vessels in real-time. However, it has several disadvantages that occur in the acquisition process, which causes inconvenience and difficulty. Here, we propose a novel segmentation and nonrigid registration method to provide useful real-time assistive images and information. A convolutional neural network is used for the segmentation of coronary arteries in 2D X-ray angiography acquired from various angles in real-time. To compensate for errors that occur during the 2D X-ray angiography acquisition process, 3D CT angiography is used to analyze the topological structure. A novel energy function-based 3D deformation and optimization is utilized to implement real-time registration. We evaluated the proposed method for 50 series from 38 patients by comparing the ground truth. The proposed segmentation method showed that Precision, Recall, and F1 score were 0.7563, 0.6922, and 0.7176 for all vessels, 0.8542, 0.6003, and 0.7035 for markers, and 0.8897, 0.6389, and 0.7386 for bifurcation points, respectively. In the nonrigid registration method, the average distance of 0.8705, 1.06, and 1. 5706 mm for all vessels, markers, and bifurcation points was achieved. The overall process execution time was 0.179 s.

ORTHOGONAL TRACE-SUM MAXIMIZATION: TIGHTNESS OF THE SEMIDEFINITE RELAXATION AND GUARANTEE OF LOCALLY OPTIMAL SOLUTIONS

This paper studies an optimization problem on the sum of traces of matrix quadratic forms in m semiorthogonal matrices, which can be considered as a generalization of the synchronization of rotations. While the problem is nonconvex, this paper shows that its semidefinite programming relaxation solves the original nonconvex problems exactly with high probability under an additive noise model with small noise in the order of O(m^1/4). In addition, it shows that, with high probability, the sufficient condition for global optimality considered in Won, Zhou, and Lange [SIAM J. Matrix Anal. Appl., 2 (2021), pp. 859-882] is also necessary under a similar small noise condition. These results can be considered as a generalization of existing results on phase synchronization.

A Bayesian Formulation of Coherent Point Drift

Coherent point drift is a well-known algorithm for solving point set registration problems, i.e., finding corresponding points between shapes represented as point sets. Despite its advantages over other state-of-the-art algorithms, theoretical and practical issues remain. Among theoretical issues, (1) it is unknown whether the algorithm always converges, and (2) the meaning of the parameters concerning motion coherence is unclear. Among practical issues, (3) the algorithm is relatively sensitive to target shape rotation, and (4) acceleration of the algorithm is restricted to the use of the Gaussian kernel. To overcome these issues and provide a different and more general perspective to the algorithm, we formulate coherent point drift in a Bayesian setting. The formulation brings the following consequences and advances to the field: convergence of the algorithm is guaranteed by variational Bayesian inference; the definition of motion coherence as a prior distribution provides a basis for interpretation of the parameters; rigid and non-rigid registration can be performed in a single algorithm, enhancing robustness against target rotation. We also propose an acceleration scheme for the algorithm that can be applied to non-Gaussian kernels and that provides greater efficiency than coherent point drift.

Robust Point Set Registration Using Signature Quadratic Form Distance

Point set registration is a problem with a long history in many pattern recognition tasks. This paper presents a robust point set registration algorithm based on optimizing the distance between two probability distributions. A major problem in point to point algorithms is defining the correspondence between two point sets. This paper follows the idea of some probability-based point set registration methods by representing the point sets as Gaussian mixture models (GMMs). By optimizing the distance between the two GMMs, rigid transformations (rotation and translation) between two point sets can be obtained without having to find a correspondence. Previous studies have used L2, Kullback Leibler, etc. distance to measure similarity between two GMMs; however, these methods have problems with robustness to noise and outliers, especially when the covariance matrix is large, or a local minimum exists. Therefore, in this paper, the signature quadratic form distance is derived to measure the distribution similarity. The contribution of this paper lies in adopting the signature quadratic form distance for the point set registration algorithm. The experimental results show the precision and robustness of this algorithm and demonstrate that it outperforms other state-of-the-art point set registration algorithms regarding factors, such as noise, outliers, missing partial structures, and initial misalignment.

Rapid and Accurate Registration Method between Intraoperative 2D XA and Preoperative 3D CTA Images for Guidance of Percutaneous Coronary Intervention

In this paper, we propose a rapid rigid registration method for the fusion visualization of intraoperative 2D X-ray angiogram (XA) and preoperative 3D computed tomography angiography (CTA) images. First, we perform the cardiac cycle alignment of a patient's 2D XA and 3D CTA images obtained from a different apparatus. Subsequently, we perform the initial registration through alignment of the registration space and optimal boundary box. Finally, the two images are registered where the distance between two vascular structures is minimized by using the local distance map, selective distance measure, and optimization of transformation function. To improve the accuracy and robustness of the registration process, the normalized importance value based on the anatomical information of the coronary arteries is utilized. The experimental results showed fast, robust, and accurate registration using 10 cases, each of the left coronary artery (LCA) and right coronary artery (RCA). Our method can be used as a computer-aided technology for percutaneous coronary intervention (PCI). Our method can be applied to the study of other types of vessels.

Intensity-based 2D-3D registration for an ACL reconstruction navigation system

To improve the positioning accuracy of tunnels for anterior cruciate ligament (ACL) reconstruction, we proposed an intensity-based 2D-3D registration method for an ACL reconstruction navigation system. Methods for digitally reconstructed radiograph (DRR) generation, similarity measurement, and optimization are crucial to 2D-3D registration. We evaluated the accuracy, success rate, and processing time of different methods: (a) ray-casting and splating were compared for DRR generation; (b) normalized mutual information (NMI), Mattes mutual information (MMI), and Spearman's rank correlation coefficient (SRC) were assessed for similarity between registrations; and (c) gradient descent (GD) and downhill simplex (DS) were compared for optimization. The combination of splating, SRC, and GD provided the best composite performance and was applied in an augmented reality (AR) ACL reconstruction navigation system. The accuracy of the navigation system could fulfill the clinical needs of ACL reconstruction, with an end pose error of 2.50 mm and an angle error of 2.74°.

2D-3D Point Set Registration Based on Global Rotation Search

Simultaneously determining the relative pose and correspondence between a set of 3D points and its 2D projection is a fundamental problem in computer vision, and the problem becomes more difficult when the point sets are contaminated by noise and outliers. Traditionally, this problem is solved by local optimization methods, which usually start from an initial guess of the pose and alternately optimize the pose and the correspondence. In this paper, we formulate the problem as optimizing the pose of the 3D points in the SE(3) space to make its 2D projection best align with the 2D point set, which is measured by the cardinality of the inlier set on the 2D projection plane. We propose four geometric bounds for the position of the projection of a 3D point on the 2D projection plane and solve the 2D-3D point set registration problem by combining a global optimal rotation search and a grid search of translation. Compared with existing global optimization approaches, the proposed method utilizes a different problem formulation and more efficiently searches the translation space, which improves the registration speed. Experiments with synthetic and real data showed that the proposed approach significantly outperformed state-of-the-art local and global methods.