Robustness and accuracy of feature-based single image 2-D-3-D registration without correspondences for image-guided intervention

CAR-Net: A Deep Learning-Based Deformation Model for 3D/2D Coronary Artery Registration

Percutaneous coronary intervention is widely applied for the treatment of coronary artery disease under the guidance of X-ray coronary angiography (XCA) image. However, the projective nature of XCA causes the loss of 3D structural information, which hinders the intervention. This issue can be addressed by the deformable 3D/2D coronary artery registration technique, which fuses the pre-operative computed tomography angiography volume with the intra-operative XCA image. In this study, we propose a deep learning-based neural network for this task. The registration is conducted in a segment-by-segment manner. For each vessel segment pair, the centerlines that preserve topological information are decomposed into an origin tensor and a spherical coordinate shape tensor as network input through independent branches. Features of different modalities are fused and processed for predicting angular deflections, which is a special type of deformation field implying motion and length preservation constraints for vessel segments. The proposed method achieves an average error of 1.13 mm on the clinical dataset, which shows the potential to be applied in clinical practice.

Topological recovery for non-rigid 2D/3D registration of coronary artery models

BACKGROUND AND OBJECTIVE

Intra-operative X-ray angiography, the current standard method for visualizing and diagnosing cardiovascular disease, is limited in its ability to provide essential 3D information. These limitations are disadvantages in treating patients. For example, it is a cause of lowering the success rate of interventional procedures. Here, we propose a novel 2D-3D non-rigid registration method to understand vascular geometry during percutaneous coronary intervention.

METHODS

The proposed method uses the local bijection pair distance as a cost function to minimize the effect of inconsistencies from center-line extraction. Moreover, novel cage-based 3D deformation and multi-threaded particle swarm optimization are utilized to implement real-time registration. We evaluated the proposed method for 154 examinations from 10 anonymous patients by coverage percentage, comparing the average distance of the 2D extracted center-line with that of the registered 3D center-line.

RESULTS

The proposed 2D-3D non-rigid registration method achieved an average distance of 1.98 mm with a 0.54 s computation time. Additionally, in aiming to reduce the uncertainty of XA images, we used the proposed method to retrospectively visualize the connections between 2D vascular segments and the distal part of occlusions.

CONCLUSIONS

Ultimately, the proposed 2D/3D non-rigid registration method can successfully register the 3D center-line of coronary arteries with corresponding 2D XA images, and is computationally sufficient for online usage. Therefore, this method can improve the success rate of such procedures as a percutaneous coronary intervention and provide the information necessary to diagnose cardiovascular diseases better.

Iterative closest graph matching for non-rigid 3D/2D coronary arteries registration

Background and objective Fusion of the preoperative computed tomography angiography and intraoperative X-ray angiography images can considerably enhance the visual perception of physicians during percutaneous coronary interventions. This technique can provide 3D information of the arteries and reduce the uncertainty of 2D guidance images. For this purpose, 3D/2D vascular registration with high accuracy and robustness is crucial for performing accurate surgery. Methods In this study, we propose an iterative closest graph matching (ICGM) method that utilizes an alternative iteration framework including correspondence and transformation phases. A coarse-to-fine matching approach based on redundant graph matching is proposed for the correspondence phase. The transformation phase involves rigid and non-rigid transformations, in which rigid transformation is calculated using a closed-form solution, and non-rigid transformation is achieved using a statistical shape model established from a synthetic deformation dataset. Results The proposed method is evaluated and compared with nine state-of-the-art methods on simulated data and clinical datasets. Experiments demonstrate that our method is insensitive to the pose of data and robust to noise and deformation. Moreover, it outperforms other methods in terms of registering real data. Conclusions Given its high capture range, the proposed method can register 3D vessels without prior initialization in clinical practice.

Fast and automatic periacetabular osteotomy fragment pose estimation using intraoperatively implanted fiducials and single-view fluoroscopy

Accurate and consistent mental interpretation of fluoroscopy to determine the position and orientation of acetabular bone fragments in 3D space is difficult. We propose a computer assisted approach that uses a single fluoroscopic view and quickly reports the pose of an acetabular fragment without any user input or initialization. Intraoperatively, but prior to any osteotomies, two constellations of metallic ball-bearings (BBs) are injected into the wing of a patient's ilium and lateral superior pubic ramus. One constellation is located on the expected acetabular fragment, and the other is located on the remaining, larger, pelvis fragment. The 3D locations of each BB are reconstructed using three fluoroscopic views and 2D/3D registrations to a preoperative CT scan of the pelvis. The relative pose of the fragment is established by estimating the movement of the two BB constellations using a single fluoroscopic view taken after osteotomy and fragment relocation. BB detection and inter-view correspondences are automatically computed throughout the processing pipeline. The proposed method was evaluated on a multitude of fluoroscopic images collected from six cadaveric surgeries performed bilaterally on three specimens. Mean fragment rotation error was 2.4 ± 1.0 degrees, mean translation error was 2.1 ± 0.6 mm, and mean 3D lateral center edge angle error was 1.0 ± 0.5 degrees. The average runtime of the single-view pose estimation was 0.7 ± 0.2 s. The proposed method demonstrates accuracy similar to other state of the art systems which require optical tracking systems or multiple-view 2D/3D registrations with manual input. The errors reported on fragment poses and lateral center edge angles are within the margins required for accurate intraoperative evaluation of femoral head coverage.

Fully automatic estimation of pelvic sagittal inclination from anterior-posterior radiography image using deep learning framework

BACKGROUND AND OBJECTIVE

Malposition of the acetabular component causes dislocation and prosthetic impingement after Total Hip Arthroplasty (THA), which significantly affects the postoperative quality of life and implant longevity. The position of the acetabular component is determined by the Pelvic Sagittal Inclination (PSI), which not only varies among different people but also changes in different positions. It is important to recognize individual dynamic changes of the PSI for patient-specific planning of the THA. Previously PSI was estimated by registering the CT and radiography images. In this study, we introduce a new method for accurate estimation of functional PSI without requiring CT image in order to lower radiation exposure of the patient which opens up the possibility of increasing its application in a larger number of hospitals where CT is not acquired as a routine protocol.

METHODS

The proposed method consists of two main steps: First, the Mask R-CNN framework was employed to segment the pelvic shape from the background in the radiography images. Then, following the segmentation network, another convolutional network regressed the PSI angle. We employed a transfer learning paradigm where the network weights were initialized by non-medical images followed by fine-tuning using radiography images. Furthermore, in the training process, augmented data was generated to improve the performance of both networks. We analyzed the role of segmentation network in our system and investigated the Mask R-CNN performance in comparison with the U-Net, which is commonly used for the medical image segmentation.

RESULTS

In this study, the Mask R-CNN utilizing multi-task learning, transfer learning, and data augmentation techniques achieve 0.960 ± 0.008 DICE coefficient, which significantly outperforms the U-Net. The cascaded system is capable of estimating the PSI with 4.04° ± 3.39° error for the radiography images.

CONCLUSIONS

The proposed framework suggests a fully automatic and robust estimation of the PSI using only an anterior-posterior radiography image.

Biomechanical Guidance System for Periacetabular Osteotomy

This chapter presents a biomechanical guidance navigation system for performing periacetabular osteotomy (PAO) to treat developmental dysplasia of the hip. The main motivation of the biomechanical guidance system (BGS) is to plan and track the osteotomized fragment in real time during PAO while simplifying this challenging procedure. The BGS computes the three-dimensional position of the osteotomized fragment in terms of conventional anatomical angles and simulates biomechanical states of the joint. This chapter describes the BGS structure and its application using two different navigation approaches including optical tracking of the fragment and x-ray-based navigation. Both cadaver studies and preliminary clinical studies showed that the biomechanical planning is consistent with traditional PAO planning techniques and that the additional information provided by accurate 3D positioning of the fragment does not adversely impact the surgery.

Non-Iterative Rigid 2D/3D Point-Set Registration Using Semidefinite Programming

We describe a convex programming framework for pose estimation in 2D/3D point-set registration with unknown point correspondences. We give two mixed-integer nonlinear program (MINLP) formulations of the 2D/3D registration problem when there are multiple 2D images, and propose convex relaxations for both the MINLPs to semidefinite programs that can be solved efficiently by interior point methods. Our approach to the 2D/3D registration problem is non-iterative in nature as we jointly solve for pose and correspondence. Furthermore, these convex programs can readily incorporate feature descriptors of points to enhance registration results. We prove that the convex programs exactly recover the solution to the MINLPs under certain noiseless condition. We apply these formulations to the registration of 3D models of coronary vessels to their 2D projections obtained from multiple intra-operative fluoroscopic images. For this application, we experimentally corroborate the exact recovery property in the absence of noise and further demonstrate robustness of the convex programs in the presence of noise.

Fast Rotation-Free Feature-Based Image Registration Using Improved N-SIFT and GMM-Based Parallel Optimization

Image registration is a key problem in a variety of applications, such as computer vision, medical image processing, pattern recognition, etc., while the application of registration is limited by time consumption and the accuracy in the case of large pose differences. Aimed at these two kinds of problems, we propose a fast rotation-free feature-based rigid registration method based on our proposed accelerated-NSIFT and GMM registration-based parallel optimization (PO-GMMREG). Our method is accelerated by using the GPU/CUDA programming and preserving only the location information without constructing the descriptor of each interest point, while its robustness to missing correspondences and outliers is improved by converting the interest point matching to Gaussian mixture model alignment. The accuracy in the case of large pose differences is settled by our proposed PO-GMMREG algorithm by constructing a set of initial transformations. Experimental results demonstrate that our proposed algorithm can fast rigidly register 3-D medical images and is reliable for aligning 3-D scans even when they exhibit a poor initialization.

Generalized iterative most likely oriented-point (G-IMLOP) registration

PURPOSE

The need to align multiple representations of anatomy is a problem frequently encountered in clinical applications. A new algorithm for feature-based registration is presented that solves this problem by aligning both position and orientation information of the shapes being registered.

METHODS

The iterative most likely oriented-point (IMLOP) algorithm and its generalization (G-IMLOP) to the anisotropic noise case are described. These algorithms may be understood as probabilistic variants of the popular iterative closest point (ICP) algorithm. A probabilistic model provides the framework, wherein both position information and orientation information are simultaneously optimized. Like ICP, the proposed algorithms iterate between correspondence and registration subphases. Efficient and optimal solutions are presented for implementing each subphase of the proposed methods.

RESULTS

Experiments based on human femur data demonstrate that the IMLOP and G-IMLOP algorithms provide a strong accuracy advantage over ICP, with G-IMLOP providing additional accuracy improvement over IMLOP for registering data characterized by anisotropic noise. Furthermore, the proposed algorithms have increased ability to robustly identify an accurate versus inaccurate registration result.

CONCLUSION

The IMLOP and G-IMLOP algorithms provide a cohesive framework for incorporating orientation data into the registration problem, thereby enabling improvement in accuracy as well as increased confidence in the quality of registration outcomes. For shape data having anisotropic uncertainty in position and/or orientation, the anisotropic noise model of G-IMLOP enables further gains in registration accuracy to be achieved.

Minimally invasive registration for computer-assisted orthopedic surgery: combining tracked ultrasound and bone surface points via the P-IMLOP algorithm

PURPOSE

We present a registration method for computer-assisted total hip replacement (THR) surgery, which we demonstrate to improve the state of the art by both reducing the invasiveness of current methods and increasing registration accuracy. A critical element of computer-guided procedures is the determination of the spatial correspondence between the patient and a computational model of patient anatomy. The current method for establishing this correspondence in robot-assisted THR is to register points intraoperatively sampled by a tracked pointer from the exposed proximal femur and, via auxiliary incisions, from the distal femur.

METHODS

In this paper, we demonstrate a noninvasive technique for sampling points on the distal femur using tracked B-mode ultrasound imaging and present a new algorithm for registering these data called Projected Iterative Most-Likely Oriented Point (P-IMLOP). Points and normal orientations of the distal bone surface are segmented from ultrasound images and registered to the patient model along with points sampled from the exposed proximal femur via a tracked pointer.

RESULTS

The proposed approach is evaluated using a bone- and tissue-mimicking leg phantom constructed to enable accurate assessment of experimental registration accuracy with respect to a CT-image-based model of the phantom. These experiments demonstrate that localization of the femur shaft is greatly improved by tracked ultrasound. The experiments further demonstrate that, for ultrasound-based data, the P-IMLOP algorithm significantly improves registration accuracy compared to the standard ICP algorithm.

CONCLUSION

Registration via tracked ultrasound and the P-IMLOP algorithm has high potential to reduce the invasiveness and improve the registration accuracy of computer-assisted orthopedic procedures.

Iterative most likely oriented point registration

A new algorithm for model based registration is presented that optimizes both position and surface normal information of the shapes being registered. This algorithm extends the popular Iterative Closest Point (ICP) algorithm by incorporating the surface orientation at each point into both the correspondence and registration phases of the algorithm. For the correspondence phase an efficient search strategy is derived which computes the most probable correspondences considering both position and orientation differences in the match. For the registration phase an efficient, closed-form solution provides the maximum likelihood rigid body alignment between the oriented point matches. Experiments by simulation using human femur data demonstrate that the proposed Iterative Most Likely Oriented Point (IMLOP) algorithm has a strong accuracy advantage over ICP and has increased ability to robustly identify a successful registration result.