Evaluation of similarity measures for reconstruction-based registration in image-guided radiotherapy and surgery

Review of 2-D/3-D Reconstruction Using Statistical Shape and Intensity Models and X-Ray Image Synthesis: Toward a Unified Framework

Patient-specific three-dimensional (3-D) bone models are useful for a number of clinical applications such as surgery planning, postoperative evaluation, as well as implant and prosthesis design. Two-dimensional-to-3-D (2-D/3-D) reconstruction, also known as model-to-modality or atlas-based 2-D/3-D registration, provides a means of obtaining a 3-D model of a patient's bones from their 2-D radiographs when 3-D imaging modalities are not available. The preferred approach for estimating both shape and density information (that would be present in a patient's computed tomography data) for 2-D/3-D reconstruction makes use of digitally reconstructed radiographs and deformable models in an iterative, non-rigid, intensity-based approach. Based on a large number of state-of-the-art 2-D/3-D bone reconstruction methods, a unified mathematical formulation of the problem is proposed in a common conceptual framework, using unambiguous terminology. In addition, shortcomings, recent adaptations, and persisting challenges are discussed along with insights for future research.

Comparison of automatic image registration uncertainty for three IGRT systems using a male pelvis phantom

A series of phantom images using the CIRS Virtual Human Male Pelvis was acquired across available dose ranges for three image-guided radiotherapy (IGRT) imaging systems: Elekta XVI CBCT, Varian TrueBeam CBCT, and TomoTherapy MV CT. Each image was registered to a fan-beam CT within the XVI software 100 times with random initial offsets. The residual registration error was analyzed to assess the role of imaging hardware and reconstruction in the uncertainty of the IGRT process. Residual translation errors were similar for all systems and < 0.5 mm. Over the clinical dose range for prostate IGRT images (10-25 mGy), all imaging systems provided acceptable matches in > 90% of registrations when incorporating residual rotational error using a dual quaternion derived distance metric. Outside normal dose settings, large uncertainties were observed at very low and very high dose levels. No trend between initial offset and residual registration error was observed. Patient images may incur higher uncertainties than this phantom study; however, these results encourage automatic matching for standard dose settings with review by treatment staff.

Registration of 2D C-Arm and 3D CT Images for a C-Arm Image-Assisted Navigation System for Spinal Surgery

C-Arm image-assisted surgical navigation system has been broadly applied to spinal surgery. However, accurate path planning on the C-Arm AP-view image is difficult. This research studies 2D-3D image registration methods to obtain the optimum transformation matrix between C-Arm and CT image frames. Through the transformation matrix, the surgical path planned on preoperative CT images can be transformed and displayed on the C-Arm images for surgical guidance. The positions of surgical instruments will also be displayed on both CT and C-Arm in the real time. Five similarity measure methods of 2D-3D image registration including Normalized Cross-Correlation, Gradient Correlation, Pattern Intensity, Gradient Difference Correlation, and Mutual Information combined with three optimization methods including Powell's method, Downhill simplex algorithm, and genetic algorithm are applied to evaluate their performance in converge range, efficiency, and accuracy. Experimental results show that the combination of Normalized Cross-Correlation measure method with Downhill simplex algorithm obtains maximum correlation and similarity in C-Arm and Digital Reconstructed Radiograph (DRR) images. Spine saw bones are used in the experiment to evaluate 2D-3D image registration accuracy. The average error in displacement is 0.22 mm. The success rate is approximately 90% and average registration time takes 16 seconds.

A digitally reconstructed radiograph algorithm calculated from first principles

PURPOSE

To develop an algorithm for computing realistic digitally reconstructed radiographs (DRRs) that match real cone-beam CT (CBCT) projections with no artificial adjustments.

METHODS

The authors used measured attenuation data from cone-beam CT projection radiographs of different materials to obtain a function to convert CT number to linear attenuation coefficient (LAC). The effects of scatter, beam hardening, and veiling glare were first removed from the attenuation data. Using this conversion function the authors calculated the line integral of LAC through a CT along rays connecting the radiation source and detector pixels with a ray-tracing algorithm, producing raw DRRs. The effects of scatter, beam hardening, and veiling glare were then included in the DRRs through postprocessing.

RESULTS

The authors compared actual CBCT projections to DRRs produced with all corrections (scatter, beam hardening, and veiling glare) and to uncorrected DRRs. Algorithm accuracy was assessed through visual comparison of projections and DRRs, pixel intensity comparisons, intensity histogram comparisons, and correlation plots of DRR-to-projection pixel intensities. In general, the fully corrected algorithm provided a small but nontrivial improvement in accuracy over the uncorrected algorithm. The authors also investigated both measurement- and computation-based methods for determining the beam hardening correction, and found the computation-based method to be superior, as it accounted for nonuniform bowtie filter thickness. The authors benchmarked the algorithm for speed and found that it produced DRRs in about 0.35 s for full detector and CT resolution at a ray step-size of 0.5 mm.

CONCLUSIONS

The authors have demonstrated a DRR algorithm calculated from first principles that accounts for scatter, beam hardening, and veiling glare in order to produce accurate DRRs. The algorithm is computationally efficient, making it a good candidate for iterative CT reconstruction techniques that require a data fidelity term based on the matching of DRRs and projections.

Validation for 2D/3D registration. I: A new gold standard data set

PURPOSE

In this article, the authors propose a new gold standard data set for the validation of two-dimensional/three-dimensional (2D/3D) and 3D/3D image registration algorithms.

METHODS

A gold standard data set was produced using a fresh cadaver pig head with attached fiducial markers. The authors used several imaging modalities common in diagnostic imaging or radiotherapy, which include 64-slice computed tomography (CT), magnetic resonance imaging using T1, T2, and proton density sequences, and cone beam CT imaging data. Radiographic data were acquired using kilovoltage and megavoltage imaging techniques. The image information reflects both anatomy and reliable fiducial marker information and improves over existing data sets by the level of anatomical detail, image data quality, and soft-tissue content. The markers on the 3D and 2D image data were segmented using ANALYZE 10.0 (AnalyzeDirect, Inc., Kansas City, KN) and an in-house software.

RESULTS

The projection distance errors and the expected target registration errors over all the image data sets were found to be less than 2.71 and 1.88 mm, respectively.

CONCLUSIONS

The gold standard data set, obtained with state-of-the-art imaging technology, has the potential to improve the validation of 2D/3D and 3D/3D registration algorithms for image guided therapy.

A neural network based 3D/3D image registration quality evaluator for the head-and-neck patient setup in the absence of a ground truth

PURPOSE

To develop a neural network based registration quality evaluator (RQE) that can identify unsuccessful 3D/3D image registrations for the head-and-neck patient setup in radiotherapy.

METHODS

A two-layer feed-forward neural network was used as a RQE to classify 3D/3D rigid registration solutions as successful or unsuccessful based on the features of the similarity surface near the point-of-solution. The supervised training and test data sets were generated by rigidly registering daily cone-beam CTs to the treatment planning fan-beam CTs of six patients with head-and-neck tumors. Two different similarity metrics (mutual information and mean-squared intensity difference) and two different types of image content (entire image versus bony landmarks) were used. The best solution for each registration pair was selected from 50 optimizing attempts that differed only by the initial transformation parameters. The distance from each individual solution to the best solution in the normalized parametrical space was compared to a user-defined error threshold to determine whether that solution was successful or not. The supervised training was then used to train the RQE. The performance of the RQE was evaluated using the test data set that consisted of registration results that were not used in training.

RESULTS

The RQE constructed using the mutual information had very good performance when tested using the test data sets, yielding the sensitivity, the specificity, the positive predictive value, and the negative predictive value in the ranges of 0.960-1.000, 0.993-1.000, 0.983-1.000, and 0.909-1.000, respectively. Adding a RQE into a conventional 3D/3D image registration system incurs only about 10%-20% increase of the overall processing time.

CONCLUSIONS

The authors' patient study has demonstrated very good performance of the proposed RQE when used with the mutual information in identifying unsuccessful 3D/3D registrations for daily patient setup. The classifier had very good generality and required only to be trained once for each implementation. When the RQE is incorporated with an automated 3D/3D image registration system, it can improve the robustness of the system.

Assessing the intrinsic precision of 3D/3D rigid image registration results for patient setup in the absence of a ground truth

PURPOSE

To assess the precision and robustness of patient setup corrections computed from 3D/3D rigid registration methods using image intensity, when no ground truth validation is possible.

METHODS

Fifteen pairs of male pelvic CTs were rigidly registered using four different in-house registration methods. Registration results were compared for different resolutions and image content by varying the image down-sampling ratio and by thresholding out soft tissue to isolate bony landmarks. Intrinsic registration precision was investigated by comparing the different methods and by reversing the source and the target roles of the two images being registered.

RESULTS

The translational reversibility errors for successful registrations ranged from 0.0 to 1.69 mm. Rotations were less than 1 degrees. Mutual information failed in most registrations that used only bony landmarks. The magnitude of the reversibility error was strongly correlated with the success/ failure of each algorithm to find the global minimum.

CONCLUSIONS

Rigid image registrations have an intrinsic uncertainty and robustness that depends on the imaging modality, the registration algorithm, the image resolution, and the image content. In the absence of an absolute ground truth, the variation in the shifts calculated by several different methods provides a useful estimate of that uncertainty. The difference observed by reversing the source and target images can be used as an indication of robust convergence.

Evaluation of similarity measures for use in the intensity-based rigid 2D-3D registration for patient positioning in radiotherapy

PURPOSE

Rigid 2D-3D registration is an alternative to 3D-3D registration for cases where largely bony anatomy can be used for patient positioning in external beam radiation therapy. In this article, the authors evaluated seven similarity measures for use in the intensity-based rigid 2D-3D registration using a variation in Skerl's similarity measure evaluation protocol.

METHODS

The seven similarity measures are partitioned intensity uniformity, normalized mutual information (NMI), normalized cross correlation (NCC), entropy of the difference image, pattern intensity (PI), gradient correlation (GC), and gradient difference (GD). In contrast to traditional evaluation methods that rely on visual inspection or registration outcomes, the similarity measure evaluation protocol probes the transform parameter space and computes a number of similarity measure properties, which is objective and optimization method independent. The variation in protocol offers an improved property in the quantification of the capture range. The authors used this protocol to investigate the effects of the downsampling ratio, the region of interest, and the method of the digitally reconstructed radiograph (DRR) calculation [i.e., the incremental ray-tracing method implemented on a central processing unit (CPU) or the 3D texture rendering method implemented on a graphics processing unit (GPU)] on the performance of the similarity measures. The studies were carried out using both the kilovoltage (kV) and the megavoltage (MV) images of an anthropomorphic cranial phantom and the MV images of a head-and-neck cancer patient.

RESULTS

Both the phantom and the patient studies showed the 2D-3D registration using the GPU-based DRR calculation yielded better robustness, while providing similar accuracy compared to the CPU-based calculation. The phantom study using kV imaging suggested that NCC has the best accuracy and robustness, but its slow function value change near the global maximum requires a stricter termination condition for an optimization method. The phantom study using MV imaging indicated that PI, GD, and GC have the best accuracy, while NCC and NMI have the best robustness. The clinical study using MV imaging showed that NCC and NMI have the best robustness.

CONCLUSIONS

The authors evaluated the performance of seven similarity measures for use in 2D-3D image registration using the variation in Skerl's similarity measure evaluation protocol. The generalized methodology can be used to select the best similarity measures, determine the optimal or near optimal choice of parameter, and choose the appropriate registration strategy for the end user in his specific registration applications in medical imaging.

Robust gradient-based 3-D/2-D registration of CT and MR to X-ray images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.

Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.