Fast generation of digitally reconstructed radiographs using attenuation fields with application to 2D-3D image registration

Robust gradient-based 3-D/2-D registration of CT and MR to X-ray images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.

Automatic 3D-to-2D registration for CT and dual-energy digital radiography for calcification detection

We are investigating three-dimensional (3D) to two-dimensional (2D) registration methods for computed tomography (CT) and dual-energy digital radiography (DEDR). CT is an established tool for the detection of cardiac calcification. DEDR could be a cost-effective alternative screening tool. In order to utilize CT as the "gold standard" to evaluate the capability of DEDR images for the detection and localization of calcium, we developed an automatic, intensity-based 3D-to-2D registration method for 3D CT volumes and 2D DEDR images. To generate digitally reconstructed radiography (DRR) from the CT volumes, we developed several projection algorithms using the fast shear-warp method. In particular, we created a Gaussian-weighted projection for this application. We used normalized mutual information (NMI) as the similarity measurement. Simulated projection images from CT values were fused with the corresponding DEDR images to evaluate the localization of cardiac calcification. The registration method was evaluated by digital phantoms, physical phantoms, and clinical data sets. The results from the digital phantoms show that the success rate is 100% with a translation difference of less than 0.8 mm and a rotation difference of less than 0.2 degrees. For physical phantom images, the registration accuracy is 0.43 +/- 0.24 mm. Color overlay and 3D visualization of clinical images show that the two images registered well. The NMI values between the DRR and DEDR images improved from 0.21 +/- 0.03 before registration to 0.25 +/- 0.03 after registration. Registration errors measured from anatomic markers decreased from 27.6 +/- 13.6 mm before registration to 2.5 +/- 0.5 mm after registration. Our results show that the automatic 3D-to-2D registration is accurate and robust. This technique can provide a useful tool for correlating DEDR with CT images for screening coronary artery calcification.

Automatic registration of CT volumes and dual-energy digital radiography for detection of cardiac and lung diseases

We are investigating image processing and analysis techniques to improve the ability of dual-energy digital radiography (DR) for the detection of cardiac calcification. Computed tomography (CT) is an established tool for the diagnosis of coronary artery diseases. Dual-energy digital radiography could be a cost-effective alternative. In this study, we use three-dimensional (3D) CT images as the "gold standard" to evaluate the DR X-ray images for calcification detection. To this purpose, we developed an automatic registration method for 3D CT volumes and two-dimensional (2D) X-ray images. We call this 3D-to-2D registration. We first use a 3D CT image volume to simulate X-ray projection images and then register them with X-ray images. The registered CT projection images are then used to aid the interpretation dual-energy X-ray images for the detection of cardiac calcification. We acquired both CT and X-ray images from patients with coronary artery diseases. Experimental results show that the 3D-to-2D registration is accurate and useful for this new application.

Fast DRR splat rendering using common consumer graphics hardware

Digitally rendered radiographs (DRR) are a vital part of various medical image processing applications such as 2D/3D registration for patient pose determination in image-guided radiotherapy procedures. This paper presents a technique to accelerate DRR creation by using conventional graphics hardware for the rendering process. DRR computation itself is done by an efficient volume rendering method named wobbled splatting. For programming the graphics hardware, NVIDIAs C for Graphics (Cg) is used. The description of an algorithm used for rendering DRRs on the graphics hardware is presented, together with a benchmark comparing this technique to a CPU-based wobbled splatting program. Results show a reduction of rendering time by about 70%-90% depending on the amount of data. For instance, rendering a volume of 2 x 10(6) voxels is feasible at an update rate of 38 Hz compared to 6 Hz on a common Intel-based PC using the graphics processing unit (GPU) of a conventional graphics adapter. In addition, wobbled splatting using graphics hardware for DRR computation provides higher resolution DRRs with comparable image quality due to special processing characteristics of the GPU. We conclude that DRR generation on common graphics hardware using the freely available Cg environment is a major step toward 2D/3D registration in clinical routine.

A new validation method for X-ray mammogram registration algorithms using a projection model of breast X-ray compression

Establishing spatial correspondence between features visible in X-ray mammograms obtained at different times has great potential to aid assessment and quantitation of change in the breast indicative of malignancy. The literature contains numerous nonrigid registration algorithms developed for this purpose, but existing approaches are flawed by the assumption of inappropriate 2-D transformation models and quantitative estimation of registration accuracy is limited. In this paper, we describe a novel validation method which simulates plausible mammographic compressions of the breast using a magnetic resonance imaging (MRI) derived finite element model. By projecting the resulting known 3-D displacements into 2-D and generating pseudo-mammograms from these same compressed magnetic resonance (MR) volumes, we can generate convincing images with known 2-D displacements with which to validate a registration algorithm. We illustrate this approach by computing the accuracy for two conventional nonrigid 2-D registration algorithms applied to mammographic test images generated from three patient MR datasets. We show that the accuracy of these algorithms is close to the best achievable using a 2-D one-to-one correspondence model but that new algorithms incorporating more representative transformation models are required to achieve sufficiently accurate registrations for this application.

Novel image registration quality evaluator (RQE) with an implementation for automated patient positioning in cranial radiation therapy

In external beam radiation therapy, digitally reconstructed radiographs (DRRs) and portal images are used to verify patient setup based either on a visual comparison or, less frequently, with automated registration algorithms. A registration algorithm can be trapped in local optima due to irregularity of patient anatomy, image noise and artifacts, and/or out-of-plane shifts, resulting in an incorrect solution. Thus, human observation, which is subjective, is still required to check the registration result. We propose to use a novel image registration quality evaluator (RQE) to automatically identify misregistrations as part of an algorithm-based decision-making process for verification of patient positioning. A RQE, based on an adaptive pattern classifier, is generated from a pair of reference and target images to determine the acceptability of a registration solution given an optimization process. Here we applied our RQE to patient positioning for cranial radiation therapy. We constructed two RQEs-one for the evaluation of intramodal registrations (i.e., portal-portal); the other for intermodal registrations (i.e., portal-DRR). Mutual information, because of its high discriminatory ability compared with other measures (i.e., correlation coefficient and partitioned intensity uniformity), was chosen as the test function for both RQEs. We adopted 1 mm translation and 1 degree rotation as the maximal acceptable registration errors, reflecting desirable clinical setup tolerances for cranial radiation therapy. Receiver operating characteristic analysis was used to evaluate the performance of the RQE, including computations of sensitivity and specificity. The RQEs showed very good performance for both intramodal and intermodal registrations using simulated and phantom data. The sensitivity and the specificity were 0.973 and 0.936, respectively, for the intramodal RQE using phantom data. Whereas the sensitivity and the specificity were 0.961 and 0.758, respectively, for the intermodal RQE using phantom data. Phantom experiments also indicated our RQEs detected out-of-plane deviations exceeding 2.5 mm and 2.50. A preliminary retrospective clinical study of the RQE on cranial portal imaging also yielded good sensitivity > or = 0.857) and specificity (> or = 0.987). Clinical implementation of a RQE could potentially reduce the involvement of the human observer for routine patient positioning verification, while increasing setup accuracy and reducing setup verification time.

Targeted 2D/3D registration using ray normalization and a hybrid optimizer

X-ray images are often used to guide minimally invasive procedures in interventional radiology. The use of a preoperatively obtained 3D volume can enhance the visualization needed for guiding catheters and other surgical devices. However, for intraoperative usefulness, the 3D dataset needs to be registered to the 2D x-ray images of the patient. We investigated the effect of targeting subvolumes of interest in the 3D datasets and registering the projections with C-arm x-ray images. We developed an intensity-based 2D/3D rigid-body registration using a Monte Carlo-based hybrid algorithm as the optimizer, using a single view for registration. Pattern intensity (PI) and mutual information (MI) were two metrics tested. We used normalization of the rays to address the problems due to truncation in 3D necessary for targeting. We tested the algorithm on a C-arm x-ray image of a pig's head and a 3D dataset reconstructed from multiple views of the C-arm. PI and MI were comparable in performance. For two subvolumes starting with a set of initial poses from +/-15 mm in x, from +/-3 mm (random), in y and z and +/-4 deg in the three angles, the robustness was 94% for PI and 91% for MI, with accuracy of 2.4 mm (PI) and 2.6 mm (MI), using the hybrid algorithm. The hybrid optimizer, when compared with a standard Powell's direction set method, increased the robustness from 59% (Powell) to 94% (hybrid). Another set of 50 random initial conditions from [+/-20] mm in x,y,z and [+/-10] deg in the three angles, yielded robustness of 84% (hybrid) versus 38% (Powell) using PI as metric, with accuracies 2.1 mm (hybrid) versus 2.0 mm (Powell).

A Study of the Accuracy of CyberKnife Spinal Radiosurgery Using Skeletal Structure Tracking

Objective:

New technology has enabled the increasing use of radiosurgery to ablate spinal lesions. The first generation of the CyberKnife (Accuray, Inc., Sunnyvale, CA) image-guided radiosurgery system required implanted radiopaque markers (fiducials) to localize spinal targets. A recently developed and now commercially available spine tracking technology called Xsight (Accuray, Inc.) tracks skeletal structures and eliminates the need for implanted fiducials. The Xsight system localizes spinal targets by direct reference to the adjacent vertebral elements. This study sought to measure the accuracy of Xsight spine tracking and provide a qualitative assessment of overall system performance.

Methods:

Total system error, which is defined as the distance between the centroids of the planned and delivered dose distributions and represents all possible treatment planning and delivery errors, was measured using a realistic, anthropomorphic head-and-neck phantom. The Xsight tracking system error component of total system error was also computed by retrospectively analyzing image data obtained from eleven patients with a total of 44 implanted fiducials who underwent CyberKnife spinal radiosurgery.

Results:

The total system error of the Xsight targeting technology was measured to be 0.61 mm. The tracking system error component was found to be 0.49 mm.

Conclusion:

The Xsight spine tracking system is practically important because it is accurate and eliminates the use of implanted fiducials. Experience has shown this technology to be robust under a wide range of clinical circumstances.

Image-guidance for surgical procedures

Contemporary imaging modalities can now provide the surgeon with high quality three- and four-dimensional images depicting not only normal anatomy and pathology, but also vascularity and function. A key component of image-guided surgery (IGS) is the ability to register multi-modal pre-operative images to each other and to the patient. The other important component of IGS is the ability to track instruments in real time during the procedure and to display them as part of a realistic model of the operative volume. Stereoscopic, virtual- and augmented-reality techniques have been implemented to enhance the visualization and guidance process. For the most part, IGS relies on the assumption that the pre-operatively acquired images used to guide the surgery accurately represent the morphology of the tissue during the procedure. This assumption may not necessarily be valid, and so intra-operative real-time imaging using interventional MRI, ultrasound, video and electrophysiological recordings are often employed to ameliorate this situation. Although IGS is now in extensive routine clinical use in neurosurgery and is gaining ground in other surgical disciplines, there remain many drawbacks that must be overcome before it can be employed in more general minimally-invasive procedures. This review overviews the roots of IGS in neurosurgery, provides examples of its use outside the brain, discusses the infrastructure required for successful implementation of IGS approaches and outlines the challenges that must be overcome for IGS to advance further.

An Optimized Spline-Based Registration of a 3D CT to a Set of C-Arm Images

We have developed an algorithm for the rigid-body registration of a CT volume to a set of C-arm images. The algorithm uses a gradient-based iterative minimization of a least-squares measure of dissimilarity between the C-arm images and projections of the CT volume. To compute projections, we use a novel method for fast integration of the volume along rays. To improve robustness and speed, we take advantage of a coarse-to-fine processing of the volume/image pyramids. To compute the projections of the volume, the gradient of the dissimilarity measure, and the multiresolution data pyramids, we use a continuous image/volume model based on cubic B-splines, which ensures a high interpolation accuracy and a gradient of the dissimilarity measure that is well defined everywhere. We show the performance of our algorithm on a human spine phantom, where the true alignment is determined using a set of fiducial markers.

Markerless real-time 3-D target region tracking by motion backprojection from projection images

Accurate and fast localization of a predefined target region inside the patient is an important component of many image-guided therapy procedures. This problem is commonly solved by registration of intraoperative 2-D projection images to 3-D preoperative images. If the patient is not fixed during the intervention, the 2-D image acquisition is repeated several times during the procedure, and the registration problem can be cast instead as a 3-D tracking problem. To solve the 3-D problem, we propose in this paper to apply 2-D region tracking to first recover the components of the transformation that are in-plane to the projections. The 2-D motion estimates of all projections are backprojected into 3-D space, where they are then combined into a consistent estimate of the 3-D motion. We compare this method to intensity-based 2-D to 3-D registration and a combination of 2-D motion backprojection followed by a 2-D to 3-D registration stage. Using clinical data with a fiducial marker-based gold-standard transformation, we show that our method is capable of accurately tracking vertebral targets in 3-D from 2-D motion measured in X-ray projection images. Using a standard tracking algorithm (hyperplane tracking), tracking is achieved at video frame rates but fails relatively often (32% of all frames tracked with target registration error (TRE) better than 1.2 mm, 82% of all frames tracked with TRE better than 2.4 mm). With intensity-based 2-D to 2-D image registration using normalized mutual information (NMI) and pattern intensity (PI), accuracy and robustness are substantially improved. NMI tracked 82% of all frames in our data with TRE better than 1.2 mm and 96% of all frames with TRE better than 2.4 mm. This comes at the cost of a reduced frame rate, 1.7 s average processing time per frame and projection device. Results using PI were slightly more accurate, but required on average 5.4 s time per frame. These results are still substantially faster than 2-D to 3-D registration. We conclude that motion backprojection from 2-D motion tracking is an accurate and efficient method for tracking 3-D target motion, but tracking 2-D motion accurately and robustly remains a challenge.