“Gold standard” data for evaluation and comparison of 3D/2D registration methods

Biomechanical guidance can improve accuracy of reduction for intra-articular tibia plafond fractures and reduce joint contact stress

Articular fracture malreduction increases posttraumatic osteoarthritis (PTOA) risk by elevating joint contact stress. A new biomechanical guidance system (BGS) that provides intraoperative assessment of articular fracture reduction and joint contact stress based solely on a preoperative computed tomography (CT) and intraoperative fluoroscopy may facilitate better fracture reduction. The objective of this proof-of-concept cadaveric study was to test this premise while characterizing BGS performance. Articular tibia plafond fractures were created in five cadaveric ankles. CT scans were obtained to provide digital models. Indirect reduction was performed in a simulated operating room once with and once without BGS guidance. CT scans after fixation provided models of the reduced ankles for assessing reduction accuracy, joint contact stresses, and BGS accuracy. BGS was utilized 4.8 ± 1.3 (mean ± SD) times per procedure, increasing operative time by 10 min (39%), and the number of fluoroscopy images by 31 (17%). Errors in BGS reduction assessment compared to CT-derived models were 0.45 ± 0.57 mm in translation and 2.0 ± 2.5° in rotation. For the four ankles that were successfully reduced and fixed, associated absolute errors in computed mean and maximum contact stress were 0.40 ± 0.40 and 0.96 ± 1.12 MPa, respectively. BGS reduced mean and maximum contact stress by 1.1 and 2.6 MPa, respectively. BGS thus improved the accuracy of articular fracture reduction and significantly reduced contact stress. Statement of Clinical Significance: Malreduction of articular fractures is known to lead to PTOA. The BGS described in this work has potential to improve quality of articular fracture reduction and clinical outcomes for patients with a tibia plafond fracture.

Assessing the accuracy of a new 3D2D registration algorithm based on a non-invasive skin marker model for navigated spine surgery

Purpose

We assessed the accuracy of a new 3D2D registration algorithm to be used for navigated spine surgery and explored anatomical and radiologic parameters affecting the registration accuracy. Compared to existing 3D2D registration algorithms, the algorithm does not need bone-mounted or table-mounted instruments for registration. Neither does the intraoperative imaging device have to be tracked or calibrated.

Methods

The rigid registration algorithm required imaging data (a pre-existing CT scan (3D) and two angulated fluoroscopic images (2D)) to register positions of vertebrae in 3D and is based on non-invasive skin markers. The algorithm registered five adjacent vertebrae and was tested in the thoracic and lumbar spine from three human cadaveric specimens. The registration accuracy was calculated for each registered vertebra and measured with the target registration error (TRE) in millimeters. We used multivariable analysis to identify parameters independently affecting the algorithm’s accuracy such as the angulation between the two fluoroscopic images (between 40° and 90°), the detector-skin distance, the number of skin markers applied, and waist circumference.

Results

The algorithm registered 780 vertebrae with a median TRE of 0.51 mm [interquartile range 0.32–0.73 mm] and a maximum TRE of 2.06 mm. The TRE was most affected by the angulation between the two fluoroscopic images obtained (p < 0.001): larger angulations resulted in higher accuracy. The algorithm was more accurate in thoracic vertebrae (p = 0.004) and in the specimen with the smallest waist circumference (p = 0.003). The algorithm registered all five adjacent vertebrae with similar accuracy.

Conclusion

We studied the accuracy of a new 3D2D registration algorithm based on non-invasive skin markers. The algorithm registered five adjacent vertebrae with similar accuracy in the thoracic and lumbar spine and showed a maximum target registration error of approximately 2 mm. To further evaluate its potential for navigated spine surgery, the algorithm may now be integrated into a complete navigation system.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-022-02733-w.

A new 2D-3D registration gold-standard dataset for the hip joint based on uncertainty modeling

Purpose

Estimation of the accuracy of 2D‐3D registration is paramount for a correct evaluation of its outcome in both research and clinical studies. Publicly available datasets with standardized evaluation methodology are necessary for validation and comparison of 2D‐3D registration techniques. Given the large use of 2D‐3D registration in biomechanics, we introduced the first gold standard validation dataset for computed tomography (CT)‐to‐x‐ray registration of the hip joint, based on fluoroscopic images with large rotation angles. As the ground truth computed with fiducial markers is affected by localization errors in the image datasets, we proposed a new methodology based on uncertainty propagation to estimate the accuracy of a gold standard dataset.

Methods

The gold standard dataset included a 3D CT scan of a female hip phantom and 19 2D fluoroscopic images acquired at different views and voltages. The ground truth transformations were estimated based on the corresponding pairs of extracted 2D and 3D fiducial locations. These were assumed to be corrupted by Gaussian noise, without any restrictions of isotropy. We devised the multiple projective points criterion (MPPC) that jointly optimizes the transformations and the noisy 3D fiducial locations for all views. The accuracy of the transformations obtained with the MPPC was assessed in both synthetic and real experiments using different formulations of the target registration error (TRE), including a novel formulation of the TRE (uTRE) derived from the uncertainty analysis of the MPPC.

Results

The proposed MPPC method was statistically more accurate compared to the validation methods for 2D‐3D registration that did not optimize the 3D fiducial positions or wrongly assumed the isotropy of the noise. The reported results were comparable to previous published works of gold standard datasets. However, a formulation of the TRE commonly found in these gold standard datasets was found to significantly miscalculate the true TRE computed in synthetic experiments with known ground truths. In contrast, the uncertainty‐based uTRE was statistically closer to the true TRE.

Conclusions

We proposed a new gold standard dataset for the validation of CT‐to‐X‐ray registration of the hip joint. The gold standard transformations were derived from a novel method modeling the uncertainty in extracted 2D and 3D fiducials. Results showed that considering possible noise anisotropy and including corrupted 3D fiducials in the optimization resulted in improved accuracy of the gold standard. A new uncertainty‐based formulation of the TRE also appeared as a good alternative to the unknown true TRE that has been replaced in previous works by an alternative TRE not fully reflecting the gold standard accuracy.

Learning an Attention Model for Robust 2-D/3-D Registration Using Point-To-Plane Correspondences

Minimally invasive procedures rely on image guidance for navigation at the operation site to avoid large surgical incisions. X-ray images are often used for guidance, but important structures may be not well visible. These structures can be overlaid from pre-operative 3-D images and accurate alignment can be established using 2-D/3-D registration. Registration based on the point-to-plane correspondence model was recently proposed and shown to achieve state-of-the-art performance. However, registration may still fail in challenging cases due to a large portion of outliers. In this paper, we describe a learning-based correspondence weighting scheme to improve the registration performance. By learning an attention model, inlier correspondences get higher attention in the motion estimation while the outlier correspondences are suppressed. Instead of using per-correspondence labels, our objective function allows to train the model directly by minimizing the registration error. We demonstrate a highly increased robustness, e.g. increasing the success rate from 84.9% to 97.0% for spine registration. In contrast to previously proposed learning-based methods, we also achieve a high accuracy of around 0.5mm mean re-projection distance. In addition, our method requires a relatively small amount of training data, is able to learn from simulated data, and generalizes to images with additional structures which are not present during training. Furthermore, a single model can be trained for both, different views and different anatomical structures.

Interactive initialization of 2D/3D rigid registration

PURPOSE

Registration is one of the key technical components in an image-guided navigation system. A large number of 2D/3D registration algorithms have been previously proposed, but have not been able to transition into clinical practice. The authors identify the primary reason for the lack of adoption with the prerequisite for a sufficiently accurate initial transformation, mean target registration error of about 10 mm or less. In this paper, the authors present two interactive initialization approaches that provide the desired accuracy for x-ray/MR and x-ray/CT registration in the operating room setting.

METHODS

The authors have developed two interactive registration methods based on visual alignment of a preoperative image, MR, or CT to intraoperative x-rays. In the first approach, the operator uses a gesture based interface to align a volume rendering of the preoperative image to multiple x-rays. The second approach uses a tracked tool available as part of a navigation system. Preoperatively, a virtual replica of the tool is positioned next to the anatomical structures visible in the volumetric data. Intraoperatively, the physical tool is positioned in a similar manner and subsequently used to align a volume rendering to the x-ray images using an augmented reality (AR) approach. Both methods were assessed using three publicly available reference data sets for 2D/3D registration evaluation.

RESULTS

In the authors' experiments, the authors show that for x-ray/MR registration, the gesture based method resulted in a mean target registration error (mTRE) of 9.3 ± 5.0 mm with an average interaction time of 146.3 ± 73.0 s, and the AR-based method had mTREs of 7.2 ± 3.2 mm with interaction times of 44 ± 32 s. For x-ray/CT registration, the gesture based method resulted in a mTRE of 7.4 ± 5.0 mm with an average interaction time of 132.1 ± 66.4 s, and the AR-based method had mTREs of 8.3 ± 5.0 mm with interaction times of 58 ± 52 s.

CONCLUSIONS

Based on the authors' evaluation, the authors conclude that the registration approaches are sufficiently accurate for initializing 2D/3D registration in the OR setting, both when a tracking system is not in use (gesture based approach), and when a tracking system is already in use (AR based approach).

Robust 3D-2D image registration: application to spine interventions and vertebral labeling in the presence of anatomical deformation

We present a framework for robustly estimating registration between a 3D volume image and a 2D projection image and evaluate its precision and robustness in spine interventions for vertebral localization in the presence of anatomical deformation. The framework employs a normalized gradient information similarity metric and multi-start covariance matrix adaptation evolution strategy optimization with local-restarts, which provided improved robustness against deformation and content mismatch. The parallelized implementation allowed orders-of-magnitude acceleration in computation time and improved the robustness of registration via multi-start global optimization. Experiments involved a cadaver specimen and two CT datasets (supine and prone) and 36 C-arm fluoroscopy images acquired with the specimen in four positions (supine, prone, supine with lordosis, prone with kyphosis), three regions (thoracic, abdominal, and lumbar), and three levels of geometric magnification (1.7, 2.0, 2.4). Registration accuracy was evaluated in terms of projection distance error (PDE) between the estimated and true target points in the projection image, including 14 400 random trials (200 trials on the 72 registration scenarios) with initialization error up to ±200 mm and ±10°. The resulting median PDE was better than 0.1 mm in all cases, depending somewhat on the resolution of input CT and fluoroscopy images. The cadaver experiments illustrated the tradeoff between robustness and computation time, yielding a success rate of 99.993% in vertebral labeling (with 'success' defined as PDE <5 mm) using 1,718 664 ± 96 582 function evaluations computed in 54.0 ± 3.5 s on a mid-range GPU (nVidia, GeForce GTX690). Parameters yielding a faster search (e.g., fewer multi-starts) reduced robustness under conditions of large deformation and poor initialization (99.535% success for the same data registered in 13.1 s), but given good initialization (e.g., ±5 mm, assuming a robust initial run) the same registration could be solved with 99.993% success in 6.3 s. The ability to register CT to fluoroscopy in a manner robust to patient deformation could be valuable in applications such as radiation therapy, interventional radiology, and an assistant to target localization (e.g., vertebral labeling) in image-guided spine surgery.

3D-2D registration of cerebral angiograms: a method and evaluation on clinical images

Endovascular image-guided interventions (EIGI) involve navigation of a catheter through the vasculature followed by application of treatment at the site of anomaly using live 2D projection images for guidance. 3D images acquired prior to EIGI are used to quantify the vascular anomaly and plan the intervention. If fused with the information of live 2D images they can also facilitate navigation and treatment. For this purpose 3D-2D image registration is required. Although several 3D-2D registration methods for EIGI achieve registration accuracy below 1 mm, their clinical application is still limited by insufficient robustness or reliability. In this paper, we propose a 3D-2D registration method based on matching a 3D vasculature model to intensity gradients of live 2D images. To objectively validate 3D-2D registration methods, we acquired a clinical image database of 10 patients undergoing cerebral EIGI and established "gold standard" registrations by aligning fiducial markers in 3D and 2D images. The proposed method had mean registration accuracy below 0.65 mm, which was comparable to tested state-of-the-art methods, and execution time below 1 s. With the highest rate of successful registrations and the highest capture range the proposed method was the most robust and thus a good candidate for application in EIGI.

Accuracy validation for medical image registration algorithms: a review

Accuracy validation is essential to clinical application of medical image registration techniques. Registration validation remains a challenging problem in practice mainly due to lack of 'ground truth'.In this paper, an overview of current validation methods for medical image registration is presented with detailed discussion of their benefits and drawbacks.Special focus is on non-rigid registration validation. Promising solution is also discussed.

High-performance GPU-based rendering for real-time, rigid 2D/3D-image registration and motion prediction in radiation oncology

A common problem in image-guided radiation therapy (IGRT) of lung cancer as well as other malignant diseases is the compensation of periodic and aperiodic motion during dose delivery. Modern systems for image-guided radiation oncology allow for the acquisition of cone-beam computed tomography data in the treatment room as well as the acquisition of planar radiographs during the treatment. A mid-term research goal is the compensation of tumor target volume motion by 2D/3D Registration. In 2D/3D registration, spatial information on organ location is derived by an iterative comparison of perspective volume renderings, so-called digitally rendered radiographs (DRR) from computed tomography volume data, and planar reference x-rays. Currently, this rendering process is very time consuming, and real-time registration, which should at least provide data on organ position in less than a second, has not come into existence. We present two GPU-based rendering algorithms which generate a DRR of 512×512 pixels size from a CT dataset of 53 MB size at a pace of almost 100 Hz. This rendering rate is feasible by applying a number of algorithmic simplifications which range from alternative volume-driven rendering approaches - namely so-called wobbled splatting - to sub-sampling of the DRR-image by means of specialized raycasting techniques. Furthermore, general purpose graphics processing unit (GPGPU) programming paradigms were consequently utilized. Rendering quality and performance as well as the influence on the quality and performance of the overall registration process were measured and analyzed in detail. The results show that both methods are competitive and pave the way for fast motion compensation by rigid and possibly even non-rigid 2D/3D registration and, beyond that, adaptive filtering of motion models in IGRT.

Validation for 2D/3D registration. I: A new gold standard data set

PURPOSE

In this article, the authors propose a new gold standard data set for the validation of two-dimensional/three-dimensional (2D/3D) and 3D/3D image registration algorithms.

METHODS

A gold standard data set was produced using a fresh cadaver pig head with attached fiducial markers. The authors used several imaging modalities common in diagnostic imaging or radiotherapy, which include 64-slice computed tomography (CT), magnetic resonance imaging using T1, T2, and proton density sequences, and cone beam CT imaging data. Radiographic data were acquired using kilovoltage and megavoltage imaging techniques. The image information reflects both anatomy and reliable fiducial marker information and improves over existing data sets by the level of anatomical detail, image data quality, and soft-tissue content. The markers on the 3D and 2D image data were segmented using ANALYZE 10.0 (AnalyzeDirect, Inc., Kansas City, KN) and an in-house software.

RESULTS

The projection distance errors and the expected target registration errors over all the image data sets were found to be less than 2.71 and 1.88 mm, respectively.

CONCLUSIONS

The gold standard data set, obtained with state-of-the-art imaging technology, has the potential to improve the validation of 2D/3D and 3D/3D registration algorithms for image guided therapy.

Validation for 2D/3D registration. II: The comparison of intensity- and gradient-based merit functions using a new gold standard data set

PURPOSE

A new gold standard data set for validation of 2D/3D registration based on a porcine cadaver head with attached fiducial markers was presented in the first part of this article. The advantage of this new phantom is the large amount of soft tissue, which simulates realistic conditions for registration. This article tests the performance of intensity- and gradient-based algorithms for 2D/3D registration using the new phantom data set.

METHODS

Intensity-based methods with four merit functions, namely, cross correlation, rank correlation, correlation ratio, and mutual information (MI), and two gradient-based algorithms, the backprojection gradient-based (BGB) registration method and the reconstruction gradient-based (RGB) registration method, were compared. Four volumes consisting of CBCT with two fields of view, 64 slice multidetector CT, and magnetic resonance-T1 weighted images were registered to a pair of kV x-ray images and a pair of MV images. A standardized evaluation methodology was employed. Targets were evenly spread over the volumes and 250 starting positions of the 3D volumes with initial displacements of up to 25 mm from the gold standard position were calculated. After the registration, the displacement from the gold standard was retrieved and the root mean square (RMS), mean, and standard deviation mean target registration errors (mTREs) over 250 registrations were derived. Additionally, the following merit properties were computed: Accuracy, capture range, number of minima, risk of nonconvergence, and distinctiveness of optimum for better comparison of the robustness of each merit.

RESULTS

Among the merit functions used for the intensity-based method, MI reached the best accuracy with an RMS mTRE down to 1.30 mm. Furthermore, it was the only merit function that could accurately register the CT to the kV x rays with the presence of tissue deformation. As for the gradient-based methods, BGB and RGB methods achieved subvoxel accuracy (RMS mTRE down to 0.56 and 0.70 mm, respectively). Overall, gradient-based similarity measures were found to be substantially more accurate than intensity-based methods and could cope with soft tissue deformation and enabled also accurate registrations of the MR-T1 volume to the kV x-ray image.

CONCLUSIONS

In this article, the authors demonstrate the usefulness of a new phantom image data set for the evaluation of 2D/3D registration methods, which featured soft tissue deformation. The author's evaluation shows that gradient-based methods are more accurate than intensity-based methods, especially when soft tissue deformation is present. However, the current nonoptimized implementations make them prohibitively slow for practical applications. On the other hand, the speed of the intensity-based method renders these more suitable for clinical use, while the accuracy is still competitive.

Standardized evaluation methodology for 3D/2D registration based on the Visible Human data set

PURPOSE

A new image database with a reference-based standardized evaluation methodology for objective evaluation and comparison of three-dimensional/two-dimensional (3D/2D) registration methods has been introduced.

METHODS

Computed tomography (CT) images of a male and female from the Visible Human Project were used and 16 subvolumes, each containing one of vertebrae T3-T12 and L1-L5 and the pelvis, were defined from the CTs. Six pairs of 2D fluoroscopic x-ray images from different views, showing the thoracic, lumbar, and pelvic regions, were rendered from the CT data using a ray-casting algorithm with an energy conversion function. Furthermore, a single 13-gauge needle was analytically simulated and projected onto the 2D images. By the novel standardized evaluation methodology, a 3D/2D registration method is evaluated by four evaluation criteria: Accuracy, reliability, robustness, and algorithm complexity.

RESULTS

To demonstrate the usefulness of the proposed data set and the standardized evaluation methodology, a part of the data set was used in an evaluation study of two gradient-based 3D/2D registration methods. It was shown that the use of a failure criterion to calculate the registration accuracy and reliability is not required, since all the information about a registration method can be determined from the estimated distribution of registration errors.

CONCLUSIONS

The proposed simulated image data set with quite realistic synthetic 2D images, depicting soft tissues and outliers, is especially suitable for preliminary testing of 3D/2D registration algorithms. Since the aim of this article is to provide objective comparison and unbiased evaluation of 3D/2D registration methods, the standardized evaluation methodology is available upon request from the authors.