Validation of deformable registration in head and neck cancer using analysis of variance

Evaluation of DIR algorithm performance in real patients for radiotherapy treatments: A systematic review of operator-dependent strategies

PURPOSE

The performance of deformable medical image registration (DIR) algorithms has become a major concern.

METHODS

We aimed to obtain updated information on DIR algorithm performance quantification through a literature review of articles published between 2010 and 2022. We focused only on studies using operator-based methods to treat real patients. The PubMed, Google Scholar and Embase databases were searched following PRISMA guidelines.

RESULTS

One hundred and seven articles were identified. The mean number of patients and registrations per publication was 20 and 63, respectively. We found 23 different geometric metrics appearing at least twice, and the dosimetric impact of DIR was quantified in 32 articles. Forty-eight different at-risk organs were described, and target volumes were studied in 43 publications. Prostate, head-and-neck and thoracic locations represented more than ¾ of the studied locations. We summarized the type of DIR and the images used, and other key elements. Intra/interobserver variability, threshold values and the correlation between metrics were also discussed.

CONCLUSIONS

This literature review covers the past decade and should facilitate the implementation of DIR algorithms in clinical practice by providing practical and pertinent information to quantify their performance on real patients.

MIRSIG position paper: the use of image registration and fusion algorithms in radiotherapy

The report of the American Association of Physicists in Medicine (AAPM) Task Group No. 132 published in 2017 reviewed rigid image registration and deformable image registration (DIR) approaches and solutions to provide recommendations for quality assurance and quality control of clinical image registration and fusion techniques in radiotherapy. However, that report did not include the use of DIR for advanced applications such as dose warping or warping of other matrices of interest. Considering that DIR warping tools are now readily available, discussions were hosted by the Medical Image Registration Special Interest Group (MIRSIG) of the Australasian College of Physical Scientists & Engineers in Medicine in 2018 to form a consensus on best practice guidelines. This position statement authored by MIRSIG endorses the recommendations of the report of AAPM task group 132 and expands on the best practice advice from the ‘Deforming to Best Practice’ MIRSIG publication to provide guidelines on the use of DIR for advanced applications.

Feasibility of Continual Deep Learning-Based Segmentation for Personalized Adaptive Radiation Therapy in Head and Neck Area

Simple Summary

We analyzed the contouring data of 23 organs-at-risk from 100 patients with head and neck cancer who underwent definitive radiation therapy (RT). Deep learning-based segmentation (DLS) with continual training was compared to DLS with conventional training and deformable image registration (DIR) in both quantitative and qualitative (Turing’s test) methods. Results indicate the effectiveness of DLS over DIR and that of DLS with continual training over DLS with conventional training in contouring for head and neck region, especially for glandular structures. DLS with continual training might be beneficial for optimizing personalized adaptive RT in head and neck region.

Abstract

This study investigated the feasibility of deep learning-based segmentation (DLS) and continual training for adaptive radiotherapy (RT) of head and neck (H&N) cancer. One-hundred patients treated with definitive RT were included. Based on 23 organs-at-risk (OARs) manually segmented in initial planning computed tomography (CT), modified FC-DenseNet was trained for DLS: (i) using data obtained from 60 patients, with 20 matched patients in the test set (DLSm); (ii) using data obtained from 60 identical patients with 20 unmatched patients in the test set (DLSu). Manually contoured OARs in adaptive planning CT for independent 20 patients were provided as test sets. Deformable image registration (DIR) was also performed. All 23 OARs were compared using quantitative measurements, and nine OARs were also evaluated via subjective assessment from 26 observers using the Turing test. DLSm achieved better performance than both DLSu and DIR (mean Dice similarity coefficient; 0.83 vs. 0.80 vs. 0.70), mainly for glandular structures, whose volume significantly reduced during RT. Based on subjective measurements, DLS is often perceived as a human (49.2%). Furthermore, DLSm is preferred over DLSu (67.2%) and DIR (96.7%), with a similar rate of required revision to that of manual segmentation (28.0% vs. 29.7%). In conclusion, DLS was effective and preferred over DIR. Additionally, continual DLS training is required for an effective optimization and robustness in personalized adaptive RT.

A multivariable study of deformable image registration evaluation metrics in 4DCT of thoracic cancer patients

Deformable image registration (DIR) accuracy is often validated using manually identified landmarks or known deformations generated using digital or physical phantoms. In daily practice, the application of these approaches is limited since they are time-consuming or require additional equipment. An alternative is the use of metrics automatically derived from the registrations, but their interpretation is not straightforward. In this work we aim to determine the suitability of DIR-derived metrics to validate the accuracy of 4 commonly used DIR algorithms. First, we investigated the DIR accuracy using a landmark-based metric (target registration error (TRE)) and a digital phantom-based metric (known deformation recovery error (KDE)). 4DCT scans of 16 thoracic cancer patients along with corresponding pairwise anatomical landmarks (AL) locations were collected from two public databases. Digital phantoms with known deformations were generated by each DIR algorithm to test all other algorithms and compute KDE. TRE and KDE were evaluated at AL. KDE was additionally quantified in coordinates randomly sampled (RS) inside the lungs. Second, we investigated the associations of 5 DIR-derived metrics (distance discordance metric (DDM), inverse consistency error (ICE), transitivity (TE), spatial (SS) and temporal smoothness (TS)) with DIR accuracy through uni- and multivariable linear regression models. TRE values were found higher compared to KDE values and these varied depending on the phantom used. The algorithm with the best accuracy achieved average values of TRE = 1.1 mm and KDE ranging from 0.3 to 0.8 mm. DDM was the best predictor of DIR accuracy, with moderate correlations (R ² < 0.61). Poor correlations were obtained at AL for algorithms with better accuracy, which improved when evaluated at RS. Only slight correlation improvement was obtained with a multivariable analysis (R ² < 0.64). DDM can be a useful metric to identify inaccuracies for different DIR algorithms without employing landmarks or digital phantoms.

An automated, quantitative, and case-specific evaluation of deformable image registration in computed tomography images

A prerequisite for adaptive dose-tracking in radiotherapy is the assessment of the deformable image registration (DIR) quality. In this work, various metrics that quantify DIR uncertainties are investigated using realistic deformation fields of 26 head and neck and 12 lung cancer patients. Metrics related to the physiologically feasibility (the Jacobian determinant, harmonic energy (HE), and octahedral shear strain (OSS)) and numerically robustness of the deformation (the inverse consistency error (ICE), transitivity error (TE), and distance discordance metric (DDM)) were investigated. The deformable registrations were performed using a B-spline transformation model. The DIR error metrics were log-transformed and correlated (Pearson) against the log-transformed ground-truth error on a voxel level. Correlations of r  ⩾  0.5 were found for the DDM and HE. Given a DIR tolerance threshold of 2.0 mm and a negative predictive value of 0.90, the DDM and HE thresholds were 0.49 mm and 0.014, respectively. In conclusion, the log-transformed DDM and HE can be used to identify voxels at risk for large DIR errors with a large negative predictive value. The HE and/or DDM can therefore be used to perform automated quality assurance of each CT-based DIR for head and neck and lung cancer patients.

A method to detect landmark pairs accurately between intra-patient volumetric medical images

PURPOSES

An image processing procedure was developed in this study to detect large quantity of landmark pairs accurately in pairs of volumetric medical images. The detected landmark pairs can be used to evaluate of deformable image registration (DIR) methods quantitatively.

METHODS

Landmark detection and pair matching were implemented in a Gaussian pyramid multi-resolution scheme. A 3D scale-invariant feature transform (SIFT) feature detection method and a 3D Harris-Laplacian corner detection method were employed to detect feature points, i.e., landmarks. A novel feature matching algorithm, Multi-Resolution Inverse-Consistent Guided Matching or MRICGM, was developed to allow accurate feature pairs matching. MRICGM performs feature matching using guidance by the feature pairs detected at the lower resolution stage and the higher confidence feature pairs already detected at the same resolution stage, while enforces inverse consistency.

RESULTS

The proposed feature detection and feature pair matching algorithms were optimized to process 3D CT and MRI images. They were successfully applied between the inter-phase abdomen 4DCT images of three patients, between the original and the re-scanned radiation therapy simulation CT images of two head-neck patients, and between inter-fractional treatment MRIs of two patients. The proposed procedure was able to successfully detect and match over 6300 feature pairs on average. The automatically detected landmark pairs were manually verified and the mismatched pairs were rejected. The automatic feature matching accuracy before manual error rejection was 99.4%. Performance of MRICGM was also evaluated using seven digital phantom datasets with known ground truth of tissue deformation. On average, 11855 feature pairs were detected per digital phantom dataset with TRE = 0.77 ± 0.72 mm.

CONCLUSION

A procedure was developed in this study to detect large number of landmark pairs accurately between two volumetric medical images. It allows a semi-automatic way to generate the ground truth landmark datasets that allow quantitatively evaluation of DIR algorithms for radiation therapy applications.

Open source deformable image registration system for treatment planning and recurrence CT scans : Validation in the head and neck region

BACKGROUND

Clinical application of deformable registration (DIR) of medical images remains limited due to sparse validation of DIR methods in specific situations, e. g. in case of cancer recurrences. In this study the accuracy of DIR for registration of planning CT (pCT) and recurrence CT (rCT) images of head and neck squamous cell carcinoma (HNSCC) patients was evaluated.

PATIENTS AND MATERIALS

Twenty patients treated with definitive IMRT for HNSCC in 2010-2012 were included. For each patient, a pCT and an rCT scan were used. Median interval between the scans was 8.5 months. One observer manually contoured eight anatomical regions-of-interest (ROI) twice on pCT and once on rCT.

METHODS

pCT and rCT images were deformably registered using the open source software elastix. Mean surface distance (MSD) and Dice similarity coefficient (DSC) between contours were used for validation of DIR. A measure for delineation uncertainty was estimated by assessing MSD from the re-delineations of the same ROI on pCT. DIR and manual contouring uncertainties were correlated with tissue volume and rigidity.

RESULTS

MSD varied 1-3 mm for different ROIs for DIR and 1-1.5 mm for re-delineated ROIs performed on pCT. DSC for DIR varied between 0.58 and 0.79 for soft tissues and was 0.79 or higher for bony structures, and correlated with the volumes of ROIs (r = 0.5, p < 0.001) and tissue rigidity (r = 0.54, p < 0.001).

CONCLUSION

DIR using elastix in HNSCC on planning and recurrence CT scans is feasible; an uncertainty of the method is close to the voxel size length of the planning CT images.

Deformable image registration and interobserver variation in contour propagation for radiation therapy planning

Deformable image registration (DIR) and interobserver variation inevitably introduce uncertainty into the treatment planning process. The purpose of the current work was to measure deformable image registration (DIR) errors and interobserver variability for regions of interest (ROIs) in the head and neck and pelvic regions. Measured uncertainties were combined to examine planning margin adequacy for contours propagated for adaptive therapy and to assess the trade‐off of DIR and interobserver uncertainty in atlas‐based automatic segmentation. Two experienced dosimetrists retrospectively contoured brainstem, spinal cord, anterior oral cavity, larynx, right and left parotids, optic nerves, and eyes on the planning CT (CT1) and attenuation‐correction CT of diagnostic PET/CT (CT2) for 30 patients who received radiation therapy for head and neck cancer. Two senior radiation oncology residents retrospectively contoured prostate, bladder, and rectum on the postseed‐implant CT (CT1) and planning CT (CT2) for 20 patients who received radiation therapy for prostate cancer. Interobserver variation was measured by calculating mean Hausdorff distances between the two observers' contours. CT2 was deformably registered to CT1 via commercially available multipass B‐spline DIR. CT2 contours were propagated and compared with CT1 contours via mean Hausdorff distances. These values were summed in quadrature with interobserver variation for margin analysis and compared with interobserver variation for statistical significance using two‐tailed t‐tests for independent samples (α=0.05). Combined uncertainty ranged from 1.5‐5.8 mm for head and neck structures and 3.1‐3.7 mm for pelvic structures. Conventional 5 mm margins may not be adequate to cover this additional uncertainty. DIR uncertainty was significantly less than interobserver variation for four head and neck and one pelvic ROI. DIR uncertainty was not significantly different than interobserver variation for four head and neck and one pelvic ROI. DIR uncertainty was significantly greater than interobserver variation for two head and neck and one pelvic ROI. The introduction of DIR errors may offset any reduction in interobserver variation by using atlas‐based automatic segmentation.

PACS number(s): 87.57.nj, 87.55.D‐

Validation of Varian's SmartAdapt® deformable image registration algorithm for clinical application

Background

Re-contouring of structures on consecutive planning computed tomography (CT) images for patients that exhibit anatomical changes is elaborate and may negatively impact the turn-around time if this is required for many patients. This study was therefore initiated to validate the accuracy and usefulness of automatic contour propagation for head and neck cancer patients using SmartAdapt® which is the deformable image registration (DIR) application in Varian’s Eclipse™ treatment planning system.

Methods

CT images of eight head and neck cancer patients with multiple planning CTs were registered using SmartAdapt®. The contoured structures of target volumes and OARs of the primary planning CT were deformed accordingly and subsequently compared with a reference structure set being either: 1) a structure set independently contoured by the treating Radiation Oncologist (RO), or 2) the DIR-generated structure set after being reviewed and modified by the RO.

Results

Application of DIR offered a considerable time saving for ROs in delineation of structures on CTs that were acquired mid-treatment. Quantitative analysis showed that 84% of the volume of the DIR-generated structures overlapped with the independently re-contoured structures, while 94% of the volume overlapped with the DIR-generated structures after review by the RO. This apparent intra-observer variation was further investigated resulting in the identification of several causes. Qualitative analysis showed that 92% of the DIR-generated structures either need no or only minor modification during RO reviews.

Conclusions

SmartAdapt is a powerful tool with sufficient accuracy that saves considerable time in re-contouring structures on re-CTs. However, careful review of the DIR-generated structures is mandatory, in particular in areas where tumour regression plays a role.

Validation of a dose warping algorithm using clinically realistic scenarios

Objective:

Dose warping following deformable image registration (DIR) has been proposed for interfractional dose accumulation. Robust evaluation workflows are vital to clinically implement such procedures. This study demonstrates such a workflow and quantifies the accuracy of a commercial DIR algorithm for this purpose under clinically realistic scenarios.

Methods:

12 head and neck (H&N) patient data sets were used for this retrospective study. For each case, four clinically relevant anatomical changes have been manually generated. Dose distributions were then calculated on each artificially deformed image and warped back to the original anatomy following DIR by a commercial algorithm. Spatial registration was evaluated by quantitative comparison of the original and warped structure sets, using conformity index and mean distance to conformity (MDC) metrics. Dosimetric evaluation was performed by quantitative comparison of the dose–volume histograms generated for the calculated and warped dose distributions, which should be identical for the ideal “perfect” registration of mass-conserving deformations.

Results:

Spatial registration of the artificially deformed image back to the planning CT was accurate (MDC range of 1–2 voxels or 1.2–2.4 mm). Dosimetric discrepancies introduced by the DIR were low (0.02 ± 0.03 Gy per fraction in clinically relevant dose metrics) with no statistically significant difference found (Wilcoxon test, 0.6 ≥ p ≥ 0.2).

Conclusion:

The reliability of CT-to-CT DIR-based dose warping and image registration was demonstrated for a commercial algorithm with H&N patient data.

Advances in knowledge:

This study demonstrates a workflow for validation of dose warping following DIR that could assist physicists and physicians in quantifying the uncertainties associated with dose accumulation in clinical scenarios.

Accuracy quantification of a deformable image registration tool applied in a clinical setting

The purpose of this study was to test the accuracy of a commercially available deformable image registration tool in a clinical situation. In addition, to demonstrate a method to evaluate the resulting transformation of such a tool to a reference defined by multiple experts. For 16 patients (seven head and neck, four thoracic, five abdominal), 30-50 anatomical landmarks were defined on recognizable spots of a planning CT and a corresponding fraction CT. A commercially available deformable image registration tool, Velocity AI, was used to align all fraction CTs with the respective planning CTs. The registration accuracy was quantified by means of the target registration error in respect to expert-defined landmarks, considering the interobserver variation of five observers. The interobserver uncertainty of the landmark definition in our data sets is found to be 1.2 ± 1.1 mm. In general the deformable image registration tool decreases the extent of observable misalignments from 4-8 mm to 1-4 mm for nearly 50% of the landmarks (to 77% in sum). Only small differences are observed in the alignment quality of scans with different tumor location. Smallest residual deviations were achieved in scans of the head and neck region (79%, ≤ 4 mm) and the thoracic cases (79%, ≤ 4 mm), followed by the abdominal cases (59%, ≤ 4 mm). No difference is observed in the alignment quality of different tissue types (bony vs. soft tissue). The investigated commercially available deformable image registration tool is capable of reducing a mean target registration error to a level that is clinically acceptable for the evaluation of retreatment plans and replanning in case of gross tumor change during treatment. Yet, since the alignment quality needs to be improved further, the individual result of the deformable image registration tool has still to be judged by the physician prior to application.

An optimised IGRT correction vector determined from a displacement vector field: a proof of principle of a decision-making aid for re-planning

BACKGROUND

To present a new method that determines an optimised IGRT couch correction vector from a displacement vector field (DVF). The DVF is computed by a deformable image registration (DIR) method. The proposed method can improve the quality of volume-of-interest (VOI) alignment in image guided radiation therapy (IGRT), and can serve as a decision-making aid for re-planning.

MATERIAL AND METHODS

The proposed method was demonstrated using the CT data sets of 11 head-and-neck cancer patients with daily kilovoltage control-CTs. A DVF was computed for each control-CT using a DIR method. The DVF was used for voxel tracking and re-contouring of the VOIs in the control-CTs. Then a rigid body transformation, which could be used as couch correction vector, was optimised. The aim of the optimisation process was to find a vector and rotations that map the deformed VOIs into a specified territory. This territory was defined by a margin extension of the VOIs at the time of the planning process. Within this extension, VOI motion and deformation was tolerated. The objective function in the optimisation process was the sum of all volume fractions outside the defined territories.

RESULTS

The proposed method was able to find a correction vector, which resulted in a coverage of the target volumes of at least 98% in 52.3% of all fractions. In contrast, a standard IGRT correction using a rigid registration method only fulfilled this criterion in 22.6% of all fractions. The optimisation process took an average of 1.5 minutes per fraction.

CONCLUSION

The knowledge of the deformation of the anatomy allows the determination of an optimised rigid correction vector using our method. The method ensures controlled mapping of the VOIs despite small deformations. If no optimised vector can be determined, re-planning should be considered. Thus, our method can also serve as a decision-making aid for re-planning.

Estimation of the uncertainty of elastic image registration with the demons algorithm

The accuracy of elastic image registration is limited. We propose an approach to detect voxels where registration based on the demons algorithm is likely to perform inaccurately, compared to other locations of the same image. The approach is based on the assumption that the local reproducibility of the registration can be regarded as a measure of uncertainty of the image registration. The reproducibility is determined as the standard deviation of the displacement vector components obtained from multiple registrations. These registrations differ in predefined initial deformations. The proposed approach was tested with artificially deformed lung images, where the ground truth on the deformation is known. In voxels where the result of the registration was less reproducible, the registration turned out to have larger average registration errors as compared to locations of the same image, where the registration was more reproducible. The proposed method can show a clinician in which area of the image the elastic registration with the demons algorithm cannot be expected to be accurate.

A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive

Landmark point-pairs provide a strategy to assess deformable image registration (DIR) accuracy in terms of the spatial registration of the underlying anatomy depicted in medical images. In this study, we propose to augment a publicly available database (www.dir-lab.com) of medical images with large sets of manually identified anatomic feature pairs between breath-hold computed tomography (BH-CT) images for DIR spatial accuracy evaluation. Ten BH-CT image pairs were randomly selected from the COPDgene study cases. Each patient had received CT imaging of the entire thorax in the supine position at one-fourth dose normal expiration and maximum effort full dose inspiration. Using dedicated in-house software, an imaging expert manually identified large sets of anatomic feature pairs between images. Estimates of inter- and intra-observer spatial variation in feature localization were determined by repeat measurements of multiple observers over subsets of randomly selected features. 7298 anatomic landmark features were manually paired between the 10 sets of images. Quantity of feature pairs per case ranged from 447 to 1172. Average 3D Euclidean landmark displacements varied substantially among cases, ranging from 12.29 (SD: 6.39) to 30.90 (SD: 14.05) mm. Repeat registration of uniformly sampled subsets of 150 landmarks for each case yielded estimates of observer localization error, which ranged in average from 0.58 (SD: 0.87) to 1.06 (SD: 2.38) mm for each case. The additions to the online web database (www.dir-lab.com) described in this work will broaden the applicability of the reference data, providing a freely available common dataset for targeted critical evaluation of DIR spatial accuracy performance in multiple clinical settings. Estimates of observer variance in feature localization suggest consistent spatial accuracy for all observers across both four-dimensional CT and COPDgene patient cohorts.