Vasculature-Driven Biomechanical Deformable Image Registration of Longitudinal Liver Cholangiocarcinoma Computed Tomographic Scans

Analysis and prediction of liver volume change maps derived from computational tomography scans acquired pre- and post-radiation therapy

External beam radiation therapy (EBRT) of liver cancers can cause local liver atrophy as a result of tissue damage or hypertrophy as a result of liver regeneration. Predicting those volumetric changes would enable new strategies for liver function preservation during treatment planning. However, understanding of the spatial dose/volume relationship is still limited. This study leverages the use of deep learning-based segmentation and biomechanical deformable image registration (DIR) to analyze and predict this relationship. Pre- and Post-EBRT imaging data were collected for 100 patients treated for hepatocellular carcinomas, cholangiocarcinoma or CRC with intensity-modulated radiotherapy (IMRT) with prescription doses ranging from 50 to 100 Gy delivered in 10-28 fractions. For each patient, DIR between the portal and venous (PV) phase of a diagnostic computed tomography (CT) scan acquired before radiation therapy (RT) planning, and a PV phase of a diagnostic CT scan acquired after the end of RT (on average 147 ± 36 d) was performed to calculate Jacobian maps representing volume changes in the liver. These volume change maps were used: (i): to analyze the dose/volume relationship in the whole liver and individual Couinaud's segments; and (ii): to investigate the use of deep-learning to predict a Jacobian map solely based on the pre-RT diagnostic CT and planned dose distribution. Moderate correlations between mean equivalent dose in 2 Gy fractions (EQD2) and volume change was observed for all liver sub-regions analyzed individually with Pearson correlationrranging from -0.36 to -067. The predicted volume change maps showed a significantly stronger voxel-wise correlation with the DIR-based volume change maps than when considering the original EQD2 distribution (0.63 ± 0.24 versus 0.55 ± 23, respectively), demonstrating the ability of the proposed approach to establish complex relationships between planned dose and liver volume response months after treatment, which represents a promising prediction tool for the development of future adaptive and personalized liver radiation therapy strategies.

Registration-based biomarkers for neoadjuvant treatment response of pancreatic cancer via longitudinal image registration

Purpose

Pancreatic ductal adenocarcinoma (PDAC) frequently presents as hypo- or iso-dense masses with poor contrast delineation from surrounding parenchyma, which decreases reproducibility of manual dimensional measurements obtained during conventional radiographic assessment of treatment response. Longitudinal registration between pre- and post-treatment images may produce imaging biomarkers that more reliably quantify treatment response across serial imaging.

Approach

Thirty patients who prospectively underwent a neoadjuvant chemotherapy regimen as part of a clinical trial were retrospectively analyzed in this study. Two image registration methods were applied to quantitatively assess longitudinal changes in tumor volume and tumor burden across the neoadjuvant treatment interval. Longitudinal registration errors of the pancreas were characterized, and registration-based treatment response measures were correlated to overall survival (OS) and recurrence-free survival (RFS) outcomes over 5-year follow-up. Corresponding biomarker assessments via manual tumor segmentation, the standardized response evaluation criteria in solid tumors (RECIST), and pathological examination of post-resection tissue samples were analyzed as clinical comparators.

Results

Average target registration errors were 2.56 ± 2.45    mm for a biomechanical image registration algorithm and 4.15 ± 3.63    mm for a diffeomorphic intensity-based algorithm, corresponding to 1-2 times voxel resolution. Cox proportional hazards analysis showed that registration-derived changes in tumor burden were significant predictors of OS and RFS, while none of the alternative comparators, including manual tumor segmentation, RECIST, or pathological variables were associated with consequential hazard ratios. Additional ROC analysis at 1-, 2-, 3-, and 5-year follow-up revealed that registration-derived changes in tumor burden between pre- and post-treatment imaging were better long-term predictors for OS and RFS than the clinical comparators.

Conclusions

Volumetric changes measured by longitudinal deformable image registration may yield imaging biomarkers to discriminate neoadjuvant treatment response in ill-defined tumors characteristic of PDAC. Registration-based biomarkers may help to overcome visual limits of radiographic evaluation to improve clinical outcome prediction and inform treatment selection.

Optimization of mesh generation for geometric accuracy, robustness, and efficiency of biomechanical-model-based deformable image registration

BACKGROUND

Successful generation of biomechanical-model-based deformable image registration (BM-DIR) relies on user-defined parameters that dictate surface mesh quality. The trial-and-error process to determine the optimal parameters can be labor-intensive and hinder DIR efficiency and clinical workflow.

PURPOSE

To identify optimal parameters in surface mesh generation as boundary conditions for a BM-DIR in longitudinal liver and lung CT images to facilitate streamlined image registration processes.

METHODS

Contrast-enhanced CT images of 29 colorectal liver cancer patients and end-exhale four-dimensional CT images of 26 locally advanced non-small cell lung cancer patients were collected. Different combinations of parameters that determine the triangle mesh quality (voxel side length and triangle edge length) were investigated. The quality of DIRs generated using these parameters was evaluated with metrics for geometric accuracy, robustness, and efficiency. Metrics for geometric accuracy included target registration error (TRE) of internal vessel bifurcations, dice similar coefficient (DSC), mean distance to agreement (MDA), Hausdorff distance (HD) for organ contours, and number of vertices in the triangle mesh. American Association of Physicists in Medicine Task Group 132 was used to ensure parameters met TRE, DSC, MDA recommendations before the comparison among the parameters. Robustness was evaluated as the success rate of DIR generation, and efficiency was evaluated as the total time to generate boundary conditions and compute finite element analysis.

RESULTS

Voxel side length of 0.2 cm and triangle edge length of 3 were found to be the optimal parameters for both liver and lung, with success rate of 1.00 and 0.98 and average DIR computation time of 100 and 143 s, respectively. For this combination, the average TRE, DSC, MDA, and HD were 0.38-0.40, 0.96-0.97, 0.09-0.12, and 0.87-1.17 mm, respectively.

CONCLUSION

The optimal parameters were found for the analyzed patients. The decision-making process described in this study serves as a recommendation for BM-DIR algorithms to be used for liver and lung. These parameters can facilitate consistence in the evaluation of published studies and more widespread utilization of BM-DIR in clinical practice.

Leveraging deep learning-based segmentation and contours-driven deformable registration for dose accumulation in abdominal structures

PURPOSE

Discrepancies between planned and delivered dose to GI structures during radiation therapy (RT) of liver cancer may hamper the prediction of treatment outcomes. The purpose of this study is to develop a streamlined workflow for dose accumulation in a treatment planning system (TPS) during liver image-guided RT and to assess its accuracy when using different deformable image registration (DIR) algorithms.

MATERIALS AND METHODS

Fifty-six patients with primary and metastatic liver cancer treated with external beam radiotherapy guided by daily CT-on-rails (CTOR) were retrospectively analyzed. The liver, stomach and duodenum contours were auto-segmented on all planning CTs and daily CTORs using deep-learning methods. Dose accumulation was performed for each patient using scripting functionalities of the TPS and considering three available DIR algorithms based on: (i) image intensities only; (ii) intensities + contours; (iii) a biomechanical model (contours only). Planned and accumulated doses were converted to equivalent dose in 2Gy (EQD2) and normal tissue complication probabilities (NTCP) were calculated for the stomach and duodenum. Dosimetric indexes for the normal liver, GTV, stomach and duodenum and the NTCP values were exported from the TPS for analysis of the discrepancies between planned and the different accumulated doses.

RESULTS

Deep learning segmentation of the stomach and duodenum enabled considerable acceleration of the dose accumulation process for the 56 patients. Differences between accumulated and planned doses were analyzed considering the 3 DIR methods. For the normal liver, stomach and duodenum, the distribution of the 56 differences in maximum doses (D2%) presented a significantly higher variance when a contour-driven DIR method was used instead of the intensity only-based method. Comparing the two contour-driven DIR methods, differences in accumulated minimum doses (D98%) in the GTV were >2Gy for 15 (27%) of the patients. Considering accumulated dose instead of planned dose in standard NTCP models of the duodenum demonstrated a high sensitivity of the duodenum toxicity risk to these dose discrepancies, whereas smaller variations were observed for the stomach.

CONCLUSION

This study demonstrated a successful implementation of an automatic workflow for dose accumulation during liver cancer RT in a commercial TPS. The use of contour-driven DIR methods led to larger discrepancies between planned and accumulated doses in comparison to using an intensity only based DIR method, suggesting a better capability of these approaches in estimating complex deformations of the GI organs.

Segmentation-guided multi-modal registration of liver images for dose estimation in SIRT

Purpose

Selective internal radiation therapy (SIRT) requires a good liver registration of multi-modality images to obtain precise dose prediction and measurement. This study investigated the feasibility of liver registration of CT and MR images, guided by segmentation of the liver and its landmarks. The influence of the resulting lesion registration on dose estimation was evaluated.

Methods

The liver segmentation was done with a convolutional neural network (CNN), and the landmarks were segmented manually. Our image-based registration software and its liver-segmentation-guided extension (CNN-guided) were tuned and evaluated with 49 CT and 26 MR images from 20 SIRT patients. Each liver registration was evaluated by the root mean square distance (RMSD) of mean surface distance between manually delineated liver contours and mass center distance between manually delineated landmarks (lesions, clips, etc.). The root mean square of RMSDs (RRMSD) was used to evaluate all liver registrations. The CNN-guided registration was further extended by incorporating landmark segmentations (CNN&LM-guided) to assess the value of additional landmark guidance. To evaluate the influence of segmentation-guided registration on dose estimation, mean dose and volume percentages receiving at least 70 Gy (V70) estimated on the ^99mTc-labeled macro-aggregated albumin (^99mTc-MAA) SPECT were computed, either based on lesions from the reference ^99mTc-MAA CT (reference lesions) or from the registered floating CT or MR images (registered lesions) using the CNN- or CNN&LM-guided algorithms.

Results

The RRMSD decreased for the floating CTs and MRs by 1.0 mm (11%) and 3.4 mm (34%) using CNN guidance for the image-based registration and by 2.1 mm (26%) and 1.4 mm (21%) using landmark guidance for the CNN-guided registration. The quartiles for the relative mean dose difference (the V70 difference) between the reference and registered lesions and their correlations [25th, 75th; r] are as follows: [− 5.5% (− 1.3%), 5.6% (3.4%); 0.97 (0.95)] and [− 12.3% (− 2.1%), 14.8% (2.9%); 0.96 (0.97)] for the CNN&LM- and CNN-guided CT to CT registrations, [− 7.7% (− 6.6%), 7.0% (3.1%); 0.97 (0.90)] and [− 15.1% (− 11.3%), 2.4% (2.5%); 0.91 (0.78)] for the CNN&LM- and CNN-guided MR to CT registrations.

Conclusion

Guidance by CNN liver segmentations and landmarks markedly improves the performance of the image-based registration. The small mean dose change between the reference and registered lesions demonstrates the feasibility of applying the CNN&LM- or CNN-guided registration to volume-level dose prediction. The CNN&LM- and CNN-guided registrations for CTs can be applied to voxel-level dose prediction according to their small V70 change for most lesions. The CNN-guided MR to CT registration still needs to incorporate landmark guidance for smaller change of voxel-level dose estimation.

Automatic liver tumor localization using deep learning-based liver boundary motion estimation and biomechanical modeling (DL-Bio)

PURPOSE

Recently, two-dimensional-to-three-dimensional (2D-3D) deformable registration has been applied to deform liver tumor contours from prior reference images onto estimated cone-beam computed tomography (CBCT) target images to automate on-board tumor localizations. Biomechanical modeling has also been introduced to fine-tune the intra-liver deformation-vector-fields (DVFs) solved by 2D-3D deformable registration, especially at low-contrast regions, using tissue elasticity information and liver boundary DVFs. However, the caudal liver boundary shows low contrast from surrounding tissues in the cone-beam projections, which degrades the accuracy of the intensity-based 2D-3D deformable registration there and results in less accurate boundary conditions for biomechanical modeling. We developed a deep-learning (DL)-based method to optimize the liver boundary DVFs after 2D-3D deformable registration to further improve the accuracy of subsequent biomechanical modeling and liver tumor localization.

METHODS

The DL-based network was built based on the U-Net architecture. The network was trained in a supervised fashion to learn motion correlation between cranial and caudal liver boundaries to optimize the liver boundary DVFs. Inputs of the network had three channels, and each channel featured the 3D DVFs estimated by the 2D-3D deformable registration along one Cartesian direction (x, y, z). To incorporate patient-specific liver boundary information into the DVFs, the DVFs were masked by a liver boundary ring structure generated from the liver contour of the prior reference image. The network outputs were the optimized DVFs along the liver boundary with higher accuracy. From these optimized DVFs, boundary conditions were extracted for biomechanical modeling to further optimize the solution of intra-liver tumor motion. We evaluated the method using 34 liver cancer patient cases, with 24 for training and 10 for testing. We evaluated and compared the performance of three methods: 2D-3D deformable registration, 2D-3D-Bio (2D-3D deformable registration with biomechanical modeling), and DL-Bio (DL model prediction with biomechanical modeling). The tumor localization errors were quantified through calculating the center-of-mass-errors (COMEs), DICE coefficients, and Hausdorff distance between deformed liver tumor contours and manually segmented "gold-standard" contours.

RESULTS

The predicted DVFs by the DL model showed improved accuracy at the liver boundary, which translated into more accurate liver tumor localizations through biomechanical modeling. On a total of 90 evaluated images and tumor contours, the average (± sd) liver tumor COMEs of the 2D-3D, 2D-3D-Bio, and DL-Bio techniques were 4.7 ± 1.9 mm, 2.9 ± 1.0 mm, and 1.7 ± 0.4 mm. The corresponding average (± sd) DICE coefficients were 0.60 ± 0.12, 0.71 ± 0.07, and 0.78 ± 0.03; and the average (± sd) Hausdorff distances were 7.0 ± 2.6 mm, 5.4 ± 1.5 mm, and 4.5 ± 1.3 mm, respectively.

CONCLUSION

DL-Bio solves a general correlation model to improve the accuracy of the DVFs at the liver boundary. With improved boundary conditions, the accuracy of biomechanical modeling can be further increased for accurate intra-liver low-contrast tumor localization.

Detection of vessel bifurcations in CT scans for automatic objective assessment of deformable image registration accuracy

PURPOSE

Objective assessment of deformable image registration (DIR) accuracy often relies on the identification of anatomical landmarks in image pairs, a manual process known to be extremely time-expensive. The goal of this study is to propose a method to automatically detect vessel bifurcations in images and assess their use for the computation of target registration errors (TREs).

MATERIALS AND METHODS

Three image datasets were retrospectively analyzed. The first dataset included 10 pairs of inhale/exhale phases from lung 4DCTs and full inhale and exhale breath-hold CT scans from 10 patients presenting with chronic obstructive pulmonary disease, with 300 corresponding landmarks available for each case (DIR-Lab). The second dataset included six pairs of inhale/exhale phases from lung 4DCTs (POPI dataset), with 100 pairs of landmarks for each case. The third dataset included 28 pairs of pre/post-radiotherapy liver contrast-enhanced CT scans, each with five manually picked vessel bifurcation correspondences. For all images, the vasculature was autosegmented by computing and thresholding a vesselness image. Images of the vasculature centerline were computed, and bifurcations were detected based on centerline voxel neighbors' count. The vasculature segmentations were independently registered using a Demons algorithm between representations of their surface with distance maps. Detected bifurcations were considered as corresponding when distant by less than 5 mm after vasculature DIR. The selected pairs of bifurcations were used to calculate TRE after registration of the images considering three algorithms: rigid registration, Anaconda, and a Demons algorithm. For comparison with the ground truth, TRE values calculated using the automatically detected correspondences were interpolated in the whole organs to generate TRE maps. The performance of the method in automatically calculating TRE after image registration was quantified by measuring the correlation with the TRE obtained when using the ground truth landmarks.

RESULTS

The median Pearson correlation coefficients between ground truth TRE and corresponding values in the generated TRE maps were r = 0.81 and r = 0.67 for the lung and liver cases, respectively. The correlation coefficients between mean TRE for each case were r = 0.99 and r = 0.64 for the lung and liver cases, respectively.

CONCLUSION

For lungs or liver CT scans DIR, a strong correlation was obtained between TRE calculated using manually picked or landmarks automatically detected with the proposed method. This tool should be particularly useful in studies requiring assessing the reliability of a high number of DIRs.

Technical Note: Histological validation of anatomical imaging for breast modeling using a novel cryo-microtome

PURPOSE

Precise correlation between three-dimensional (3D) imaging and histology can aid biomechanical modeling of the breast. We develop a framework to register ex vivo images to histology using a novel cryo-fluorescence tomography (CFT) device.

METHODS

A formalin-fixed cadaveric breast specimen, including chest wall, was subjected to high-resolution magnetic resonance (MR) imaging. The specimen was then frozen and embedded in an optimal cutting temperature (OCT) compound. The OCT block was placed in a CFT device with an overhead camera and 50 μm thick slices were successively shaved off the block. After each shaving, the block-face was photographed. At select locations including connective/adipose tissue, muscle, skin, and fibroglandular tissue, 20 μm sections were transferred onto cryogenic tape for manual hematoxylin and eosin staining, histological assessment, and image capture. A 3D white-light image was automatically reconstructed from the photographs by aligning fiducial markers embedded in the OCT block. The 3D MR image, 3D white-light image, and photomicrographs were rigidly registered. Target registration errors (TREs) were computed based on 10 pairs of points marked at fibroglandular intersections. The overall MR-histology registration was used to compare the MR intensities at tissue extraction sites with a one-way analysis of variance.

RESULTS

The MR image to CFT-captured white-light image registration achieved a mean TRE of 0.73 ± 0.25 mm (less than the 1 mm MR slice resolution). The block-face white-light image and block-face photomicrograph registration showed visually indistinguishable alignment of anatomical structures and tissue boundaries. The MR intensities at the four tissue sites identified from histology differed significantly (p < 0.01). Each tissue pair, except the skin-connective/adipose tissue pair, also had significantly different MR intensities (p < 0.01).

CONCLUSIONS

Fine sectioning in a highly controlled imaging/sectioning environment enables accurate registration between the MR image and histology. Statistically significant differences in MR signal intensities between histological tissues are indicators for the specificity of correlation between MRI and histology.

A novel use of biomechanical model-based deformable image registration (DIR) for assessing colorectal liver metastases ablation outcomes

Purpose:

Colorectal cancer is the third most common form of cancer in the United States, and up to 60% of these patients develop liver metastasis. While hepatic resection is the curative treatment of choice, only 20% of patients are candidates at the time of diagnosis. While percutaneous thermal ablation (PTA) has demonstrated 24%–51% overall 5-year survival rates, assurance of sufficient ablation margin delivery (5 mm) can be challenging, with current methods of 2D distance measurement not ensuring 3D minimum margin. We hypothesized that biomechanical model-based deformable image registration (DIR) can reduce spatial uncertainties and differentiate local tumor progression (LTP) patients from LTP-free patients.

Methods:

We retrospectively acquired 30 patients (16 LTP and 14 LTP-free) at our institution who had undergone PTA and had a contrast-enhanced pre-treatment and post-ablation CT scan. Liver, disease, and ablation zone were manually segmented. Biomechanical model-based DIR between the pre-treatment and post-ablation CT mapped the gross tumor volume onto the ablation zone and measured 3D minimum delivered margin (MDM). An in-house cone-tracing algorithm determined if progression qualitatively collocated with insufficient 5 mm margin achieved.

Results:

Mann–Whitney U test showed a significant difference (p < 0.01) in MDM from the LTP and LTP-free groups. A total of 93% (13/14) of patients with LTP had a correlation between progression and missing 5 mm of margin volume.

Conclusions:

Biomechanical DIR is able to reduce spatial uncertainty and allow measurement of delivered 3D MDM. This minimum margin can help ensure sufficient ablation delivery, and our workflow can provide valuable information in a clinically useful timeframe.