Deep Learning-Based Dose Prediction for Automated, Individualized Quality Assurance of Head and Neck Radiation Therapy Plans

Analyzing the Relationship between Dose and Geometric Agreement Metrics for Auto-Contouring in Head and Neck Normal Tissues

This study aimed to determine the relationship between geometric and dosimetric agreement metrics in head and neck (H&N) cancer radiotherapy plans. A total 287 plans were retrospectively analyzed, comparing auto-contoured and clinically used contours using a Dice similarity coefficient (DSC), surface DSC (sDSC), and Hausdorff distance (HD). Organs-at-risk (OARs) with ≥200 cGy dose differences from the clinical contour in terms of Dmax (D0.01cc) and Dmean were further examined against proximity to the planning target volume (PTV). A secondary set of 91 plans from multiple institutions validated these findings. For 4995 contour pairs across 19 OARs, 90% had a DSC, sDSC, and HD of at least 0.75, 0.86, and less than 7.65 mm, respectively. Dosimetrically, the absolute difference between the two contour sets was <200 cGy for 95% of OARs in terms of Dmax and 96% in terms of Dmean. In total, 97% of OARs exhibiting significant dose differences between the clinically edited contour and auto-contour were within 2.5 cm PTV regardless of geometric agreement. There was an approximately linear trend between geometric agreement and identifying at least 200 cGy dose differences, with higher geometric agreement corresponding to a lower fraction of cases being identified. Analysis of the secondary dataset validated these findings. Geometric indices are approximate indicators of contour quality and identify contours exhibiting significant dosimetric discordance. For a small subset of OARs within 2.5 cm of the PTV, geometric agreement metrics can be misleading in terms of contour quality.

Assessment of bias in scoring of AI-based radiotherapy segmentation and planning studies using modified TRIPOD and PROBAST guidelines as an example

BACKGROUND AND PURPOSE

Studies investigating the application of Artificial Intelligence (AI) in the field of radiotherapy exhibit substantial variations in terms of quality. The goal of this study was to assess the amount of transparency and bias in scoring articles with a specific focus on AI based segmentation and treatment planning, using modified PROBAST and TRIPOD checklists, in order to provide recommendations for future guideline developers and reviewers.

MATERIALS AND METHODS

The TRIPOD and PROBAST checklist items were discussed and modified using a Delphi process. After consensus was reached, 2 groups of 3 co-authors scored 2 articles to evaluate usability and further optimize the adapted checklists. Finally, 10 articles were scored by all co-authors. Fleiss' kappa was calculated to assess the reliability of agreement between observers.

RESULTS

Three of the 37 TRIPOD items and 5 of the 32 PROBAST items were deemed irrelevant. General terminology in the items (e.g., multivariable prediction model, predictors) was modified to align with AI-specific terms. After the first scoring round, further improvements of the items were formulated, e.g., by preventing the use of sub-questions or subjective words and adding clarifications on how to score an item. Using the final consensus list to score the 10 articles, only 2 out of the 61 items resulted in a statistically significant kappa of 0.4 or more demonstrating substantial agreement. For 41 items no statistically significant kappa was obtained indicating that the level of agreement among multiple observers is due to chance alone.

CONCLUSION

Our study showed low reliability scores with the adapted TRIPOD and PROBAST checklists. Although such checklists have shown great value during development and reporting, this raises concerns about the applicability of such checklists to objectively score scientific articles for AI applications. When developing or revising guidelines, it is essential to consider their applicability to score articles without introducing bias.

Optimal timing of re-planning for head and neck adaptive radiotherapy

BACKGROUND AND PURPOSE

Adaptive radiotherapy (ART) relies on re-planning to correct treatment variations, but the optimal timing of re-planning to account for dose changes in head and neck organs at risk (OARs) is still under investigation. We aimed to find out the optimal timing of re-planning in head and neck ART.

MATERIALS AND METHODS

A total of 110 head and neck cancer patients were retrospectively enrolled. A semi auto-segmentation method was applied to obtain the weekly mean dose (Dmean) to OARs. The K-nearest-neighbour method was used for missing data imputation of weekly Dmean. A dose deviation map was built using the planning Dmean and weekly Dmean values and then used to simulate different ART scenarios consisting of 1 to 6 re-plannings. The difference between accumulated Dmean and planning Dmean before re-planning (ΔDmean_acc_noART) and after re-planning (ΔDmean_acc_ART) were evaluated and compared.

RESULTS

Among all the OARs, supraglottic showed the largest ΔDmean_acc_noART (1.23 ± 3.13 Gy) and most cases of ΔDmean_acc_noART > 3 Gy (26 patients). The 3rd week is suggested in the optimal timing of re-planning for 10 OARs. For all the organs except arytenoid, 2 re-plannings were able to guarantee the ΔDmean_acc_ART below 3 Gy while the average |ΔDmean_acc_ART| was below 1 Gy. ART scenarios of 2_4, 3_4, 3_5 (week of re-planning separated with "_") were able to guarantee ΔDmean_acc_ART of 99 % of patients below 3 Gy simultaneously for 19 OARs.

CONCLUSIONS

The optimal timing of re-planning was suggested for different organs at risk in head and neck adaptive radiotherapy. Generic scenarios of timing and frequency for re-planning can be applied to guarantee the increase of accumulated mean dose within 3 Gy simultaneously for multiple organs.

Deep learning for high-resolution dose prediction in high dose rate brachytherapy for breast cancer treatment

Objective.Monte Carlo (MC) simulations are the benchmark for accurate radiotherapy dose calculations, notably in patient-specific high dose rate brachytherapy (HDR BT), in cases where considering tissue heterogeneities is critical. However, the lengthy computational time limits the practical application of MC simulations. Prior research used deep learning (DL) for dose prediction as an alternative to MC simulations. While accurate dose predictions akin to MC were attained, graphics processing unit limitations constrained these predictions to large voxels of 3 mm × 3 mm × 3 mm. This study aimed to enable dose predictions as accurate as MC simulations in 1 mm × 1 mm × 1 mm voxels within a clinically acceptable timeframe.Approach.Computed tomography scans of 98 breast cancer patients treated with Iridium-192-based HDR BT were used: 70 for training, 14 for validation, and 14 for testing. A new cropping strategy based on the distance to the seed was devised to reduce the volume size, enabling efficient training of 3D DL models using 1 mm × 1 mm × 1 mm dose grids. Additionally, novel DL architecture with layer-level fusion were proposed to predict MC simulated dose to medium-in-medium (Dm,m). These architectures fuse information from TG-43 dose to water-in-water (Dw,w) with patient tissue composition at the layer-level. Different inputs describing patient body composition were investigated.Main results.The proposed approach demonstrated state-of-the-art performance, on par with the MCDm,mmaps, but 300 times faster. The mean absolute percent error for dosimetric indices between the MC and DL-predicted complete treatment plans was 0.17% ± 0.15% for the planning target volumeV100, 0.30% ± 0.32% for the skinD2cc, 0.82% ± 0.79% for the lungD2cc, 0.34% ± 0.29% for the chest wallD2ccand 1.08% ± 0.98% for the heartD2cc.Significance.Unlike the time-consuming MC simulations, the proposed novel strategy efficiently converts TG-43Dw,wmaps into preciseDm,mmaps at high resolution, enabling clinical integration.

Artificial neural network-assisted prediction of radiobiological indices in head and neck cancer

Background and purpose

We proposed an artificial neural network model to predict radiobiological parameters for the head and neck squamous cell carcinoma patients treated with radiation therapy. The model uses the tumor specification, demographics, and radiation dose distribution to predict the tumor control probability and the normal tissue complications probability. These indices are crucial for the assessment and clinical management of cancer patients during treatment planning.

Methods

Two publicly available datasets of 31 and 215 head and neck squamous cell carcinoma patients treated with conformal radiation therapy were selected. The demographics, tumor specifications, and radiation therapy treatment parameters were extracted from the datasets used as inputs for the training of perceptron. Radiobiological indices are calculated by open-source software using dosevolume histograms from radiation therapy treatment plans. Those indices were used as output in the training of a single-layer neural network. The distribution of data used for training, validation, and testing purposes was 70, 15, and 15%, respectively.

Results

The best performance of the neural network was noted at epoch number 32 with the mean squared error of 0.0465. The accuracy of the prediction of radiobiological indices by the artificial neural network in training, validation, and test phases were determined to be 0.89, 0.87, and 0.82, respectively. We also found that the percentage volume of parotid inside the planning target volume is the significant parameter for the prediction of normal tissue complications probability.

Conclusion

We believe that the model has significant potential to predict radiobiological indices and help clinicians in treatment plan evaluation and treatment management of head and neck squamous cell carcinoma patients.

A new predictive parameter for dose-volume metrics in intensity-modulated radiation therapy planning for prostate cancer: Initial phantom study

BACKGROUND

Organ-at-risk (OAR) sparing is often assessed using an overlap volume-based parameter, defined as the ratio of the volume of OAR that overlaps the planning target volume (PTV) to the whole OAR volume. However, this conventional overlap-based predictive parameter (COPP) does not consider the volume relationship between the PTV and OAR.

PURPOSE

We propose a new overlap-based predictive parameter that consider the PTV volume. The effectiveness of proposed overlap-based predictive parameter (POPP) is evaluated compared with COPP.

METHODS

We defined as POPP = (overlap volume between OAR and PTV/OAR volume) × (PTV volume/OAR volume). We generated intensity modulated radiation therapy (IMRT) based on step and shoot technique, and volumetric modulated arc therapy (VMAT) plans with the Auto-Planning module of Pinnacle3 treatment planning system (v14.0, Philips Medical Systems, Fitchburg, WI) using the American Association of Physicists in Medicine Task Group (TG119) prostate phantom. The relationship between the position and size of the prostate phantom was systematically modified to simulate various geometric arrangements. The correlation between overlap-based predictive parameters (COPP and POPP) and dose-volume metrics (mean dose, V70Gy, V60Gy, and V37.5 Gy for rectum and bladder) was investigated using linear regression analysis.

RESULTS

Our results indicated POPP was better than COPP in predicting intermediate-dose metrics. The bladder results showed a trend similar to that of the rectum. The correlation coefficient of POPP was significantly greater than that of COPP in < 62 Gy (82% of the prescribed dose) region for IMRT and in < 55 Gy (73% of the prescribed dose) region for VMAT regarding the rectum (p < 0.05).

CONCLUSIONS

POPP is superior to COPP for creating predictive models at an intermediate-dose level. Because rectal bleeding and bladder toxicity can be associated with intermediate-doses as well as high-doses, it is important to predict dose-volume metrics for various dose levels. POPP is a useful parameter for predicting dose-volume metrics and assisting the generation of treatment plans.

Predicting voxel-level dose distributions of single-isocenter volumetric modulated arc therapy treatment plan for multiple brain metastases

Purpose

Brain metastasis is a common, life-threatening neurological problem for patients with cancer. Single-isocenter volumetric modulated arc therapy (VMAT) has been popularly used due to its highly conformal dose and short treatment time. Accurate prediction of its dose distribution can provide a general standard for evaluating the quality of treatment plan. In this study, a deep learning model is applied to the dose prediction of a single-isocenter VMAT treatment plan for radiotherapy of multiple brain metastases.

Method

A U-net with residual networks (U-ResNet) is employed for the task of dose prediction. The deep learning model is first trained from a database consisting of hundreds of historical treatment plans. The 3D dose distribution is then predicted with the input of the CT image and contours of regions of interest (ROIs). A total of 150 single-isocenter VMAT plans for multiple brain metastases are used for training and testing. The model performance is evaluated based on mean absolute error (MAE) and mean absolute differences of multiple dosimetric indexes (DIs), including (D max and D mean) for OARs, (D 98, D 95, D 50, and D 2) for PTVs, homogeneity index, and conformity index. The similarity between the predicted and clinically approved plan dose distribution is also evaluated.

Result

For 20 tested patients, the largest and smallest MAEs are 3.3% ± 3.6% and 1.3% ± 1.5%, respectively. The mean MAE for the 20 tested patients is 2.2% ± 0.7%. The mean absolute differences of D 98, D 95, D 50, and D2 for PTV60, PTV52, PTV50, and PTV40 are less than 2.5%, 3.0%, 2.0%, and 3.0%, respectively. The prediction accuracy of OARs for D max and D mean is within 3.2% and 1.2%, respectively. The average DSC ranges from 0.86 to 1 for all tested patients.

Conclusion

U-ResNet is viable to produce accurate dose distribution that is comparable to those of the clinically approved treatment plans. The predicted results can be used to improve current treatment planning design, plan quality, efficiency, etc.

Deep learning-based dose prediction to improve the plan quality of volumetric modulated arc therapy for gynecologic cancers

BACKGROUND

In recent years, deep-learning models have been used to predict entire three-dimensional dose distributions. However, the usability of dose predictions to improve plan quality should be further investigated.

PURPOSE

To develop a deep-learning model to predict high-quality dose distributions for volumetric modulated arc therapy (VMAT) plans for patients with gynecologic cancer and to evaluate their usability in driving plan quality improvements.

METHODS

A total of 79 VMAT plans for the female pelvis were used to train (47 plans), validate (16 plans), and test (16 plans) 3D dense dilated U-Net models to predict 3D dose distributions. The models received the normalized CT scan, dose prescription, and target and normal tissue contours as inputs. Three models were used to predict the dose distributions for plans in the test set. A radiation oncologist specializing in the treatment of gynecologic cancers scored the test set predictions using a 5-point scale (5, acceptable as-is; 4, prefer minor edits; 3, minor edits needed; 2, major edits needed; and 1, unacceptable). The clinical plans for which the dose predictions indicated that improvements could be made were reoptimized with constraints extracted from the predictions.

RESULTS

The predicted dose distributions in the test set were of comparable quality to the clinical plans. The mean voxel-wise dose difference was -0.14 ± 0.46 Gy. The percentage dose differences in the predicted target metrics ofD 1 % ${D}_{1{\mathrm{\% }}}$ andD 98 % ${D}_{98{\mathrm{\% }}}$ were -1.05% ± 0.59% and 0.21% ± 0.28%, respectively. The dose differences in the predicted organ at risk mean and maximum doses were -0.30 ± 1.66 Gy and -0.42 ± 2.07 Gy, respectively. A radiation oncologist deemed all of the predicted dose distributions clinically acceptable; 12 received a score of 5, and four received a score of 4. Replanning of flagged plans (five plans) showed that the original plans could be further optimized to give dose distributions close to the predicted dose distributions.

CONCLUSIONS

Deep-learning dose prediction can be used to predict high-quality and clinically acceptable dose distributions for VMAT female pelvis plans, which can then be used to identify plans that can be improved with additional optimization.

Consistency in contouring of organs at risk by artificial intelligence vs oncologists in head and neck cancer patients

BACKGROUND

In the Danish Head and Neck Cancer Group (DAHANCA) 35 trial, patients are selected for proton treatment based on simulated reductions of Normal Tissue Complication Probability (NTCP) for proton compared to photon treatment at the referring departments. After inclusion in the trial, immobilization, scanning, contouring and planning are repeated at the national proton centre. The new contours could result in reduced expected NTCP gain of the proton plan, resulting in a loss of validity in the selection process. The present study evaluates if contour consistency can be improved by having access to AI (Artificial Intelligence) based contours.

MATERIALS AND METHODS

The 63 patients in the DAHANCA 35 pilot trial had a CT from the local DAHANCA centre and one from the proton centre. A nationally validated convolutional neural network, based on nnU-Net, was used to contour OARs on both scans for each patient. Using deformable image registration, local AI and oncologist contours were transferred to the proton centre scans for comparison. Consistency was calculated with the Dice Similarity Coefficient (DSC) and Mean Surface Distance (MSD), comparing contours from AI to AI and oncologist to oncologist, respectively. Two NTCP models were applied to calculate NTCP for xerostomia and dysphagia.

RESULTS

The AI contours showed significantly better consistency than the contours by oncologists. The median and interquartile range of DSC was 0.85 [0.78 - 0.90] and 0.68 [0.51 - 0.80] for AI and oncologist contours, respectively. The median and interquartile range of MSD was 0.9 mm [0.7 - 1.1] mm and 1.9 mm [1.5 - 2.6] mm for AI and oncologist contours, respectively. There was no significant difference in Δ NTCP.

CONCLUSIONS

The study showed that OAR contours made by the AI algorithm were more consistent than those made by oncologists. No significant impact on the Δ NTCP calculations could be discerned.

Automated evaluation for rapid implementation of knowledge-based radiotherapy planning models

PURPOSE

Knowledge-based planning (KBP) offers the ability to predict dose-volume metrics based on information extracted from previous plans, reducing plan variability and improving plan quality. As clinical integration of KBP is increasing there is a growing need for quantitative evaluation of KBP models. A .NET-based application, RapidCompare, was created for automated plan creation and analysis of Varian RapidPlan models.

METHODS

RapidCompare was designed to read calculation parameters and a list of reference plans. The tool copies the reference plan field geometry and structure set, applies the RapidPlan model, optimizes the KBP plan, and generates data for quantitative evaluation of dose-volume metrics. A cohort of 85 patients, divided into training (50), testing (10), and validation (25) groups, was used to demonstrate the utility of RapidCompare. After training and tuning, the KBP model was paired with three different optimization templates to compare various planning strategies in the validation cohort. All templates used the same set of constraints for the planning target volume (PTV). For organs-at-risk, the optimization template provided constraints using the whole dose-volume histogram (DVH), fixed-dose/volume points, or generalized equivalent uniform dose (gEUD). The resulting plans from each optimization approach were compared using DVH metrics.

RESULTS

RapidCompare allowed for the automated generation of 75 total plans for comparison with limited manual intervention. In comparing optimization techniques, the Dose/Volume and Lines optimization templates generated plans with similar DVH metrics, with a slight preference for the Lines technique with reductions in heart V30Gy and spinal cord max dose. The gEUD model produced high target heterogeneity.

CONCLUSION

Automated evaluation allowed for the exploration of multiple optimization templates in a larger validation cohort than would have been feasible using a manual approach. A final KBP model using line optimization objectives produced the highest quality plans without human intervention.

Automated contouring and statistical process control for plan quality in a breast clinical trial

Background and purpose

Automatic review of breast plan quality for clinical trials is time-consuming and has some unique challenges due to the lack of target contours for some planning techniques. We propose using an auto-contouring model and statistical process control to independently assess planning consistency in retrospective data from a breast radiotherapy clinical trial.

Materials and methods

A deep learning auto-contouring model was created and tested quantitatively and qualitatively on 104 post-lumpectomy patients' computed tomography images (nnUNet; train/test: 80/20). The auto-contouring model was then applied to 127 patients enrolled in a clinical trial. Statistical process control was used to assess the consistency of the mean dose to auto-contours between plans and treatment modalities by setting control limits within three standard deviations of the data's mean. Two physicians reviewed plans outside the limits for possible planning inconsistencies.

Results

Mean Dice similarity coefficients comparing manual and auto-contours was above 0.7 for breast clinical target volume, supraclavicular and internal mammary nodes. Two radiation oncologists scored 95% of contours as clinically acceptable. The mean dose in the clinical trial plans was more variable for lymph node auto-contours than for breast, with a narrower distribution for volumetric modulated arc therapy than for 3D conformal treatment, requiring distinct control limits. Five plans (5%) were flagged and reviewed by physicians: one required editing, two had clinically acceptable variations in planning, and two had poor auto-contouring.

Conclusions

An automated contouring model in a statistical process control framework was appropriate for assessing planning consistency in a breast radiotherapy clinical trial.

Intentional deep overfit learning for patient-specific dose predictions in adaptive radiotherapy

BACKGROUND

The framework of adaptive radiation therapy (ART) was crafted to address the underlying sources of intra-patient variation that were observed throughout numerous patients' radiation sessions. ART seeks to minimize the consequential dosimetric uncertainty resulting from this daily variation, commonly through treatment planning re-optimization. Re-optimization typically consists of manual evaluation and modification of previously utilized optimization criteria. Ideally, frequent treatment plan adaptation through re-optimization on each day's computed tomography (CT) scan may improve dosimetric accuracy and minimize dose delivered to organs at risk (OARs) as the planning target volume (PTV) changes throughout the course of treatment.

PURPOSE

Re-optimization in its current form is time-consuming and inefficient. In response to this ART bottleneck, we propose a deep learning based adaptive dose prediction model that utilizes a head and neck (H&N) patient's initial planning data to fine-tune a previously trained population model towards a patient-specific model. Our fine-tuned, patient-specific (FT-PS) model, which is trained using the intentional deep overfit learning (IDOL) method, may enable clinicians and treatment planners to rapidly evaluate relevant dosimetric changes daily and re-optimize accordingly.

METHODS

An adaptive population (AP) model was trained using adaptive data from 33 patients. Separately, 10 patients were selected for training FT-PS models. The previously trained AP model was utilized as the base model weights prior to re-initializing model training for each FT-PS model. Ten FT-PS models were separately trained by fine-tuning the previous model weights based on each respective patient's initial treatment plan. From these 10 patients, 26 ART treatment plans were withheld from training as the test dataset for retrospective evaluation of dose prediction performance between the AP and FT-PS models. Each AP and FT-PS dose prediction was compared against the ground truth dose distribution as originally generated during the patient's course of treatment. Mean absolute percent error (MAPE) evaluated the dose differences between a model's prediction and the ground truth.

RESULTS

MAPE was calculated within the 10% isodose volume region of interest for each of the AP and FT-PS models dose predictions and averaged across all test adaptive sessions, yielding 5.759% and 3.747% respectively. MAPE differences were compared between AP and FT-PS models across each test session in a test of statistical significance. The differences were statistically significant in a paired t-test with two-tailed p-value equal to3.851 × 10 - 9 $3.851 \times {10}^{ - 9}$ and 95% confidence interval (CI) equal to [-2.483, -1.542]. Furthermore, MAPE was calculated using each individually segmented structure as an ROI. Nineteen of 24 structures demonstrated statistically significant differences between the AP and FT-PS models.

CONCLUSION

We utilized the IDOL method to fine-tune a population-based dose prediction model into an adaptive, patient-specific model. The averaged MAPE across the test dataset was 5.759% for the population-based model versus 3.747% for the fine-tuned, patient-specific model, and the difference in MAPE between models was found to be statistically significant. Our work demonstrates the feasibility of patient-specific models in adaptive radiotherapy, and offers unique clinical benefit by utilizing initial planning data that contains the physician's treatment intent.

Deep-Learning-Based Dose Predictor for Glioblastoma-Assessing the Sensitivity and Robustness for Dose Awareness in Contouring

External beam radiation therapy requires a sophisticated and laborious planning procedure. To improve the efficiency and quality of this procedure, machine-learning models that predict these dose distributions were introduced. The most recent dose prediction models are based on deep-learning architectures called 3D U-Nets that give good approximations of the dose in 3D almost instantly. Our purpose was to train such a 3D dose prediction model for glioblastoma VMAT treatment and test its robustness and sensitivity for the purpose of quality assurance of automatic contouring. From a cohort of 125 glioblastoma (GBM) patients, VMAT plans were created according to a clinical protocol. The initial model was trained on a cascaded 3D U-Net. A total of 60 cases were used for training, 15 for validation and 20 for testing. The prediction model was tested for sensitivity to dose changes when subject to realistic contour variations. Additionally, the model was tested for robustness by exposing it to a worst-case test set containing out-of-distribution cases. The initially trained prediction model had a dose score of 0.94 Gy and a mean DVH (dose volume histograms) score for all structures of 1.95 Gy. In terms of sensitivity, the model was able to predict the dose changes that occurred due to the contour variations with a mean error of 1.38 Gy. We obtained a 3D VMAT dose prediction model for GBM with limited data, providing good sensitivity to realistic contour variations. We tested and improved the model's robustness by targeted updates to the training set, making it a useful technique for introducing dose awareness in the contouring evaluation and quality assurance process.

Automated Contouring and Planning in Radiation Therapy: What Is 'Clinically Acceptable'?

Developers and users of artificial-intelligence-based tools for automatic contouring and treatment planning in radiotherapy are expected to assess clinical acceptability of these tools. However, what is 'clinical acceptability'? Quantitative and qualitative approaches have been used to assess this ill-defined concept, all of which have advantages and disadvantages or limitations. The approach chosen may depend on the goal of the study as well as on available resources. In this paper, we discuss various aspects of 'clinical acceptability' and how they can move us toward a standard for defining clinical acceptability of new autocontouring and planning tools.