Variations in inter-observer contouring and its impact on dosimetric and radiobiological parameters for intensity-modulated radiotherapy planning in treatment of localised prostate cancer

Evaluation of an automated clinical decision system with deep learning dose prediction and NTCP model for prostate cancer proton therapy

Objective.To evaluate the feasibility of using a deep learning dose prediction approach to identify patients who could benefit most from proton therapy based on the normal tissue complication probability (NTCP) model.Approach.Two 3D UNets were established to predict photon and proton doses. A dataset of 95 patients with localized prostate cancer was randomly partitioned into 55, 10, and 30 for training, validation, and testing, respectively. We selected NTCP models for late rectum bleeding and acute urinary urgency of grade 2 or higher to quantify the benefit of proton therapy. Propagated uncertainties of predicted ΔNTCPs resulting from the dose prediction errors were calculated. Patient selection accuracies for a single endpoint and a composite evaluation were assessed under different ΔNTCP thresholds.Main results.Our deep learning-based dose prediction technique can reduce the time spent on plan comparison from approximately 2 days to as little as 5 seconds. The expanded uncertainty of predicted ΔNTCPs for rectum and bladder endpoints propagated from the dose prediction error were 0.0042 and 0.0016, respectively, which is less than one-third of the acceptable tolerance. The averaged selection accuracies for rectum bleeding, urinary urgency, and composite evaluation were 90%, 93.5%, and 93.5%, respectively.Significance.Our study demonstrates that deep learning dose prediction and NTCP evaluation scheme could distinguish the NTCP differences between photon and proton treatment modalities. In addition, the dose prediction uncertainty does not significantly influence the decision accuracy of NTCP-based patient selection for proton therapy. Therefore, automated deep learning dose prediction and NTCP evaluation schemes can potentially be used to screen large patient populations and to avoid unnecessary delays in the start of prostate cancer radiotherapy in the future.

Random Forest Modeling of Acute Toxicity in Anal Cancer: Effects of Peritoneal Cavity Contouring Approaches on Model Performance

PURPOSE

Our purpose was to analyze the effect on gastrointestinal (GI) toxicity models when their dose-volume metrics predictors are derived from segmentations of the peritoneal cavity after different contouring approaches.

METHODS AND MATERIALS

A random forest machine learning approach was used to predict acute grade ≥3 GI toxicity from dose-volume metrics and clinicopathologic factors for 246 patients (toxicity incidence = 9.5%) treated with definitive chemoradiation for squamous cell carcinoma of the anus. Three types of random forest models were constructed based on different bowel bag segmentation approaches: (1) physician-delineated after Radiation Therapy Oncology Group (RTOG) guidelines, (2) autosegmented by a deep learning model (nnU-Net) following RTOG guidelines, and (3) autosegmented but spanning the entire bowel space. Each model type was evaluated using repeated cross-validation (100 iterations; 50%/50% training/test split). The performance of the models was assessed using area under the precision-recall curve (AUPRC) and the receiver operating characteristic curve (AUROCC), as well as optimal F1 score.

RESULTS

When following RTOG guidelines, the models based on the nnU-Net auto segmentations (mean values: AUROCC, 0.71 ± 0.07; AUPRC, 0.42 ± 0.09; F1 score, 0.46 ± 0.08) significantly outperformed (P < .001) those based on the physician-delineated contours (mean values: AUROCC, 0.67 ± 0.07; AUPRC, 0.34 ± 0.08; F1 score, 0.36 ± 0.07). When spanning the entire bowel space, the performance of the autosegmentation models improved considerably (mean values: AUROCC, 0.87 ± 0.05; AUPRC, 0.70 ± 0.09; F1 score, 0.68 ± 0.09).

CONCLUSIONS

Random forest models were superior at predicting acute grade ≥3 GI toxicity when based on RTOG-defined bowel bag autosegmentations rather than physician-delineated contours. Models based on autosegmentations spanning the entire bowel space show further considerable improvement in model performance. The results of this study should be further validated using an external data set.

Evaluating the Effectiveness of Deep Learning Contouring across Multiple Radiotherapy Centres

Background and purpose

Deep learning contouring (DLC) has the potential to decrease contouring time and variability of organ contours. This work evaluates the effectiveness of DLC for prostate and head and neck across four radiotherapy centres using a commercial system.

Materials and methods

Computed tomography scans of 123 prostate and 310 head and neck patients were evaluated. Besides one head and neck model, generic DLC models were used. Contouring time using centres' existing clinical methods and contour editing time after DLC were compared. Timing was evaluated using paired and non-paired studies. Commercial software or in-house scripts assessed dice similarity coefficient (DSC) and distance to agreement (DTA). One centre assessed head and neck inter-observer variability.

Results

The mean contouring time saved for prostate structures using DLC compared to the existing clinical method was 5.9 ± 3.5 min. The best agreement was shown for the femoral heads (median DSC 0.92 ± 0.03, median DTA 1.5 ± 0.3 mm) and the worst for the rectum (median DSC 0.68 ± 0.04, median DTA 4.6 ± 0.6 mm). The mean contouring time saved for head and neck structures using DLC was 16.2 ± 8.6 min. For one centre there was no DLC time-saving compared to an atlas-based method. DLC contours reduced inter-observer variability compared to manual contours for the brainstem, left parotid gland and left submandibular gland.

Conclusions

Generic prostate and head and neck DLC models can provide time-savings which can be assessed with paired or non-paired studies to integrate with clinical workload. Reducing inter-observer variability potential has been shown.

Knowledge-based quality control of organ delineations in radiation therapy

PURPOSE

To reduce the likelihood of errors in organ delineations used for radiotherapy treatment planning, a knowledge-based quality control (KBQC) system, which discriminates between valid and anomalous delineations is developed.

METHOD AND MATERIALS

The KBQC is comprised of a group-wise inference system and anomaly detection modules trained using historical priors from 296 locally advanced lung and prostate cancer patient computational tomographies (CTs). The inference system discriminates different organs based on shape, relational, and intensity features. For a given delineated image set, the inference system solves a combinatorial optimization problem that results in an organ group whose relational features follow those of the training set considering the posterior probabilities obtained from support vector machine (SVM), discriminant subspace ensemble (DSE), and artificial neural network (ANN) classifiers. These classifiers are trained on nonrelational features with a 10-fold cross-validation scheme. The anomaly detection module is a bank of ANN autoencoders, each corresponding with an organ, trained on nonrelational features. A heuristic rule detects anomalous organs that exceed predefined organ-specific tolerances for the feature reconstruction error and the classifier's posterior probabilities. Independent data sets with anomalous delineations were used to test the overall performance of the KBQC system. The anomalous delineations were manually manipulated, computer-generated, or propagated based on a transformation obtained by imperfect registrations. Both peer-review-based scoring system and shape similarity coefficient (DSC) were used to label regions of interest (ROIs) as normal or anomalous in two independent test cohorts.

RESULTS

The accuracy of the classifiers was ≥ $\ge$ 99.8%, and the minimum per-class F1-scores were 0.99, 0.99, and 0.98 for SVM, DSE, and ANN, respectively. The group-wise inference system reduced the miss-classification likelihood for the test data set with anomalous delineations compared to each individual classifier and a fused classifier that used the average posterior probability of all classifiers. For 15 independent locally advanced lung patients, the system detected > $>$ 79% of the anomalous ROIs. For 1320 auto-segmented abdominopelvic organs, the anomaly detection system identified anomalous delineations, which also had low Dice similarity coefficient values with respect to manually delineated organs in the training data set.

CONCLUSION

The KBQC system detected anomalous delineations with superior accuracy compared to classification methods that judge only based on posterior probabilities.

Quantifying the dosimetric impact of organ-at-risk delineation variability in head and neck radiation therapy in the context of patient setup uncertainty

The purpose of this study was to quantify the potential dosimetric impact of delineation variability (DV) in head and neck radiation therapy (RT) when inherent patient setup variability (SV) is also considered. The impact of DV was assessed by generating plans with multiple structure sets, cross-evaluating them, including SV, across sets, and determining P _PQM: the probability of achieving organ-specific plan quality metrics (PQM). DV was incorporated by: (1) using multiple organ at risk (OAR) structure sets delineated by independent manual observers; and (2) randomly perturbing manually generated OARs to generate alternatives with varying levels of uncertainty (low, medium, and high DV). For each structure set, independent VMAT plans were auto-generated to meet clinical PQMs. Each plan was cross-evaluated using OARs from multiple structure sets with simulated SV including per-fraction random (σ _s) and per-treatment-course systematic (Σ_s) setup errors. The dosimetric impact of DV was assessed by examining P _PQM with and without SV/DV. Clinically significant differences were defined by those that exceeded differences caused by a  +2% output variation. Without including SV, simulated DV at the medium level reduced P _PQM by an average of 5.5% for all OARs with D _max PQMs. This reduction decreased to 2.8% for SV  =  2 mm and 2.4% for SV  =  4 mm (the average P _PQM reduction due to 2% output errors was 2.7%). For OARs with D _mean PQMs, the average P _PQM reduction was 0.9% for SV  =  0 and  ⩽0.1% for SV  ⩾  2 mm. The effect of DV was larger for OARs that directly abutted a target volume than for those that did not. These trends were also observed with real DV from multi-observer delineations. The dosimetric impact of DV appeared to decrease when random and systematic SV was considered. Sensitivity to DV was affected by OAR objective type (i.e. D _mean versus D _max objectives) as well as distance from the target volume.

Clinical adequacy assessment of autocontours for prostate IMRT with meaningful endpoints

PURPOSE

To determine if radiation treatment plans created based on autosegmented (AS) regions-of-interest (ROI)s are clinically equivalent to plans created based on manually segmented ROIs, where equivalence is evaluated using probabilistic dosimetric metrics and probabilistic biological endpoints for prostate IMRT.

METHOD AND MATERIALS

Manually drawn contours and autosegmented ROIs were created for 167 CT image sets acquired from 19 prostate patients. Autosegmentation was performed utilizing Pinnacle's Smart Probabilistic Image Contouring Engine. For each CT set, 78 Gy/39 fraction 7-beam IMRT treatment plans with 1 cm CTV-to-PTV margins were created for each of the three contour scenarios; PMD using manually delineated (MD) ROIs, PAS using autosegmented ROIs, and PAM using autosegmented organ-at-risks (OAR)s and the manually drawn target. For each plan, 1000 virtual treatment simulations with different systematic errors for each simulation and a different random error for each fraction were performed. The statistical probability of achieving dose-volume metrics (coverage probability (CP)), expectation values for normal tissue complication probability (NTCP), and tumor control probability (TCP) metrics for all possible cross-evaluation pairs of ROI types and planning scenarios were reported. In evaluation scenarios, the root mean square loss (RMSL) and maximum absolute loss (MAL) of coverage probability of dose-volume objectives, E[TCP], and E[NTCP] were compared with respect to the base plan created and evaluated with manually drawn contours.

RESULTS

Femoral head dose objectives were satisfied in all situations, as well as the maximum dose objectives for all ROIs. Bladder metrics were within the clinical coverage tolerances except D35Gy for the autosegmented plan evaluated with the manual contours. Dosimetric indices for CTV and rectum could be highly compromised when the definition of the ROIs switched from manually delineated to autosegmented. Seventy-two percent of CT image sets satisfied the worst-case CP thresholds for all dosimetric objectives in all scenarios, the percentage dropped to 50% if biological indices were taken into account. Among evaluation scenarios, (MD,PAM ) bore the highest resemblance to (MD,PMD ) where 99% and 88% of cases met all CP thresholds for bladder and rectum, respectively.

CONCLUSIONS

When including daily setup variations in prostate IMRT, the dose-volume metric CP, and biological indices of ROIs were approximately equivalent for the plans created based on manually drawn targets and autosegmented OARs in 88% of cases. The accuracy of autosegmented prostates and rectums are impediment to attain statistically equivalent plans created based on manually drawn ROIs.

Coverage-based treatment planning to accommodate delineation uncertainties in prostate cancer treatment

PURPOSE

To compare two coverage-based planning (CP) techniques with fixed margin-based (FM) planning for high-risk prostate cancer treatments, with the exclusive consideration of the dosimetric impact of delineation uncertainties of target structures and normal tissues.

METHODS

In this work, 19-patient data sets were involved. To estimate structure dose for each delineated contour under the influence of interobserver contour variability and CT image quality limitations, 1000 alternative structures were simulated by an average-surface-of-standard-deviation model, which utilized the patient-specific information of delineated structure and CT image contrast. An IMRT plan with zero planning-target-volume (PTV) margin on the delineated prostate and seminal vesicles [clinical-target-volume (CTV prostate) and CTVSV] was created and dose degradation due to contour variability was quantified by the dosimetric consequences of 1000 alternative structures. When D98 failed to achieve a 95% coverage probability objective D98,95 ≥ 78 Gy (CTV prostate) or D98,95 ≥ 66 Gy (CTVSV), replanning was performed using three planning techniques: (1) FM (PTV prostate margin = 4,5,6 mm and PTVSV margin = 4,5,7 mm for RL, PA, and SI directions, respectively), (2) CPOM which optimized uniform PTV margins for CTV prostate and CTVSV to meet the D98,95 objectives, and (3) CPCOP which directly optimized coverage-based objectives for all the structures. These plans were intercompared by computing percentile dose-volume histograms and tumor-control probability/normal tissue complication probability (TCP/NTCP) distributions.

RESULTS

Inherent contour variability resulted in unacceptable CTV coverage for the zero-PTV-margin plans for all patients. For plans designed to accommodate contour variability, 18/19 CP plans were most favored by achieving desirable D98,95 and TCP/NTCP values. The average improvement of probability of complication free control was 9.3% for CPCOP plans and 3.4% for CPOM plans.

CONCLUSIONS

When the delineation uncertainties need to be considered for prostate patients, CP techniques can produce more desirable plans than FM plans for most patients. The relative advantages between CPCOP and CPOM techniques are patient specific.