Evaluation of dose-volume histogram prediction for organ-at risk and planning target volume based on machine learning

Assessment of bias in scoring of AI-based radiotherapy segmentation and planning studies using modified TRIPOD and PROBAST guidelines as an example

BACKGROUND AND PURPOSE

Studies investigating the application of Artificial Intelligence (AI) in the field of radiotherapy exhibit substantial variations in terms of quality. The goal of this study was to assess the amount of transparency and bias in scoring articles with a specific focus on AI based segmentation and treatment planning, using modified PROBAST and TRIPOD checklists, in order to provide recommendations for future guideline developers and reviewers.

MATERIALS AND METHODS

The TRIPOD and PROBAST checklist items were discussed and modified using a Delphi process. After consensus was reached, 2 groups of 3 co-authors scored 2 articles to evaluate usability and further optimize the adapted checklists. Finally, 10 articles were scored by all co-authors. Fleiss' kappa was calculated to assess the reliability of agreement between observers.

RESULTS

Three of the 37 TRIPOD items and 5 of the 32 PROBAST items were deemed irrelevant. General terminology in the items (e.g., multivariable prediction model, predictors) was modified to align with AI-specific terms. After the first scoring round, further improvements of the items were formulated, e.g., by preventing the use of sub-questions or subjective words and adding clarifications on how to score an item. Using the final consensus list to score the 10 articles, only 2 out of the 61 items resulted in a statistically significant kappa of 0.4 or more demonstrating substantial agreement. For 41 items no statistically significant kappa was obtained indicating that the level of agreement among multiple observers is due to chance alone.

CONCLUSION

Our study showed low reliability scores with the adapted TRIPOD and PROBAST checklists. Although such checklists have shown great value during development and reporting, this raises concerns about the applicability of such checklists to objectively score scientific articles for AI applications. When developing or revising guidelines, it is essential to consider their applicability to score articles without introducing bias.

Predictive modeling of dose-volume parameters of carcinoma tongue cases using machine learning models

The aim of this study is to create a single institution-based machine learning model for a dose prediction generation tool for post-operative carcinoma of the tongue cases prospectively. Intensity-modulated radiotherapy (IMRT) plans for 20 patients with carcinoma of the tongue were generated using the Eclipse treatment planning system. A machine learning model was generated using a Python 3.10 computer language in a Jupyter notebook using Anaconda software. The PTVs and OARs doses obtained from the clinical treatment plans were used as a primary dataset. Machine learning models are built with two different datasets (10 and 20) for each selected volume. Volumes from 10 new sets of patients were fed into the software for predicting the corresponding dose values. Through the input given, the plan generated dose values of 10 patients were compared with the predicted outcomes of the 10 and 20 dataset models. The model created using the PTVs volume data predicted the dose values with increased accuracy. By verifying the model prediction with the TPS generated value, both the 10 and 20 dataset models predict all the 10 PTVs data within an error bound of 3% and most of the OARs data within an error bound of 5%. The dosimetric features implemented in the machine learning models reasonably predict both the PTVs dose parameter and OARs constraints and give confidence in decision-making during the clinical planning process.

Prediction of dose deposition matrix using voxel features driven machine learning approach

OBJECTIVES

A dose deposition matrix (DDM) prediction method using several voxel features and a machine learning (ML) approach is proposed for plan optimization in radiation therapy.

METHODS

Head and lung cases with the inhomogeneous medium are used as training and testing data. The prediction model is a cascade forward backprop neural network where the input is the features of the voxel, including 1) voxel to body surface distance along the beamlet axis, 2) voxel to beamlet axis distance, 3) voxel density, 4) heterogeneity corrected voxel to body surface distance, 5) heterogeneity corrected voxel to beamlet axis, and (6) the dose of voxel obtained from the pencil beam (PB) algorithm. The output is the predicted voxel dose corresponding to a beamlet. The predicted DDM was used for plan optimization (ML method) and compared with the dose of MC-based plan optimization (MC method) and the dose of pencil beam-based plan optimization (PB method). The mean absolute error (MAE) value was calculated for full volume relative to the dose of the MC method to evaluate the overall dose performance of the final plan.

RESULTS

For patient with head tumor, the ML method achieves MAE value 0.49 × 10-4 and PB has MAE 1.86 × 10-4. For patient with lung tumor, the ML method has MAE 1.42 × 10-4 and PB has MAE 3.72 × 10-4. The maximum percentage difference in PTV dose coverage (D98) between ML and MC methods is no more than 1.2% for patient with head tumor, while the difference is larger than 10% using the PB method. For patient with lung tumor, the maximum percentage difference in PTV dose coverage (D98) between ML and MC methods is no more than 2.1%, while the difference is larger than 16% using the PB method.

CONCLUSIONS

In this work, a reliable DDM prediction method is established for plan optimization by applying several voxel features and the ML approach. The results show that the ML method based on voxel features can obtain plans comparable to the MC method and is better than the PB method in achieving accurate dose to the patient, which is helpful for rapid plan optimization and accurate dose calculation.

ADVANCES IN KNOWLEDGE

Establishment of a new machine learning method based on the relationship between the voxel and beamlet features for dose deposition matrix prediction in radiation therapy.

Comparison of Jaw Mode and Field Width for Left-Breast Cancer Using TomoDirect Three-Dimensional Conformal Radiation Therapy: A Phantom Study

It is very important to use effective parameters in the treatment plan of breast cancer patients in TomoDirect (TD)-three-dimensional conformal radiation therapy (TD-3DCRT). The objective of this study was to compare the radiation treatment plans to the parameters (jaw width and jaw mode) of TD-3DCRT for left-breast cancer. This study was conducted using the phantom, the jaw mode (fixed and dynamic) and field width (2.5 cm and 5.0 cm) were controlled to compare the TD-3DCRT treatment plans. There was small difference in the conformity index (CI) and homogeneity index (HI) values for target according to the jaw mode for each field width. As a result of observation in terms of dose, treatment time and unnecessary damage to surrounding normal organs could be minimized when dynamic jaw with a field width of 5.0 cm was used. In conclusion, we verified that the use of dynamic jaws and 5.0 cm field width was effective in left-breast cancer radiotherapy plan using TD-3DCRT.

Knowledge-based planning using both the predicted DVH of organ-at risk and planning target volume

PURPOSE

The purpose of this study was to evaluate the performance of a knowledge-based planning (KBP) method in nasopharyngeal cancer radiotherapy using the predicted dose-volume histogram (DVH) of organ-at risk (OAR) and planning target volume (PTV).

METHODS AND MATERIALS

A total of 85 patients previously treated for nasopharyngeal cancer using 9-field 6-MV intensity-modulated radiation therapy (IMRT) were identified for training and 30 similar patients were identified for testing. The dosimetric deposition information, individual dose-volume histograms (IDVHs) induced by a series of fields with uniform-intensity irradiation, was used to predict both OAR and PTV DVH. Two KBP methods (KBP_OAR and KBP_OAR+PTV) were established for plan generation based on the DVH prediction. The KBP_OAR method utilized the dose constraints based on the predicted OAR DVH and the PTV dose constraints obtained according to the planning experience, while the KBP_OAR+PTV method applied the dose constraints based on the predicted OAR and PTV DVH. For the plan evaluation, the PTV dose coverage was used D₉₈ and D₂, and the maximum dose, mean dose or dose-volume parameters were used for the OARs. Statistical differences of the two KBP methods were tested with the Wilcoxon signed rank test.

RESULTS

For patients with T3 tumors, there was no significant difference between the KBP_OAR and KBP_OAR+PTV methods in dosimetric results at most OARs and PTVs. Both KBP methods achieved a similar number of plans meeting the dose requirements. For patients with T4 tumors, KBP_OAR+PTV reduced the maximum dose by more than 1 Gy in the body, spinal cord, optic nerve, eye and temporal lobes and reduced the V₅₀ value by more than 3.9% in the larynx and tongue without reducing the PTV dose compared with KBP_OAR. The KBP_OAR+PTV method increased the plans by more than 14.2% in meeting the maximum dose requirements at the body, optic nerve, mandible and eye and increased the plans by more than 21.4% in meeting the V₅₀ of the larynx and V₅₀ of the tongue when compared with the KBP_OAR method.

CONCLUSIONS

For patients with T3 tumors, no significant difference was found between the KBP_OAR and KBP_OAR+PTV methods in dosimetric results at most OARs and PTVs. For patients with T4 tumors, the KBP_OAR+PTV method performs better than the KBP_OAR method in improving the quality of the plans. Compared with the KBP_OAR method, dose sparing of some OARs was achieved without reducing PTV dose coverage and helped to increase the number of plans meeting the dose requirements when the KBP_OAR+PTV method was utilized.

A personalized DVH prediction model for HDR brachytherapy in cervical cancer treatment

Purpose

Although the knowledge-based dose-volume histogram (DVH) prediction has been largely researched and applied in External Beam Radiation Therapy, it is still less investigated in the domain of brachytherapy. The purpose of this study is to develop a reliable DVH prediction method for high-dose-rate brachytherapy plans.

Method

A DVH prediction workflow combining kernel density estimation (KDE), k-nearest neighbor (kNN), and principal component analysis (PCA) was proposed. PCA and kNN were first employed together to select similar patients based on principal component directions. 79 cervical cancer patients with different applicators inserted was included in this study. The KDE model was built based on the relationship between distance-to-target (DTH) and the dose in selected cases, which can be subsequently used to estimate the dose probability distribution in the validation set. Model performance of bladder and rectum was quantified by |ΔD_2cc|, |ΔD_1cc|, |ΔD_0.1cc|, |ΔD_max|, and |ΔD_mean| in the form of mean and standard deviation. The model performance between KDE only and the combination of kNN, PCA, and KDE was compared.

Result

20, 30 patients were selected for rectum and bladder based on KNN and PCA, respectively. The absolute residual between the actual plans and the predicted plans were 0.38 ± 0.29, 0.4 ± 0.32, 0.43 ± 0.36, 0.97 ± 0.66, and 0.13 ± 0.99 for |ΔD_2cc|, |ΔD_1cc|, |ΔD_0.1cc|, |ΔD_max|, and |ΔD_mean| in the bladder, respectively. For rectum, the corresponding results were 0.34 ± 0.27, 0.38 ± 0.33, 0.63 ± 0.57, 1.41 ± 0.99 and 0.23 ± 0.17, respectively. The combination of kNN, PCA, and KDE showed a significantly better prediction performance than KDE only, with an improvement of 30.3% for the bladder and 33.3% for the rectum.

Conclusion

In this study, a knowledge-based machine learning model was proposed and verified to accurately predict the DVH for new patients. This model is proved to be effective in our testing group in the workflow of HDR brachytherapy.

Knowledge-based planning for the radiation therapy treatment plan quality assurance for patients with head and neck cancer

This study aimed to investigate the feasibility of using a knowledge‐based planning technique to detect poor quality VMAT plans for patients with head and neck cancer. We created two dose–volume histogram (DVH) prediction models using a commercial knowledge‐based planning system (RapidPlan, Varian Medical Systems, Palo Alto, CA) from plans generated by manual planning (MP) and automated planning (AP) approaches. DVHs were predicted for evaluation cohort 1 (EC1) of 25 patients and compared with achieved DVHs of MP and AP plans to evaluate prediction accuracy. Additionally, we predicted DVHs for evaluation cohort 2 (EC2) of 25 patients for which we intentionally generated plans with suboptimal normal tissue sparing while satisfying dose–volume limits of standard practice. Three radiation oncologists reviewed these plans without seeing the DVH predictions. We found that predicted DVH ranges (upper–lower predictions) were consistently wider for the MP model than for the AP model for all normal structures. The average ranges of mean dose predictions among all structures was 9.7 Gy (MP model) and 3.4 Gy (AP model) for EC1 patients. RapidPlan models identified 7 MP plans as outliers according to mean dose or D1% for at least one structure, while none of AP plans were flagged. For EC2 patients, 22 suboptimal plans were identified by prediction. While re‐generated AP plans validated that these suboptimal plans could be improved, 40 out of 45 structures with predicted poor sparing were also identified by oncologist reviews as requiring additional planning to improve sparing in the clinical setting. Our study shows that knowledge‐based DVH prediction models can be sufficiently accurate for plan quality assurance purposes. A prediction model built by a small cohort automatically‐generated plans was effective in detecting suboptimal plans. Such tools have potential to assist the plan quality assurance workflow for individual patients in the clinic.

Organ-at-risk dose prediction using a machine learning algorithm: Clinical validation and treatment planning benefit for lung SBRT

OBJECTIVE

To quantify the clinical performance of a machine learning (ML) algorithm for organ-at-risk (OAR) dose prediction for lung stereotactic body radiation therapy (SBRT) and estimate the treatment planning benefit from having upfront access to these dose predictions.

METHODS

ML models were trained using multi-center data consisting of 209 patients previously treated with lung SBRT. Two prescription levels were investigated, 50 Gy in five fractions and 54 Gy in three fractions. Models were generated using a gradient-boosted regression tree algorithm using grid searching with fivefold cross-validation. Twenty patients not included in the training set were used to test OAR dose prediction performance, ten for each prescription. We also performed blinded re-planning based on OAR dose predictions but without access to clinically delivered plans. Differences between predicted and delivered doses were assessed by root-mean square deviation (RMSD), and statistical differences between predicted, delivered, and re-planned doses were evaluated with one-way analysis of variance (ANOVA) tests.

RESULTS

ANOVA tests showed no significant differences between predicted, delivered, and replanned OAR doses (all p ≥ 0.36). The RMSD was 2.9, 3.9, 4.3, and 1.7Gy for max dose to the spinal cord, great vessels, heart, and trachea, respectively, for 50 Gy in five fractions. Average improvements of 1.0, 1.4, and 2.0 Gy were seen for spinal cord, esophagus, and trachea max doses in blinded replans compared to clinically delivered plans with 54 Gy in three fractions, and 1.8, 0.7, and 1.5 Gy, respectively, for the esophagus, heart and bronchus max doses with 50 Gy in five fractions. Target coverage was similar with an average PTV V100% of 94.7% for delivered plans compared to 97.3% for blinded re-plans for 50 Gy in five fractions, and respectively 98.4% versus 99.2% for 54 Gy in three fractions.

CONCLUSION

This study validated ML-based OAR dose prediction for lung SBRT, showing potential for improved OAR dose sparing and more consistent plan quality using dose predictions for patient-specific planning guidance.

Machine Learning for Future Subtyping of the Tumor Microenvironment of Gastro-Esophageal Adenocarcinomas

Tumor progression involves an intricate interplay between malignant cells and their surrounding tumor microenvironment (TME) at specific sites. The TME is dynamic and is composed of stromal, parenchymal, and immune cells, which mediate cancer progression and therapy resistance. Evidence from preclinical and clinical studies revealed that TME targeting and reprogramming can be a promising approach to achieve anti-tumor effects in several cancers, including in GEA. Thus, it is of great interest to use modern technology to understand the relevant components of programming the TME. Here, we discuss the approach of machine learning, which recently gained increasing interest recently because of its ability to measure tumor parameters at the cellular level, reveal global features of relevance, and generate prognostic models. In this review, we discuss the relevant stromal composition of the TME in GEAs and discuss how they could be integrated. We also review the current progress in the application of machine learning in different medical disciplines that are relevant for the management and study of GEA.