Offline Quality Assurance for Intensity Modulated Radiation Therapy Treatment Plans for NRG-HN001 Head and Neck Clinical Trial Using Knowledge-Based Planning

Radiotherapy Plan Quality Assurance in NRG Oncology Trials for Brain and Head/Neck Cancers: An AI-Enhanced Knowledge-Based Approach

The quality of radiation therapy (RT) treatment plans directly affects the outcomes of clinical trials. KBP solutions have been utilized in RT plan quality assurance (QA). In this study, we evaluated the quality of RT plans for brain and head/neck cancers enrolled in multi-institutional clinical trials utilizing a KBP approach. The evaluation was conducted on 203 glioblastoma (GBM) patients enrolled in NRG-BN001 and 70 nasopharyngeal carcinoma (NPC) patients enrolled in NRG-HN001. For each trial, fifty high-quality photon plans were utilized to build a KBP photon model. A KBP proton model was generated using intensity-modulated proton therapy (IMPT) plans generated on 50 patients originally treated with photon RT. These models were then applied to generate KBP plans for the remaining patients, which were compared against the submitted plans for quality evaluation, including in terms of protocol compliance, target coverage, and organ-at-risk (OAR) doses. RT plans generated by the KBP models were demonstrated to have superior quality compared to the submitted plans. KBP IMPT plans can decrease the variation of proton plan quality and could possibly be used as a tool for developing improved plans in the future. Additionally, the KBP tool proved to be an effective instrument for RT plan QA in multi-center clinical trials.

NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions

Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.

Knowledge-based versus deep learning based treatment planning for breast radiotherapy

Background and Purpose

To improve radiotherapy (RT) planning efficiency and plan quality, knowledge-based planning (KBP) and deep learning (DL) solutions have been developed. We aimed to make a direct comparison of these models for breast cancer planning using the same training, validation, and testing sets.

Materials and Methods

Two KBP models were trained and validated with 90 RT plans for left-sided breast cancer with 15 fractions of 2.6 Gy. The versions either used the full dataset (non-clean model) or a cleaned dataset (clean model), thus eliminating geometric and dosimetric outliers. Results were compared with a DL U-net model (previously trained and validated with the same 90 RT plans) and manually produced RT plans, for the same independent dataset of 15 patients. Clinically relevant dose volume histogram parameters were evaluated according to established consensus criteria.

Results

Both KBP models underestimated the mean heart and lung dose equally 0.4 Gy (0.3-1.1 Gy) and 1.4 Gy (1.1-2.8 Gy) compared to the clinical plans 0.8 Gy (0.5-1.8 Gy) and 1.7 Gy (1.3-3.2 Gy) while in the final calculations the mean lung dose was higher 1.9-2.0 Gy (1.5-3.5 Gy) for both KPB models. The U-Net model resulted in a mean planning target volume dose of 40.7 Gy (40.4-41.3 Gy), slightly higher than the clinical plans 40.5 Gy (40.1-41.0 Gy).

Conclusions

Only small differences were observed between the estimated and final dose calculation and the clinical results for both KPB models and the DL model. With a good set of breast plans, the data cleaning module is not needed and both KPB and DL models lead to clinically acceptable results.

Knowledge-Based Quality Assurance and Model Maintenance in Lung Cancer Radiation Therapy in a Statewide Quality Consortium of Academic and Community Practice Centers

PURPOSE

Locally advanced lung cancer (LALC) treatment planning is often complex due to challenging tradeoffs related to large targets near organs at risk, making the judgment of plan quality difficult. The purpose of this work was to update and maintain a multi-institutional knowledge-based planning (KBP) model developed by a statewide consortium of academic and community practices for use as a plan quality assurance (QA) tool.

METHODS AND MATERIALS

Sixty LALC volumetric-modulated arc therapy plans from 2021 were collected from 24 institutions. Plan quality was scored, with high-quality clinical (HQC) plans selected to update a KBP model originally developed in 2017. The model was validated via automated KBP planning, with 20 cases excluded from the model. Differences in dose-volume histogram metrics in the clinical plans, 2017 KBP model plans, and 2022 KBP model plans were compared. Twenty recent clinical cases not meeting consortium quality metrics were replanned with the 2022 model to investigate potential plan quality improvements.

RESULTS

Forty-seven plans were included in the final KBP model. Compared with the clinical plans, the 2022 model validation plans improved 60%, 65%, and 65% of the lung V20Gy, mean heart dose, and spinal canal D0.03cc metrics, respectively. The 2022 model showed improvements from the 2017 model in hot spot management at the cost of greater lung doses. Of the 20 recent cases not meeting quality metrics, 40% of the KBP model-replanned cases resulted in acceptable plans, suggesting potential clinical plan improvements.

CONCLUSIONS

A multi-institutional KBP model was updated using plans from a statewide consortium. Multidisciplinary plan review resulted in HQC model training plans and model validation resulted in acceptable quality plans. The model proved to be effective at identifying potential plan quality improvements. Work is ongoing to develop web-based training plan review tools and vendor-agnostic platforms to provide the model as a QA tool statewide.

Implementation of Machine Learning Models to Ensure Radiotherapy Quality for Multicenter Clinical Trials: Report from a Phase III Lung Cancer Study

The outcome of the patient and the success of clinical trials involving RT is dependent on the quality assurance of the RT plans. Knowledge-based Planning (KBP) models using data from a library of high-quality plans have been utilized in radiotherapy to guide treatment. In this study, we report on the use of these machine learning tools to guide the quality assurance of multicenter clinical trial plans. The data from 130 patients submitted to RTOG1308 were included in this study. Fifty patient cases were used to train separate photon and proton models on a commercially available platform based on principal component analysis. Models evaluated 80 patient cases. Statistical comparisons were made between the KBP plans and the original plans submitted for quality evaluation. Both photon and proton KBP plans demonstrate a statistically significant improvement of quality in terms of organ-at-risk (OAR) sparing. Proton KBP plans, a relatively emerging technique, show more improvements compared with photon plans. The KBP proton model is a useful tool for creating proton plans that adhere to protocol requirements. The KBP tool was also shown to be a useful tool for evaluating the quality of RT plans in the multicenter clinical trial setting.

Development and Clinical Implementation of an Automated Virtual Integrative Planner for Radiation Therapy of Head and Neck Cancer

Purpose

Head and neck (HN) radiation (RT) treatment planning is complex and resource intensive. Deviations and inconsistent plan quality significantly affect clinical outcomes. We sought to develop a novel automated virtual integrative (AVI) knowledge-based planning application to reduce planning time, increase consistency, and improve baseline quality.

Methods and Materials

An in-house write-enabled script was developed from a library of 668 previously treated HN RT plans. Prospective hazard analysis was performed, and mitigation strategies were implemented before clinical release. The AVI-planner software was retrospectively validated in a cohort of 52 recent HN cases. A physician panel evaluated planning limitations during initial deployment, and feedback was enacted via software refinements. A final second set of plans was generated and evaluated. Kolmogorov-Smirnov test in addition to generalized evaluation metric and weighted experience score were used to compare normal tissue sparing between final AVI planner versus respective clinically treated and historically accepted plans. A t test was used to compare the interactive time, complexity, and monitor units for AVI planner versus manual optimization.

Results

Initially, 86% of plans were acceptable to treat, with 10% minor and 4% major revisions or rejection recommended. Variability was noted in plan quality among HN subsites, with high initial quality for oropharynx and oral cavity plans. Plans needing revisions were comprised of sinonasal, nasopharynx, P-16 negative squamous cell carcinoma unknown primary, or cutaneous primary sites. Normal tissue sparing varied within subsites, but AVI planner significantly lowered mean larynx dose (median, 18.5 vs 19.7 Gy; P < .01) compared with clinical plans. AVI planner significantly reduced interactive optimization time (mean, 2 vs 85 minutes; P < .01).

Conclusions

AVI planner reliably generated clinically acceptable RT plans for oral cavity, salivary, oropharynx, larynx, and hypopharynx cancers. Physician-driven iterative learning processes resulted in favorable evolution in HN RT plan quality with significant time savings and improved consistency using AVI planner.

Practice changing data and emerging concepts from recent radiation therapy randomised clinical trials

INTRODUCTION

Oncology treatments are constantly and rapidly evolving. We aimed at highlighting the latest radiation therapy practice changing trials and emerging concepts, through an overview of recent randomised clinical trials (RCTs).

MATERIALS AND METHODS

Requests were performed in the Medline database to identify all publications reporting radiation therapy RCTs from 2018 to 2021.

RESULTS

Recent RCTs sustained the role of newer combinatorial strategies through radioimmunotherapy for early stage or metastatic lung cancer, newer pro-apoptotic agents (e.g. debio 1143 in locoregionally advanced head and neck squamous cell carcinoma) or nanoparticles (e.g. NBTXR3 in locally advanced soft-tissue sarcoma). High-tech radiotherapy allows intensifying treatments and gaining ground in some indications through the development of stereotactic body radiotherapy for example. First randomised evidence on personalised radiation therapy through imaging-based (¹⁸FDG positron emission tomography-computed tomography for lung cancer or early stage unfavourable Hodgkin lymphoma, PMSA positron emission tomography-computed tomography or magnetic resonance imaging for high-risk prostate cancer) or biological biomarkers (PSA for prostate cancer, HPV for head and neck cancer, etc) were conducted to more tailored treatments, with more favourable outcomes. Patients' quality of life and satisfaction appeared to be increasing aims. RCTs have validated (ultra)hypofractionated schemes in many indications as for breast, prostate or rectal cancer, resulting in equivalent outcomes and toxicities, more convenient for patients and favouring shared decision making.

CONCLUSION

Radiation therapy is a dynamic field of research, and many RCTs have greatly impacted therapeutic standards over the last years. Investments in radiotherapy research should facilitate the transfer of innovation to clinic.

Knowledge-based planning for the radiation therapy treatment plan quality assurance for patients with head and neck cancer

This study aimed to investigate the feasibility of using a knowledge‐based planning technique to detect poor quality VMAT plans for patients with head and neck cancer. We created two dose–volume histogram (DVH) prediction models using a commercial knowledge‐based planning system (RapidPlan, Varian Medical Systems, Palo Alto, CA) from plans generated by manual planning (MP) and automated planning (AP) approaches. DVHs were predicted for evaluation cohort 1 (EC1) of 25 patients and compared with achieved DVHs of MP and AP plans to evaluate prediction accuracy. Additionally, we predicted DVHs for evaluation cohort 2 (EC2) of 25 patients for which we intentionally generated plans with suboptimal normal tissue sparing while satisfying dose–volume limits of standard practice. Three radiation oncologists reviewed these plans without seeing the DVH predictions. We found that predicted DVH ranges (upper–lower predictions) were consistently wider for the MP model than for the AP model for all normal structures. The average ranges of mean dose predictions among all structures was 9.7 Gy (MP model) and 3.4 Gy (AP model) for EC1 patients. RapidPlan models identified 7 MP plans as outliers according to mean dose or D1% for at least one structure, while none of AP plans were flagged. For EC2 patients, 22 suboptimal plans were identified by prediction. While re‐generated AP plans validated that these suboptimal plans could be improved, 40 out of 45 structures with predicted poor sparing were also identified by oncologist reviews as requiring additional planning to improve sparing in the clinical setting. Our study shows that knowledge‐based DVH prediction models can be sufficiently accurate for plan quality assurance purposes. A prediction model built by a small cohort automatically‐generated plans was effective in detecting suboptimal plans. Such tools have potential to assist the plan quality assurance workflow for individual patients in the clinic.

Radiotherapy Standardisation and Artificial Intelligence within the National Cancer Institute's Clinical Trials Network

Artificial intelligence in healthcare refers to the use of complex algorithms designed to conduct certain tasks in an automated manner. Artificial intelligence has a transformative power in radiation oncology to improve the quality and efficiency of patient care, given the increase in volume and complexity of digital data, as well as the multi-faceted and highly technical nature of this field of medicine. However, artificial intelligence alone will not be able to fix healthcare's problem, because new technologies bring unexpected and potentially underappreciated obstacles. The inclusion of multicentre datasets, the incorporation of time-varying data, the assessment of missing data as well as informative censoring and the addition of clinical utility could significantly improve artificial intelligence models. Standardisation plays a crucial, supportive and leading role in artificial intelligence. Clinical trials are the most reliable method of demonstrating the efficacy and safety of a treatment or clinical approach, as well as providing high-level evidence to justify artificial intelligence. The National Surgical Adjuvant Breast and Bowel Project, the Radiation Therapy Oncology Group and the Gynecologic Oncology Group collaborated to form NRG Oncology (acronym NRG derived from the names of the parental groups). NRG Oncology is one of the adult cancer clinical trial groups containing radiotherapy specialty of the National Cancer Institute's Clinical Trials Network (NCTN). Standardisation from NRG/NCTN has the potential to reduce variation in clinical treatment and patient outcome by eliminating potential errors, enabling broader application of artificial intelligence tools. NCTN, NRG and Imaging and Radiation Oncology Core are in a unique position to help with standards development, advocacy and enforcement, all of which can benefit from artificial intelligence, as artificial intelligence has the ability to improve trial success rates by transforming crucial phases in clinical trial design, from study planning through to execution. Here we will examine: (i) how to conduct technical and clinical evaluations before adopting artificial intelligence technologies, (ii) how to obtain high-quality data for artificial intelligence, (iii) the NCTN infrastructure and standards, (iv) radiotherapy standardisation for clinical trials and (v) artificial intelligence applications in standardisation.

Knowledge-based dose prediction models to inform gynecologic brachytherapy needle supplementation for locally advanced cervical cancer

PURPOSE

The use of interstitial needles, combined with intracavitary applicators, enables customized dose distributions and is beneficial for complex cases, but increases procedure time. Overall, applicator selection is not standardized and depends on physician expertise and preference. The purpose of this study is to determine whether dose prediction models can guide needle supplementation decision-making for cervical cancer.

MATERIALS AND METHODS

Intracavitary knowledge-based models for organ-at-risk (OAR) dose estimation were trained and validated for tandem-and-ring/ovoids (T&R/T&O) implants. Models were applied to hybrid cases with 1-3 implanted needles to predict OAR dose without needles. As a reference, 70/67 hybrid T&R/T&O cases were replanned without needles, following a standardized procedure guided by dose predictions. If a replanned dose exceeded the dose objective, the case was categorized as requiring needles. Receiver operating characteristic (ROC) curves of needle classification accuracy were generated. Optimal classification thresholds were determined from the Youden Index.

RESULTS

Needle supplementation reduced dose to OARs. However, 67%/39% of replans for T&R/T&O met all dose constraints without needles. The ROC for T&R/T&O models had an area-under-curve of 0.89/0.86, proving high classification accuracy. The optimal threshold of 99%/101% of the dose limit for T&R/T&O resulted in classification sensitivity and specificity of 78%/86% and 85%/78%.

CONCLUSIONS

Needle supplementation reduced OAR dose for most cases but was not always required to meet standard dose objectives, particularly for T&R cases. Our knowledge-based dose prediction model accurately identified cases that could have met constraints without needle supplementation, suggesting that such models may be beneficial for applicator selection.

Evaluation of dose differences between intracavitary applicators for cervical brachytherapy using knowledge-based models

PURPOSE

Currently, there is a lack of patient-specific tools to guide brachytherapy planning and applicator choice for cervical cancer. The purpose of this study is to evaluate the accuracy of organ-at-risk (OAR) dose predictions using knowledge-based intracavitary models, and the use of these models and clinical data to determine the dosimetric differences of tandem-and-ring (T&R) and tandem-and-ovoids (T&O) applicators.

MATERIALS AND METHODS

Knowledge-based models, which predict organ D_2cc, were trained on 77/75 cases and validated on 32/38 for T&R/T&O applicators. Model performance was quantified using ΔD_2cc=D_2cc,actual-D_{2cc,predicted}, with standard deviation (σ(ΔD_2cc)) representing precision. Model-predicted applicator dose differences were determined by applying T&O models to T&R cases, and vice versa, and compared to clinically-achieved D_2cc differences. Applicator differences were assessed using a Student's t-test (p < 0.05 significant).

RESULTS

Validation T&O/T&R model precision was 0.65/0.55 Gy, 0.55/0.38 Gy, and 0.43/0.60 Gy for bladder, rectum and sigmoid, respectively, and similar to training. When applying T&O/T&R models to T&R/T&O cases, bladder, rectum and sigmoid D_2cc values in EQD2 were on average 5.69/2.62 Gy, 7.31/6.15 Gy and 3.65/0.69 Gy lower for T&R, with similar HRCTV volume and coverage. Clinical data also showed lower T&R OAR doses, with mean EQD2 D_2cc deviations of 0.61 Gy, 7.96 Gy (p < 0.01) and 5.86 Gy (p < 0.01) for bladder, rectum and sigmoid.

CONCLUSIONS

Accurate knowledge-based dose prediction models were developed for two common intracavitary applicators. These models could be beneficial for standardizing and improving the quality of brachytherapy plans. Both models and clinical data suggest that significant OAR sparing can be achieved with T&R over T&O applicators, particularly for the rectum.

A comparison of two methodologies for radiotherapy treatment plan optimization and QA for clinical trials

Background and purpose

The efficacy of clinical trials and the outcome of patient treatment are dependent on the quality assurance (QA) of radiation therapy (RT) plans. There are two widely utilized approaches that include plan optimization guidance created based on patient‐specific anatomy. This study examined these two techniques for dose‐volume histogram predictions, RT plan optimizations, and prospective QA processes, namely the knowledge‐based planning (KBP) technique and another first principle (FP) technique.

Methods

This analysis included 60, 44, and 10 RT plans from three Radiation Therapy Oncology Group (RTOG) multi‐institutional trials: RTOG 0631 (Spine SRS), RTOG 1308 (NSCLC), and RTOG 0522 (H&N), respectively. Both approaches were compared in terms of dose prediction and plan optimization. The dose predictions were also compared to the original plan submitted to the trials for the QA procedure.

Results

For the RTOG 0631 (Spine SRS) and RTOG 0522 (H&N) plans, the dose predictions from both techniques have correlation coefficients of >0.9. The RT plans that were re‐optimized based on the predictions from both techniques showed similar quality, with no statistically significant differences in target coverage or organ‐at‐risk sparing. The predictions of mean lung and heart doses from both methods for RTOG1308 patients, on the other hand, have a discrepancy of up to 14 Gy.

Conclusions

Both methods are valuable tools for optimization guidance of RT plans for Spine SRS and Head and Neck cases, as well as for QA purposes. On the other hand, the findings suggest that KBP may be more feasible in the case of inoperable lung cancer patients who are treated with IMRT plans that have spatially unevenly distributed beam angles.

Personalising treatment plan quality review with knowledge-based planning in the TROG 15.03 trial for stereotactic ablative body radiotherapy in primary kidney cancer

INTRODUCTION

Quality assurance (QA) of treatment plans in clinical trials improves protocol compliance and patient outcomes. Retrospective use of knowledge-based-planning (KBP) in clinical trials has demonstrated improved treatment plan quality and consistency. We report the results of prospective use of KBP for real-time QA of treatment plan quality in the TROG 15.03 FASTRACK II trial, which evaluates efficacy of stereotactic ablative body radiotherapy (SABR) for kidney cancer.

METHODS

A KBP model was generated based on single institution data. For each patient in the KBP phase (open to the last 31 patients in the trial), the treating centre submitted treatment plans 7 days prior to treatment. A treatment plan was created by using the KBP model, which was compared with the submitted plan for each organ-at-risk (OAR) dose constraint. A report comparing each plan for each OAR constraint was provided to the submitting centre within 24 h of receiving the plan. The centre could then modify the plan based on the KBP report, or continue with the existing plan.

RESULTS

Real-time feedback using KBP was provided in 24/31 cases. Consistent plan quality was in general achieved between KBP and the submitted plan. KBP review resulted in replan and improvement of OAR dosimetry in two patients. All centres indicated that the feedback was a useful QA check of their treatment plan.

CONCLUSION

KBP for real-time treatment plan review was feasible for 24/31 cases, and demonstrated ability to improve treatment plan quality in two cases. Challenges include integration of KBP feedback into clinical timelines, interpretation of KBP results with respect to clinical trade-offs, and determination of appropriate plan quality improvement criteria.

Knowledge-based radiation treatment planning: A data-driven method survey

This paper surveys the data-driven dose prediction methods investigated for knowledge-based planning (KBP) in the last decade. These methods were classified into two major categories-traditional KBP methods and deep-learning (DL) methods-according to their techniques of utilizing previous knowledge. Traditional KBP methods include studies that require geometric or anatomical features to either find the best-matched case(s) from a repository of prior treatment plans or to build dose prediction models. DL methods include studies that train neural networks to make dose predictions. A comprehensive review of each category is presented, highlighting key features, methods, and their advancements over the years. We separated the cited works according to the framework and cancer site in each category. Finally, we briefly discuss the performance of both traditional KBP methods and DL methods, then discuss future trends of both data-driven KBP methods to dose prediction.