Quality assurance of radiotherapy in the ongoing EORTC 1219-DAHANCA-29 trial for HPV/p16 negative squamous cell carcinoma of the head and neck: Results of the benchmark case procedure

Radiotherapy trial quality assurance processes: a systematic review

Quality assurance remains a neglected component of many trials, particularly for technical interventions, such as surgery and radiotherapy, for which quality of treatment is an important component in defining outcomes. We aimed to evaluate evidence for the processes used in radiotherapy quality assurance of clinical trials. A systematic review was undertaken focusing on use of a pre-trial outlining benchmark case and subsequent on-trial individual case reviews of outlining for recruited patients. These pre-trial and on-trial checks are used to ensure consistency and standardisation of treatment for each patient recruited to the trial by confirming protocol compliance. Non-adherence to the trial protocol has been shown to have a negative effect on trial outcomes. 29 studies published between January, 2000, and December, 2022, were identified that reported on either outlining benchmark case results or outlining individual case review results, or both. The trials identified varied in their use of radiotherapy quality assurance practices and reporting of outcomes was inconsistent. Deviations from trial protocols were frequent, particularly regarding outlining. Studies correlating benchmark case results with on-trial individual case reviews provided mixed results, meaning firm conclusions could not be drawn regarding the influence of the pre-trial benchmark case on subsequent on-trial performance. The optimal radiotherapy quality assurance processes were unclear, and there was little evidence available. Improved reporting of outcomes from radiotherapy quality assurance programmes is needed to develop an evidence base for the optimal approach to radiotherapy quality assurance in trials.

Extensive clinical testing of Deep Learning Segmentation models for thorax and breast cancer radiotherapy planning

BACKGROUND

The performance of deep learning segmentation (DLS) models for automatic organ extraction from CT images in the thorax and breast regions was investigated. Furthermore, the readiness and feasibility of integrating DLS into clinical practice were addressed by measuring the potential time savings and dosimetric impact.

MATERIAL AND METHODS

Thirty patients referred to radiotherapy for breast cancer were prospectively included. A total of 23 clinically relevant left- and right-sided organs were contoured manually on CT images according to ESTRO guidelines. Next, auto-segmentation was executed, and the geometric agreement between the auto-segmented and manually contoured organs was qualitatively assessed applying a scale in the range [0-not acceptable, 3-no corrections]. A quantitative validation was carried out by calculating Dice coefficients (DSC) and the 95% percentile of Hausdorff distances (HD95). The dosimetric impact of optimizing the treatment plans on the uncorrected DLS contours, was investigated from a dose coverage analysis using DVH values of the manually delineated contours as references.

RESULTS

The qualitative analysis showed that 93% of the DLS generated OAR contours did not need corrections, except for the heart where 67% of the contours needed corrections. The majority of DLS generated CTVs needed corrections, whereas a minority were deemed not acceptable. Still, using the DLS-model for CTV and heart delineation is on average 14 minutes faster. An average DSC=0.91 and H95=9.8 mm were found for the left and right breasts, respectively. Likewise, and average DSC in the range [0.66, 0.76]mm and HD95 in the range [7.04, 12.05]mm were found for the lymph nodes.

CONCLUSION

The validation showed that the DLS generated OAR contours can be used clinically. Corrections were required to most of the DLS generated CTVs, and therefore warrants more attention before possibly implementing the DLS models clinically.

Moving Experience from North America to Developing Country to Approach a Desired Level of Quality Assurance in Head and Neck Radiation Therapy

Evidence is lacking correlation between head and neck (HN) radiation oncology fellowship training and quality assurance (QA) round decision for plan modifications. This study was conducted to identify the association between training and changes in QA decisions. From 2007 to 2018, data on HN cancer cases presented at departmental QA rounds were prospectively collected. Then, we retrospectively analyzed the collected data to determine the impact of fellowship training on QA decisions. Cases were divided into pre-fellowship group (January 2007-September 2014) and post-fellowship group (October 2014-December 2018). Multivariable analysis (MVA) evaluated variables that could be associated with decreased frequencies of QA modification rates. From 2007 to 2018, 1266 HN cancer patients were identified; 728 patients were in the pre-fellowship group and 538 patients in the post-fellowship group. On MVA, fellowship training transformed QA decisions from more to less modifications (modified vs. approved; OR, 0.135; 95% CI, 0.076-0.240; p = 0.0001), increased approval rate for advanced T and N categories (T3-4 vs. T0-T2; OR, 0.798; 95% CI, 1.892-4.929; p = 0.0001 and N2-3 vs. N0-1; OR, 0.865; 95% CI, 1.454-3.423; p = 0.0002). By type of modification, fellowship training demonstrated a statistically significant reduction in rates of several types of modification that include target volume definition, target volume delineation, and dose (all p < 0.05). Our study determines the impact of the HN radiation oncology fellowship on decreased QA modification rates.

International assessment of interobserver reproducibility of flap delineation in head and neck carcinoma

Background: Several reports have suggested that radiotherapy after reconstructive surgery for head and neck cancer (HNC), could have deleterious effects on the flaps with respect to functional outcomes. To predict and prevent toxicities, flap delineation should be accurate and reproducible. The objective of the present study was to evaluate the interobserver variability of frequent types of flaps used in HNC, based on the recent GORTEC atlas.Materials and methods: Each member of an international working group (WG) consisting of 14 experts delineated the flaps on a CT set from six patients. Each patient had one of the five most commonly used flaps in HNC: a regional pedicled pectoralis major myocutaneous flap, a local pedicled rotational soft tissue facial artery musculo-mucosal (FAMM) (2 patients), a fasciocutaneous radial forearm free flap, a soft tissue anterolateral thigh (ALT) free flap, or a fibular free flap. The WG's contours were compared to a reference contour, validated by a surgeon and a radiologist specializing in HNC. Contours were considered as reproducible if the median Dice Similarity Coefficient (DSC) was > 0.7.Results: The median volumes of the six flaps delineated by the WG were close to the reference contour value, with approximately 50 cc for the pectoral, fibula, and ALT flaps, 20 cc for the radial forearm, and up to 10 cc for the FAMM. The volumetric ratio was thus close to the optimal value of 100% for all flaps. The median DSC obtained by the WG compared to the reference for the pectoralis flap, the FAMM, the radial forearm flap, ALT flap, and the fibular flap were 0.82, 0.40, 0.76, 0.81, and 0.76, respectively.Conclusions: This study showed that the delineation of four main flaps used for HNC was reproducible. The delineation of the FAMM, however, requires close cooperation between radiologist, surgeon and radiation oncologist because of the poor visibility of this flap on CT and its small size.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Investigating the potential of deep learning for patient-specific quality assurance of salivary gland contours using EORTC-1219-DAHANCA-29 clinical trial data

INTRODUCTION

Manual quality assurance (QA) of radiotherapy contours for clinical trials is time and labor intensive and subject to inter-observer variability. Therefore, we investigated whether deep-learning (DL) can provide an automated solution to salivary gland contour QA.

MATERIAL AND METHODS

DL-models were trained to generate contours for parotid (PG) and submandibular glands (SMG). Sørensen-Dice coefficient (SDC) and Hausdorff distance (HD) were used to assess agreement between DL and clinical contours and thresholds were defined to highlight cases as potentially sub-optimal. 3 types of deliberate errors (expansion, contraction and displacement) were gradually applied to a test set, to confirm that SDC and HD were suitable QA metrics. DL-based QA was performed on 62 patients from the EORTC-1219-DAHANCA-29 trial. All highlighted contours were visually inspected.

RESULTS

Increasing the magnitude of all 3 types of errors resulted in progressively severe deterioration/increase in average SDC/HD. 19/124 clinical PG contours were highlighted as potentially sub-optimal, of which 5 (26%) were actually deemed clinically sub-optimal. 2/19 non-highlighted contours were false negatives (11%). 15/69 clinical SMG contours were highlighted, with 7 (47%) deemed clinically sub-optimal and 2/15 non-highlighted contours were false negatives (13%). For most incorrectly highlighted contours causes for low agreement could be identified.

CONCLUSION

Automated DL-based contour QA is feasible but some visual inspection remains essential. The substantial number of false positives were caused by sub-optimal performance of the DL-model. Improvements to the model will increase the extent of automation and reliability, facilitating the adoption of DL-based contour QA in clinical trials and routine practice.

Quality assurance of radiotherapy in the ongoing EORTC 1420 "Best of" trial for early stage oropharyngeal, supraglottic and hypopharyngeal carcinoma: results of the benchmark case procedure

INTRODUCTION

The current phase III EORTC 1420 Best-of trial (NCT02984410) compares the swallowing function after transoral surgery versus intensity modulated radiotherapy (RT) in patients with early-stage carcinoma of the oropharynx, supraglottis and hypopharynx. We report the analysis of the Benchmark Case (BC) procedures before patient recruitment with special attention to dysphagia/aspiration related structures (DARS).

MATERIALS AND METHODS

Submitted RT volumes and plans from participating centers were analyzed and compared against the gold-standard expert delineations and dose distributions. Descriptive analysis of protocol deviations was conducted. Mean Sorensen-Dice similarity index (mDSI) and Hausdorff distance (mHD) were applied to evaluate the inter-observer variability (IOV).

RESULTS

65% (23/35) of the institutions needed more than one submission to achieve Quality assurance (RTQA) clearance. OAR volume delineations were the cause for rejection in 53% (40/76) of cases. IOV could be improved in 5 out of 12 OARs by more than 10 mm after resubmission (mHD). Despite this, final IOV for critical OARs in delineation remained significant among DARS by choosing an aleatory threshold of 0.7 (mDSI) and 15 mm (mHD).

CONCLUSIONS

This is to our knowledge the largest BC analysis among Head and neck RTQA programs performed in the framework of a prospective trial. Benchmarking identified non-common OARs and target delineations errors as the main source of deviations and IOV could be reduced in a significant number of cases after this process. Due to the substantial resources involved with benchmarking, future benchmark analyses should assess fully the impact on patients' clinical outcome.

Radiotherapy Quality Assurance for Head and Neck Squamous Cell Carcinoma

The impact of radiotherapy (RT) quality assurance (QA) has been demonstrated by numerous studies and is particularly important for head and neck cancer (HNC) treatment due to the complexity of RT target volumes in this region and the multiple adjacent organs at risk. The RT planning process includes many critical steps including interpretation of diagnostic imaging, image fusion, target volume delineation (tumor, lymph nodes, and organs at risk), and planning. Each step has become highly complex, and precise and rigorous QA throughout the planning process is essential. The ultimate aim is to precisely deliver radiation dose to the target, maximizing the tumor dose and minimizing the dose to surrounding organs at risk, in order to improve the therapeutic index. It is imperative that RT QA programs should systematically control all aspects of the RT planning pathway and include regular end-to-end tests and external audits. However, comprehensive QA should not be limited to RT and should, where possible, also be implemented for surgery, systemic therapy, pathology, as well as other aspects involved in the interdisciplinary treatment of HNC.

Definitive chemoradiotherapy in patients with squamous cell cancers of the head and neck - results from an unselected cohort of the clinical cooperation group "Personalized Radiotherapy in Head and Neck Cancer"

BACKGROUND

Definitive chemoradiotherapy (dCRT) is a standard treatment for patients with locally advanced head and neck cancer. There is a clinical need for a stratification of this prognostically heterogeneous group of tumors in order to optimize treatment of individual patients. We retrospectively reviewed all patients with head and neck squamous cell carcinoma (HNSCC) of the oral cavity, oropharynx, hypopharynx, or larynx, treated with dCRT from 09/2008 until 03/2016 at the Department of Radiation Oncology, LMU Munich. Here we report the clinical results of the cohort which represent the basis for biomarker discovery and molecular genetic research within the framework of a clinical cooperation group.

METHODS

Patient data were collected and analyzed for outcome and treatment failures with regard to previously described and established risk factors.

RESULTS

We identified 184 patients with a median follow-up of 65 months and a median age of 64 years. Patients received dCRT with a median dose of 70 Gy and simultaneous chemotherapy in 90.2% of cases, mostly mitomycin C / 5-FU in concordance with the ARO 95-06 trial. The actuarial 3-year overall survival (OS), local, locoregional and distant failure rates were 42.7, 29.8, 34.0 and 23.4%, respectively. Human papillomavirus-associated oropharynx cancer (HPVOPC) and smaller gross tumor volume were associated with significantly improved locoregional tumor control rate, disease-free survival (DFS) and OS in multivariate analysis. Additionally, lower hemoglobin levels were significantly associated with impaired DFS und OS in univariate analysis. The extent of lymph node involvement was associated with distant failure, DFS and OS. Moreover, 92 patients (50%) of our cohort have been treated in concordance with the ARO 95-06 study, corroborating the results of this study.

CONCLUSION

Our cohort is a large unselected monocentric cohort of HNSCC patients treated with dCRT. Tumor control rates and survival rates compare favorably with the results of previously published reports. The clinical data, together with the available tumor samples from biopsies, will allow translational research based on molecular genetic analyses.

Dummy-run for standardizing plan quality of intensity-modulated radiotherapy for postoperative uterine cervical cancer: Japan Clinical Oncology Group study (JCOG1402)

BACKGROUND

The purpose of this study was to assess compliance with treatment planning in a dummy-run for a multicenter clinical trial involving patients with high-risk postoperative uterine cervical cancer using intensity-modulated radiation therapy (IMRT) (JCOG1402 trial).

METHODS

For the dummy-run, we prepared a computed tomography dataset comprising two anonymized cases of post-hysterectomy cervical cancer. These were sent to the 47 participating institutions to assess institutional plan quality such as delineations and dose distributions.

RESULTS

Central review showed 3 and 4 deviations per treatment plan on average. The deviations related to the nodal and vaginal cuff clinical target volume (CTV) delineation, which accounted for approximately 50% of the total deviations. The CTV vaginal cuff showed considerable differences in delineation compared with the nodal CTV. For the Dice similarity coefficient, case 1 showed a mean ± 1σ of 0.81 ± 0.03 and 0.60 ± 0.09 for the nodal and the CTV vaginal cuff, respectively, while these were 0.81 ± 0.04 and 0.54 ± 0.14, respectively, for case two. Of the 47 institutions, 10 were required to resubmit their treatment plan because the delineations, planning target volume margin, and required dose distributions were not in accordance with the JCOG1402 protocol.

CONCLUSIONS

The dummy-run test in postoperative uterine cervical cancer demonstrated substantial deviations in the delineations, particularly for the CTV vaginal cuff. The analysis data could provide helpful information on delineation and planning, allowing standardization of IMRT planning for postoperative uterine cervical cancer.

TRIAL REGISTRATION

Japanese Clinical Trial Registry #: UMIN000027017 at https://upload.umin.ac.jp/cgi-open-bin/ctr/ctr_view.cgi?recptno=R000030672;language=J.

Repeat FMISO-PET imaging weakly correlates with hypoxia-associated gene expressions for locally advanced HNSCC treated by primary radiochemotherapy

BACKGROUND

Hypoxia is an important factor of tumour resistance to radiotherapy, chemotherapy and potentially immunotherapy. It can be measured e.g. by positron emission tomography (PET) imaging or hypoxia-associated gene expressions from tumour biopsies. Here we correlate [¹⁸F]fluoromisonidazole (FMISO)-PET/CT imaging with hypoxia-associated gene expressions on a cohort of 50 head and neck squamous cell carcinoma (HNSCC) patients and compare their prognostic value for response to radiochemotherapy (RCTx).

METHODS

FMISO-PET/CT images of 50 HNSCC patients were acquired at four time-points before and during RCTx. For 42 of these patients, hypoxia-associated gene expressions were evaluated by nanoString technology based on a biopsy obtained before any treatment. The FMISO-PET parameters tumour-to-background ratio and hypoxic volume were correlated to the expressions of 58 hypoxia-associated genes using the Spearman correlation coefficient ρ. Three hypoxia-associated gene signatures were compared regarding their correlation with the FMISO-PET parameters using their median expression. In addition, the correlation with tumour volume was analysed. The impact of both hypoxia measurement methods on loco-regional tumour control (LRC) and overall survival (OS) was assessed by Cox regression.

RESULTS

The median expression of hypoxia-associated genes was weakly correlated to hypoxia measured by FMISO-PET imaging (ρ ≤ 0.43), with higher correlations to imaging after weeks 1 and 2 of treatment (p < 0.001). Moderate correlations were obtained between FMISO-PET imaging and tumour volume (ρ ≤ 0.69). Prognostic models for LRC and OS based on the FMISO-PET parameters could not be improved by including hypoxia classifiers.

CONCLUSION

We observed low correlations between hypoxia FMISO-PET parameters and expressions of hypoxia-associated genes. Since FMISO-PET showed a superior patient stratification, it may be the preferred biomarker over hypoxia-associated genes for stratifying patients with locally advanced HNSCC treated by primary RCTx.

Radiation therapy quality assurance in head and neck radiotherapy - Moving forward

Head and Neck Cancer (HNC) radiation oncologists (ROs) enjoy the immense pleasure of curing patients, working within a large multidisciplinary team to effectively deliver curative intent treatment whilst also aiming to minimise late treatment toxicity. Secondary analyses of large-scale HNC clinical trials have shown the critical impact of the quality of radiotherapy plans, where protocol non-compliant plans have yielded inferior survival rates approximating 20%. The peer review process in routine day-to-day HNC practice shows that even in major academic centers a significant proportion of RT plans may require changes to the radiotherapy planning volume. Optimising the therapeutic ratio in HNC has been dramatically facilitated by intensity modulated radiotherapy (IMRT), but that technology has also increased the complexity of HNC radiotherapy treatment and high-volume centers with experienced clinicians may be best placed to deliver this most accurately. International consensus guidelines to standardise or benchmark best practice with respect to the RT-QA process in HNC are needed. The aim of this paper is to highlight the importance of the RT-QA process in the HNC treatment process and to make some recommendations for its inclusion in both clinical trials and routine clinical practice.

Big Data Readiness in Radiation Oncology: An Efficient Approach for Relabeling Radiation Therapy Structures With Their TG-263 Standard Name in Real-World Data Sets

Purpose

To prepare for big data analyses on radiation therapy data, we developed Stature, a tool-supported approach for standardization of structure names in existing radiation therapy plans. We applied the widely endorsed nomenclature standard TG-263 as the mapping target and quantified the structure name inconsistency in 2 real-world data sets.

Methods and Materials

The clinically relevant structures in the radiation therapy plans were identified by reference to randomized controlled trials. The Stature approach was used by clinicians to identify the synonyms for each relevant structure, which was then mapped to the corresponding TG-263 name. We applied Stature to standardize the structure names for 654 patients with prostate cancer (PCa) and 224 patients with head and neck squamous cell carcinoma (HNSCC) who received curative radiation therapy at our institution between 2007 and 2017. The accuracy of the Stature process was manually validated in a random sample from each cohort. For the HNSCC cohort we measured the resource requirements for Stature, and for the PCa cohort we demonstrated its impact on an example clinical analytics scenario.

Results

All but 1 synonym group (“Hydrogel”) was mapped to the corresponding TG-263 name, resulting in a TG-263 relabel rate of 99% (8837 of 8925 structures). For the PCa cohort, Stature matched a total of 5969 structures. Of these, 5682 structures were exact matches (ie, following local naming convention), 284 were matched via a synonym, and 3 required manual matching. This original radiation therapy structure names therefore had a naming inconsistency rate of 4.81%. For the HNSCC cohort, Stature mapped a total of 2956 structures (2638 exact, 304 synonym, 14 manual; 10.76% inconsistency rate) and required 7.5 clinician hours. The clinician hours required were one-fifth of those that would be required for manual relabeling. The accuracy of Stature was 99.97% (PCa) and 99.61% (HNSCC).

Conclusions

The Stature approach was highly accurate and had significant resource efficiencies compared with manual curation.

A prospective in silico analysis of interdisciplinary and interobserver spatial variability in post-operative target delineation of high-risk oral cavity cancers: Does physician specialty matter?

Background

The aim of this study was to determine the interdisciplinary agreement in identifying the post-operative tumor bed.

Methods

Three radiation oncologists (ROs), four surgeons, and three radiologists segmented post-operative tumor and nodal beds for three patients with oral cavity cancer. Specialty cohort composite contours were created by STAPLE algorithm implementation results for interspecialty comparison. Dice similarity coefficient and Hausdorff distance were utilized to compare spatial differentials between specialties.

Results

There were significant differences between disciplines in target delineation. There was unacceptable variation in Dice similarity coefficient for each observer and discipline when compared to the STAPLE contours. Within surgery and radiology disciplines, there was good consistency in volumes. ROs and radiologists have similar Dice similarity coefficient scores compared to surgeons.

Conclusion

There were significant interdisciplinary differences in perceptions of tissue-at-risk. Better communication and explicit description of at-risk areas between disciplines is required to ensure high-risk areas are adequately targeted.