Quality improvements in prostate radiotherapy: Outcomes and impact of comprehensive quality assurance during the TROG 03.04 ‘RADAR’ trial

Radiotherapy trial quality assurance processes: a systematic review

Quality assurance remains a neglected component of many trials, particularly for technical interventions, such as surgery and radiotherapy, for which quality of treatment is an important component in defining outcomes. We aimed to evaluate evidence for the processes used in radiotherapy quality assurance of clinical trials. A systematic review was undertaken focusing on use of a pre-trial outlining benchmark case and subsequent on-trial individual case reviews of outlining for recruited patients. These pre-trial and on-trial checks are used to ensure consistency and standardisation of treatment for each patient recruited to the trial by confirming protocol compliance. Non-adherence to the trial protocol has been shown to have a negative effect on trial outcomes. 29 studies published between January, 2000, and December, 2022, were identified that reported on either outlining benchmark case results or outlining individual case review results, or both. The trials identified varied in their use of radiotherapy quality assurance practices and reporting of outcomes was inconsistent. Deviations from trial protocols were frequent, particularly regarding outlining. Studies correlating benchmark case results with on-trial individual case reviews provided mixed results, meaning firm conclusions could not be drawn regarding the influence of the pre-trial benchmark case on subsequent on-trial performance. The optimal radiotherapy quality assurance processes were unclear, and there was little evidence available. Improved reporting of outcomes from radiotherapy quality assurance programmes is needed to develop an evidence base for the optimal approach to radiotherapy quality assurance in trials.

Pretrial Quality Assurance for Hypofractionated Salvage Radiation Therapy After Prostatectomy in the Multi-Institutional PERYTON-trial

Purpose

The PERYTON trial is a multicenter randomized controlled trial that will investigate whether the treatment outcome of salvage external beam radiation therapy (sEBRT) will be improved with hypofractionated radiation therapy. A pretrial quality assurance (QA) program was undertaken to ensure protocol compliance within the PERYTON trial and to assess variation in sEBRT treatment protocols between the participating centers.

Methods and Materials

Completion of the QA program was mandatory for each participating center (N = 8) to start patient inclusion. The pretrial QA program included (1) a questionnaire on the center-specific sEBRT protocol, (2) a delineation exercise of the clinical target volume (CTV) and organs at risk, and (3) a treatment planning exercise. All contours were analyzed using the pairwise dice similarity coefficient (DSC) and the 50th and 95th percentile Hausdorff distance (HD50 and HD95, respectively). The submitted treatment plans were reviewed for protocol compliance.

Results

The results of the questionnaire showed that high-quality, state-of-the-art radiation therapy techniques were used in the participating centers and identified variations of the sEBRT protocols used concerning the position verification and preparation techniques. The submitted CTVs showed significant variation, with a range in volume of 29 cm3 to 167 cm3, a mean pairwise DSC of 0.52, and a mean HD50 and HD95 of 2.3 mm and 24.4 mm, respectively. Only in 1 center the treatment plan required adaptation before meeting all constraints of the PERYTON protocol.

Conclusions

The pretrial QA of the PERYTON trial demonstrated that high-quality, but variable, radiation techniques were used in the 8 participating centers. The treatment planning exercise confirmed that the dose constraints of the PERYTON protocol were feasible for all participating centers. The observed variation in CTV delineation led to agreement on a new (image-based) delineation guideline to be used by all participating centers within the PERYTON trial.

Assessment of deep learning-based auto-contouring on interobserver consistency in target volume and organs-at-risk delineation for breast cancer: Implications for RTQA program in a multi-institutional study

PURPOSE

To quantify interobserver variation (IOV) in target volume and organs-at-risk (OAR) contouring across 31 institutions in breast cancer cases and to explore the clinical utility of deep learning (DL)-based auto-contouring in reducing potential IOV.

METHODS AND MATERIALS

In phase 1, two breast cancer cases were randomly selected and distributed to multiple institutions for contouring six clinical target volumes (CTVs) and eight OAR. In Phase 2, auto-contour sets were generated using a previously published DL Breast segmentation model and were made available for all participants. The difference in IOV of submitted contours in phases 1 and 2 was investigated quantitatively using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). The qualitative analysis involved using contour heat maps to visualize the extent and location of these variations and the required modification.

RESULTS

Over 800 pairwise comparisons were analysed for each structure in each case. Quantitative phase 2 metrics showed significant improvement in the mean DSC (from 0.69 to 0.77) and HD (from 34.9 to 17.9 mm). Quantitative analysis showed increased interobserver agreement in phase 2, specifically for CTV structures (5-19 %), leading to fewer manual adjustments. Underlying IOV differences causes were reported using a questionnaire and hierarchical clustering analysis based on the volume of CTVs.

CONCLUSION

DL-based auto-contours improved the contour agreement for OARs and CTVs significantly, both qualitatively and quantitatively, suggesting its potential role in minimizing radiation therapy protocol deviation.

The Importance of Quality Assurance in Radiation Oncology Clinical Trials

Clinical trials have been the center of progress in modern medicine. In oncology, we are fortunate to have a structure in place through the National Clinical Trials Network (NCTN). The NCTN provides the infrastructure and a forum for scientific discussion to develop clinical concepts for trial design. The NCTN also provides a network group structure to administer trials for successful trial management and outcome analyses. There are many important aspects to trial design and conduct. Modern trials need to ensure appropriate trial conduct and secure data management processes. Of equal importance is the quality assurance of a clinical trial. If progress is to be made in oncology clinical medicine, investigators and patient care providers of service need to feel secure that trial data is complete, accurate, and well-controlled in order to be confident in trial analysis and move trial outcome results into daily practice. As our technology has matured, so has our need to apply technology in a uniform manner for appropriate interpretation of trial outcomes. In this article, we review the importance of quality assurance in clinical trials involving radiation therapy. We will include important aspects of institution and investigator credentialing for participation as well as ongoing processes to ensure that each trial is being managed in a compliant manner. We will provide examples of the importance of complete datasets to ensure study interpretation. We will describe how successful strategies for quality assurance in the past will support new initiatives moving forward.

Validation of prediction models for radiation-induced late rectal bleeding: evidence from a large pooled population of prostate cancer patients

PURPOSE

To validate published models for the risk estimate of grade≥1 (G1+), grade≥2 (G2+) and grade=3 (G3) late rectal bleeding (LRB) after radical radiotherapy for prostate cancer in a large pooled population from three prospective trials.

MATERIALS AND METHODS

The external validation population included patients from Europe, and Oceanian centres enrolled between 2003 and 2014. Patients received 3DCRT or IMRT at doses between 66-80 Gy. IMRT was administered with conventional or hypofractionated schemes (2.35-2.65 Gy/fr). LRB was prospectively scored using patient-reported questionnaires (LENT/SOMA scale) with a 3-year follow-up. All Normal Tissue Complication Probability (NTCP) models published until 2021 based on the Equivalent Uniform Dose (EUD) from the rectal Dose Volume Histogram (DVH) were considered for validation. Model performance in validation was evaluated through calibration and discrimination.

RESULTS

Sixteen NTCP models were tested on data from 1633 patients. G1+ LRB was scored in 465 patients (28.5%), G2+ in 255 patients (15.6%) and G3 in 112 patients (6.8%). The best performances for G2+ and G3 LRB highlighted the importance of the medium-high doses to the rectum (volume parameters n=0.24 and n=0.18, respectively). Good performance was seen for models of severe LRB. Moreover, a multivariate model with two clinical factors found the best calibration slope.

CONCLUSION

Five published NTCP models developed on non-contemporary cohorts were able to predict a relative increase in the toxicity response in a more recent validation population. Compared to QUANTEC findings, dosimetric results pointed toward mid-high doses of rectal DVH. The external validation cohort confirmed abdominal surgery and cardiovascular diseases as risk factors.

Dummy run quality assurance study in the Korean Radiation Oncology Group 19 − 09 multi-institutional prospective cohort study of breast cancer

Background

The Korean Radiation Oncology Group (KROG) 19 − 09 prospective cohort study aims to determine the effect of regional nodal irradiation on regional recurrence rates in ypN0 breast cancer patients. Dosimetric variations between radiotherapy (RT) plans of participating institutions may affect the clinical outcome of the study. We performed this study to assess inter-institutional dosimetric variations by dummy run.

Methods

Twelve participating institutions created RT plans for four clinical scenarios using computed tomography images of two dummy cases. Based on a reference structure set, we analyzed dose-volume histograms after collecting the RT plans.

Results

We found variations in dose distribution between institutions, especially in the regional nodal areas. Whole breast and regional nodal irradiation (WBI + RNI) plans had lower inter-institutional agreement and similarity for 95% isodose lines than WBI plans. Fleiss’s kappa values, which were used to measure inter-institutional agreement for the 95% isodose lines, were 0.830 and 0.767 for the large and medium breast WBI plans, respectively, and 0.731 and 0.679 for the large and medium breast WBI + RNI plans, respectively. There were outliers in minimum dose delivered to 95% of the structure (D95%) of axillary level 1 among WBI plans and in D95% of the interpectoral region and axillary level 4 among WBI + RNI plans.

Conclusion

We found inter-institutional and inter-case variations in radiation dose delivered to target volumes and organs at risk. As KROG 19 − 09 is a prospective cohort study, we accepted the dosimetric variation among the different institutions. Actual patient RT plan data should be collected to achieve reliable KROG 19 − 09 study results.

Autosegmentation based on different-sized training datasets of consistently-curated volumes and impact on rectal contours in prostate cancer radiation therapy

Background and purpose

Autosegmentation techniques are emerging as time-saving means for radiation therapy (RT) contouring, but the understanding of their performance on different datasets is limited. The aim of this study was to determine agreement between rectal volumes by an existing autosegmentation algorithm and manually-delineated rectal volumes in prostate cancer RT. We also investigated contour quality by different-sized training datasets and consistently-curated volumes for retrained versions of this same algorithm.

Materials and methods

Single-institutional data from 624 prostate cancer patients treated to 50–70 Gy were used. Manually-delineated clinical rectal volumes (clinical) and consistently-curated volumes recontoured to one anatomical guideline (reference) were compared to autocontoured volumes by a commercial autosegmentation tool based on deep-learning (v1; n = 891, multiple-institutional data) and retrained versions using subsets of the curated volumes (v32/64/128/256; n = 32/64/128/256). Evaluations included dose-volume histogram metrics, Dice similarity coefficients, and Hausdorff distances; differences between groups were quantified using parametric or non-parametric hypothesis testing.

Results

Volumes by v1-256 (76–78 cm³) were larger than reference (75 cm³) and clinical (76 cm³). Mean doses by v1-256 (24.2–25.2 Gy) were closer to reference (24.2 Gy) than to clinical (23.8 Gy). Maximum doses were similar for all volumes (65.7–66.0 Gy). Dice for v1-256 and reference (0.87–0.89) were higher than for v1-256 and clinical (0.86–0.87) with corresponding Hausdorff comparisons including reference smaller than comparisons including clinical (5–6 mm vs. 7–8 mm).

Conclusion

Using small single-institutional RT datasets with consistently-defined rectal volumes when training autosegmentation algorithms created contours of similar quality as the same algorithm trained on large multi-institutional datasets.

Adherence to contouring and treatment planning requirements within a multicentric trial -results of the quality assurance of the SAKK 09/10 trial

PURPOSE

To evaluate the results of the radiation therapy (RT) quality assurance (QA) program of the phase III randomized "XXXX-Anonymized for Review" trial in biochemically recurrent prostate cancer (PC) patients after prostatectomy.

METHODS AND MATERIALS

Within the "XXXX-Anonymized for Review" trial testing 64Gy versus 70Gy salvage RT, a central collection of treatment plans were performed, which were thoroughly reviewed by a dedicated medical physicist and radiation oncologist. Adherence to the treatment protocol and specifically to the European Organization for the Research and Treatment of Cancer (EORTC) guidelines for target volume definition (classified as deviation observed yes vs. no) and its potential correlation with acute and late toxicity (Common Terminology Criteria for Adverse Events (CTCAE) v4.0) and freedom from biochemical progression (FFBP) were investigated.

RESULTS

The treatment plans of 344 patients treated between February 2011 and April 2014 depicted important deviations to the EORTC guidelines and to the recommendations per trial protocol. For example, in up to half of the cases, the delineated structures deviated from the protocol (e.g., prostate bed (PB) in 48.8%, rectal wall (RW) in 41%). In addition, variations in clinical (CTV) - and planning target volume (PTV) occurred frequently (e.g., CTV and PTV deviations in up to 42.4% and 25.9%, respectively). The detected deviations showed a significant association with a lower risk of grade ≥ 2 gastrointestinal (GI) acute toxicity when CTV not overlapped RW vs. CTV overlapping RW, (OR 0.43; CI [0.22, 0.85]; p= 0.014), and a higher rate of grade ≥ 2 late genitourinary (GU) toxicity in case of the CTV overlapped with RW, (OR 2.58; CI [1.17, 5.72]; p= 0.019). A marginally significant lower risk of grade ≥ 2 late GU toxicity in patients when PB not overlapping RW versus overlapping RW was observed (OR 0.51; CI [0.25, 1.03]; p= 0.06). In addition, a marginally significant decrease of FFBP in patients with PTV not including surgical clips as potential markers of the limits of the prostate bed, (HR 1.44; CI [0.96, 2.17]; p= 0.07) was observed.

CONCLUSIONS

Despite a thorough QA program, the central review of a phase-III trial showed limited adherence to treatment protocol recommendations which was associated with a higher risk of toxicity by means of acute or late GI or GU toxicity and showed a trend towards worse FFBP. Data from this QA review may help refine future QA programs and prostate bed delineation guidelines.

The radiotherapy quality assurance gap among phase III cancer clinical trials

PURPOSE

Quality assurance (QA) practices improve the quality level of oncology trials by ensuring that the protocol is followed and the results are valid and reproducible. This study investigated the utilization of QA among randomized controlled trials that involve radiotherapy (RT).

METHODS AND MATERIALS

We searched ClinicalTrials.gov in February 2020 for all phase III oncology randomized clinical trials (RCTs). These trials were screened for RT-specific RCTs that had published primary trial results. Information regarding QA in each trial was collected from the study publications and trial protocol if available. Two individuals independently performed trial screening and data collection. Pearson's Chi-square tests analyses were used to assess factors that were associated with QA inclusion in RT trials.

RESULTS

Forty-two RCTs with RT as the primary intervention or as a mandatory component of the protocol were analyzed; the earliest was started in 1994 and one trial was still active though not recruiting. Twenty-nine (69%) trials mandated RT quality assurance (RTQA) practices as part of the trial protocol, with 19 (45%) trials requiring institutional credentialing. Twenty-one (50%) trials published protocol deviation outcomes. Clinical trials involving advanced radiation techniques (IMRT, VMAT, SRS, SBRT) did not include more RTQA than trials without these advanced techniques (73% vs. 65%, p = 0.55). Trials that reported protocol deviation outcomes were associated with mandating RTQA in their protocols as compared to trials that did not report these outcomes (100% vs. 38%, p < 0.001).

CONCLUSIONS

There is a lack of RTQA utilization and transparency in RT clinical trials. It is imperative for RT trials to include increased QA for safe, consistent, and high-quality RT planning and delivery.

Automatic radiotherapy delineation quality assurance on prostate MRI with deep learning in a multicentre clinical trial

Volume delineation quality assurance (QA) is particularly important in clinical trial settings where consistent protocol implementation is required, as outcomes will affect future as well current patients. Currently, where feasible, this is conducted manually, which is time consuming and resource intensive. Although previous studies mostly focused on automating delineation QA on CT, magnetic resonance imaging (MRI) is being increasingly used in radiotherapy treatment. In this work, we propose to perform automatic delineation QA on prostate MRI for both the clinical target volume (CTV) and organs-at-risk (OARs) by using delineations generated by 3D Unet variants as benchmarks for QA. These networks were trained on a small gold standard atlas set and applied on a multicentre radiotherapy clinical trial dataset to generate benchmark delineations. Then, a QA stage was designed to recommend 'pass', 'minor correction' and 'major correction' for each manual delineation in the trial set by thresholding its Dice similarity coefficient to the network generated delineation. Among all 3D Unet variants explored, the Unet with anatomical gates in an AtlasNet architecture performed the best in delineation QA, achieving an area under the receiver operating characteristics curve of 0.97, 0.92, 0.89 and 0.97 for identifying unacceptable (major correction) delineations with a sensitivity of 0.93, 0.73, 0.74 and 0.90 at a specificity of 0.93, 0.86, 0.86 and 0.95 for bladder, prostate CTV, rectum and gel spacer respectively. To the best of our knowledge, this is the first study to propose automated delineation QA for a multicentre radiotherapy clinical trial with treatment planning MRI. The methods proposed in this work can potentially improve the accuracy and consistency of CTV and OAR delineation in radiotherapy treatment planning.

Single-fraction prostate stereotactic body radiotherapy: Dose reconstruction with electromagnetic intrafraction motion tracking

PURPOSE

To reconstruct the dose delivered during single-fraction urethra-sparing prostate stereotactic body radiotherapy (SBRT) accounting for intrafraction motion monitored by intraprostatic electromagnetic transponders (EMT).

METHODS

We analyzed data of 15 patients included in the phase I/II "ONE SHOT" trial and treated with a single fraction of 19 Gy to the planning target volume (PTV) and 17 Gy to the urethra planning risk volume. During delivery, prostate motion was tracked with implanted EMT. SBRT was interrupted when a 3-mm threshold was trespassed and corrected unless the offset was transient. Motion-encoded reconstructed (MER) plans were obtained by splitting the original plans into multiple sub-beams with isocenter shifts based on recorded EMT positions, mimicking prostate motion during treatment. We analyzed intrafraction motion and compared MER to planned doses.

RESULTS

The median EMT motion range (±SD) during delivery was 0.26 ± 0.09, 0.22 ± 0.14 and 0.18 ± 0.10 cm in the antero-posterior, supero-inferior, and left-right axes, respectively. Treatment interruptions were needed for 8 patients because of target motion beyond limits in the antero-posterior (n = 6) and/or supero-inferior directions (n = 4). Comparing MER vs. original plan there was a median relative dose difference of -1.9% (range, -7.9 to -1.0%) and of +0.5% (-0.3-1.7%) for PTV D_98% and D_2%, respectively. The clinical target volume remained sufficiently covered with a median D_98% difference of -0.3% (-1.6-0.5%). Bladder and rectum dosimetric parameters showed significant differences between original and MER plans, but mostly remained within acceptable limits.

CONCLUSIONS

The dosimetric impact of intrafraction prostate motion was minimal for target coverage for single-fraction prostate SBRT with real-time electromagnetic tracking combined with beam gating.

Urethra-Sparing Stereotactic Body Radiation Therapy for Prostate Cancer: Quality Assurance of a Randomized Phase 2 Trial

PURPOSE

To present the radiation therapy quality assurance results from a prospective multicenter phase 2 randomized trial of short versus protracted urethra-sparing stereotactic body radiation therapy (SBRT) for localized prostate cancer.

METHODS AND MATERIALS

Between 2012 and 2015, 165 patients with prostate cancer from 9 centers were randomized and treated with SBRT delivered either every other day (arm A, n = 82) or once a week (arm B, n = 83); 36.25 Gy in 5 fractions were prescribed to the prostate with (n = 92) or without (n = 73) inclusion of the seminal vesicles (SV), and the urethra planning-risk volume received 32.5 Gy. Patients were treated either with volumetric modulated arc therapy (VMAT; n = 112) or with intensity modulated radiation therapy (IMRT; n = 53). Deviations from protocol dose constraints, planning target volume (PTV) homogeneity index, PTV Dice similarity coefficient, and number of monitor units for each treatment plan were retrospectively analyzed. Dosimetric results of VMAT versus IMRT and treatment plans with versus without inclusion of SV were compared.

RESULTS

At least 1 major protocol deviation occurred in 51 patients (31%), whereas none was observed in 41. Protocol violations were more frequent in the IMRT group (P < .001). Furthermore, the use of VMAT yielded better dosimetric results than IMRT for urethra planning-risk volume D_98% (31.1 vs 30.8 Gy, P < .0001), PTV D_2% (37.9 vs 38.7 Gy, P < .0001), homogeneity index (0.09 vs 0.10, P < .0001), Dice similarity coefficient (0.83 vs 0.80, P < .0001), and bladder wall V_50% (24.5% vs 33.5%, P = .0001). To achieve its goals volumetric modulated arc therapy required fewer monitor units than IMRT (2275 vs 3378, P <.0001). The inclusion of SV in the PTV negatively affected the rectal wall V_90% (9.1% vs 10.4%, P = .0003) and V_80% (13.2% vs 15.7%, P = .0003).

CONCLUSIONS

Protocol deviations with potential impact on tumor control or toxicity occurred in 31% of patients in this prospective clinical trial. Protocol deviations were more frequent with IMRT. Prospective radiation therapy quality assurance protocols should be strongly recommended for SBRT trials to minimize potential protocol deviations.

Radiotherapy Quality Assurance for the CHHiP Trial: Conventional Versus Hypofractionated High-Dose Intensity-Modulated Radiotherapy in Prostate Cancer

AIMS

The CHHiP trial investigated the use of moderate hypofractionation for the treatment of localised prostate cancer using intensity-modulated radiotherapy (IMRT). A radiotherapy quality assurance programme was developed to assess compliance with treatment protocol and to audit treatment planning and dosimetry of IMRT. This paper considers the outcome and effectiveness of the programme.

MATERIALS AND METHODS

Quality assurance exercises included a pre-trial process document and planning benchmark cases, prospective case reviews and a dosimetry site visit on-trial and a post-trial feedback questionnaire.

RESULTS

In total, 41 centres completed the quality assurance programme (37 UK, four international) between 2005 and 2010. Centres used either forward-planned (field-in-field single phase) or inverse-planned IMRT (25 versus 17). For pre-trial quality assurance exercises, 7/41 (17%) centres had minor deviations in their radiotherapy processes; 45/82 (55%) benchmark plans had minor variations and 17/82 (21%) had major variations. One hundred prospective case reviews were completed for 38 centres. Seventy-one per cent required changes to clinical outlining pre-treatment (primarily prostate apex and base, seminal vesicles and penile bulb). Errors in treatment planning were reduced relative to pre-trial quality assurance results (49% minor and 6% major variations). Dosimetry audits were conducted for 32 centres. Ion chamber dose point measurements were within ±2.5% in the planning target volume and ±8% in the rectum. 28/36 films for combined fields passed gamma criterion 3%/3 mm and 11/15 of IMRT fluence film sets passed gamma criterion 4%/4 mm using a 98% tolerance. Post-trial feedback showed that trial participation was beneficial in evolving clinical practice and that the quality assurance programme helped some centres to implement and audit prostate IMRT.

CONCLUSION

Overall, quality assurance results were satisfactory and the CHHiP quality assurance programme contributed to the success of the trial by auditing radiotherapy treatment planning and protocol compliance. Quality assurance supported the introduction of IMRT in UK centres, giving additional confidence and external review of IMRT where it was a newly adopted technique.

Impact of deviations in target volume delineation - Time for a new RTQA approach?

The international radiotherapy community has recognised that non-adherence to RT protocols can influence trial endpoints. However this conclusion is based on studies predominantly assessing the impact of deviations in dosimetric or treatment delivery protocol parameters rather than target volume delineation (TVD). This review evaluates the assessment of TVD within Radiation Therapy Quality Assurance (RTQA) programmes in clinical trials and the clinical impact of TVD protocol deviations. The implications for RTQA programmes are discussed. MEDLINE, PreMEDLINE, Embase, Cochrane Library, Web of Science, OpenGrey, WHO International Clinical Trials Registry Platform portal and ClinicalTrials.gov were searched. Full-length articles and conference abstracts were included to avoid publication bias. 5864 abstracts were screened for relevance; 94 full-length articles were reviewed and 5 relevant trials identified. Various classification systems were used to assess protocol deviations; 'unacceptable' or 'major' deviations in TVD occurred in 2.9-13.4% of assessed RT plans (when reported). It was often not possible to establish deviation rates specifically related to TVD as these were frequently combined with other types of protocol deviations. Details on the nature of unacceptable deviations was also not routinely reported and difficulties in establishing a 'consensus' for appropriate TVD for on-trial patients highlighted. Results suggest that deviations in TVD were associated with poorer outcomes for overall survival, local control and treatment-related toxicity; however the data were heterogeneous. RTQA of TVD was retrospective and feedback on the quality of TVD to recruiting centres was not standard. In summary, few trials have published outcomes on the impact of assessing the quality of TVD in trials. We propose that a new approach is now required. Unacceptable TVD deviations must be clearly defined at the time of protocol development to minimise interobserver variation, thereby promoting consistency in RTQA feedback. Prospective TVD reviews should be implemented for trials involving novel or complex RT techniques to identify deviations that require modification prior to treatment delivery. Furthermore, the consistent reporting of RTQA programme outcomes, both within and across trial groups, is of paramount importance to accelerate the evidence-base for the best RTQA approach when assessing TVD and to enable the impact on clinical outcomes within RT trials to be assessed.

Estimation of Hounsfield unit conversion parameters for pelvic CT images

Linear scaling is used to convert raw computed tomography (CT) pixel values into Hounsfield units corresponding to different tissue values. Analysis of a benchmarking study, presented here, where the same CT scan was imported into and then exported from multiple radiotherapy treatment planning systems, found inconsistencies in HU scaling parameter values exported along with the images, particularly when images were exported using the Radiation Therapy Oncology Group format. Several methods of estimating conversion parameters, based on estimating pixel values corresponding to air and water within the image, for pelvic CT images from a large multi-centre trial were compared against original Digital Imaging and Communications in Medicine export parameters. In general using the mean of a sample region at the centroid of the bladder to estimate the value of water was more accurate than using the minimum or maximum or a single value at the centroid. Accuracy of methods of air estimation tested were dependent in part on features of the CT scanners and treatment planning systems, making it difficult to pick one method as superior that was independent of scanner and treatment planning system type. Based on the above analysis, methods for estimating air and water were selected for use in performing linear scaling of a set of pelvic CT images prior to their use in an interpatient image registration application. The selected methods were validated against a more recent and homogeneous dataset. Estimation error was found to be much lower within the validation set.

A new era for clinical trial quality assurance: A credentialing programme for RTT led adaptive radiotherapy

•A multi-centre QA programme incorporating adaptive plan selection has been developed.•This novel QA approach has been validated by 71 RTTs from ten UK centres.•A multidisciplinary approach is essential in the development of a credentialing programme.

Association between measures of treatment quality and disease progression in prostate cancer radiotherapy: An exploratory analysis from the TROG 03.04 RADAR trial

INTRODUCTION

Quality assurance methods are incorporated into multicentre radiotherapy clinical trials for ensuring consistent application of trial protocol and quantifying treatment uncertainties. The study's purpose was to determine whether post-treatment disease progression is associated with measures of the quality of radiotherapy treatment.

METHODS

The TROG 03.04 RADAR trial tested the impact of androgen deprivation on prostate cancer patients receiving dose-escalated external beam radiation therapy. The trial incorporated a plan-review process and Level III dosimetric intercomparison at each centre, from which variables suggestive of treatment quality were collected. Kaplan-Meier statistics and Fine and Gray competing risk modelling were employed to test for associations between quality-related variables and the participant outcome local composite progression.

RESULTS

Increased 'dose-difference' at the prostatic apex and at the anterior rectal wall, between planned and measured dose, was associated with reduced progression. Participants whose treatment plans included clinical target volume (CTV) to planning target volume (PTV) margins exceeding protocol requirements also experienced reduced progression. Other quality-related variables, including total accrual from participating centres, measures of target coverage and other variations from protocol, were not significantly associated with progression.

CONCLUSIONS

This analysis has revealed the association of several treatment quality factors with disease progression. Increased dose and dose margin coverage in the prostate region can reduce disease progression. Extensive and rigorous monitoring has helped to maximise treatment quality, reducing the incidence of quality-indicator outliers, and thus reduce the chance of observing significant associations with progression rates.

Statistical-learning strategies generate only modestly performing predictive models for urinary symptoms following external beam radiotherapy of the prostate: A comparison of conventional and machine-learning methods

PURPOSE

Given the paucity of available data concerning radiotherapy-induced urinary toxicity, it is important to ensure derivation of the most robust models with superior predictive performance. This work explores multiple statistical-learning strategies for prediction of urinary symptoms following external beam radiotherapy of the prostate.

METHODS

The performance of logistic regression, elastic-net, support-vector machine, random forest, neural network, and multivariate adaptive regression splines (MARS) to predict urinary symptoms was analyzed using data from 754 participants accrued by TROG03.04-RADAR. Predictive features included dose-surface data, comorbidities, and medication-intake. Four symptoms were analyzed: dysuria, haematuria, incontinence, and frequency, each with three definitions (grade ≥ 1, grade ≥ 2 and longitudinal) with event rate between 2.3% and 76.1%. Repeated cross-validations producing matched models were implemented. A synthetic minority oversampling technique was utilized in endpoints with rare events. Parameter optimization was performed on the training data. Area under the receiver operating characteristic curve (AUROC) was used to compare performance using sample size to detect differences of ≥0.05 at the 95% confidence level.

RESULTS

Logistic regression, elastic-net, random forest, MARS, and support-vector machine were the highest-performing statistical-learning strategies in 3, 3, 3, 2, and 1 endpoints, respectively. Logistic regression, MARS, elastic-net, random forest, neural network, and support-vector machine were the best, or were not significantly worse than the best, in 7, 7, 5, 5, 3, and 1 endpoints. The best-performing statistical model was for dysuria grade ≥ 1 with AUROC ± standard deviation of 0.649 ± 0.074 using MARS. For longitudinal frequency and dysuria grade ≥ 1, all strategies produced AUROC>0.6 while all haematuria endpoints and longitudinal incontinence models produced AUROC<0.6.

CONCLUSIONS

Logistic regression and MARS were most likely to be the best-performing strategy for the prediction of urinary symptoms with elastic-net and random forest producing competitive results. The predictive power of the models was modest and endpoint-dependent. New features, including spatial dose maps, may be necessary to achieve better models.

Intervention quality is not routinely assessed in Cochrane systematic reviews of radiation therapy interventions

INTRODUCTION

The aim of this study was to maximise the benefits from clinical trials involving technological interventions such as radiation therapy. High compliance to the quality assurance protocols is crucial. We assessed whether the quality of radiation therapy intervention was evaluated in Cochrane systematic reviews.

METHODS

We searched 416 published Cochrane systematic reviews and identified 67 Cochrane systematic reviews that investigated radiation therapy or radiotherapy as an intervention. For each systematic review, either quality assurance or quality control for the intervention was identified by a description of such processes in the published systematic reviews.

RESULTS

Of the 67 Cochrane systematic reviews studied, only two mentioned quality assurance or quality control.

CONCLUSIONS

Our findings revealed that 65 of 67 (97%) Cochrane systematic reviews of radiation therapy interventions failed to consider the quality of the intervention. We suggest that advice about the evaluation of intervention quality be added to author support materials.

Modeling Urinary Dysfunction After External Beam Radiation Therapy of the Prostate Using Bladder Dose-Surface Maps: Evidence of Spatially Variable Response of the Bladder Surface

PURPOSE

We assessed the association of the spatial distribution of dose to the bladder surface, described using dose-surface maps, with the risk of urinary dysfunction.

METHODS AND MATERIALS

The bladder dose-surface maps of 754 participants from the TROG 03.04-RADAR trial were generated from the volumetric data by virtually cutting the bladder at the sagittal slice, intersecting the bladder center-of-mass through to the bladder posterior and projecting the dose information on a 2-dimensional plane. Pixelwise dose comparisons were performed between patients with and without symptoms (dysuria, hematuria, incontinence, and an International Prostate Symptom Score increase of ≥10 [ΔIPSS10]). The results with and without permutation-based multiple-comparison adjustments are reported. The pixelwise multivariate analysis findings (peak-event model for dysuria, hematuria, and ΔIPSS10; event-count model for incontinence), with adjustments for clinical factors, are also reported.

RESULTS

The associations of the spatially specific dose measures to urinary dysfunction were dependent on the presence of specific symptoms. The doses received by the anteroinferior and, to lesser extent, posterosuperior surface of the bladder had the strongest relationship with the incidence of dysuria, hematuria, and ΔIPSS10, both with and without adjustment for clinical factors. For the doses to the posteroinferior region corresponding to the area of the trigone, the only symptom with significance was incontinence.

CONCLUSIONS

A spatially variable response of the bladder surface to the dose was found for symptoms of urinary dysfunction. Limiting the dose extending anteriorly might help reduce the risk of urinary dysfunction.

Modelling late stool frequency and rectal pain after radical radiotherapy in prostate cancer patients: Results from a large pooled population

AIM

To investigate late gastrointestinal toxicity in a large pooled population of prostate cancer patients treated with radical radiotherapy. Normal tissue complication probability models were developed for late stool frequency and late rectal pain.

METHODS AND MATERIALS

Population included 1336 patients, 3-year minimum follow-up, treated with 66-80Gy. Toxicity was scored with LENT-SOMA-scale. Two toxicity endpoints were considered: grade ⩾2 rectal pain and mean grade (average score during follow-up) in stool frequency >1. DVHs of anorectum were reduced to equivalent uniform dose (EUD). The best-value of the volume parameter n was determined through numerical optimization. Association between EUD/clinical factors and the endpoints was investigated by logistic analyses. Likelihood, Brier-score and calibration were used to evaluate models. External calibration was also carried out.

RESULTS

4% of patients (45/1122) reported mean stool frequency grade >1; grade ⩾2 rectal pain was present in the TROG 03.04 RADAR population only (21/677, 3.1%): for this endpoint, the analysis was limited to this population. Analysis of DVHs highlighted the importance of mid-range doses (30-50Gy) for both endpoints. EUDs calculated with n=1 (OR=1.04) and n=0.35 (OR=1.06) were the most suitable dosimetric descriptors for stool frequency and rectal pain respectively. The final models included EUD and cardiovascular diseases (OR=1.78) for stool frequency and EUD and presence of acute gastrointestinal toxicity (OR=4.2) for rectal pain.

CONCLUSION

Best predictors of stool frequency and rectal pain are consistent with findings previously reported for late faecal incontinence, indicating an important role in optimization of mid-range dose region to minimize these symptoms highly impacting the quality-of-life of long surviving patients.

Urinary symptoms following external beam radiotherapy of the prostate: Dose-symptom correlates with multiple-event and event-count models

BACKGROUND AND PURPOSE

This study aimed to compare urinary dose-symptom correlates after external beam radiotherapy of the prostate using commonly utilised peak-symptom models to multiple-event and event-count models which account for repeated events.

MATERIALS AND METHODS

Urinary symptoms (dysuria, haematuria, incontinence and frequency) from 754 participants from TROG 03.04-RADAR trial were analysed. Relative (R1-R75 Gy) and absolute (A60-A75Gy) bladder dose-surface area receiving more than a threshold dose and equivalent uniform dose using exponent a (range: a ∈[1 … 100]) were derived. The dose-symptom correlates were analysed using; peak-symptom (logistic), multiple-event (generalised estimating equation) and event-count (negative binomial regression) models.

RESULTS

Stronger dose-symptom correlates were found for incontinence and frequency using multiple-event and/or event-count models. For dysuria and haematuria, similar or better relationships were found using peak-symptom models. Dysuria, haematuria and high grade (⩾ 2) incontinence were associated to high dose (R61-R71 Gy). Frequency and low grade (⩾ 1) incontinence were associated to low and intermediate dose-surface parameters (R13-R41Gy). Frequency showed a parallel behaviour (a=1) while dysuria, haematuria and incontinence showed a more serial behaviour (a=4 to a ⩾ 100). Relative dose-surface showed stronger dose-symptom associations.

CONCLUSIONS

For certain endpoints, the multiple-event and event-count models provide stronger correlates over peak-symptom models. Accounting for multiple events may be advantageous for a more complete understanding of urinary dose-symptom relationships.

Dosimetry, clinical factors and medication intake influencing urinary symptoms after prostate radiotherapy: An analysis of data from the RADAR prostate radiotherapy trial

PURPOSE/OBJECTIVE

To identify dosimetry, clinical factors and medication intake impacting urinary symptoms after prostate radiotherapy.

MATERIAL AND METHODS

Data describing clinical factors and bladder dosimetry (reduced with principal component (PC) analysis) for 754 patients treated with external beam radiotherapy accrued by TROG 03.04 RADAR prostate radiotherapy trial were available for analysis. Urinary symptoms (frequency, incontinence, dysuria and haematuria) were prospectively assessed using LENT-SOMA to a median of 72months. The endpoints assessed were prevalence (grade ⩾1) at the end of radiotherapy (representing acute symptoms), at 18-, 36- and 54-month follow-ups (representing late symptoms) and peak late incidence including only grade ⩾2. Impact of factors was assessed using multivariate logistic regression models with correction for over-optimism.

RESULTS

Baseline symptoms, non-insulin dependent diabetes mellitus, age and PC1 (correlated to the mean dose) impact symptoms at >1 timepoints. Associations at a single timepoint were found for cerebrovascular condition, ECOG status and non-steroidal anti-inflammatory drug intake. Peak incidence analysis shows the impact of baseline, bowel and cerebrovascular condition and smoking status.

CONCLUSIONS

The prevalence and incidence analysis provide a complementary view for urinary symptom prediction. Sustained impacts across time points were found for several factors while some associations were not repeated at different time points suggesting poorer or transient impact.

Quality improvement process to assess tattoo alignment, set-up accuracy and isocentre reproducibility in pelvic radiotherapy patients

INTRODUCTION

This quality improvement study tested three methods of tattoo alignment and isocentre definition to investigate if aligning lateral tattoos to minimise pitch, roll and yaw decreased set-up error, and if defining the isocentre using the lateral tattoos for cranio-caudal (CC) position improved isocentre reproducibility. The study population was patients receiving curative external beam radiotherapy (EBRT) for prostate cancer. The results are applicable to all supine pelvic EBRT patients.

METHODS

The three sequential cohorts recruited 11, 11 and 10 patients respectively. A data set of 20 orthogonal pairs of electronic portal images (EPI) was acquired for each patient. EPIs were matched offline to digitally reconstructed radiographs. In cohort 1, lateral tattoos were adjusted to minimise roll. The anterior tattoo was used to define the isocentre. In cohort 2, lateral tattoos were aligned to minimise roll and yaw. Isocentre was defined as per cohort 1. In cohort 3, lateral tattoos were aligned as per cohort 2 and the anterior tattoo was adjusted to minimise pitch. Isocentre was defined by the lateral tattoos for CC position and the anterior tattoo for the left-right position.

RESULTS

Cohort 3 results were superior as CC systematic and random set-up errors reduced from -1.3 mm to -0.5 mm, and 3.1 mm to 1.4 mm respectively, from cohort 1 to cohort 3. Isocentre reproducibility also improved from 86.7% to 92.1% of treatment isocentres within 5 mm of the planned isocentre.

CONCLUSION

The methods of tattoo alignment and isocentre definition in cohort 3 reduced set-up errors and improved isocentre reproducibility.

Impact of treatment planning and delivery factors on gastrointestinal toxicity: an analysis of data from the RADAR prostate radiotherapy trial

BACKGROUND

To assess the impact of incremental modifications of treatment planning and delivery technique, as well as patient anatomical factors, on late gastrointestinal toxicity using data from the TROG 03.04 RADAR prostate radiotherapy trial.

METHODS

The RADAR trial accrued 813 external beam radiotherapy participants during 2003-2008 from 23 centres. Following review and archive to a query-able database, digital treatment plans and data describing treatment technique for 754 patients were available for analysis. Treatment demographics, together with anatomical features, were assessed using uni- and multivariate regression models against late gastrointestinal toxicity at 18-, 36- and 54-month follow-up. Regression analyses were reviewed in the context of dose-volume data for the rectum and anal canal.

RESULTS

A multivariate analysis at 36-month follow-up shows that patients planned using a more rigorous dose calculation algorithm (DCA) was associated with a lower risk of stool frequency (OR: 0.435, CI: 0.242-0.783, corrected p = 0.04). Patients using laxative as a method of bowel preparation had higher risk of having increased stool frequency compared to patients with no dietary intervention (OR: 3.639, CI: 1.502-8.818, corrected p = 0.04). Despite higher risks of toxicities, the anorectum, anal canal and rectum dose-volume histograms (DVH) indicate patients using laxative had unremarkably different planned dose distributions. Patients planned with a more rigorous DCA had lower median DVH values between EQD23 = 15 Gy and EQD23 = 35 Gy. Planning target volume (PTV), conformity index, rectal width and prescription dose were not significant when adjusted for false discovery rate. Number of beams, beam energy, treatment beam definition, positioning orientation, rectum-PTV separation, rectal length and mean cross sectional area did not affect the risk of toxicities.

CONCLUSIONS

The RADAR study dataset has allowed an assessment of technical modifications on gastrointestinal toxicity. A number of interesting associations were subsequently found and some factors, previously hypothesised to influence toxicity, did not demonstrate any significant impact. We recommend trial registries be encouraged to record technical modifications introduced during the trial in order for more powerful evidence to be gathered regarding the impact of the interventions.

Automatic segmentation of male pelvic anatomy on computed tomography images: a comparison with multiple observers in the context of a multicentre clinical trial

Background

This study investigates the variation in segmentation of several pelvic anatomical structures on computed tomography (CT) between multiple observers and a commercial automatic segmentation method, in the context of quality assurance and evaluation during a multicentre clinical trial.

Methods

CT scans of two prostate cancer patients (‘benchmarking cases’), one high risk (HR) and one intermediate risk (IR), were sent to multiple radiotherapy centres for segmentation of prostate, rectum and bladder structures according to the TROG 03.04 “RADAR” trial protocol definitions. The same structures were automatically segmented using iPlan software for the same two patients, allowing structures defined by automatic segmentation to be quantitatively compared with those defined by multiple observers. A sample of twenty trial patient datasets were also used to automatically generate anatomical structures for quantitative comparison with structures defined by individual observers for the same datasets.

Results

There was considerable agreement amongst all observers and automatic segmentation of the benchmarking cases for bladder (mean spatial variations < 0.4 cm across the majority of image slices). Although there was some variation in interpretation of the superior-inferior (cranio-caudal) extent of rectum, human-observer contours were typically within a mean 0.6 cm of automatically-defined contours. Prostate structures were more consistent for the HR case than the IR case with all human observers segmenting a prostate with considerably more volume (mean +113.3%) than that automatically segmented. Similar results were seen across the twenty sample datasets, with disagreement between iPlan and observers dominant at the prostatic apex and superior part of the rectum, which is consistent with observations made during quality assurance reviews during the trial.

Conclusions

This study has demonstrated quantitative analysis for comparison of multi-observer segmentation studies. For automatic segmentation algorithms based on image-registration as in iPlan, it is apparent that agreement between observer and automatic segmentation will be a function of patient-specific image characteristics, particularly for anatomy with poor contrast definition. For this reason, it is suggested that automatic registration based on transformation of a single reference dataset adds a significant systematic bias to the resulting volumes and their use in the context of a multicentre trial should be carefully considered.