Impact of deviations in target volume delineation - Time for a new RTQA approach?

Radiotherapy and theranostics: a Lancet Oncology Commission

Following on from the 2015 Lancet Oncology Commission on expanding global access to radiotherapy, Radiotherapy and theranostics: a Lancet Oncology Commission was created to assess the access and availability of radiotherapy to date and to address the important issue of access to the promising field of theranostics at a global level. A marked disparity in the availability of radiotherapy machines between high-income countries and low-income and middle-income countries (LMICs) has been identified previously and remains a major problem. The availability of a suitably trained and credentialled workforce has also been highlighted as a major limiting factor to effective implementation of radiotherapy, particularly in LMICs. We investigated initiatives that could mitigate these issues in radiotherapy, such as extended treatment hours, hypofractionation protocols, and new technologies. The broad implementation of hypofractionation techniques compared with conventional radiotherapy in prostate cancer and breast cancer was projected to provide radiotherapy for an additional 2·2 million patients (0·8 million patients with prostate cancer and 1·4 million patients with breast cancer) with existing resources, highlighting the importance of implementing new technologies in LMICs. A global survey undertaken for this Commission revealed that use of radiopharmaceutical therapy-other than 131I-was highly variable in high-income countries and LMICs, with supply chains, workforces, and regulatory issues affecting access and availability. The capacity for radioisotope production was highlighted as a key issue, and training and credentialling of health professionals involved in theranostics is required to ensure equitable access and availability for patient treatment. New initiatives-such as the International Atomic Energy Agency's Rays of Hope programme-and interest by international development banks in investing in radiotherapy should be supported by health-care systems and governments, and extended to accelerate the momentum generated by recognising global disparities in access to radiotherapy. In this Commission, we propose actions and investments that could enhance access to radiotherapy and theranostics worldwide, particularly in LMICs, to realise health and economic benefits and reduce the burden of cancer by accessing these treatments.

Automated segmentation in pelvic radiotherapy: A comprehensive evaluation of ATLAS-, machine learning-, and deep learning-based models

Artificial intelligence can standardize and automatize highly demanding procedures, such as manual segmentation, especially in an anatomical site as common as the pelvis. This study investigated four automated segmentation tools on computed tomography (CT) images in female and male pelvic radiotherapy (RT) starting from simpler and well-known atlas-based methods to the most recent neural networks-based algorithms. The evaluation included quantitative, qualitative and time efficiency assessments. A mono-institutional consecutive series of 40 cervical cancer and 40 prostate cancer structure sets were retrospectively selected. After a preparatory phase, the remaining 20 testing sets per each site were auto-segmented by the atlas-based model STAPLE, a Random Forest-based model, and two Deep Learning-based tools (DL), MVision and LimbusAI. Setting manual segmentation as the Ground Truth, 200 structure sets were compared in terms of Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Distance-to-Agreement Portion (DAP). Automated segmentation and manual correction durations were recorded. Expert clinicians performed a qualitative evaluation. In cervical cancer CTs, DL outperformed the other tools with higher quantitative metrics, qualitative scores, and shorter correction times. On the other hand, in prostate cancer CTs, the performance across all the analyzed tools was comparable in terms of both quantitative and qualitative metrics. Such discrepancy in performance outcome could be explained by the wide range of anatomical variability in cervical cancer with respect to the strict bladder and rectum filling preparation in prostate Stereotactic Body Radiation Therapy (SBRT). Decreasing segmentation times can reduce the burden of pelvic radiation therapy routine in an automated workflow.

Pilot study of radiation oncology peer review in low middle income country (LMIC) through cloud-based platform

PURPOSE

Peer review is an essential step in clinical quality assurance for radiation therapy. There are very little data on peer reviews from low-middle-income countries (LMIC). With increasing access to advanced technologies in LMIC also, peer review is becoming more important to ensure quality and standard of care. We evaluated cloud-based e-Peer review in our network of cancer centers in India with an aim to study its feasibility and impact on care.

MATERIALS AND METHODS

Four out of 13 cancer centers across India were selected for this pilot study. All team members were trained on the e-Peer review platform before the initiation of the study. A lead dosimetrist from a centralized planning site was selected to share new cases every week. Cases treated with only definitive intent were selected. The link to the cases was sent through an email to reviewing physicians. The following aspects were reviewed for each case. 1) Work up and staging. 2) Treatment intent and prescription. 3) Target contours. 4) Normal organ at risk contours. 5) Dose-volume-histogram (DVH) with clinical goals attached. Cases were marked as "Not Appropriate," "Appropriate," "Appropriate with minor finding," and "Represent with major revisions" as per volume and plan review.

RESULTS

Over a period of 3 months, 100 cases underwent peer review before the start of treatment. Median turnover time was 48 (interquartile range: 24-96) hours. The median time for review was 8 min with time to review cases requiring major and minor changes being 12 and 6 min, respectively (P < 0.001). Of all the cases reviewed, no changes, minor changes, and major changes were suggested for 36%, 48%, and 16% of cases, respectively. The most frequent reason for major changes was contouring corrections (15%). Also, 31.3% of major changes underwent recontouring and replanning before initiation of treatment.

CONCLUSION

Peer review was feasible in our setting through this cloud-based peer review system, with median turnover time and time taken for review being 48 h and 8 min, respectively. Like published data from the Western world, peer review led to changes that could impact patient care delivery and outcome. We plan to implement this across the remaining centers in our network.

Evaluation of a deep image-to-image network (DI2IN) auto-segmentation algorithm across a network of cancer centers

PURPOSE/OBJECTIVE S

Due to manual OAR contouring challenges, various automatic contouring solutions have been introduced. Historically, common clinical auto-segmentation algorithms used were atlas-based, which required maintaining a library of self-made contours. Searching the collection was computationally intensive and could take several minutes to complete. Deep learning approaches have shown significant benefits compared to atlas-based methods in improving segmentation accuracy and efficiency in auto-segmentation algorithms. This work represents the first multi-institutional study to describe and evaluate an AI algorithm for the auto-segmentation of organs at risk (OARs) based on a deep image-to-image network (DI2IN).

MATERIALS/METHODS

The AI-Rad Companion Organs RT (AIRC) algorithm (Siemens Healthineers, Erlangen, Germany) uses a two-step approach for segmentation. In the first step, the target organ region in the optimal input image is extracted using a trained deep reinforcement learning network (DRL), which is then used as input to create the contours in the second step based on DI2IN. The study was initially designed as a prospective single-center evaluation. The automated contours generated by AIRC were evaluated by three experienced board-certified radiation oncologists using a four-point scale where 4 is clinically usable and 1 requires re-contouring. After seeing favorable results in a single-center pilot study, we decided to expand the study to six additional institutions, encompassing eight additional evaluators for a total of 11 physician evaluators across seven institutions.

RESULTS

One hundred and fifty-six patients and 1366 contours were prospectively evaluated. The five most commonly contoured organs were the lung (136 contours, average rating = 4.0), spinal cord (106 contours, average rating = 3.1), eye globe (80 contours, average rating = 3.9), lens (77 contours, average rating = 3.9), and optic nerve (75 contours, average rating = 4.0). The average rating per evaluator per contour was 3.6. On average, 124 contours were evaluated by each evaluator. 65% of the contours were rated as 4, and 31% were rated as 3. Only 4% of contours were rated as 1 or 2. Thirty-three organs were evaluated in the study, with 19 structures having a 3.5 or above average rating (ribs, abdominopelvic cavity, skeleton, larynx, lung, aorta, brachial plexus, lens, eye globe, glottis, heart, parotid glands, bladder, kidneys, supraglottic larynx, submandibular glands, esophagus, optic nerve, oral cavity) and the remaining organs having a rating of 3.0 or greater (female breast, proximal femur, seminal vesicles, rectum, sternum, brainstem, prostate, brain, lips, mandible, liver, optic chiasm, spinal cord, spleen). No organ had an average rating below 3.

CONCLUSION

AIRC performed well with greater than 95% of contours accepted by treating physicians with no or minor edits. It supported a fully automated workflow with the potential for time savings and increased standardization with the use of AI-powered algorithms for high-quality OAR contouring.

Radiotherapy trial quality assurance processes: a systematic review

Quality assurance remains a neglected component of many trials, particularly for technical interventions, such as surgery and radiotherapy, for which quality of treatment is an important component in defining outcomes. We aimed to evaluate evidence for the processes used in radiotherapy quality assurance of clinical trials. A systematic review was undertaken focusing on use of a pre-trial outlining benchmark case and subsequent on-trial individual case reviews of outlining for recruited patients. These pre-trial and on-trial checks are used to ensure consistency and standardisation of treatment for each patient recruited to the trial by confirming protocol compliance. Non-adherence to the trial protocol has been shown to have a negative effect on trial outcomes. 29 studies published between January, 2000, and December, 2022, were identified that reported on either outlining benchmark case results or outlining individual case review results, or both. The trials identified varied in their use of radiotherapy quality assurance practices and reporting of outcomes was inconsistent. Deviations from trial protocols were frequent, particularly regarding outlining. Studies correlating benchmark case results with on-trial individual case reviews provided mixed results, meaning firm conclusions could not be drawn regarding the influence of the pre-trial benchmark case on subsequent on-trial performance. The optimal radiotherapy quality assurance processes were unclear, and there was little evidence available. Improved reporting of outcomes from radiotherapy quality assurance programmes is needed to develop an evidence base for the optimal approach to radiotherapy quality assurance in trials.

Assessment of deep learning-based auto-contouring on interobserver consistency in target volume and organs-at-risk delineation for breast cancer: Implications for RTQA program in a multi-institutional study

PURPOSE

To quantify interobserver variation (IOV) in target volume and organs-at-risk (OAR) contouring across 31 institutions in breast cancer cases and to explore the clinical utility of deep learning (DL)-based auto-contouring in reducing potential IOV.

METHODS AND MATERIALS

In phase 1, two breast cancer cases were randomly selected and distributed to multiple institutions for contouring six clinical target volumes (CTVs) and eight OAR. In Phase 2, auto-contour sets were generated using a previously published DL Breast segmentation model and were made available for all participants. The difference in IOV of submitted contours in phases 1 and 2 was investigated quantitatively using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). The qualitative analysis involved using contour heat maps to visualize the extent and location of these variations and the required modification.

RESULTS

Over 800 pairwise comparisons were analysed for each structure in each case. Quantitative phase 2 metrics showed significant improvement in the mean DSC (from 0.69 to 0.77) and HD (from 34.9 to 17.9 mm). Quantitative analysis showed increased interobserver agreement in phase 2, specifically for CTV structures (5-19 %), leading to fewer manual adjustments. Underlying IOV differences causes were reported using a questionnaire and hierarchical clustering analysis based on the volume of CTVs.

CONCLUSION

DL-based auto-contours improved the contour agreement for OARs and CTVs significantly, both qualitatively and quantitatively, suggesting its potential role in minimizing radiation therapy protocol deviation.

ARCHERY: a prospective observational study of artificial intelligence-based radiotherapy treatment planning for cervical, head and neck and prostate cancer - study protocol

INTRODUCTION

Fifty per cent of patients with cancer require radiotherapy during their disease course, however, only 10%-40% of patients in low-income and middle-income countries (LMICs) have access to it. A shortfall in specialised workforce has been identified as the most significant barrier to expanding radiotherapy capacity. Artificial intelligence (AI)-based software has been developed to automate both the delineation of anatomical target structures and the definition of the position, size and shape of the radiation beams. Proposed advantages include improved treatment accuracy, as well as a reduction in the time (from weeks to minutes) and human resources needed to deliver radiotherapy.

METHODS

ARCHERY is a non-randomised prospective study to evaluate the quality and economic impact of AI-based automated radiotherapy treatment planning for cervical, head and neck, and prostate cancers, which are endemic in LMICs, and for which radiotherapy is the primary curative treatment modality. The sample size of 990 patients (330 for each cancer type) has been calculated based on an estimated 95% treatment plan acceptability rate. Time and cost savings will be analysed as secondary outcome measures using the time-driven activity-based costing model. The 48-month study will take place in six public sector cancer hospitals in India (n=2), Jordan (n=1), Malaysia (n=1) and South Africa (n=2) to support implementation of the software in LMICs.

ETHICS AND DISSEMINATION

The study has received ethical approval from University College London (UCL) and each of the six study sites. If the study objectives are met, the AI-based software will be offered as a not-for-profit web service to public sector state hospitals in LMICs to support expansion of high quality radiotherapy capacity, improving access to and affordability of this key modality of cancer cure and control. Public and policy engagement plans will involve patients as key partners.

Systematic Review and Meta-analysis of the Association Between Radiation Therapy Treatment Volume and Patient Outcomes

PURPOSE

Evidence of a volume-outcome association in cancer surgery has shaped the centralization of cancer services; however, it is unknown whether a similar association exists for radiation therapy. The objective of this study was to determine the association between radiation therapy treatment volume and patient outcomes.

METHODS AND MATERIALS

This systematic review and meta-analysis included studies that compared outcomes of patients who underwent definitive radiation therapy at high-volume radiation therapy facilities (HVRFs) versus low-volume facilities (LVRFs). The systematic review used Ovid MEDLINE and Embase. For the meta-analysis, a random effects model was used. Absolute effects and hazard ratios (HRs) were used to compare patient outcomes.

RESULTS

The search identified 20 studies assessing the association between radiation therapy volume and patient outcomes. Seven of the studies looked at head and neck cancers (HNCs). The remaining studies covered cervical (4), prostate (4), bladder (3), lung (2), anal (2), esophageal (1), brain (2), liver (1), and pancreatic cancer (1). The meta-analysis demonstrated that HVRFs were associated with a lower chance of death compared with LVRFs (pooled HR, 0.90; 95% CI, 0.87- 0.94). HNCs had the strongest evidence of a volume-outcome association for both nasopharyngeal cancer (pooled HR, 0.74; 95% CI, 0.62-0.89) and nonnasopharyngeal HNC subsites (pooled HR, 0.80; 95% CI, 0.75-0.84), followed by prostate cancer (pooled HR, 0.92; 95% CI, 0.86-0.98). The remaining cancer types showed weak evidence of an association. The results also demonstrate that some centers defined as HVRFs are undertaking very few procedures per annum (<5 radiation therapy cases per year).

CONCLUSIONS

An association between radiation therapy treatment volume and patient outcomes exists for most cancer types. Centralization of radiation therapy services should be considered for cancer types with the strongest volume-outcome association, but the effect on equitable access to services needs to be explicitly considered.

A Quality Control Study on Involved Node Radiation Therapy in the European Organisation for Research and Treatment of Cancer/Lymphoma Study Association/Fondazione Italiana Linfomi H10 Trial on Stages I and II Hodgkin Lymphoma: Lessons Learned

PURPOSE

Involved node radiation therapy (INRT) was introduced in the European Organisation for Research and Treatment of Cancer/Lymphoma Study Association/Fondazione Italiana Linfomi H10 trial, a large multicenter trial in early-stage Hodgkin Lymphoma. The present study aimed to evaluate the quality of INRT in this trial.

METHODS AND MATERIALS

A retrospective, descriptive study was initiated to evaluate INRT in a representative sample encompassing approximately 10% of all irradiated patients in the H10 trial. Sampling was stratified by academic group, year of treatment, size of the treatment center, and treatment arm, and it was done proportional to the size of the strata. The sample was completed for all patients with known recurrences to enable future research on relapse patterns. Radiation therapy principle, target volume delineation and coverage, and applied technique and dose were evaluated using the EORTC Radiation Therapy Quality Assurance platform. Each case was reviewed by 2 reviewers and, in case of disagreement also by an adjudicator for a consensus evaluation.

RESULTS

Data were retrieved for 66 of 1294 irradiated patients (5.1%). Data collection and analysis were hampered more than anticipated by changes in archiving of diagnostic imaging and treatment planning systems during the running period of the trial. A review could be performed on 61 patients. The INRT principle was applied in 86.6%. Overall, 88.5% of cases were treated according to protocol. Unacceptable variations were predominately due to geographic misses of the target volume delineations. The rate of unacceptable variations decreased during trial recruitment.

CONCLUSIONS

The principle of INRT was applied in most of the reviewed patients. Almost 90% of the evaluated patients were treated according to the protocol. The present results should, however, be interpreted with caution because the number of patients evaluated was limited. Individual case reviews should be done in a prospective fashion in future trials. Radiation therapy Quality Assurance tailored to the clinical trial objectives is strongly recommended.

Advancing herbal medicine: enhancing product quality and safety through robust quality control practices

This manuscript provides an in-depth review of the significance of quality control in herbal medication products, focusing on its role in maintaining efficiency and safety. With a historical foundation in traditional medicine systems, herbal remedies have gained widespread popularity as natural alternatives to conventional treatments. However, the increasing demand for these products necessitates stringent quality control measures to ensure consistency and safety. This comprehensive review explores the importance of quality control methods in monitoring various aspects of herbal product development, manufacturing, and distribution. Emphasizing the need for standardized processes, the manuscript delves into the detection and prevention of contaminants, the authentication of herbal ingredients, and the adherence to regulatory standards. Additionally, it highlights the integration of traditional knowledge and modern scientific approaches in achieving optimal quality control outcomes. By emphasizing the role of quality control in herbal medicine, this manuscript contributes to promoting consumer trust, safeguarding public health, and fostering the responsible use of herbal medication products.

Radiotherapy quality assurance in paediatric clinical trials: first report from six QUARTET-affiliated trials

BACKGROUND AND PURPOSE

SIOP Europe's QUARTET project launched in 2016; aiming to improve access to high-quality radiotherapy for children and adolescents treated within clinical trials across Europe. The aim of this report is to present the profile of institutions participating in six QUARTET-affiliated trials and a description of the initial individual case review (ICR) outcomes.

METHODS

This is a two-part analysis. Firstly, using facility questionnaires, beam output audit certificates, and advanced technique credentialing records to create a profile of approved institutions, and secondly, collating trial records for ICRs submitted prior to 31/10/2022. Trials included are: SIOPEN HR-NBL1, SIOPEN-LINES, SIOPEN- VERITAS, SIOP-BTG HRMB, EpSSG-FaR-RMS, and SIOPEN HR-NBL2.

RESULTS

By 31/10/2022, a total of 103 institutions had commenced QUARTET site approval procedures to participate in QUARTET-affiliated trials; 66 sites across 20 countries were approved. These participating institutions were often paediatric referral sites with intensity modulated radiotherapy or proton beam therapy, designated paediatric radiation oncologists, and paediatric adapted facilities and imaging protocols available. In total, 263 patient plans were submitted for ICR, 254 ICRs from 15 countries were completed. ICRs had a rejection rate of 39.8%, taking an average of 1.4 submissions until approval was achieved. Target delineation was the most frequent reason for rejection.

CONCLUSION

The QUARTET facility questionnaire is a valuable tool for mapping resources, personnel, and technology available to children and adolescents receiving radiotherapy. Prospective ICR is essential for paediatric oncology clinical trials and should be prioritised to reduce protocol violations.

Automatic gross tumor segmentation of canine head and neck cancer using deep learning and cross-species transfer learning

Background

Radiotherapy (RT) is increasingly being used on dogs with spontaneous head and neck cancer (HNC), which account for a large percentage of veterinary patients treated with RT. Accurate definition of the gross tumor volume (GTV) is a vital part of RT planning, ensuring adequate dose coverage of the tumor while limiting the radiation dose to surrounding tissues. Currently the GTV is contoured manually in medical images, which is a time-consuming and challenging task.

Purpose

The purpose of this study was to evaluate the applicability of deep learning-based automatic segmentation of the GTV in canine patients with HNC.

Materials and methods

Contrast-enhanced computed tomography (CT) images and corresponding manual GTV contours of 36 canine HNC patients and 197 human HNC patients were included. A 3D U-Net convolutional neural network (CNN) was trained to automatically segment the GTV in canine patients using two main approaches: (i) training models from scratch based solely on canine CT images, and (ii) using cross-species transfer learning where models were pretrained on CT images of human patients and then fine-tuned on CT images of canine patients. For the canine patients, automatic segmentations were assessed using the Dice similarity coefficient (Dice), the positive predictive value, the true positive rate, and surface distance metrics, calculated from a four-fold cross-validation strategy where each fold was used as a validation set and test set once in independent model runs.

Results

CNN models trained from scratch on canine data or by using transfer learning obtained mean test set Dice scores of 0.55 and 0.52, respectively, indicating acceptable auto-segmentations, similar to the mean Dice performances reported for CT-based automatic segmentation in human HNC studies. Automatic segmentation of nasal cavity tumors appeared particularly promising, resulting in mean test set Dice scores of 0.69 for both approaches.

Conclusion

In conclusion, deep learning-based automatic segmentation of the GTV using CNN models based on canine data only or a cross-species transfer learning approach shows promise for future application in RT of canine HNC patients.

QUARTET: A SIOP Europe project for quality and excellence in radiotherapy and imaging for children and adolescents with cancer

The European Society for Paediatric Oncology (SIOPE) Radiation Oncology Working Group presents the QUARTET Project: a centralised quality assurance programme designed to standardise care and improve the quality of radiotherapy and imaging for international clinical trials recruiting children and adolescents with cancer throughout Europe. QUARTET combines the paediatric radiation oncology expertise of SIOPE with the infrastructure and experience of the European Organisation for Research and Treatment of Cancer to deliver radiotherapy quality assurance programmes for large, prospective, international clinical trials. QUARTET-affiliated trials include children and adolescents with brain tumours, neuroblastoma, sarcomas including rhabdomyosarcoma, and renal tumours including Wilms' tumour. With nine prospective clinical trials and two retrospective studies within the active portfolio in March 2022, QUARTET will collect one of the largest repositories of paediatric radiotherapy and imaging data, support the clinical assessment of radiotherapy, and evaluate the role and benefit of radiotherapy quality assurance for this cohort of patients within the context of clinical trials.

Consensus Quality Measures and Dose Constraints for Head and Neck Cancer with an emphasis on Oropharyngeal and Laryngeal Cancer from the Veterans Affairs Radiation Oncology Quality Surveillance Program and American Society for Radiation Oncology Expert Panel

PURPOSE

Safeguarding high-quality care using evidence-based radiation therapy for patients with head and neck cancer is crucial to improving oncologic outcomes, including survival and quality of life.

METHODS AND MATERIALS

The Veterans Administration (VA) National Radiation Oncology Program established the VA Radiation Oncology Quality Surveillance Program (VAROQS) to develop clinical quality measures (QM) in head and neck cancer. As part of the development of QM, the VA commissioned, along with the American Society for Radiation Oncology, a blue-ribbon panel comprising experts in head and neck cancer, to develop QM.

RESULTS

We describe the methods used to develop QM and the final consensus QM, as well as aspirational and surveillance QM, which capture all aspects of the continuum of patient care from initial patient work-up, radiation treatment planning and delivery, and follow-up care, as well as dose volume constraints.

CONCLUSION

These QM are intended for use as part of ongoing quality surveillance for veterans receiving radiation therapy throughout the VA as well as outside the VA. They may also be used by the non-VA community as a basic measure of quality care for head and neck cancer patients receiving radiation.

Machine Learning for Auto-Segmentation in Radiotherapy Planning

Manual segmentation of target structures and organs at risk is a crucial step in the radiotherapy workflow. It has the disadvantages that it can require several hours of clinician time per patient and is prone to inter- and intra-observer variability. Automatic segmentation (auto-segmentation), using computer algorithms, seeks to address these issues. Advances in machine learning and computer vision have led to the development of methods for accurate and efficient auto-segmentation. This review surveys auto-segmentation techniques and applications in radiotherapy planning. It provides an overview of traditional approaches to auto-segmentation, including intensity analysis, shape modelling and atlas-based methods. The focus, though, is on uses of machine learning and deep learning, including convolutional neural networks. Finally, the future of machine-learning-driven auto-segmentation in clinical settings is considered, and the barriers that must be overcome for it to be widely accepted into routine practice are highlighted.

Deep learning-based automatic delineation of anal cancer gross tumour volume: a multimodality comparison of CT, PET and MRI

BACKGROUND

Accurate target volume delineation is a prerequisite for high-precision radiotherapy. However, manual delineation is resource-demanding and prone to interobserver variation. An automatic delineation approach could potentially save time and increase delineation consistency. In this study, the applicability of deep learning for fully automatic delineation of the gross tumour volume (GTV) in patients with anal squamous cell carcinoma (ASCC) was evaluated for the first time. An extensive comparison of the effects single modality and multimodality combinations of computed tomography (CT), positron emission tomography (PET), and magnetic resonance imaging (MRI) have on automatic delineation quality was conducted.

MATERIAL AND METHODS

18F-fluorodeoxyglucose PET/CT and contrast-enhanced CT (ceCT) images were collected for 86 patients with ASCC. A subset of 36 patients also underwent a study-specific 3T MRI examination including T2- and diffusion-weighted imaging. The resulting two datasets were analysed separately. A two-dimensional U-Net convolutional neural network (CNN) was trained to delineate the GTV in axial image slices based on single or multimodality image input. Manual GTV delineations constituted the ground truth for CNN model training and evaluation. Models were evaluated using the Dice similarity coefficient (Dice) and surface distance metrics computed from five-fold cross-validation.

RESULTS

CNN-generated automatic delineations demonstrated good agreement with the ground truth, resulting in mean Dice scores of 0.65-0.76 and 0.74-0.83 for the 86 and 36-patient datasets, respectively. For both datasets, the highest mean Dice scores were obtained using a multimodal combination of PET and ceCT (0.76-0.83). However, models based on single modality ceCT performed comparably well (0.74-0.81). T2W-only models performed acceptably but were somewhat inferior to the PET/ceCT and ceCT-based models.

CONCLUSION

CNNs provided high-quality automatic GTV delineations for both single and multimodality image input, indicating that deep learning may prove a versatile tool for target volume delineation in future patients with ASCC.

The radiotherapy quality assurance gap among phase III cancer clinical trials

PURPOSE

Quality assurance (QA) practices improve the quality level of oncology trials by ensuring that the protocol is followed and the results are valid and reproducible. This study investigated the utilization of QA among randomized controlled trials that involve radiotherapy (RT).

METHODS AND MATERIALS

We searched ClinicalTrials.gov in February 2020 for all phase III oncology randomized clinical trials (RCTs). These trials were screened for RT-specific RCTs that had published primary trial results. Information regarding QA in each trial was collected from the study publications and trial protocol if available. Two individuals independently performed trial screening and data collection. Pearson's Chi-square tests analyses were used to assess factors that were associated with QA inclusion in RT trials.

RESULTS

Forty-two RCTs with RT as the primary intervention or as a mandatory component of the protocol were analyzed; the earliest was started in 1994 and one trial was still active though not recruiting. Twenty-nine (69%) trials mandated RT quality assurance (RTQA) practices as part of the trial protocol, with 19 (45%) trials requiring institutional credentialing. Twenty-one (50%) trials published protocol deviation outcomes. Clinical trials involving advanced radiation techniques (IMRT, VMAT, SRS, SBRT) did not include more RTQA than trials without these advanced techniques (73% vs. 65%, p = 0.55). Trials that reported protocol deviation outcomes were associated with mandating RTQA in their protocols as compared to trials that did not report these outcomes (100% vs. 38%, p < 0.001).

CONCLUSIONS

There is a lack of RTQA utilization and transparency in RT clinical trials. It is imperative for RT trials to include increased QA for safe, consistent, and high-quality RT planning and delivery.

Evaluation of the impact of teaching on delineation variation during a virtual stereotactic ablative radiotherapy contouring workshop

Introduction:

Variation in delineation of target volumes/organs at risk (OARs) is well recognised in radiotherapy and may be reduced by several methods including teaching. We evaluated the impact of teaching on contouring variation for thoracic/pelvic stereotactic ablative radiotherapy (SABR) during a virtual contouring workshop.

Materials and methods:

Target volume/OAR contours produced by workshop participants for three cases were evaluated against reference contours using DICE similarity coefficient (DSC) and line domain error (LDE) metrics. Pre- and post-workshop DSC results were compared using Wilcoxon signed ranks test to determine the impact of teaching during the workshop.

Results:

Of 50 workshop participants, paired pre- and post-workshop contours were available for 21 (42%), 20 (40%) and 22 (44%) participants for primary lung cancer, pelvic bone metastasis and pelvic node metastasis cases, respectively. Statistically significant improvements post-workshop in median DSC and LDE results were observed for 6 (50%) and 7 (58%) of 12 structures, respectively, although the magnitude of DSC/LDE improvement was modest in most cases. An increase in median DSC post-workshop ≥0·05 was only observed for GTVbone, IGTVlung and SacralPlex, and reduction in median LDE > 1 mm was only observed for GTVbone, CTVbone and SacralPlex. Post-workshop, median DSC values were >0·7 for 75% of structures. For 92% of the structures, post-workshop contours were considered to be acceptable or within acceptable variation following review by the workshop faculty.

Conclusions:

This study has demonstrated that virtual SABR contouring training is feasible and was associated with some improvements in contouring variation for multiple target volumes/OARs.

ProCaLung - Peer review in stage III, mediastinal node-positive, non-small-cell lung cancer: How to benchmark clinical practice of nodal target volume definition and delineation in Belgium☆

BACKGROUND AND PURPOSE

The Quality Assurance project for stage III non-small cell lung cancer radiotherapy ProCaLung performed a multicentric two-step exercise evaluating mediastinal nodal Target Volume Definition and Delineation (TVD) variability and the opportunity for standardization. The TVD variability before and after providing detailed guidelines and the value of qualitative contour reviewing before applying quantitative measures were investigated.

MATERIALS AND METHODS

The case of a patient with stage III NSCLC and involved mediastinal lymph nodes was used as a basis for this study. Twenty-two radiation oncologists from nineteen centers in Belgium and Luxembourg participated in at least one of two phases of the project (before and after introduction of ProCaLung contouring guidelines). The resulting thirty-three mediastinal nodal GTV and CTV contours were then evaluated using a qualitative-before-quantitative (QBQ) approach. First, a qualitative analysis was performed, evaluating adherence to most recent guidelines. From this, a list of observed deviations was created and these were used to evaluate contour conformity. The second analysis was quantitative, using overlap and surface distance measures to compare contours within qualitative groups and between phases. A 'most robust' reference volume for these analyses was created using the STAPLE-algorithm and an averaging method.

RESULTS

Five GTV and seven CTV qualitative groups were identified. Second step contours were more often in higher-conformity groups (p = 0.012 for GTV and p = 0.024 for CTV). Median Residual Mean Square Distances improved from 2.34 mm to 1.36 mm for GTV (p = 0.01) and from 4.53 mm to 1.58 mm for CTV (p < 0.0001). Median Dice coefficients increased from 0.81 to 0.84 for GTV (p = 0.07) and from 0.82 to 0.89 for CTV (p ≤ 0.001). Using HC-contours only to generate references translated in more robust quantitative evaluations.

CONCLUSION

Variability of mediastinal nodal TVD was reduced after providing the ProCaLung consensus guidelines. A qualitative review was essential for providing meaningful quantitative measures.

FIELDRT: an open-source platform for the assessment of target volume delineation in radiation therapy

Objectives:

Target volume delineation (TVD) has been identified as a weakness in the accuracy of radiotherapy, both within and outside of clinical trials due to the intra/interobserver variations affecting the TVD quality. Sources of variations such as poor compliance or protocol violation may have adverse effect on treatment outcomes. In this paper, we present and describe the FIELD^RT software developed for the ARENA project to improve the quality of TVD through qualitative and quantitative feedbacks and individual and personalized summary of trainee”s performance.

Methods:

For each site-specific clinical case included in the FIELD^RT software, reference volumes, minimum and maximum “acceptable” volumes and organ at risk were derived by outlines of consultants and senior trainees. The software components currently developed include: (a) user-friendly importing interface (b) analysis toolbox to compute quantitative and qualitative (c) visualiser and (d) structured report generator for personalised feedback. The FIELD^RT software was validated by comparing the performance of 63 trainees and by measuring performance over time. In addition, a trainee evaluation day was held in 2019 to collect feedback on FIELD^RT.

Results:

Results show the trainees’ improvement when reoutlining a case after reviewing the feedback generated from the FIELD^RT software. Comments and feedback received after evaluation day were positive and confirmed that FIELD^RT can be a useful application for training purposes.

Conclusion:

We presented a new open-source software to support education in TVD and ongoing continuous professional development for clinical oncology trainees and consultants. ARENA in combination with FIELD^RT implements site-specific modules with reference target and organs at risk volumes and automatically evaluates individual performance using several quantitative and qualitative feedbacks. Pilot results suggests this software could be used as an education tool to reduce variation in TVD so to guarantee high quality in radiotherapy.

Advances in knowledge:

FIELD^RT is a new easy and free to use software aiming at supporting education in TVD and ongoing continuous professional development. The software provides quantitative/qualitative feedback and an exportable report with an individual and personalised summary of trainee’s performance.

Geometric contour variation in clinical target volume of axillary lymph nodes in breast cancer radiotherapy: an AIRO multi-institutional study

OBJECTIVES

To determine interobserver variability in axillary nodal contouring in breast cancer (BC) radiotherapy (RT) by comparing the clinical target volume of participating single centres (SC-CTV) with a gold-standard CTV (GS-CTV).

METHODS

The GS-CTV of three patients (P1, P2, P3) with increasing complexity was created in DICOM format from the median contour of axillary CTVs drawn by BC experts, validated using the simultaneous truth and performance-level estimation and peer-reviewed. GS-CTVs were compared with the correspondent SC-CTVs drawn by radiation oncologists, using validated metrics and a total score (TS) integrating all of them.

RESULTS

Eighteen RT centres participated in the study. Comparative analyses revealed that, on average, the SC-CTVs were smaller than GS-CTV for P1 and P2 (by -29.25% and -27.83%, respectively) and larger for P3 (by +12.53%). The mean Jaccard index was greater for P1 and P2 compared to P3, but the overlap extent value was around 0.50 or less. Regarding nodal levels, L4 showed the highest concordance with the GS. In the intra-patient comparison, L2 and L3 achieved lower TS than L4. Nodal levels showed discrepancy with GS, which was not statistically significant for P1, and negligible for P2, while P3 had the worst agreement. DICE similarity coefficient did not exceed the minimum threshold for agreement of 0.70 in all the measurements.

CONCLUSIONS

Substantial differences were observed between SC- and GS-CTV, especially for P3 with altered arm setup. L2 and L3 were the most critical levels. The study highlighted these key points to address.

ADVANCES IN KNOWLEDGE

The present study compares, by means of validated geometric indexes, manual segmentations of axillary lymph nodes in breast cancer from different observers and different institutions made on radiotherapy planning CT images. Assessing such variability is of paramount importance, as geometric uncertainties might lead to incorrect dosimetry and compromise oncological outcome.

Quality assurance of radiotherapy in the ongoing EORTC 1420 "Best of" trial for early stage oropharyngeal, supraglottic and hypopharyngeal carcinoma: results of the benchmark case procedure

INTRODUCTION

The current phase III EORTC 1420 Best-of trial (NCT02984410) compares the swallowing function after transoral surgery versus intensity modulated radiotherapy (RT) in patients with early-stage carcinoma of the oropharynx, supraglottis and hypopharynx. We report the analysis of the Benchmark Case (BC) procedures before patient recruitment with special attention to dysphagia/aspiration related structures (DARS).

MATERIALS AND METHODS

Submitted RT volumes and plans from participating centers were analyzed and compared against the gold-standard expert delineations and dose distributions. Descriptive analysis of protocol deviations was conducted. Mean Sorensen-Dice similarity index (mDSI) and Hausdorff distance (mHD) were applied to evaluate the inter-observer variability (IOV).

RESULTS

65% (23/35) of the institutions needed more than one submission to achieve Quality assurance (RTQA) clearance. OAR volume delineations were the cause for rejection in 53% (40/76) of cases. IOV could be improved in 5 out of 12 OARs by more than 10 mm after resubmission (mHD). Despite this, final IOV for critical OARs in delineation remained significant among DARS by choosing an aleatory threshold of 0.7 (mDSI) and 15 mm (mHD).

CONCLUSIONS

This is to our knowledge the largest BC analysis among Head and neck RTQA programs performed in the framework of a prospective trial. Benchmarking identified non-common OARs and target delineations errors as the main source of deviations and IOV could be reduced in a significant number of cases after this process. Due to the substantial resources involved with benchmarking, future benchmark analyses should assess fully the impact on patients' clinical outcome.

Challenges in the target volume definition of lung cancer radiotherapy

Radiotherapy, with or without systemic treatment has an important role in the management of lung cancer. In order to deliver the treatment accurately, the clinician must precisely outline the gross tumour volume (GTV), mostly on computed tomography (CT) images. However, due to the limited contrast between tumour and non-malignant changes in the lung tissue, it can be difficult to distinguish the tumour boundaries on CT images leading to large interobserver variation and differences in interpretation. Therefore the definition of the GTV has often been described as the weakest link in radiotherapy with its inaccuracy potentially leading to missing the tumour or unnecessarily irradiating normal tissue. In this article, we review the various techniques that can be used to reduce delineation uncertainties in lung cancer.

Proposal for the delineation of neoadjuvant target volumes in oesophageal cancer

PURPOSE

To define instructions for delineation of target volumes in the neoadjuvant setting in oesophageal cancer.

MATERIALS AND METHODS

Radiation oncologists of five European centres participated in the following consensus process: [1] revision of published (MEDLINE) and national/institutional delineation guidelines; [2] first delineation round of five cases (patient 1-5) according to national/institutional guidelines; [3] consensus meeting to discuss the results of step 1 and 2, followed by a target volume delineation proposal; [4] circulation of proposed instructions for target volume delineation and atlas for feedback; [5] second delineation round of five new cases (patient 6-10) to peer review and validate (two additional centres) the agreed delineation guidelines and atlas; [6] final consensus on the delineation guidelines depicted in an atlas. Target volumes of the delineation rounds were compared between centres by Dice similarity coefficient (DSC) and maximum/mean undirected Hausdorff distances (H_max/H_mean).

RESULTS

In the first delineation round, the consistency between centres was moderate (CTVtotal: DSC = 0.59-0.88; H_mean = 0.2-0.4 cm). Delineations in the second round were much more consistent. Lowest variability was obtained between centres participating in the consensus meeting (CTVtotal: DSC: p < 0.050 between rounds for patients 6/7/8/10; H_mean: p < 0.050 for patients 7/8/10), compared to validation centres (CTVtotal: DSC: p < 0.050 between validation and consensus meeting centres for patients 6/7/8; H_mean: p < 0.050 for patients 7/10). A proposal for delineation of target volumes and an atlas were generated.

CONCLUSION

We proposed instructions for target volume delineation and an atlas for the neoadjuvant radiation treatment in oesophageal cancer. These will enable a more uniform delineation of patients in clinical practice and clinical trials.

Clinical evaluation of atlas- and deep learning-based automatic segmentation of multiple organs and clinical target volumes for breast cancer

Manual segmentation is the gold standard method for radiation therapy planning; however, it is time-consuming and prone to inter- and intra-observer variation, giving rise to interests in auto-segmentation methods. We evaluated the feasibility of deep learning-based auto-segmentation (DLBAS) in comparison to commercially available atlas-based segmentation solutions (ABAS) for breast cancer radiation therapy. This study used contrast-enhanced planning computed tomography scans from 62 patients with breast cancer who underwent breast-conservation surgery. Contours of target volumes (CTVs), organs, and heart substructures were generated using two commercial ABAS solutions and DLBAS using fully convolutional DenseNet. The accuracy of the segmentation was assessed using 14 test patients using the Dice Similarity Coefficient and Hausdorff Distance referencing the expert contours. A sensitivity analysis was performed using non-contrast planning CT from 14 additional patients. Compared to ABAS, the proposed DLBAS model yielded more consistent results and the highest average Dice Similarity Coefficient values and lowest Hausdorff Distances, especially for CTVs and the substructures of the heart. ABAS showed limited performance in soft-tissue-based regions, such as the esophagus, cardiac arteries, and smaller CTVs. The results of sensitivity analysis between contrast and non-contrast CT test sets showed little difference in the performance of DLBAS and conversely, a large discrepancy for ABAS. The proposed DLBAS algorithm was more consistent and robust in its performance than ABAS across the majority of structures when examining both CTVs and normal organs. DLBAS has great potential to aid a key process in the radiation therapy workflow, helping optimise and reduce the clinical workload.

A Systematic Review of Contouring Guidelines in Radiation Oncology: Analysis of Frequency, Methodology, and Delivery of Consensus Recommendations

PURPOSE

Clinical trials have described variation in radiation therapy plan quality, of which contour delineation is a key component, and linked this to inferior patient outcomes. In response, consensus guidelines have been developed to standardize contour delineation. This investigation assesses trends in contouring guidelines and examines the methodologies used to generate and deliver recommendations.

METHODS AND MATERIALS

We conducted a literature search for contouring guidelines published after 1995. Of 11,124 citations, 332 were identified for full-text review to determine inclusion. We abstracted articles for the intent of the consensus process, key elements of the methodology, and mode of information delivery. A Fisher exact test was used to identify elements that differed among the guidelines generated for clinical trials and routine care.

RESULTS

Overall, 142 guidelines were included, of which 16 (11%) were developed for a clinical trial. There was an increase in guideline publication over time (0 from 1995-1999 vs 65 from 2015- 2019; P = .03), particularly among recommendations for stereotactic radiation and brachytherapy. The most common disease sites were head and neck (24%), gastrointestinal (12%), and gynecologic (12%). Methods used to develop recommendations included literature review (50%) and image-based methods (45%). Panels included a median of 10 physicians (interquartile range, 7-16); 70% of panels represented multidisciplinary expertise. Guidelines developed for a clinical trial were more likely to include an image-based approach, with quantitative analysis of contours submitted by the panel members and to publish a full set of image-based recommendations (P < .005).

CONCLUSIONS

This review highlights an increase in consensus contouring recommendations over time. Guidelines focus on disease sites, such as head and neck, with evidence supporting a correlation between treatment planning and patient outcomes, although variation exists in the approach to the consensus process. Elements that may improve guideline acceptance (ie, image-based consensus contour analysis) and usability (ie, inclusion of a full image set) are more common in guidelines developed for clinical trials.

Impact of Peer Review in Reducing Uncertainty in the Definition of the Lung Target Volume Among Trainee Oncologists

AIMS

To evaluate the impact of peer review and contouring workshops on reducing uncertainty in target volume delineation for lung cancer radiotherapy.

MATERIALS AND METHODS

Data from two lung cancer target volume delineation courses were analysed. In total, 22 trainees in clinical oncology working across different UK centres attended these courses with priori experience in lung cancer radiotherapy. The courses were made up of short presentations and contouring practice sessions. The participants were divided into two groups and asked to first individually delineate (IND) and then individually peer review (IPR) the contours of another participant. The contours were discussed with an expert panel consisting of two consultant clinical oncologists and a consultant radiologist. Contours were analysed quantitatively by measuring the volume and local distance standard deviation (localSD) from the reference expert consensus contour and qualitatively through visual analysis. Feedback from the participants was obtained using a questionnaire.

RESULTS

All participants applied minor editing to the contours during IPR, leading to a non-statistically significant reduction in the mean delineated volume (IND = 140.92 cm3, IPR = 125.26 cm3, P = 0.211). The overall interobserver variation was similar, with a localSD of 0.33 cm and 0.38 cm for the IND and IPR, respectively (P = 0.848). Six participants (29%) carried out correct major changes by either including tumour or excluding healthy tissue. One participant (5%) carried out an incorrect edit by excluding parts of the tumour, while another observer failed to identify a major contour error. The participants' level of confidence in target volume delineation increased following the course and identified the discussions with the radiologist and colleagues as the most important highlights of the course.

CONCLUSION

IPR could improve target volume delineation quality among trainee oncologists by identifying most major contour errors. However, errors were also introduced after IPR, suggesting the need to further discuss major changes with a multidisciplinary team.