A Systematic Review of Contouring Guidelines in Radiation Oncology: Analysis of Frequency, Methodology, and Delivery of Consensus Recommendations

Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation

PURPOSE

The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors.

METHODS

Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure.

RESULTS

Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations.

CONCLUSION

Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Impact of a contouring atlas on radiographer inter-observer variation in male pelvis radiotherapy

PURPOSE/OBJECTIVE

To determine the impact of a MR-based contouring atlas for male pelvis radiotherapy delineation on inter-observer variation to support radiographer led real-time magnetic resonance image guided adaptive radiotherapy (MRgART).

MATERIAL/METHODS

Eight RTTs contoured 25 MR images in the Monaco treatment planning system (Monaco 5.40.01), from 5 patients. The prostate, seminal vesicles, bladder, and rectum were delineated before and after the introduction of an atlas developed through multi-disciplinary consensus. Inter-observer contour variations (volume), time to contour and observer contouring confidence were determined at both time-points using a 5-point Likert scale. Descriptive statistics were used to analyse both continuous and categorical variables. Dice similarity coefficient (DSC), Dice-Jaccard coefficient (DJC) and Hausdorff distance were used to calculate similarity between observers.

RESULTS

Although variation in volume definition decreased for all structures among all observers post intervention, the change was not statistically significant. DSC and DJC measurements remained consistent following the introduction of the atlas for all observers. The highest similarity was found in the bladder and prostate whilst the lowest was the seminal vesicles. The mean contouring time for all observers was reduced by 50% following the introduction of the atlas (53 to 27 minutes, p=0.01). For all structures across all observers, the mean contouring confidence increased significantly from 2.3 to 3.5 out of 5 (p=0.02).

CONCLUSION

Although no significant improvements were observed in contour variation amongst observers, the introduction of the consensus-based contouring atlas improved contouring confidence and speed; key factors for a real-time RTT-led MRgART.

Evaluation of multiple-vendor AI autocontouring solutions

BACKGROUND

Multiple artificial intelligence (AI)-based autocontouring solutions have become available, each promising high accuracy and time savings compared with manual contouring. Before implementing AI-driven autocontouring into clinical practice, three commercially available CT-based solutions were evaluated.

MATERIALS AND METHODS

The following solutions were evaluated in this work: MIM-ProtégéAI+ (MIM), Radformation-AutoContour (RAD), and Siemens-DirectORGANS (SIE). Sixteen organs were identified that could be contoured by all solutions. For each organ, ten patients that had manually generated contours approved by the treating physician (AP) were identified, totaling forty-seven different patients. CT scans in the supine position were acquired using a Siemens-SOMATOMgo 64-slice helical scanner and used to generate autocontours. Physician scoring of contour accuracy was performed by at least three physicians using a five-point Likert scale. Dice similarity coefficient (DSC), Hausdorff distance (HD) and mean distance to agreement (MDA) were calculated comparing AI contours to "ground truth" AP contours.

RESULTS

The average physician score ranged from 1.00, indicating that all physicians reviewed the contour as clinically acceptable with no modifications necessary, to 3.70, indicating changes are required and that the time taken to modify the structures would likely take as long or longer than manually generating the contour. When averaged across all sixteen structures, the AP contours had a physician score of 2.02, MIM 2.07, RAD 1.96 and SIE 1.99. DSC ranged from 0.37 to 0.98, with 41/48 (85.4%) contours having an average DSC ≥ 0.7. Average HD ranged from 2.9 to 43.3 mm. Average MDA ranged from 0.6 to 26.1 mm.

CONCLUSIONS

The results of our comparison demonstrate that each vendor's AI contouring solution exhibited capabilities similar to those of manual contouring. There were a small number of cases where unusual anatomy led to poor scores with one or more of the solutions. The consistency and comparable performance of all three vendors' solutions suggest that radiation oncology centers can confidently choose any of the evaluated solutions based on individual preferences, resource availability, and compatibility with their existing clinical workflows. Although AI-based contouring may result in high-quality contours for the majority of patients, a minority of patients require manual contouring and more in-depth physician review.

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review

Background/purpose

The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions.

Methods

We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics.

Results

We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets.

Conclusion

Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions

Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.

vOARiability: Interobserver and intermodality variability analysis in OAR contouring from head and neck CT and MR images

BACKGROUND

Accurate and consistent contouring of organs-at-risk (OARs) from medical images is a key step of radiotherapy (RT) cancer treatment planning. Most contouring approaches rely on computed tomography (CT) images, but the integration of complementary magnetic resonance (MR) modality is highly recommended, especially from the perspective of OAR contouring, synthetic CT and MR image generation for MR-only RT, and MR-guided RT. Although MR has been recognized as valuable for contouring OARs in the head and neck (HaN) region, the accuracy and consistency of the resulting contours have not been yet objectively evaluated.

PURPOSE

To analyze the interobserver and intermodality variability in contouring OARs in the HaN region, performed by observers with different level of experience from CT and MR images of the same patients.

METHODS

In the final cohort of 27 CT and MR images of the same patients, contours of up to 31 OARs were obtained by a radiation oncology resident (junior observer, JO) and a board-certified radiation oncologist (senior observer, SO). The resulting contours were then evaluated in terms of interobserver variability, characterized as the agreement among different observers (JO and SO) when contouring OARs in a selected modality (CT or MR), and intermodality variability, characterized as the agreement among different modalities (CT and MR) when OARs were contoured by a selected observer (JO or SO), both by the Dice coefficient (DC) and 95-percentile Hausdorff distance (HD95 $_{95}$ ).

RESULTS

The mean (±standard deviation) interobserver variability was 69.0 ± 20.2% and 5.1 ± 4.1 mm, while the mean intermodality variability was 61.6 ± 19.0% and 6.1 ± 4.3 mm in terms of DC and HD95 $_{95}$ , respectively, across all OARs. Statistically significant differences were only found for specific OARs. The performed MR to CT image registration resulted in a mean target registration error of 1.7 ± 0.5 mm, which was considered as valid for the analysis of intermodality variability.

CONCLUSIONS

The contouring variability was, in general, similar for both image modalities, and experience did not considerably affect the contouring performance. However, the results indicate that an OAR is difficult to contour regardless of whether it is contoured in the CT or MR image, and that observer experience may be an important factor for OARs that are deemed difficult to contour. Several of the differences in the resulting variability can be also attributed to adherence to guidelines, especially for OARs with poor visibility or without distinctive boundaries in either CT or MR images. Although considerable contouring differences were observed for specific OARs, it can be concluded that almost all OARs can be contoured with a similar degree of variability in either the CT or MR modality, which works in favor of MR images from the perspective of MR-only and MR-guided RT.

Multicenter Study of Pelvic Nodal Autosegmentation Algorithm of Siemens Healthineers: Comparison of Male Versus Female Pelvis

Purpose

The autosegmentation algorithm of Siemens Healthineers version VA 30 (AASH) (Siemens Healthineers, Erlangen, Germany) was trained and developed in the male pelvis, with no published data on its usability in the female pelvis. This is the first multi-institutional study to describe and evaluate an artificial intelligence algorithm for autosegmentation of the pelvic nodal region by gender.

Methods and Materials

We retrospectively evaluated AASH pelvic nodal autosegmentation in both male and female patients treated at our network of institutions. The automated pelvic nodal contours generated by AASH were evaluated by 1 board-certified radiation oncologist. A 4-point scale was used for each nodal region contour: a score of 4 is clinically usable with minimal edits; a score of 3 requires minor edits (missing nodal contour region, cutting through vessels, or including bowel loops) in 3 or fewer computed tomography slices; a score of 2 requires major edits, as previously defined but in 4 or more computed tomography slices; and a score of 1 requires complete recontouring of the region. Pelvic nodal regions included the right and left side of the common iliac, external iliac, internal iliac, obturator, and midline presacral nodes. In addition, patients were graded based on their lowest nodal contour score. Statistical analysis was performed using Fisher exact tests and Yates-corrected χ2 tests.

Results

Fifty-two female and 51 male patients were included in the study, representing a total of 468 and 447 pelvic nodal regions, respectively. Ninety-six percent and 99% of contours required minor edits at most (score of 3 or 4) for female and male patients, respectively (P = .004 using Fisher exact test; P = .007 using Yates correction). No nodal regions had a statistically significant difference in scores between female and male patients. The percentage of patients requiring no more than minor edits was 87% (45 patients) and 92% (47 patients) for female and male patients, respectively (P = .53 using Fisher exact test; P = .55 using Yates correction).

Conclusions

AASH pelvic nodal autosegmentation performed very well in both male and female pelvic nodal regions, although with better male pelvic nodal autosegmentation. As autosegmentation becomes more widespread, it may be important to have equal representation from all sexes in training and validation of autosegmentation algorithms.

Standardizing and improving dose predictions for head and neck cancers using complete sets of OAR contours

BACKGROUND

Radiotherapy dose predictions have been trained with data from previously treated patients of similar sites and prescriptions. However, clinical datasets are often inconsistent and do not contain the same number of organ at risk (OAR) structures. The effects of missing contour data in deep learning-based dose prediction models have not been studied.

PURPOSE

The purpose of this study was to investigate the impacts of incomplete contour sets in the context of deep learning-based radiotherapy dose prediction models trained with clinical datasets and to introduce a novel data substitution method that utilizes automated contours for undefined structures.

METHODS

We trained Standard U-Nets and Cascade U-Nets to predict the volumetric dose distributions of patients with head and neck cancers (HNC) using three input variations to evaluate the effects of missing contours, as well as a novel data substitution method. Each architecture was trained with the original contour (OC) inputs, which included missing information, hybrid contour (HC) inputs, where automated OAR contours generated in software were substituted for missing contour data, and automated contour (AC) inputs containing only automated OAR contours. 120 HNC treatments were used for model training, 30 were used for validation and tuning, and 44 were used for evaluation and testing. Model performance and accuracy were evaluated with global whole body dose agreement, PTV coverage accuracy, and OAR dose agreement. The differences in these values between dataset variations were used to determine the effects of missing data and automated contour substitutions.

RESULTS

Automated contours used as substitutions for missing data were found to improve dose prediction accuracy in the Standard U-Net and Cascade U-Net, with a statistically significant difference in some global metrics and/or OAR metrics. For both models, PTV coverage between input variations was unaffected by the substitution technique. Automated contours in HC and AC datasets improved mean dose accuracy for some OAR contours, including the mandible and brainstem, with a greater improvement seen with HC datasets. Global dose metrics, including mean absolute error, mean error, and percent error were different for the Standard U-Net but not for the Cascade U-Net.

CONCLUSION

Automated contours used as a substitution for contour data improved prediction accuracy for some but not all dose prediction metrics. Compared to the Standard U-Net models, the Cascade U-Net achieved greater precision.

Two-dimensional segmentation fusion tool: an extensible, free-to-use, user-friendly tool for combining different bidimensional segmentations

Introduction: In several fields, the process of fusing multiple two-dimensional (2D) closed lines is an important step. For instance, this is fundamental in histology and oncology in general. The treatment of a tumor consists of numerous steps and activities. Among them, segmenting the cancer area, that is, the correct identification of its spatial location by the segmentation technique, is one of the most important and at the same time complex and delicate steps. The difficulty in deriving reliable segmentations stems from the lack of a standard for identifying the edges and surrounding tissues of the tumor area. For this reason, the entire process is affected by considerable subjectivity. Given a tumor image, different practitioners can associate different segmentations with it, and the diagnoses produced may differ. Moreover, experimental data show that the analysis of the same area by the same physician at two separate timepoints may result in different lines being produced. Accordingly, it is challenging to establish which contour line is the ground truth. Methods: Starting from multiple segmentations related to the same tumor, statistical metrics and computational procedures could be exploited to combine them for determining the most reliable contour line. In particular, numerous algorithms have been developed over time for this procedure, but none of them is validated yet. Accordingly, in this field, there is no ground truth, and research is still active. Results: In this work, we developed the Two-Dimensional Segmentation Fusion Tool (TDSFT), a user-friendly tool distributed as a free-to-use standalone application for MAC, Linux, and Windows, which offers a simple and extensible interface where numerous algorithms are proposed to "compute the mean" (i.e., the process to fuse, combine, and "average") multiple 2D lines. Conclusions: The TDSFT can support medical specialists, but it can also be used in other fields where it is required to combine 2D close lines. In addition, the TDSFT is designed to be easily extended with new algorithms thanks to a dedicated graphical interface for configuring new parameters. The TDSFT can be downloaded from the following link: https://sourceforge.net/p/tdsft.

Revolutionizing radiation therapy: the role of AI in clinical practice

This review provides an overview of the application of artificial intelligence (AI) in radiation therapy (RT) from a radiation oncologist's perspective. Over the years, advances in diagnostic imaging have significantly improved the efficiency and effectiveness of radiotherapy. The introduction of AI has further optimized the segmentation of tumors and organs at risk, thereby saving considerable time for radiation oncologists. AI has also been utilized in treatment planning and optimization, reducing the planning time from several days to minutes or even seconds. Knowledge-based treatment planning and deep learning techniques have been employed to produce treatment plans comparable to those generated by humans. Additionally, AI has potential applications in quality control and assurance of treatment plans, optimization of image-guided RT and monitoring of mobile tumors during treatment. Prognostic evaluation and prediction using AI have been increasingly explored, with radiomics being a prominent area of research. The future of AI in radiation oncology offers the potential to establish treatment standardization by minimizing inter-observer differences in segmentation and improving dose adequacy evaluation. RT standardization through AI may have global implications, providing world-standard treatment even in resource-limited settings. However, there are challenges in accumulating big data, including patient background information and correlating treatment plans with disease outcomes. Although challenges remain, ongoing research and the integration of AI technology hold promise for further advancements in radiation oncology.

Consensus Delineation Guidelines for Pelvic Lymph Node Radiation Therapy of Prostate Cancer: On Behalf of the Francophone Group of Urological Radiation Therapy (GFRU)

PURPOSE

Clinical target volume (CTV) delineation for pelvic lymph nodes in prostate cancer is currently based on 3 consensus guidelines with some inherent discrepancies. To improve the reproducibility in nodal delineation, the Francophone Group of Urological Radiotherapy (Groupe Francophone de Radiothérapie Urologique [GFRU]) worked toward proposing an easily applicable, reproducible, and practice-validated contouring guideline for pelvic nodal CTV.

METHODS AND MATERIALS

The nodal CTV data sets of a high-risk node-negative prostate cancer clinical case contoured by 86 radiation oncologists participating in a GFRU contouring workshop were analyzed. CTV volumes were defined before and after a structured presentation of literature data on lymphatic drainage pathways and patterns of nodal involvement and relapse, illustrated using a reference contour (CRef) defined by 3 GFRU experts. The consistency between the participants' contours and CRef was assessed quantitively by means of the Simultaneous Truth and Performance Level Estimation (STAPLE) method, the Dice coefficient, and the Hausdorff distance and qualitatively using a count map. These results combined with the literature review were thoroughly discussed among GFRU experts to reach a consensus.

RESULTS

From the 86 workshop participants, the volume of the STAPLE CTV was 591 cc compared with 502 cc for CRef. The Dice coefficient of the STAPLE CTV compared with the experts' CRef was 0.736 (±0.084) before and 0.823 (±0.070) after the workshop; the standard deviation decreased from 11.5% to 8.5% over the workshop. The Hausdorff distance of the STAPLE CTV compared with the CRef was 34.5 mm (±12.4) before the workshop and 21.8 mm (±9.3) after the workshop. Four areas of significant interobserver variability were identified, and a consensus was reached.

CONCLUSIONS

Using a robust methodology, our cooperative group proposed an easily applicable, reproducible, and practice-validated guideline for the delineation of the pelvic CTV in prostate cancer, useful for implementation in daily practice and clinical trials.

Proton and photon radiotherapy in stage III NSCLC: Effects on hematological toxicity and adjuvant immune therapy

BACKGROUND AND PURPOSE

Concurrent chemo-radiotherapy (CCRT) followed by adjuvant durvalumab is standard-of-care for fit patients with unresectable stage III NSCLC. Intensity modulated proton therapy (IMPT) results in different doses to organs than intensity modulated photon therapy (IMRT). We investigated whether IMPT compared to IMRT reduce hematological toxicity and whether it affects durvalumab treatment.

MATERIALS AND METHODS

Prospectively collected series of consecutive patients with stage III NSCLC receiving CCRT between 06.16 and 12.22 (staged with FDG-PET-CT and brain imaging) were retrospectively analyzed. The primary endpoint was the incidence of lymphopenia grade ≥ 3 in IMPT vs IMRT treated patients.

RESULTS

271 patients were enrolled (IMPT: n = 71, IMRT: n = 200) in four centers. All patients received platinum-based chemotherapy. Median age: 66 years, 58 % were male, 36 % had squamous NSCLC. The incidence of lymphopenia grade ≥ 3 during CCRT was 67 % and 47 % in the IMRT and IMPT group, respectively (OR 2.2, 95 % CI: 1.0-4.9, P = 0.03). The incidence of anemia grade ≥ 3 during CCRT was 26 % and 9 % in the IMRT and IMPT group respectively (OR = 4.9, 95 % CI: 1.9-12.6, P = 0.001). IMPT was associated with a lower rate of Performance Status (PS) ≥ 2 at day 21 and 42 after CCRT (13 % vs. 26 %, P = 0.04, and 24 % vs. 39 %, P = 0.02). Patients treated with IMPT had a higher probability of receiving adjuvant durvalumab (74 % vs. 52 %, OR 0.35, 95 % CI: 0.16-0.79, P = 0.01).

CONCLUSION

IMPT was associated with a lower incidence of severe lymphopenia and anemia, better PS after CCRT and a higher probability of receiving adjuvant durvalumab.

Analysis of Radiation Therapy Quality Assurance in NRG Oncology RTOG 0848

PURPOSE

NRG/Radiation Therapy Oncology Group 0848 is a 2-step randomized trial to evaluate the benefit of the addition of concurrent fluoropyrimidine and radiation therapy (RT) after adjuvant chemotherapy (second step) for patients with resected pancreatic head adenocarcinoma. Real-time quality assurance (QA) was performed on each patient who underwent RT. This analysis aims to evaluate adherence to protocol-specified contouring and treatment planning and to report the types and frequencies of deviations requiring revisions.

METHODS AND MATERIALS

In addition to a web-based contouring atlas, the protocol outlined step-by-step instructions for generating the clinical treatment volume through the creation of specific regions of interest. The planning target volume was a uniform 0.5 cm clinical treatment volume expansion. One of 2 radiation oncology study chairs independently reviewed each plan. Plans with unacceptable deviations were returned for revision and resubmitted until approved. Treatment started after final approval of the RT plan.

RESULTS

From 2014 to 2018, 354 patients were enrolled in the second randomization. Of these, 160 patients received RT and were included in the QA analysis. Resubmissions were more common for patients planned with 3-dimensional conformal RT (43%) than with intensity modulated RT (31%). In total, at least 1 resubmission of the treatment plan was required for 33% of patients. Among patients requiring resubmission, most only needed 1 resubmission (87%). The most common reasons for resubmission were unacceptable deviations with respect to the preoperative gross target volume (60.7%) and the pancreaticojejunostomy (47.5%).

CONCLUSION

One-third of patients required resubmission to meet protocol compliance criteria, demonstrating the continued need for expending resources on real-time, pretreatment QA in trials evaluating the use of RT, particularly for pancreas cancer. Rigorous QA is critically important for clinical trials involving RT to ensure that the true effect of RT is assessed. Moreover, RT QA serves as an educational process through providing feedback from specialists to practicing radiation oncologists on best practices.

Review and recommendations on deformable image registration uncertainties for radiotherapy applications

Deformable image registration (DIR) is a versatile tool used in many applications in radiotherapy (RT). DIR algorithms have been implemented in many commercial treatment planning systems providing accessible and easy-to-use solutions. However, the geometric uncertainty of DIR can be large and difficult to quantify, resulting in barriers to clinical practice. Currently, there is no agreement in the RT community on how to quantify these uncertainties and determine thresholds that distinguish a good DIR result from a poor one. This review summarises the current literature on sources of DIR uncertainties and their impact on RT applications. Recommendations are provided on how to handle these uncertainties for patient-specific use, commissioning, and research. Recommendations are also provided for developers and vendors to help users to understand DIR uncertainties and make the application of DIR in RT safer and more reliable.

ESTRO ACROP consensus recommendation on the target volume definition for radiation therapy of macroscopic prostate cancer recurrences after radical prostatectomy

Background

The European Society for Radiotherapy & Oncology (ESTRO) Advisory Committee for Radiation Oncology Practice (ACROP) panel on prostate bed delineation reflected on macroscopic local recurrences in patients referred for postoperative radiotherapy (PORT), a challenging situation without standardized approach, and decided to propose a consensus recommendation on target volume selection and definition.

Methods

An ESTRO ACROP contouring consensus panel consisting of 12 radiation oncologists and one radiologist, all with subspecialty expertise in prostate cancer, was established. Participants were asked to delineate the prostate bed clinical target volumes (CTVs) in two separate clinically relevant scenarios: a local recurrence at the seminal vesicle bed and one apically at the level of the anastomosis. Both recurrences were prostate-specific membrane antigen (PSMA)-avid and had an anatomical correlate on magnetic resonance imaging (MRI). Participants also answered case-specific questionnaires addressing detailed recommendations on target delineation. Discussions via electronic mails and videoconferences for final editing and consensus were performed.

Results

Contouring of the two cases confirmed considerable variation among the panelists. Finally, however, a consensus recommendation could be agreed upon. Firstly, it was proposed to always delineate the entire prostate bed as clinical target volume and not the local recurrence alone. The panel judged the risk of further microscopic disease outside of the visible recurrence too high to safely exclude the rest of the prostate bed from the CTV. A focused, "stereotactic" approach should be reserved for re-irradiation after previous PORT. Secondly, the option of a focal boost on the recurrence was discussed.

Conclusion

Radiation oncologists are increasingly confronted with macroscopic local recurrences visible on imaging in patients referred for postoperative radiotherapy. It was recommended to always delineate and irradiate the entire prostate bed, and not the local recurrence alone, whatever the exact location of that recurrence. Secondly, specific dose-escalation on the macroscopic recurrence should only be considered if an anatomic correlate is visible. Such a focal boost is probably feasible, provided that OAR constraints are prioritized. Possible dose is also dependent on the location of the recurrence. Its potential benefit should urgently be investigated in prospective clinical trials.

Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation

BACKGROUND

Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance.

METHODS

Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure.

RESULTS

After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals.

CONCLUSION

Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.

A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy

Purpose/objectives

Auto-segmentation with artificial intelligence (AI) offers an opportunity to reduce inter- and intra-observer variability in contouring, to improve the quality of contours, as well as to reduce the time taken to conduct this manual task. In this work we benchmark the AI auto-segmentation contours produced by five commercial vendors against a common dataset.

Methods and materials

The organ at risk (OAR) contours generated by five commercial AI auto-segmentation solutions (Mirada (Mir), MVision (MV), Radformation (Rad), RayStation (Ray) and TheraPanacea (Ther)) were compared to manually-drawn expert contours from 20 breast, 20 head and neck, 20 lung and 20 prostate patients. Comparisons were made using geometric similarity metrics including volumetric and surface Dice similarity coefficient (vDSC and sDSC), Hausdorff distance (HD) and Added Path Length (APL). To assess the time saved, the time taken to manually draw the expert contours, as well as the time to correct the AI contours, were recorded.

Results

There are differences in the number of CT contours offered by each AI auto-segmentation solution at the time of the study (Mir 99; MV 143; Rad 83; Ray 67; Ther 86), with all offering contours of some lymph node levels as well as OARs. Averaged across all structures, the median vDSCs were good for all systems and compared favorably with existing literature: Mir 0.82; MV 0.88; Rad 0.86; Ray 0.87; Ther 0.88. All systems offer substantial time savings, ranging between: breast 14-20 mins; head and neck 74-93 mins; lung 20-26 mins; prostate 35-42 mins. The time saved, averaged across all structures, was similar for all systems: Mir 39.8 mins; MV 43.6 mins; Rad 36.6 min; Ray 43.2 mins; Ther 45.2 mins.

Conclusions

All five commercial AI auto-segmentation solutions evaluated in this work offer high quality contours in significantly reduced time compared to manual contouring, and could be used to render the radiotherapy workflow more efficient and standardized.

ESTRO ACROP guideline on prostate bed delineation for postoperative radiotherapy in prostate cancer

Purpose/Objective

Radiotherapy to the prostate bed is a potentially curative salvage option after radical prostatectomy. Although prostate bed contouring guidelines are available in the literature, important variabilities exist. The objective of this work is to provide a contemporary consensus guideline for prostate bed delineation for postoperative radiotherapy.

Methods

An ESTRO-ACROP contouring consensus panel consisting of 11 radiation oncologists and one radiologist, all with known subspecialty expertise in prostate cancer, was established. Participants were asked to delineate the prostate bed clinical target volumes (CTVs) in 3 separate clinically relevant scenarios: adjuvant radiation, salvage radiation with PSA progression, and salvage radiation with persistently elevated PSA. These cases focused on the presence of positive surgical margin, extracapsular extension, and seminal vesicles involvement. None of the cases had radiographic evidence of local recurrence on imaging. A single computed tomography (CT) dataset was shared via FALCON platform and contours were performed using EduCaseTM software. Contours were analyzed qualitatively using heatmaps which provided a visual assessment of controversial regions and quantitatively analyzed using Sorensen-Dice similarity coefficients. Participants also answered case-specific questionnaires addressing detailed recommendations on target delineation. Discussions via electronic mails and videoconferences for final editing and consensus were performed.

Results

The mean CTV for the adjuvant case was 76 cc (SD = 26.6), salvage radiation with PSA progression was 51.80 cc (SD = 22.7), and salvage radiation with persistently elevated PSA 57.63 cc (SD = 25.2). Compared to the median, the mean Sorensen-Dice similarity coefficient for the adjuvant case was 0.60 (SD 0.10), salvage radiation with PSA progression was 0.58 (SD = 0.12), and salvage radiation with persistently elevated PSA 0.60 (SD = 0.11). A heatmap for each clinical scenario was generated. The group agreed to proceed with a uniform recommendation for all cases, independent of the radiotherapy timing. Several controversial areas of the prostate bed CTV were identified based on both heatmaps and questionnaires. This formed the basis for discussions via videoconferences where the panel achieved consensus on the prostate bed CTV to be used as a novel guideline for postoperative prostate cancer radiotherapy.

Conclusion

Variability was observed in a group formed by experienced genitourinary radiation oncologists and a radiologist. A single contemporary ESTRO-ACROP consensus guideline was developed to address areas of dissonance and improve consistency in prostate bed delineation, independent of the indication.There is important variability in existing contouring guidelines for postoperative prostate bed (PB) radiotherapy (RT) after radical prostatectomy. This work aimed at providing a contemporary consensus guideline for PB delineation. An ESTRO ACROP consensus panel including radiation oncologists and a radiologist, all with known subspecialty expertise in prostate cancer, delineated the PB CTV in 3 scenarios: adjuvant RT, salvage RT with PSA progression, and salvage RT with persistently elevated PSA. None of the cases had evidence of local recurrence. Contours were analysed qualitatively using heatmaps for visual assessment of controversial regions and quantitatively using Sorensen-Dice coefficient. Case-specific questionnaires were also discussed via e-mails and videoconferences for consensus. Several controversial areas of the PB CTV were identified based on both heatmaps and questionnaires. This formed the basis for discussions via videoconferences. Finally, a contemporary ESTRO-ACROP consensus guideline was developed to address areas of dissonance and improve consistency in PB delineation, independent of the indication.

Dynamic Cross-Task Representation Adaptation for Clinical Targets Co-Segmentation in CT Image-Guided Post-Prostatectomy Radiotherapy

Adjuvant and salvage radiotherapy after radical prostatectomy requires precise delineations of prostate bed (PB), i.e., the clinical target volume, and surrounding organs at risk (OARs) to optimize radiotherapy planning. Segmenting PB is particularly challenging even for clinicians, e.g., from the planning computed tomography (CT) images, as it is an invisible/virtual target after the operative removal of the cancerous prostate gland. Very recently, a few deep learning-based methods have been proposed to automatically contour non-contrast PB by leveraging its spatial reliance on adjacent OARs (i.e., the bladder and rectum) with much more clear boundaries, mimicking the clinical workflow of experienced clinicians. Although achieving state-of-the-art results from both the clinical and technical aspects, these existing methods improperly ignore the gap between the hierarchical feature representations needed for segmenting those fundamentally different clinical targets (i.e., PB and OARs), which in turn limits their delineation accuracy. This paper proposes an asymmetric multi-task network integrating dynamic cross-task representation adaptation (i.e., DyAdapt) for accurate and efficient co-segmentation of PB and OARs in one-pass from CT images. In the learning-to-learn framework, the DyAdapt modules adaptively transfer the hierarchical feature representations from the source task of OARs segmentation to match up with the target (and more challenging) task of PB segmentation, conditioned on the dynamic inter-task associations learned from the learning states of the feed-forward path. On a real-patient dataset, our method led to state-of-the-art results of PB and OARs co-segmentation. Code is available at https://github.com/ladderlab-xjtu/DyAdapt.

Risk-Adapted Target Delineation for Breast Cancer: Controversies and Considerations

The advent of computed tomography-based planning coupled with modern tools for target delineation and hypofractionated treatment schedules has increased efficiency and throughput for patients with breast cancer. While the benefit of adjuvant radiation therapy (RT) in reducing locoregional recurrences is established, disentangling local versus regional recurrence risks with modern treatment protocols has become an area of active research to de-escalate treatment. Delineation guidelines for nodal regions either attempt to replicate results of conventional RT techniques by translating bony landmarks to clinical target volumes or use landmarks based on the fact that lymphatic channels run along the vasculature. Because direct comparisons of both approaches are implausible, mapping studies of nodal recurrences have reported on the proportion of nodes included in these delineation guidelines, and larger, bony, landmark-based guidelines appear intuitively appealing for patients with unfavorable risk factors. A pooled analysis of these studies is reported here, along with literature supporting the exclusion of the true chest wall from postmastectomy/breast-conserving surgery clinical target volumes and the selective (versus routine) use of bolus during postmastectomy RT. The risk-adapted approach suggested here accounts for the risk of recurrence as well as toxicity and endorses nuanced target volume delineation rather than a one-size-fits-all approach.

E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation

Purpose

Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is a crowdsourced challenge engaging radiation oncologists across various expertise levels in segmentation. An obstacle to artificial intelligence (AI) development is the paucity of multiexpert datasets; consequently, we sought to characterize whether aggregate segmentations generated from multiple nonexperts could meet or exceed recognized expert agreement.

Approach

Participants who contoured ≥ 1 region of interest (ROI) for the breast, sarcoma, head and neck (H&N), gynecologic (GYN), or gastrointestinal (GI) cases were identified as a nonexpert or recognized expert. Cohort-specific ROIs were combined into single simultaneous truth and performance level estimation (STAPLE) consensus segmentations.STAPLE nonexpert ROIs were evaluated againstSTAPLE expert contours using Dice similarity coefficient (DSC). The expert interobserver DSC (IODSC expert ) was calculated as an acceptability threshold betweenSTAPLE nonexpert andSTAPLE expert . To determine the number of nonexperts required to match theIODSC expert for each ROI, a single consensus contour was generated using variable numbers of nonexperts and then compared to theIODSC expert .

Results

For all cases, the DSC values forSTAPLE nonexpert versusSTAPLE expert were higher than comparator expertIODSC expert for most ROIs. The minimum number of nonexpert segmentations needed for a consensus ROI to achieveIODSC expert acceptability criteria ranged between 2 and 4 for breast, 3 and 5 for sarcoma, 3 and 5 for H&N, 3 and 5 for GYN, and 3 for GI.

Conclusions

Multiple nonexpert-generated consensus ROIs met or exceeded expert-derived acceptability thresholds. Five nonexperts could potentially generate consensus segmentations for most ROIs with performance approximating experts, suggesting nonexpert segmentations as feasible cost-effective AI inputs.

Segmentation stability of human head and neck cancer medical images for radiotherapy applications under de-identification conditions: Benchmarking data sharing and artificial intelligence use-cases

Background

Demand for head and neck cancer (HNC) radiotherapy data in algorithmic development has prompted increased image dataset sharing. Medical images must comply with data protection requirements so that re-use is enabled without disclosing patient identifiers. Defacing, i.e., the removal of facial features from images, is often considered a reasonable compromise between data protection and re-usability for neuroimaging data. While defacing tools have been developed by the neuroimaging community, their acceptability for radiotherapy applications have not been explored. Therefore, this study systematically investigated the impact of available defacing algorithms on HNC organs at risk (OARs).

Methods

A publicly available dataset of magnetic resonance imaging scans for 55 HNC patients with eight segmented OARs (bilateral submandibular glands, parotid glands, level II neck lymph nodes, level III neck lymph nodes) was utilized. Eight publicly available defacing algorithms were investigated: afni_refacer, DeepDefacer, defacer, fsl_deface, mask_face, mri_deface, pydeface, and quickshear. Using a subset of scans where defacing succeeded (N=29), a 5-fold cross-validation 3D U-net based OAR auto-segmentation model was utilized to perform two main experiments: 1.) comparing original and defaced data for training when evaluated on original data; 2.) using original data for training and comparing the model evaluation on original and defaced data. Models were primarily assessed using the Dice similarity coefficient (DSC).

Results

Most defacing methods were unable to produce any usable images for evaluation, while mask_face, fsl_deface, and pydeface were unable to remove the face for 29%, 18%, and 24% of subjects, respectively. When using the original data for evaluation, the composite OAR DSC was statistically higher (p ≤ 0.05) for the model trained with the original data with a DSC of 0.760 compared to the mask_face, fsl_deface, and pydeface models with DSCs of 0.742, 0.736, and 0.449, respectively. Moreover, the model trained with original data had decreased performance (p ≤ 0.05) when evaluated on the defaced data with DSCs of 0.673, 0.693, and 0.406 for mask_face, fsl_deface, and pydeface, respectively.

Conclusion

Defacing algorithms may have a significant impact on HNC OAR auto-segmentation model training and testing. This work highlights the need for further development of HNC-specific image anonymization methods.

Prospective imaging comparison of anatomic delineation with rapid kV cone beam CT on a novel ring gantry radiotherapy device

INTRODUCTION

A kV imager coupled to a novel, ring-gantry radiotherapy system offers improved on-board kV-cone-beam computed tomography (CBCT) acquisition time (17-40 seconds) and image quality, which may improve CT radiotherapy image-guidance and enable online adaptive radiotherapy. We evaluated whether inter-observer contour variability over various anatomic structures was non-inferior using a novel ring gantry kV-CBCT (RG-CBCT) imager as compared to diagnostic-quality simulation CT (simCT).

MATERIALS/METHODS

Seven patients undergoing radiotherapy were imaged with the RG-CBCT system at breath hold (BH) and/or free breathing (FB) for various disease sites on a prospective imaging study. Anatomy was independently contoured by seven radiation oncologists on: 1. SimCT 2. Standard C-arm kV-CBCT (CA-CBCT), and 3. Novel RG-CBCT at FB and BH. Inter-observer contour variability was evaluated by computing simultaneous truth and performance level estimation (STAPLE) consensus contours, then computing average symmetric surface distance (ASSD) and Dice similarity coefficient (DSC) between individual raters and consensus contours for comparison across image types.

RESULTS

Across 7 patients, 18 organs-at-risk (OARs) were evaluated on 27 image sets. Both BH and FB RG-CBCT were non-inferior to simCT for inter-observer delineation variability across all OARs and patients by ASSD analysis (p < 0.001), whereas CA-CBCT was not (p = 0.923). RG-CBCT (FB and BH) also remained non-inferior for abdomen and breast subsites compared to simCT on ASSD analysis (p < 0.025). On DSC comparison, neither RG-CBCT nor CA-CBCT were non-inferior to simCT for all sites (p > 0.025).

CONCLUSIONS

Inter-observer ability to delineate OARs using novel RG-CBCT images was non-inferior to simCT by the ASSD criterion but not DSC criterion.

Magnetic Resonance Imaging-Based Delineation of Organs at Risk in the Head and Neck Region

Purpose

The aim of this article is to establish a comprehensive contouring guideline for treatment planning using only magnetic resonance images through an up-to-date set of organs at risk (OARs), recommended organ boundaries, and relevant suggestions for the magnetic resonance imaging (MRI)-based delineation of OARs in the head and neck (H&N) region.

Methods and Materials

After a detailed review of the literature, MRI data were collected from the H&N region of healthy volunteers. OARs were delineated in the axial, coronal, and sagittal planes on T2-weighted sequences. Every contour defined was revised by 4 radiation oncologists and subsequently by 2 independent senior experts (H&N radiation oncologist and radiologist). After revision, the final structures were presented to the consortium partners.

Results

A definitive consensus was reached after multi-institutional review. On that basis, we provided a detailed anatomic and functional description and specific MRI characteristics of the OARs.

Conclusions

In the era of precision radiation therapy, the need for well-built, straightforward contouring guidelines is on the rise. Precise, uniform, delineation-based, automated OAR segmentation on MRI may lead to increased accuracy in terms of organ boundaries and analysis of dose-dependent sequelae for an adequate definition of normal tissue complication probability.

Prospective clinical evaluation of integrating a radiation anatomist for contouring in routine radiation treatment planning

Purpose

A radiation anatomist was trained and integrated into clinical practice at a multi-site academic center. The primary objective of this quality improvement study was to determine whether a radiation anatomist improves the quality of organ-at-risk (OAR) contours, and secondarily to determine the impact on efficiency in the treatment planning process.

Methods and Materials

From March to August 2020, all patients undergoing computed tomography–based radiation planning at 2 clinics at Memorial Sloan Kettering Cancer Center were assigned using an “every other” process to either (1) OAR contouring by a radiation anatomist (intervention) or (2) contouring by the treating physician (standard of care). Blinded dosimetrists reported OAR contour quality using a 3-point scoring system based on a common clinical trial protocol deviation scale (1, acceptable; 2, minor deviation; and 3, major deviation). Physicians reported time spent contouring for all cases. Analyses included the Fisher exact test and multivariable ordinal logistic regression.

Results

There were 249 cases with data available for the primary endpoint (66% response rate). The mean OAR quality rating was 1.1 ± 0.4 for the intervention group and 1.4 ± 0.7 for the standard of care group (P < .001), with subset analysis showing a significant difference for gastrointestinal cases (n = 49; P <.001). Time from simulation to contour approval was reduced from 3 days (interquartile range [IQR], 1-6 days) in the control group to 2 days (IQR, 1-5 days) in the intervention group (P = .007). Both physicians and dosimetrists self-reported decreased time spent contouring in the intervention group compared with the control group, with a decreases of 8 minutes (17%; P < .001) and 5 minutes (50%; P = .002), respectively. Qualitative comments most often indicated edits required to bowel contours (n = 14).

Conclusions

These findings support improvements in both OAR contour quality and workflow efficiency with implementation of a radiation anatomist in routine practice. Findings could also inform development of autosegmentation by identifying disease sites and specific OARs contributing to low clinical efficiency. Future research is needed to determine the potential effect of reduced physician time spent contouring OARs on burnout.

Deep-Learning-Based Automatic Segmentation of Head and Neck Organs for Radiation Therapy in Dogs

Purpose: This study was conducted to develop a deep learning-based automatic segmentation (DLBAS) model of head and neck organs for radiotherapy (RT) in dogs, and to evaluate the feasibility for delineating the RT planning.

Materials and Methods: The segmentation indicated that there were potentially 15 organs at risk (OARs) in the head and neck of dogs. Post-contrast computed tomography (CT) was performed in 90 dogs. The training and validation sets comprised 80 CT data sets, including 20 test sets. The accuracy of the segmentation was assessed using both the Dice similarity coefficient (DSC) and the Hausdorff distance (HD), and by referencing the expert contours as the ground truth. An additional 10 clinical test sets with relatively large displacement or deformation of organs were selected for verification in cancer patients. To evaluate the applicability in cancer patients, and the impact of expert intervention, three methods–HA, DLBAS, and the readjustment of the predicted data obtained via the DLBAS of the clinical test sets (HA_DLBAS)–were compared.

Results: The DLBAS model (in the 20 test sets) showed reliable DSC and HD values; it also had a short contouring time of ~3 s. The average (mean ± standard deviation) DSC (0.83 ± 0.04) and HD (2.71 ± 1.01 mm) values were similar to those of previous human studies. The DLBAS was highly accurate and had no large displacement of head and neck organs. However, the DLBAS in the 10 clinical test sets showed lower DSC (0.78 ± 0.11) and higher HD (4.30 ± 3.69 mm) values than those of the test sets. The HA_DLBAS was comparable to both the HA (DSC: 0.85 ± 0.06 and HD: 2.74 ± 1.18 mm) and DLBAS presented better comparison metrics and decreased statistical deviations (DSC: 0.94 ± 0.03 and HD: 2.30 ± 0.41 mm). In addition, the contouring time of HA_DLBAS (30 min) was less than that of HA (80 min).

Conclusion: In conclusion, HA_DLBAS method and the proposed DLBAS was highly consistent and robust in its performance. Thus, DLBAS has great potential as a single or supportive tool to the key process in RT planning.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.