Contouring variations and the role of atlas in non-small cell lung cancer radiation therapy: Analysis of a multi-institutional preclinical trial planning study

A Prospective Study Measuring Resident and Faculty Contour Concordance: A Potential Tool for Quantitative Assessment of Residents' Performance in Contouring and Target Delineation in Radiation Oncology Residency

PURPOSE/OBJECTIVE(S)

Accurate target delineation (ie, contouring) is essential for radiation treatment planning and radiotherapy efficacy. As a result, improving the quality of target delineation is an important goal in the education of radiation oncology residents. The purpose of this study was to track the concordance of radiation oncology residents' contours with those of faculty physicians over the course of 1 year to assess for patterns.

MATERIALS/METHODS

Residents in postgraduate year (PGY) levels 2 to 4 were asked to contour target volumes that were then compared to the finalized, faculty physician-approved contours. Concordance between resident and faculty physician contours was determined by calculating the Jaccard concordance index (JCI), ranging from 0, meaning no agreement, to 1, meaning complete agreement. Multivariate mixed-effect models were used to assess the association of JCI to the fixed effect of PGY level and its interactions with cancer type and other baseline characteristics. Post hoc means of JCI were compared between PGY levels after accounting for multiple comparisons using Tukey's method.

RESULTS

In total, 958 structures from 314 patients collected during the 2020-2021 academic year were studied. The mean JCI was 0.77, 0.75, and 0.61 for the PGY-4, PGY-3, and PGY-2 levels, respectively. The JCI score for PGY-2 was found to be lower than those for PGY-3 and PGY-4, respectively (all P < .001). No statistically significant difference of JCI score was found between the PGY-3 and PGY-4 levels. The average JCI score was lowest (0.51) for primary head and/or neck cancers, and it was highest (0.80) for gynecologic cancers.

CONCLUSIONS

Tracking and comparing the concordance of resident contours with faculty physician contours is an intriguing method of assessing resident performance in contouring and target delineation and could potentially serve as a quantitative metric, which is lacking currently, in radiation oncology resident evaluation. However, additional study is necessary before this technique can be incorporated into residency assessments.

Artificial Intelligence for Radiotherapy Auto-Contouring: Current Use, Perceptions of and Barriers to Implementation

AIMS

Artificial intelligence has the potential to transform the radiotherapy workflow, resulting in improved quality, safety, accuracy and timeliness of radiotherapy delivery. Several commercially available artificial intelligence-based auto-contouring tools have emerged in recent years. Their clinical deployment raises important considerations for clinical oncologists, including quality assurance and validation, education, training and job planning. Despite this, there is little in the literature capturing the views of clinical oncologists with respect to these factors.

MATERIALS AND METHODS

The Royal College of Radiologists realises the transformational impact artificial intelligence is set to have on our specialty and has appointed the Artificial Intelligence for Clinical Oncology working group. The aim of this work was to survey clinical oncologists with regards to perceptions, current use of and barriers to using artificial intelligence-based auto-contouring for radiotherapy. Here we share our findings with the wider clinical and radiation oncology communities. We hope to use these insights in developing support, guidance and educational resources for the deployment of auto-contouring for clinical use, to help develop the case for wider access to artificial intelligence-based auto-contouring across the UK and to share practice from early-adopters.

RESULTS

In total, 78% of clinical oncologists surveyed felt that artificial intelligence would have a positive impact on radiotherapy. Attitudes to risk were more varied, but 49% felt that artificial intelligence will decrease risk for patients. There is a marked appetite for urgent guidance, education and training on the safe use of such tools in clinical practice. Furthermore, there is a concern that the adoption and implementation of such tools is not equitable, which risks exacerbating existing inequalities across the country.

CONCLUSION

Careful coordination is required to ensure that all radiotherapy departments, and the patients they serve, may enjoy the benefits of artificial intelligence in radiotherapy. Professional organisations, such as the Royal College of Radiologists, have a key role to play in delivering this.

E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation

Purpose

Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is a crowdsourced challenge engaging radiation oncologists across various expertise levels in segmentation. An obstacle to artificial intelligence (AI) development is the paucity of multiexpert datasets; consequently, we sought to characterize whether aggregate segmentations generated from multiple nonexperts could meet or exceed recognized expert agreement.

Approach

Participants who contoured ≥ 1 region of interest (ROI) for the breast, sarcoma, head and neck (H&N), gynecologic (GYN), or gastrointestinal (GI) cases were identified as a nonexpert or recognized expert. Cohort-specific ROIs were combined into single simultaneous truth and performance level estimation (STAPLE) consensus segmentations.STAPLE nonexpert ROIs were evaluated againstSTAPLE expert contours using Dice similarity coefficient (DSC). The expert interobserver DSC (IODSC expert ) was calculated as an acceptability threshold betweenSTAPLE nonexpert andSTAPLE expert . To determine the number of nonexperts required to match theIODSC expert for each ROI, a single consensus contour was generated using variable numbers of nonexperts and then compared to theIODSC expert .

Results

For all cases, the DSC values forSTAPLE nonexpert versusSTAPLE expert were higher than comparator expertIODSC expert for most ROIs. The minimum number of nonexpert segmentations needed for a consensus ROI to achieveIODSC expert acceptability criteria ranged between 2 and 4 for breast, 3 and 5 for sarcoma, 3 and 5 for H&N, 3 and 5 for GYN, and 3 for GI.

Conclusions

Multiple nonexpert-generated consensus ROIs met or exceeded expert-derived acceptability thresholds. Five nonexperts could potentially generate consensus segmentations for most ROIs with performance approximating experts, suggesting nonexpert segmentations as feasible cost-effective AI inputs.

Clinical validation of deep learning algorithms for radiotherapy targeting of non-small-cell lung cancer: an observational study

BACKGROUND

Artificial intelligence (AI) and deep learning have shown great potential in streamlining clinical tasks. However, most studies remain confined to in silico validation in small internal cohorts, without external validation or data on real-world clinical utility. We developed a strategy for the clinical validation of deep learning models for segmenting primary non-small-cell lung cancer (NSCLC) tumours and involved lymph nodes in CT images, which is a time-intensive step in radiation treatment planning, with large variability among experts.

METHODS

In this observational study, CT images and segmentations were collected from eight internal and external sources from the USA, the Netherlands, Canada, and China, with patients from the Maastro and Harvard-RT1 datasets used for model discovery (segmented by a single expert). Validation consisted of interobserver and intraobserver benchmarking, primary validation, functional validation, and end-user testing on the following datasets: multi-delineation, Harvard-RT1, Harvard-RT2, RTOG-0617, NSCLC-radiogenomics, Lung-PET-CT-Dx, RIDER, and thorax phantom. Primary validation consisted of stepwise testing on increasingly external datasets using measures of overlap including volumetric dice (VD) and surface dice (SD). Functional validation explored dosimetric effect, model failure modes, test-retest stability, and accuracy. End-user testing with eight experts assessed automated segmentations in a simulated clinical setting.

FINDINGS

We included 2208 patients imaged between 2001 and 2015, with 787 patients used for model discovery and 1421 for model validation, including 28 patients for end-user testing. Models showed an improvement over the interobserver benchmark (multi-delineation dataset; VD 0·91 [IQR 0·83-0·92], p=0·0062; SD 0·86 [0·71-0·91], p=0·0005), and were within the intraobserver benchmark. For primary validation, AI performance on internal Harvard-RT1 data (segmented by the same expert who segmented the discovery data) was VD 0·83 (IQR 0·76-0·88) and SD 0·79 (0·68-0·88), within the interobserver benchmark. Performance on internal Harvard-RT2 data segmented by other experts was VD 0·70 (0·56-0·80) and SD 0·50 (0·34-0·71). Performance on RTOG-0617 clinical trial data was VD 0·71 (0·60-0·81) and SD 0·47 (0·35-0·59), with similar results on diagnostic radiology datasets NSCLC-radiogenomics and Lung-PET-CT-Dx. Despite these geometric overlap results, models yielded target volumes with equivalent radiation dose coverage to those of experts. We also found non-significant differences between de novo expert and AI-assisted segmentations. AI assistance led to a 65% reduction in segmentation time (5·4 min; p<0·0001) and a 32% reduction in interobserver variability (SD; p=0·013).

INTERPRETATION

We present a clinical validation strategy for AI models. We found that in silico geometric segmentation metrics might not correlate with clinical utility of the models. Experts' segmentation style and preference might affect model performance.

FUNDING

US National Institutes of Health and EU European Research Council.

Impact of image quality on radiomics applications

Radiomics features extracted from medical images have been widely reported to be useful in the patient specific outcome modeling for variety of assessment and prediction purposes. Successful application of radiomics features as imaging biomarkers, however, is dependent on the robustness of the approach to the variation in each step of the modeling workflow. Variation in the input image quality is one of the main sources that impacts the reproducibility of radiomics analysis when a model is applied to broader range of medical imaging data. The quality of medical image is generally affected by both the scanner related factors such as image acquisition/reconstruction settings and the patient related factors such as patient motion. This article aimed to review the published literatures in this field that reported the impact of various imaging factors on the radiomics features through the change in image quality. The literatures were categorized by different imaging modalities and also tabulated based on the imaging parameters and the class of radiomics features included in the study. Strategies for image quality standardization were discussed based on the relevant literatures and recommendations for reducing the impact of image quality variation on the radiomics in multi-institutional clinical trial were summarized at the end of this article.

Dose-volume-based evaluation of convolutional neural network-based auto-segmentation of thoracic organs at risk

•

Dice score and Hausdorff distance do not correlate with dose-volume-based results.

•

Auto-contours close to the tumor or in entry/exit beams should be checked.

•

Heart and esophagus must be checked for locally advanced non-small cell lung cancer.

•

Bronchi must be checked for peripheral early-stage non-small cell lung cancer.

•

Every treatment plan still passed the clinical goals for the manual organs at risk.

Background and purpose

The geometrical accuracy of auto-segmentation using convolutional neural networks (CNNs) has been demonstrated. This study aimed to investigate the dose-volume impact of differences between automatic and manual OARs for locally advanced (LA) and peripherally located early-stage (ES) non-small cell lung cancer (NSCLC).

Material and methods

A single CNN was created for automatic delineation of the heart, lungs, main left and right bronchus, esophagus, spinal cord and trachea using 55/10/40 patients for training/validation/testing. Dice score coefficient (DSC) and 95th percentile Hausdorff distance (HD95) were used for geometrical analysis. A new treatment plan based on the auto-segmented OARs was created for each test patient using 3D for ES-NSCLC (SBRT, 3–8 fractions) and IMRT for LA-NSCLC (24–35 fractions). The correlation between geometrical metrics and dose-volume differences was investigated.

Results

The average (±1 SD) DSC and HD95 were 0.82 ± 0.07 and 16.2 ± 22.4 mm, while the average dose-volume differences were 0.5 ± 1.5 Gy (ES) and 1.5 ± 2.8 Gy (LA). The geometrical metrics did not correlate with the observed dose-volume differences (average Pearson for DSC: −0.27 ± 0.18 (ES) and −0.09 ± 0.12 (LA); HD95: 0.1 ± 0.3 mm (ES) and 0.2 ± 0.2 mm (LA)).

Conclusions

After post-processing, manual adjustments of automatic contours are only needed for clinically relevant OARs situated close to the tumor or within an entry or exit beam e.g., the heart and the esophagus for LA-NSCLC and the bronchi for ES-NSCLC. The lungs do not need to be checked further in detail.

Real-world analysis of manual editing of deep learning contouring in the thorax region

Background and purpose

User-adjustments after deep-learning (DL) contouring in radiotherapy were evaluated to get insight in real-world editing during clinical practice. This study assessed the amount, type and spatial regions of editing of auto-contouring for organs-at-risk (OARs) in routine clinical workflow for patients in the thorax region.

Materials and methods

A total of 350 lung cancer and 362 breast cancer patients, contoured between March 2020 and March 2021 using a commercial DL-contouring method followed by manual adjustments were retrospectively analyzed. Subsampling was performed for some OARs, using an inter-slice gap of 1-3 slices. Commonly-used whole-organ contouring assessment measures were calculated, and all cases were registered to a common reference shape per OAR to identify regions of manual adjustment. Results were expressed as the median, 10th-90th percentile of adjustment and visualized using 3D renderings.

Results

Per OAR, the median amount of editing was below 1 mm. However, large adjustments were found in some locations for most OARs. In general, enlarging of the auto-contours was needed. Subsampling DL-contours showed less adjustments were made in the interpolated slices compared to simulated no-subsampling for these OARs.

Conclusion

The real-world performance of automatic DL-contouring software was evaluated and proven useful in clinical practice. Specific regions-of-adjustment were identified per OAR in the thorax region, and separate models were found to be necessary for specific clinical indications different from training data. This analysis showed the need to perform routine clinical analysis especially when procedures or acquisition protocols change to have the best configuration of the workflow.

Delineation of organs at risk

The delineation of organs at risk is the basis of radiotherapy oncologists' work. Indeed, the knowledge of this delineation enables to better identify the target volumes and to optimize dose distribution, involving the prognosis of the patients but also their future. The learning of this delineation must continue throughout the clinician's career. Some contour changes have appeared with better imaging, some volumes are now required due to development of knowledge of side effects. In addition, the increasing survival time of patients requires to be more systematic and precise in the delineations, both to avoid complications until now exceptional but also because re-irradiations are becoming more and more frequent. We present the update of the recommendations of the French Society for Radiation Oncology (SFRO) on new findings or adaptations to volumes at risk.

Challenges in the target volume definition of lung cancer radiotherapy

Radiotherapy, with or without systemic treatment has an important role in the management of lung cancer. In order to deliver the treatment accurately, the clinician must precisely outline the gross tumour volume (GTV), mostly on computed tomography (CT) images. However, due to the limited contrast between tumour and non-malignant changes in the lung tissue, it can be difficult to distinguish the tumour boundaries on CT images leading to large interobserver variation and differences in interpretation. Therefore the definition of the GTV has often been described as the weakest link in radiotherapy with its inaccuracy potentially leading to missing the tumour or unnecessarily irradiating normal tissue. In this article, we review the various techniques that can be used to reduce delineation uncertainties in lung cancer.

Particle Beam Therapy for Cardiac-Sparing Radiotherapy in Non-Small Cell Lung Cancer

Radiation therapy plays an integral role in the treatment of all stages of non-small cell lung cancer. Survival outcomes are improving, but radiation therapy remains associated with long-term toxicity. Recently, it has become evident that the heart is an important organ at risk for treatment-related morbidity. In this review, we discuss the hypothesis that particle radiation therapy offers superior dosimetry compared with photon-based treatment, and that this comparative advantage translates into clinically meaningful cardiac toxicity reduction with similar local tumor control. We discuss the evidence in non-small cell lung cancer to date, the ongoing prospective trials that may provide additional insight, and the opportunities to optimally integrate particle therapy into future prospective investigation.

Validation of a new open-source method for automatic delineation and dose assessment of the heart and LADCA in breast radiotherapy with simultaneous uncertainty estimation

Radiotherapy has been shown to increase risks of cardiotoxicities for breast cancer patients. Automated delineation approaches are necessary for consistent and efficient assessment of cardiac doses in large, retrospective datasets, while patient-specific estimation of the uncertainty in these doses provides valuable additional data for modelling and understanding risks. In this work, we aim to validate the consistency of our previously described open-source software model for automatic cardiac delineation in the context of dose assessment, relative to manual contouring. We also extend our software to introduce a novel method to automatically quantify the uncertainty in cardiac doses based on expected inter-observer variability (IOV) in contouring. This method was applied to a cohort of 15 left-sided breast cancer patients treated in Denmark using modern tangential radiotherapy techniques. On each image set, the whole heart and left anterior descending coronary artery (LADCA) were contoured by nine independent experts; the range of doses to these nine volumes provided a reference for the dose uncertainties generated from the automatic method. Local and external atlas sets were used to test the method. Results give confidence in the consistency of automatic segmentations, with mean whole heart dose differences for local and external atlas sets of -0.20 ± 0.17 and -0.10 ± 0.14 Gy, respectively. Automatic estimates of uncertainties in doses are similar to those from IOV for both the whole heart and LADCA. Overall, this study confirms that our automated approach can be used to accurately assess cardiac doses, and the proposed method can provide a useful tool in estimating dose uncertainties.

Assessing tumor centrality in lung stereotactic ablative body radiotherapy (SABR): the effects of variations in bronchial tree delineation and potential for automated methods

Accurate delineation of the proximal bronchial tree (PBT) is crucial for appropriate assessment of lung tumor centrality and choice of Stereotactic Ablative Body Radiotherapy (SABR) dose prescription. Here, we investigate variabilities in manual PBT delineation and their potential to influence assessing lesion centrality. A fully automatic, intensity-based tool for PBT contouring and measuring distance to the target is also described. This retrospective analysis included a total of 61 patients treated with lung SABR. A subset of 41 patients was used as a training dataset, containing clinical PBT contour and additional subsequently generated manual contours. The tool was optimized and compared against manual contours in terms of volume, distance to the target and various overlap/similarity metrics. The remaining 20 patients were used as a validation dataset to investigate the dosimetric effects of variations between manual and automatic PBT contours. Considerable interobserver variability was observed, particularly in identifying the superior and inferior borders of the PBT. Automatic PBT contours were comparable to manual contours with average Dice of 0.63 to 0.79 and mean distance to agreement of 1.78 to 3.34 mm. No significant differences in dosimetric parameters were found between automatically and manually generated contours. A moderate negative correlation was found between PBT maximum dose and distance to the lesion (p < 0.05). Variability in manual PBT delineation may result in inconsistent assessment of tumor centrality. Automatic contouring can help standardize clinical practice, support investigations into the link between SABR outcomes and lesion proximity to central airways and the development of predictive toxicity models that incorporate precise measurements of tumor location in relation to high-risk organs.

Artificial intelligence in radiation oncology

Artificial intelligence (AI) has the potential to fundamentally alter the way medicine is practised. AI platforms excel in recognizing complex patterns in medical data and provide a quantitative, rather than purely qualitative, assessment of clinical conditions. Accordingly, AI could have particularly transformative applications in radiation oncology given the multifaceted and highly technical nature of this field of medicine with a heavy reliance on digital data processing and computer software. Indeed, AI has the potential to improve the accuracy, precision, efficiency and overall quality of radiation therapy for patients with cancer. In this Perspective, we first provide a general description of AI methods, followed by a high-level overview of the radiation therapy workflow with discussion of the implications that AI is likely to have on each step of this process. Finally, we describe the challenges associated with the clinical development and implementation of AI platforms in radiation oncology and provide our perspective on how these platforms might change the roles of radiotherapy medical professionals.

Deep Learning Improved Clinical Target Volume Contouring Quality and Efficiency for Postoperative Radiation Therapy in Non-small Cell Lung Cancer

Purpose: To investigate whether a deep learning-assisted contour (DLAC) could provide greater accuracy, inter-observer consistency, and efficiency compared with a manual contour (MC) of the clinical target volume (CTV) for non-small cell lung cancer (NSCLC) receiving postoperative radiotherapy (PORT).

Materials and Methods: A deep dilated residual network was used to achieve the effective automatic contour of the CTV. Eleven junior physicians contoured CTVs on 19 patients by using both MC and DLAC methods independently. Compared with the ground truth, the accuracy of the contour was evaluated by using the Dice coefficient and mean distance to agreement (MDTA). The coefficient of variation (CV) and standard distance deviation (SDD) were rendered to measure the inter-observer variability or consistency. The time consumed for each of the two contouring methods was also compared.

Results: A total of 418 CTV sets were generated. DLAC improved contour accuracy when compared with MC and was associated with a larger Dice coefficient (mean ± SD: 0.75 ± 0.06 vs. 0.72 ± 0.07, p < 0.001) and smaller MDTA (mean ± SD: 2.97 ± 0.91 mm vs. 3.07 ± 0.98 mm, p < 0.001). The DLAC was also associated with decreased inter-observer variability, with a smaller CV (mean ± SD: 0.129 ± 0.040 vs. 0.183 ± 0.043, p < 0.001) and SDD (mean ± SD: 0.47 ± 0.22 mm vs. 0.72 ± 0.41 mm, p < 0.001). In addition, a value of 35% of time saving was provided by the DLAC (median: 14.81 min vs. 9.59 min, p < 0.001).

Conclusions: Compared with MC, the DLAC is a promising strategy to obtain superior accuracy, consistency, and efficiency for the PORT-CTV in NSCLC.

Bridging the Gap in Global Advanced Radiation Oncology Training: Impact of a Web-Based Open-Access Interactive Three-Dimensional Contouring Atlas on Radiation Oncologist Practice in Russia

Radiation oncologists in Russia face a number of unique professional difficulties including lack of standardized training and continuing medical education. To combat this, under the auspices of the Russian Society of Clinical Oncology (RUSSCO), our group has developed a series of ongoing in-person interactive contouring workshops that are held during the major Russian oncology conferences in Moscow, Russia. Since November 2016 during each workshop, we utilized a web-based open-access interactive three-dimensional contouring atlas as part of our didactics. We sought to determine the impact of this resource on radiation oncology practice in Russia. We distributed an IRB-approved web-based survey to 172 practicing radiation oncologists in Russia. We inquired about practice demographics, RUSSCO contouring workshop attendance, and the clinical use of open-access English language interactive contouring atlas (eContour). The survey remained open for 2 months until November 2017. Eighty radiation oncologists completed the survey with a 46.5% response rate. Mean number of years in practice was 13.7. Sixty respondents (75%) attended at least one RUSSCO contouring workshop. Of those who were aware of eContour, 76% were introduced during a RUSSCO contouring workshop, and 81% continue to use it in their daily practice. The greatest obstacles to using the program were language barrier (51%) and internet access (38%). Nearly 90% reported their contouring practices changed since they started using the program, particularly for delineation of clinical target volumes (57%) and/or organs at risk (46%). More than 97% found the clinical pearls/links to cooperative group protocols in the software helpful in their daily practice. The majority used the contouring program several times per month (43%) or several times per week (41%). Face-to-face contouring instruction in combination with open-access web-based interactive contouring resource had a meaningful impact on perceived quality of radiation oncology contours among Russian practitioners and has the potential to have applications worldwide.

Preliminary Clinical Study of the Differences Between Interobserver Evaluation and Deep Convolutional Neural Network-Based Segmentation of Multiple Organs at Risk in CT Images of Lung Cancer

Background: In this study, publicly datasets with organs at risk (OAR) structures were used as reference data to compare the differences of several observers. Convolutional neural network (CNN)-based auto-contouring was also used in the analysis. We evaluated the variations among observers and the effect of CNN-based auto-contouring in clinical applications. Materials and methods: A total of 60 publicly available lung cancer CT with structures were used; 48 cases were used for training, and the other 12 cases were used for testing. The structures of the datasets were used as reference data. Three observers and a CNN-based program performed contouring for 12 testing cases, and the 3D dice similarity coefficient (DSC) and mean surface distance (MSD) were used to evaluate differences from the reference data. The three observers edited the CNN-based contours, and the results were compared to those of manual contouring. A value of P<0.05 was considered statistically significant. Results: Compared to the reference data, no statistically significant differences were observed for the DSCs and MSDs among the manual contouring performed by the three observers at the same institution for the heart, esophagus, spinal cord, and left and right lungs. The 95% confidence interval (CI) and P-values of the CNN-based auto-contouring results comparing to the manual results for the heart, esophagus, spinal cord, and left and right lungs were as follows: the DSCs were CNN vs. A: 0.914~0.939(P = 0.004), 0.746~0.808(P = 0.002), 0.866~0.887(P = 0.136), 0.952~0.966(P = 0.158) and 0.960~0.972 (P = 0.136); CNN vs. B: 0.913~0.936 (P = 0.002), 0.745~0.807 (P = 0.005), 0.864~0.894 (P = 0.239), 0.952~0.964 (P = 0.308), and 0.959~0.971 (P = 0.272); and CNN vs. C: 0.912~0.933 (P = 0.004), 0.748~0.804(P = 0.002), 0.867~0.890 (P = 0.530), 0.952~0.964 (P = 0.308), and 0.958~0.970 (P = 0.480), respectively. The P-values of MSDs are similar to DSCs. The P-values of heart and esophagus is smaller than 0.05. No significant differences were found between the edited CNN-based auto-contouring results and the manual results. Conclusion: For the spinal cord, both lungs, no statistically significant differences were found between CNN-based auto-contouring and manual contouring. Further modifications to contouring of the heart and esophagus are necessary. Overall, editing based on CNN-based auto-contouring can effectively shorten the contouring time without affecting the results. CNNs have considerable potential for automatic contouring applications.

Use of Crowd Innovation to Develop an Artificial Intelligence-Based Solution for Radiation Therapy Targeting

IMPORTANCE

Radiation therapy (RT) is a critical cancer treatment, but the existing radiation oncologist work force does not meet growing global demand. One key physician task in RT planning involves tumor segmentation for targeting, which requires substantial training and is subject to significant interobserver variation.

OBJECTIVE

To determine whether crowd innovation could be used to rapidly produce artificial intelligence (AI) solutions that replicate the accuracy of an expert radiation oncologist in segmenting lung tumors for RT targeting.

DESIGN, SETTING, AND PARTICIPANTS

We conducted a 10-week, prize-based, online, 3-phase challenge (prizes totaled $55 000). A well-curated data set, including computed tomographic (CT) scans and lung tumor segmentations generated by an expert for clinical care, was used for the contest (CT scans from 461 patients; median 157 images per scan; 77 942 images in total; 8144 images with tumor present). Contestants were provided a training set of 229 CT scans with accompanying expert contours to develop their algorithms and given feedback on their performance throughout the contest, including from the expert clinician.

MAIN OUTCOMES AND MEASURES

The AI algorithms generated by contestants were automatically scored on an independent data set that was withheld from contestants, and performance ranked using quantitative metrics that evaluated overlap of each algorithm's automated segmentations with the expert's segmentations. Performance was further benchmarked against human expert interobserver and intraobserver variation.

RESULTS

A total of 564 contestants from 62 countries registered for this challenge, and 34 (6%) submitted algorithms. The automated segmentations produced by the top 5 AI algorithms, when combined using an ensemble model, had an accuracy (Dice coefficient = 0.79) that was within the benchmark of mean interobserver variation measured between 6 human experts. For phase 1, the top 7 algorithms had average custom segmentation scores (S scores) on the holdout data set ranging from 0.15 to 0.38, and suboptimal performance using relative measures of error. The average S scores for phase 2 increased to 0.53 to 0.57, with a similar improvement in other performance metrics. In phase 3, performance of the top algorithm increased by an additional 9%. Combining the top 5 algorithms from phase 2 and phase 3 using an ensemble model, yielded an additional 9% to 12% improvement in performance with a final S score reaching 0.68.

CONCLUSIONS AND RELEVANCE

A combined crowd innovation and AI approach rapidly produced automated algorithms that replicated the skills of a highly trained physician for a critical task in radiation therapy. These AI algorithms could improve cancer care globally by transferring the skills of expert clinicians to under-resourced health care settings.

The impact of training and professional collaboration on the interobserver variation of lung cancer delineations: a multi-institutional study

BACKGROUND

To assess the impact of training and interprofessional collaboration on the interobserver variation in the delineation of the lung gross tumor volume (GTVp) and lymph node (GTVln).

MATERIAL AND METHODS

Eight target volume delineations courses were organized between 2008 and 2013. Specialists and trainees in radiation oncology were asked to delineate the GTVp and GTVln on four representative CT images of a patient diagnosed with lung cancer individually prior each course (baseline), together as group (interprofessional collaboration) and post-training. The mean delineated volume and local standard deviation (local SD) between the contours for each course group were calculated and compared with the expert delineations.

RESULTS

A total 410 delineations were evaluated. The average local SD was lowest for the interprofessional collaboration (GTVp = 0.194 cm, GTVln = 0.371 cm) followed by the post-training (GTVp = 0.244 cm, GTVln = 0.607 cm) and baseline delineations (GTVp = 0.274 cm, GTVln: 0.718 cm). The mean delineated volume was smallest for the interprofessional (GTVp = 4.93 cm³, GTVln = 4.34 cm³) followed by the post-training (GTVp = 5.68 cm³, GTVln = 5.47 cm³) and baseline delineations (GTVp = 6.65 cm³, GTVln = 6.93 cm³). All delineations were larger than the expert for both GTVp and GTVln (p < .001).

CONCLUSION

Our findings indicate that image interpretational differences can lead to large interobserver variation particularly when delineating the GTVln. Interprofessional collaboration was found to have the greatest impact on reducing interobserver variation in the delineation of the GTVln. This highlights the need to develop a clinical workflow so as to ensure that difficult cases are reviewed routinely by a second radiation oncologist or radiologist so as to minimize the risk of geographical tumor miss and unnecessary irradiation to normal tissue.

NCTN clinical trial standardization for radiotherapy through IROC and CIRO

Evidence-based practice is the cornerstone of modern medicine. Randomized clinical trials across multiple institutions are the gold standard for modern evidence collection. National Cancer Trials Network (NCTN) instruments the clinical trials through the new infrastructure for improvements in cancer treatment. Radiation therapy is an integral component of cancer treatment and is involved in many of the NCTN clinical trials. Radiotherapy is experiencing exciting developments in new treatment modalities and multi-modality image guidance. One of NCTN network groups NRG Oncology brings together the research areas of the National Surgical Adjuvant Breast and Bowel Project (NSABP), the Radiation Therapy Oncology Group (RTOG), and the Gynecologic Oncology Group (GOG). The Imaging and Radiation Oncology Core (IROC) and Center for Innovation in Radiation Oncology(CIRO) of NRG Oncology complement each other's functions in development and implementation of the new radiotherapy and imaging technologies in clinical trials with standardization and other strategies for quality. The standardization process is the essential step to make the data collected for clinical trials of high quality, interoperable, and reusable.

Magnetic resonance imaging in precision radiation therapy for lung cancer

Radiotherapy remains the cornerstone of curative treatment for inoperable locally advanced lung cancer, given concomitantly with platinum-based chemotherapy. With poor overall survival, research efforts continue to explore whether integration of advanced radiation techniques will assist safe treatment intensification with the potential for improving outcomes. One advance is the integration of magnetic resonance imaging (MRI) in the treatment pathway, providing anatomical and functional information with excellent soft tissue contrast without exposure of the patient to radiation. MRI may complement or improve the diagnostic staging accuracy of F-18 fluorodeoxyglucose position emission tomography and computerized tomography imaging, particularly in assessing local tumour invasion and is also effective for identification of nodal and distant metastatic disease. Incorporating anatomical MRI sequences into lung radiotherapy treatment planning is a novel application and may improve target volume and organs at risk delineation reproducibility. Furthermore, functional MRI may facilitate dose painting for heterogeneous target volumes and prediction of normal tissue toxicity to guide adaptive strategies. MRI sequences are rapidly developing and although the issue of intra-thoracic motion has historically hindered the quality of MRI due to the effect of motion, progress is being made in this field. Four-dimensional MRI has the potential to complement or supersede 4D CT and 4D F-18-FDG PET, by providing superior spatial resolution. A number of MR-guided radiotherapy delivery units are now available, combining a radiotherapy delivery machine (linear accelerator or cobalt-60 unit) with MRI at varying magnetic field strengths. This novel hybrid technology is evolving with many technical challenges to overcome. It is anticipated that the clinical benefits of MR-guided radiotherapy will be derived from the ability to adapt treatment on the fly for each fraction and in real-time, using 'beam-on' imaging. The lung tumour site group of the Atlantic MR-Linac consortium is working to generate a challenging MR-guided adaptive workflow for multi-institution treatment intensification trials in this patient group.

MRI-guided lung SBRT: Present and future developments

Stereotactic body radiotherapy (SBRT) is rapidly becoming an alternative to surgery for the treatment of early-stage non-small cell lung cancer patients. Lung SBRT is administered in a hypo-fractionated, conformal manner, delivering high doses to the target. To avoid normal-tissue toxicity, it is crucial to limit the exposure of nearby healthy organs-at-risk (OAR). Current image-guided radiotherapy strategies for lung SBRT are mostly based on X-ray imaging modalities. Although still in its infancy, magnetic resonance imaging (MRI) guidance for lung SBRT is not exposure-limited and MRI promises to improve crucial soft-tissue contrast. Looking beyond anatomical imaging, functional MRI is expected to inform treatment decisions and adaptations in the future. This review summarises and discusses how MRI could be advantageous to the different links of the radiotherapy treatment chain for lung SBRT: diagnosis and staging, tumour and OAR delineation, treatment planning, and inter- or intrafractional motion management. Special emphasis is placed on a new generation of hybrid MRI treatment devices and their potential for real-time adaptive radiotherapy.

Anatomical contouring variability in thoracic organs at risk

The purpose of this study was to determine whether contouring thoracic organs at risk was consistent among medical dosimetrists and to identify how trends in dosimetrist׳s education and experience affected contouring accuracy. Qualitative and quantitative methods were used to contextualize the raw data that were obtained. A total of 3 different computed tomography (CT) data sets were provided to medical dosimetrists (N = 13) across 5 different institutions. The medical dosimetrists were directed to contour the lungs, heart, spinal cord, and esophagus. The medical dosimetrists were instructed to contour in line with their institutional standards and were allowed to use any contouring tool or technique that they would traditionally use. The contours from each medical dosimetrist were evaluated against "gold standard" contours drawn and validated by 2 radiation oncology physicians. The dosimetrist-derived contours were evaluated against the gold standard using both a Dice coefficient method and a penalty-based metric scoring system. A short survey was also completed by each medical dosimetrist to evaluate their individual contouring experience. There was no significant variation in the contouring consistency of the lungs and spinal cord. Intradosimetrist contouring was consistent for those who contoured the esophagus and heart correctly; however, medical dosimetrists with a poor metric score showed erratic and inconsistent methods of contouring.

The role of delineation education programs for improving interobserver variability in target volume delineation in gastric cancer

OBJECTIVE

To assess whether delineation courses for radiation oncologists improve interobserver variability in target volume delineation for post-operative gastric cancer radiotherapy planning.

METHODS

29 radiation oncologists delineated target volumes in a gastric cancer patient. An experienced radiation oncologist lectured about delineation based on contouring atlas and delineation recommendations. After the course, the radiation oncologists, blinded to the previous delineation, provided delineation for the same patient.

RESULTS

The difference between delineated volumes and reference volumes for pre- and post-course clinical target volume (CTV) were 19.8% (-42.4 to 70.6%) and 12.3% (-12.0 to 27.3%) (p = 0.26), respectively. The planning target volume (PTV) differences pre- and post-course according to the reference volume were 20.5% (-40.7 to 93.7%) and 13.1% (-10.6 to 29.5%) (p = 0.30), respectively. The concordance volumes between the pre- and post-course CTVs and PTVs were 467.1 ± 89.2 vs 597.7 ± 54.6 cm³ (p < 0.001) and 738.6 ± 135.1 vs 893.2 ± 144.6 cm³ (p < 0.001), respectively. Minimum and maximum observer variations were seen at the cranial part and splenic hilus and at the caudal part of the CTV. The kappa indices compared with the reference contouring at pre- and post-course delineations were 0.68 and 0.82, respectively.

CONCLUSION

The delineation course improved interobserver variability for gastric cancer. However, impact of target volume changes on toxicity and local control should be evaluated for further studies. Advances in knowledge: This study demonstrated that a delineation course based on current recommendations helped physicians delineate smaller and more homogeneous target volumes. Better target volume delineation allows proper target volume irradiation and preventing unnecessary normal tissue irradiation.