1
|
Sahlsten J, Jaskari J, Wahid KA, Ahmed S, Glerean E, He R, Kann BH, Mäkitie A, Fuller CD, Naser MA, Kaski K. Application of simultaneous uncertainty quantification and segmentation for oropharyngeal cancer use-case with Bayesian deep learning. COMMUNICATIONS MEDICINE 2024; 4:110. [PMID: 38851837 PMCID: PMC11162474 DOI: 10.1038/s43856-024-00528-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 05/16/2024] [Indexed: 06/10/2024] Open
Abstract
BACKGROUND Radiotherapy is a core treatment modality for oropharyngeal cancer (OPC), where the primary gross tumor volume (GTVp) is manually segmented with high interobserver variability. This calls for reliable and trustworthy automated tools in clinician workflow. Therefore, accurate uncertainty quantification and its downstream utilization is critical. METHODS Here we propose uncertainty-aware deep learning for OPC GTVp segmentation, and illustrate the utility of uncertainty in multiple applications. We examine two Bayesian deep learning (BDL) models and eight uncertainty measures, and utilize a large multi-institute dataset of 292 PET/CT scans to systematically analyze our approach. RESULTS We show that our uncertainty-based approach accurately predicts the quality of the deep learning segmentation in 86.6% of cases, identifies low performance cases for semi-automated correction, and visualizes regions of the scans where the segmentations likely fail. CONCLUSIONS Our BDL-based analysis provides a first-step towards more widespread implementation of uncertainty quantification in OPC GTVp segmentation.
Collapse
Affiliation(s)
- Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Sara Ahmed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
- Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland.
| |
Collapse
|
2
|
Wahid KA, Sahin O, Kundu S, Lin D, Alanis A, Tehami S, Kamel S, Duke S, Sherer MV, Rasmussen M, Korreman S, Fuentes D, Cislo M, Nelms BE, Christodouleas JP, Murphy JD, Mohamed AS, He R, Naser MA, Gillespie EF, Fuller CD. Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation. JCO Clin Cancer Inform 2024; 8:e2300174. [PMID: 38870441 PMCID: PMC11214868 DOI: 10.1200/cci.23.00174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 01/08/2024] [Accepted: 04/03/2024] [Indexed: 06/15/2024] Open
Abstract
PURPOSE The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Onur Sahin
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Suprateek Kundu
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Diana Lin
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY
| | - Anthony Alanis
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Salik Tehami
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Serageldin Kamel
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Simon Duke
- Department of Radiation Oncology, Cambridge University Hospitals, Cambridge, United Kingdom
| | - Michael V. Sherer
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA
| | - Mathis Rasmussen
- Department of Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Michael Cislo
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY
| | | | - John P. Christodouleas
- Department of Radiation Oncology, The University of Pennsylvania Cancer Center, Philadelphia, PA
- Elekta, Atlanta, GA
| | - James D. Murphy
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA
| | - Abdallah S.R. Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Mohammed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Erin F. Gillespie
- Department of Radiation Oncology, University of Washington Fred Hutchinson Cancer Center, Seattle, WA
| | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| |
Collapse
|
3
|
Smits HJG, Raaijmakers CPJ, de Ridder M, Gouw ZAR, Doornaert PAH, Pameijer FA, Lodeweges JE, Ruiter LN, Kuijer KM, Schakel T, de Bree R, Dankbaar JW, Terhaard CHJ, Breimer GE, Willems SM, Philippens MEP. Improved delineation with diffusion weighted imaging for laryngeal and hypopharyngeal tumors validated with pathology. Radiother Oncol 2024; 194:110182. [PMID: 38403024 DOI: 10.1016/j.radonc.2024.110182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 02/16/2024] [Accepted: 02/18/2024] [Indexed: 02/27/2024]
Abstract
OBJECTIVE This study aims to determine the added value of a geometrically accurate diffusion-weighted (DW-) MRI sequence on the accuracy of gross tumor volume (GTV) delineations, using pathological tumor delineations as a ground truth. METHODS Sixteen patients with laryngeal or hypopharyngeal carcinoma were included. After total laryngectomy, the specimen was cut into slices. Photographs of these slices were stacked to create a 3D digital specimen reconstruction, which was registered to the in vivo imaging. The pathological tumor (tumorHE) was delineated on the specimen reconstruction. Six observers delineated all tumors twice: once with only anatomical MR imaging, and once (a few weeks later) when DW sequences were also provided. The majority voting delineation of session one (GTVMRI) and session two (GTVDW-MRI), as well as the clinical target volumes (CTVs), were compared to the tumorHE. RESULTS The mean tumorHE volume was 11.1 cm3, compared to a mean GTVMRI volume of 18.5 cm3 and a mean GTVDW-MRI volume of 15.7 cm3. The median sensitivity (tumor coverage) was comparable between sessions: 0.93 (range: 0.61-0.99) for the GTVMRI and 0.91 (range: 0.53-1.00) for the GTVDW-MRI. The CTV volume also decreased when DWI was available, with a mean CTVMR of 47.1 cm3 and a mean CTVDW-MRI of 41.4 cm3. Complete tumor coverage was achieved in 15 and 14 tumors, respectively. CONCLUSION GTV delineations based on anatomical MR imaging tend to overestimate the tumor volume. The availability of the geometrically accurate DW sequence reduces the GTV overestimation and thereby CTV volumes, while maintaining acceptable tumor coverage.
Collapse
Affiliation(s)
- Hilde J G Smits
- Department of Radiotherapy, University Medical Center Utrecht, Utrecht, the Netherlands.
| | | | - Mischa de Ridder
- Department of Radiotherapy, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Zeno A R Gouw
- Department of Radiotherapy, Netherlands Cancer Institute, Amsterdam, Netherlands
| | | | - Frank A Pameijer
- Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Joyce E Lodeweges
- Department of Radiotherapy, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Lilian N Ruiter
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Koen M Kuijer
- Department of Radiotherapy, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Tim Schakel
- Department of Radiotherapy, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Remco de Bree
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Jan W Dankbaar
- Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Chris H J Terhaard
- Department of Radiotherapy, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Gerben E Breimer
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Stefan M Willems
- Department of Pathology and Medical Biology, University Medical Center Groningen, Groningen, the Netherlands
| | | |
Collapse
|
4
|
McDonald BA, Dal Bello R, Fuller CD, Balermpas P. The Use of MR-Guided Radiation Therapy for Head and Neck Cancer and Recommended Reporting Guidance. Semin Radiat Oncol 2024; 34:69-83. [PMID: 38105096 DOI: 10.1016/j.semradonc.2023.10.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2023]
Abstract
Although magnetic resonance imaging (MRI) has become standard diagnostic workup for head and neck malignancies and is currently recommended by most radiological societies for pharyngeal and oral carcinomas, its utilization in radiotherapy has been heterogeneous during the last decades. However, few would argue that implementing MRI for annotation of target volumes and organs at risk provides several advantages, so that implementation of the modality for this purpose is widely accepted. Today, the term MR-guidance has received a much broader meaning, including MRI for adaptive treatments, MR-gating and tracking during radiotherapy application, MR-features as biomarkers and finally MR-only workflows. First studies on treatment of head and neck cancer on commercially available dedicated hybrid-platforms (MR-linacs), with distinct common features but also differences amongst them, have also been recently reported, as well as "biological adaptation" based on evaluation of early treatment response via functional MRI-sequences such as diffusion weighted ones. Yet, all of these approaches towards head and neck treatment remain at their infancy, especially when compared to other radiotherapy indications. Moreover, the lack of standardization for reporting MR-guided radiotherapy is a major obstacle both to further progress in the field and to conduct and compare clinical trials. Goals of this article is to present and explain all different aspects of MR-guidance for radiotherapy of head and neck cancer, summarize evidence, as well as possible advantages and challenges of the method and finally provide a comprehensive reporting guidance for use in clinical routine and trials.
Collapse
Affiliation(s)
- Brigid A McDonald
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Riccardo Dal Bello
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Panagiotis Balermpas
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland.
| |
Collapse
|
5
|
Gay SS, Cardenas CE, Nguyen C, Netherton TJ, Yu C, Zhao Y, Skett S, Patel T, Adjogatse D, Guerrero Urbano T, Naidoo K, Beadle BM, Yang J, Aggarwal A, Court LE. Fully-automated, CT-only GTV contouring for palliative head and neck radiotherapy. Sci Rep 2023; 13:21797. [PMID: 38066074 PMCID: PMC10709623 DOI: 10.1038/s41598-023-48944-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 12/01/2023] [Indexed: 12/18/2023] Open
Abstract
Planning for palliative radiotherapy is performed without the advantage of MR or PET imaging in many clinics. Here, we investigated CT-only GTV delineation for palliative treatment of head and neck cancer. Two multi-institutional datasets of palliative-intent treatment plans were retrospectively acquired: a set of 102 non-contrast-enhanced CTs and a set of 96 contrast-enhanced CTs. The nnU-Net auto-segmentation network was chosen for its strength in medical image segmentation, and five approaches separately trained: (1) heuristic-cropped, non-contrast images with a single GTV channel, (2) cropping around a manually-placed point in the tumor center for non-contrast images with a single GTV channel, (3) contrast-enhanced images with a single GTV channel, (4) contrast-enhanced images with separate primary and nodal GTV channels, and (5) contrast-enhanced images along with synthetic MR images with separate primary and nodal GTV channels. Median Dice similarity coefficient ranged from 0.6 to 0.7, surface Dice from 0.30 to 0.56, and 95th Hausdorff distance from 14.7 to 19.7 mm across the five approaches. Only surface Dice exhibited statistically-significant difference across these five approaches using a two-tailed Wilcoxon Rank-Sum test (p ≤ 0.05). Our CT-only results met or exceeded published values for head and neck GTV autocontouring using multi-modality images. However, significant edits would be necessary before clinical use in palliative radiotherapy.
Collapse
Affiliation(s)
- Skylar S Gay
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA.
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, USA.
| | - Carlos E Cardenas
- Department of Radiation Oncology, The University of Alabama at Birmingham, Birmingham, AL, USA
| | - Callistus Nguyen
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | - Tucker J Netherton
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | - Cenji Yu
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, USA
| | - Yao Zhao
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, USA
| | | | | | | | | | | | | | - Jinzhong Yang
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | | | - Laurence E Court
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| |
Collapse
|
6
|
Salzillo TC, Dresner MA, Way A, Wahid KA, McDonald BA, Mulder S, Naser MA, He R, Ding Y, Yoder A, Ahmed S, Corrigan KL, Manzar GS, Andring L, Pinnix C, Stafford RJ, Mohamed ASR, Christodouleas J, Wang J, Fuller CD. Development and implementation of optimized endogenous contrast sequences for delineation in adaptive radiotherapy on a 1.5T MR-linear-accelerator: a prospective R-IDEAL stage 0-2a quantitative/qualitative evaluation of in vivo site-specific quality-assurance using a 3D T2 fat-suppressed platform for head and neck cancer. J Med Imaging (Bellingham) 2023; 10:065501. [PMID: 37937259 PMCID: PMC10627232 DOI: 10.1117/1.jmi.10.6.065501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 10/06/2023] [Accepted: 10/16/2023] [Indexed: 11/09/2023] Open
Abstract
Purpose To improve segmentation accuracy in head and neck cancer (HNC) radiotherapy treatment planning for the 1.5T hybrid magnetic resonance imaging/linear accelerator (MR-Linac), three-dimensional (3D), T2-weighted, fat-suppressed magnetic resonance imaging sequences were developed and optimized. Approach After initial testing, spectral attenuated inversion recovery (SPAIR) was chosen as the fat suppression technique. Five candidate SPAIR sequences and a nonsuppressed, T2-weighted sequence were acquired for five HNC patients using a 1.5T MR-Linac. MR physicists identified persistent artifacts in two of the SPAIR sequences, so the remaining three SPAIR sequences were further analyzed. The gross primary tumor volume, metastatic lymph nodes, parotid glands, and pterygoid muscles were delineated using five segmentors. A robust image quality analysis platform was developed to objectively score the SPAIR sequences on the basis of qualitative and quantitative metrics. Results Sequences were analyzed for the signal-to-noise ratio and the contrast-to-noise ratio and compared with fat and muscle, conspicuity, pairwise distance metrics, and segmentor assessments. In this analysis, the nonsuppressed sequence was inferior to each of the SPAIR sequences for the primary tumor, lymph nodes, and parotid glands, but it was superior for the pterygoid muscles. The SPAIR sequence that received the highest combined score among the analysis categories was recommended to Unity MR-Linac users for HNC radiotherapy treatment planning. Conclusions Our study led to two developments: an optimized, 3D, T2-weighted, fat-suppressed sequence that can be disseminated to Unity MR-Linac users and a robust image quality analysis pathway that can be used to objectively score SPAIR sequences and can be customized and generalized to any image quality optimization protocol. Improved segmentation accuracy with the proposed SPAIR sequence will potentially lead to improved treatment outcomes and reduced toxicity for patients by maximizing the target coverage and minimizing the radiation exposure of organs at risk.
Collapse
Affiliation(s)
- Joint Head and Neck Radiotherapy-MRI Development Cooperative
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
- Philips Healthcare, Cleveland, Ohio, United States
- MD Anderson Cancer Center, Radiation Physics, Houston, Texas, United States
- MD Anderson Cancer Center, Imaging Physics, Houston, Texas, United States
- Elekta AB, Stockholm, Sweden
| | - Travis C. Salzillo
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | | | - Ashley Way
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Kareem A. Wahid
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Brigid A. McDonald
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Sam Mulder
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Mohamed A. Naser
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Renjie He
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Yao Ding
- MD Anderson Cancer Center, Radiation Physics, Houston, Texas, United States
| | - Alison Yoder
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Sara Ahmed
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Kelsey L. Corrigan
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Gohar S. Manzar
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Lauren Andring
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - Chelsea Pinnix
- MD Anderson Cancer Center, Radiation Oncology, Houston, Texas, United States
| | - R. Jason Stafford
- MD Anderson Cancer Center, Imaging Physics, Houston, Texas, United States
| | | | | | - Jihong Wang
- MD Anderson Cancer Center, Radiation Physics, Houston, Texas, United States
| | | |
Collapse
|
7
|
Mikalsen SG, Skjøtskift T, Flote VG, Hämäläinen NP, Heydari M, Rydén-Eilertsen K. Extensive clinical testing of Deep Learning Segmentation models for thorax and breast cancer radiotherapy planning. Acta Oncol 2023; 62:1184-1193. [PMID: 37883678 DOI: 10.1080/0284186x.2023.2270152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 10/04/2023] [Indexed: 10/28/2023]
Abstract
BACKGROUND The performance of deep learning segmentation (DLS) models for automatic organ extraction from CT images in the thorax and breast regions was investigated. Furthermore, the readiness and feasibility of integrating DLS into clinical practice were addressed by measuring the potential time savings and dosimetric impact. MATERIAL AND METHODS Thirty patients referred to radiotherapy for breast cancer were prospectively included. A total of 23 clinically relevant left- and right-sided organs were contoured manually on CT images according to ESTRO guidelines. Next, auto-segmentation was executed, and the geometric agreement between the auto-segmented and manually contoured organs was qualitatively assessed applying a scale in the range [0-not acceptable, 3-no corrections]. A quantitative validation was carried out by calculating Dice coefficients (DSC) and the 95% percentile of Hausdorff distances (HD95). The dosimetric impact of optimizing the treatment plans on the uncorrected DLS contours, was investigated from a dose coverage analysis using DVH values of the manually delineated contours as references. RESULTS The qualitative analysis showed that 93% of the DLS generated OAR contours did not need corrections, except for the heart where 67% of the contours needed corrections. The majority of DLS generated CTVs needed corrections, whereas a minority were deemed not acceptable. Still, using the DLS-model for CTV and heart delineation is on average 14 minutes faster. An average DSC=0.91 and H95=9.8 mm were found for the left and right breasts, respectively. Likewise, and average DSC in the range [0.66, 0.76]mm and HD95 in the range [7.04, 12.05]mm were found for the lymph nodes. CONCLUSION The validation showed that the DLS generated OAR contours can be used clinically. Corrections were required to most of the DLS generated CTVs, and therefore warrants more attention before possibly implementing the DLS models clinically.
Collapse
Affiliation(s)
| | | | | | | | - Mojgan Heydari
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
| | | |
Collapse
|
8
|
Clasen K, Nachbar M, Gatidis S, Zips D, Thorwarth D, Welz S. Impact of MRI on target volume definition in head and neck cancer patients. Radiat Oncol 2023; 18:148. [PMID: 37674171 PMCID: PMC10483850 DOI: 10.1186/s13014-023-02326-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 08/03/2023] [Indexed: 09/08/2023] Open
Abstract
BACKGROUND Target volume definition for curative radiochemotherapy in head and neck cancer is crucial since the predominant recurrence pattern is local. Additional diagnostic imaging like MRI is increasingly used, yet it is usually hampered by different patient positioning compared to radiotherapy. In this study, we investigated the impact of diagnostic MRI in treatment position for target volume delineation. METHODS We prospectively analyzed patients who were suitable and agreed to undergo an MRI in treatment position with immobilization devices prior to radiotherapy planning from 2017 to 2019. Target volume delineation for the primary tumor was first performed using all available information except for the MRI and subsequently with additional consideration of the co-registered MRI. The derived volumes were compared by subjective visual judgment and by quantitative mathematical methods. RESULTS Sixteen patients were included and underwent the planning CT, MRI and subsequent definitive radiochemotherapy. In 69% of the patients, there were visually relevant changes to the gross tumor volume (GTV) by use of the MRI. In 44%, the GTV_MRI would not have been covered completely by the planning target volume (PTV) of the CT-only contour. Yet, median Hausdorff und DSI values did not reflect these differences. The 3-year local control rate was 94%. CONCLUSIONS Adding a diagnostic MRI in RT treatment position is feasible and results in relevant changes in target volumes in the majority of patients.
Collapse
Affiliation(s)
- Kerstin Clasen
- Department of Radiation Oncology, University Hospital Tübingen, University of Tübingen, Tübingen, Germany.
| | - Marcel Nachbar
- Section for Biomedical Physics, Department of Radiation Oncology, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
| | - Sergios Gatidis
- Department of Radiology, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
| | - Daniel Zips
- Department of Radiation Oncology, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
| | - Daniela Thorwarth
- Section for Biomedical Physics, Department of Radiation Oncology, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
| | - Stefan Welz
- Department of Radiation Oncology, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
- Department of Radiation Oncology, Marienhospital, Stuttgart, Germany
| |
Collapse
|
9
|
Wahid KA, Sahin O, Kundu S, Lin D, Alanis A, Tehami S, Kamel S, Duke S, Sherer MV, Rasmussen M, Korreman S, Fuentes D, Cislo M, Nelms BE, Christodouleas JP, Murphy JD, Mohamed ASR, He R, Naser MA, Gillespie EF, Fuller CD. Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.08.30.23294786. [PMID: 37693394 PMCID: PMC10491357 DOI: 10.1101/2023.08.30.23294786] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/12/2023]
Abstract
BACKGROUND Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance. METHODS Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure. RESULTS After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals. CONCLUSION Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Onur Sahin
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Suprateek Kundu
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Diana Lin
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY
| | - Anthony Alanis
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Salik Tehami
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Serageldin Kamel
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Simon Duke
- Department of Radiation Oncology, Cambridge University Hospitals, Cambridge, UK
| | - Michael V. Sherer
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, USA
| | | | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Denmark
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael Cislo
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY
| | | | - John P. Christodouleas
- Department of Radiation Oncology, The University of Pennsylvania Cancer Center, Philadelphia, PA, USA
- Elekta, Atlanta, GA, USA
| | - James D. Murphy
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, USA
| | - Abdallah S. R. Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Mohammed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
10
|
Wahid KA, Lin D, Sahin O, Cislo M, Nelms BE, He R, Naser MA, Duke S, Sherer MV, Christodouleas JP, Mohamed ASR, Murphy JD, Fuller CD, Gillespie EF. Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites. Sci Data 2023; 10:161. [PMID: 36949088 PMCID: PMC10033824 DOI: 10.1038/s41597-023-02062-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 03/10/2023] [Indexed: 03/24/2023] Open
Abstract
Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.
Collapse
Affiliation(s)
- Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Diana Lin
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Onur Sahin
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael Cislo
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Mohammed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Simon Duke
- Department of Radiation Oncology, Cambridge University Hospitals, Cambridge, UK
| | - Michael V Sherer
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, USA
| | - John P Christodouleas
- Department of Radiation Oncology, The University of Pennsylvania Cancer Center, Philadelphia, PA, USA
- Elekta, Atlanta, GA, USA
| | - Abdallah S R Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - James D Murphy
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, USA
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA.
| | - Erin F Gillespie
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
- Fred Hutchinson Cancer Center, Seattle, WA, USA.
| |
Collapse
|
11
|
de Ridder M, Rijken JA, Smits HJG, Smid EJ, Doornaert PAH, de Bree R. Oncological outcome of vocal cord-only radiotherapy for cT1-T2 glottic laryngeal squamous cell carcinoma. Eur Arch Otorhinolaryngol 2023; 280:3345-3352. [PMID: 36881167 DOI: 10.1007/s00405-023-07904-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 02/27/2023] [Indexed: 03/08/2023]
Abstract
PURPOSE Early-stage glottic cancer can be treated with radiotherapy only. Modern radiotherapy solutions allow for individualized dose distributions, hypofractionation and sparing of organs at risk. The target volume used to be the entire voice box. This series describe the oncological outcome and toxicity of individualized vocal cord-only hypofractionated radiotherapy for early stage (cT1a-T2 N0). METHODS Retrospective cohort study with patients treated in a single center between 2014 and 2020. RESULTS A total of 93 patients were included. Local control rate was 100% for cT1a, 97% for cT1b and 77% for cT2. Risk factor for local recurrence was smoking during radiotherapy. Laryngectomy-free survival was 90% at 5 years. Grade III or higher late toxicity was 3.7%. CONCLUSION Vocal cord-only hypofractionated radiotherapy appears to be oncologically safe in early-stage glottic cancer. Modern, image-guided radiotherapy led to comparable results as historical series with very limited late toxicity.
Collapse
Affiliation(s)
- Mischa de Ridder
- Department of Radiotherapy, University Medical Center Utrecht, Heidelberglaan 100, Postbox 85500, 3508 GA, Utrecht, The Netherlands.
| | - Johannes A Rijken
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Hilde J G Smits
- Department of Radiotherapy, University Medical Center Utrecht, Heidelberglaan 100, Postbox 85500, 3508 GA, Utrecht, The Netherlands
| | - Ernst J Smid
- Department of Radiotherapy, University Medical Center Utrecht, Heidelberglaan 100, Postbox 85500, 3508 GA, Utrecht, The Netherlands
| | - Patricia A H Doornaert
- Department of Radiotherapy, University Medical Center Utrecht, Heidelberglaan 100, Postbox 85500, 3508 GA, Utrecht, The Netherlands
| | - Remco de Bree
- Department of Head and Neck Surgical Oncology, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|
12
|
Sahlsten J, Jaskari J, Wahid KA, Ahmed S, Glerean E, He R, Kann BH, Mäkitie A, Fuller CD, Naser MA, Kaski K. Application of simultaneous uncertainty quantification for image segmentation with probabilistic deep learning: Performance benchmarking of oropharyngeal cancer target delineation as a use-case. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.02.20.23286188. [PMID: 36865296 PMCID: PMC9980236 DOI: 10.1101/2023.02.20.23286188] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/26/2023]
Abstract
Background Oropharyngeal cancer (OPC) is a widespread disease, with radiotherapy being a core treatment modality. Manual segmentation of the primary gross tumor volume (GTVp) is currently employed for OPC radiotherapy planning, but is subject to significant interobserver variability. Deep learning (DL) approaches have shown promise in automating GTVp segmentation, but comparative (auto)confidence metrics of these models predictions has not been well-explored. Quantifying instance-specific DL model uncertainty is crucial to improving clinician trust and facilitating broad clinical implementation. Therefore, in this study, probabilistic DL models for GTVp auto-segmentation were developed using large-scale PET/CT datasets, and various uncertainty auto-estimation methods were systematically investigated and benchmarked. Methods We utilized the publicly available 2021 HECKTOR Challenge training dataset with 224 co-registered PET/CT scans of OPC patients with corresponding GTVp segmentations as a development set. A separate set of 67 co-registered PET/CT scans of OPC patients with corresponding GTVp segmentations was used for external validation. Two approximate Bayesian deep learning methods, the MC Dropout Ensemble and Deep Ensemble, both with five submodels, were evaluated for GTVp segmentation and uncertainty performance. The segmentation performance was evaluated using the volumetric Dice similarity coefficient (DSC), mean surface distance (MSD), and Hausdorff distance at 95% (95HD). The uncertainty was evaluated using four measures from literature: coefficient of variation (CV), structure expected entropy, structure predictive entropy, and structure mutual information, and additionally with our novel Dice-risk measure. The utility of uncertainty information was evaluated with the accuracy of uncertainty-based segmentation performance prediction using the Accuracy vs Uncertainty (AvU) metric, and by examining the linear correlation between uncertainty estimates and DSC. In addition, batch-based and instance-based referral processes were examined, where the patients with high uncertainty were rejected from the set. In the batch referral process, the area under the referral curve with DSC (R-DSC AUC) was used for evaluation, whereas in the instance referral process, the DSC at various uncertainty thresholds were examined. Results Both models behaved similarly in terms of the segmentation performance and uncertainty estimation. Specifically, the MC Dropout Ensemble had 0.776 DSC, 1.703 mm MSD, and 5.385 mm 95HD. The Deep Ensemble had 0.767 DSC, 1.717 mm MSD, and 5.477 mm 95HD. The uncertainty measure with the highest DSC correlation was structure predictive entropy with correlation coefficients of 0.699 and 0.692 for the MC Dropout Ensemble and the Deep Ensemble, respectively. The highest AvU value was 0.866 for both models. The best performing uncertainty measure for both models was the CV which had R-DSC AUC of 0.783 and 0.782 for the MC Dropout Ensemble and Deep Ensemble, respectively. With referring patients based on uncertainty thresholds from 0.85 validation DSC for all uncertainty measures, on average the DSC improved from the full dataset by 4.7% and 5.0% while referring 21.8% and 22% patients for MC Dropout Ensemble and Deep Ensemble, respectively. Conclusion We found that many of the investigated methods provide overall similar but distinct utility in terms of predicting segmentation quality and referral performance. These findings are a critical first-step towards more widespread implementation of uncertainty quantification in OPC GTVp segmentation.
Collapse
Affiliation(s)
- Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Sara Ahmed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| |
Collapse
|
13
|
Baroudi H, Brock KK, Cao W, Chen X, Chung C, Court LE, El Basha MD, Farhat M, Gay S, Gronberg MP, Gupta AC, Hernandez S, Huang K, Jaffray DA, Lim R, Marquez B, Nealon K, Netherton TJ, Nguyen CM, Reber B, Rhee DJ, Salazar RM, Shanker MD, Sjogreen C, Woodland M, Yang J, Yu C, Zhao Y. Automated Contouring and Planning in Radiation Therapy: What Is 'Clinically Acceptable'? Diagnostics (Basel) 2023; 13:diagnostics13040667. [PMID: 36832155 PMCID: PMC9955359 DOI: 10.3390/diagnostics13040667] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2022] [Revised: 01/21/2023] [Accepted: 01/30/2023] [Indexed: 02/12/2023] Open
Abstract
Developers and users of artificial-intelligence-based tools for automatic contouring and treatment planning in radiotherapy are expected to assess clinical acceptability of these tools. However, what is 'clinical acceptability'? Quantitative and qualitative approaches have been used to assess this ill-defined concept, all of which have advantages and disadvantages or limitations. The approach chosen may depend on the goal of the study as well as on available resources. In this paper, we discuss various aspects of 'clinical acceptability' and how they can move us toward a standard for defining clinical acceptability of new autocontouring and planning tools.
Collapse
Affiliation(s)
- Hana Baroudi
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Kristy K. Brock
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Wenhua Cao
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Xinru Chen
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Caroline Chung
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Laurence E. Court
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Correspondence:
| | - Mohammad D. El Basha
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Maguy Farhat
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Skylar Gay
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Mary P. Gronberg
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Aashish Chandra Gupta
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Soleil Hernandez
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Kai Huang
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - David A. Jaffray
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Rebecca Lim
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Barbara Marquez
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Kelly Nealon
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Callistus M. Nguyen
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Brandon Reber
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Dong Joo Rhee
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Ramon M. Salazar
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Mihir D. Shanker
- The University of Queensland, Saint Lucia 4072, Australia
- The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Carlos Sjogreen
- Department of Physics, University of Houston, Houston, TX 77004, USA
| | - McKell Woodland
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Computer Science, Rice University, Houston, TX 77005, USA
| | - Jinzhong Yang
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Cenji Yu
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Yao Zhao
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| |
Collapse
|
14
|
Lin D, Wahid KA, Nelms BE, He R, Naser MA, Duke S, Sherer MV, Christodouleas JP, Mohamed ASR, Cislo M, Murphy JD, Fuller CD, Gillespie EF. E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation. J Med Imaging (Bellingham) 2023; 10:S11903. [PMID: 36761036 PMCID: PMC9907021 DOI: 10.1117/1.jmi.10.s1.s11903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 01/02/2023] [Indexed: 02/11/2023] Open
Abstract
Purpose Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is a crowdsourced challenge engaging radiation oncologists across various expertise levels in segmentation. An obstacle to artificial intelligence (AI) development is the paucity of multiexpert datasets; consequently, we sought to characterize whether aggregate segmentations generated from multiple nonexperts could meet or exceed recognized expert agreement. Approach Participants who contoured ≥ 1 region of interest (ROI) for the breast, sarcoma, head and neck (H&N), gynecologic (GYN), or gastrointestinal (GI) cases were identified as a nonexpert or recognized expert. Cohort-specific ROIs were combined into single simultaneous truth and performance level estimation (STAPLE) consensus segmentations.STAPLE nonexpert ROIs were evaluated againstSTAPLE expert contours using Dice similarity coefficient (DSC). The expert interobserver DSC (IODSC expert ) was calculated as an acceptability threshold betweenSTAPLE nonexpert andSTAPLE expert . To determine the number of nonexperts required to match theIODSC expert for each ROI, a single consensus contour was generated using variable numbers of nonexperts and then compared to theIODSC expert . Results For all cases, the DSC values forSTAPLE nonexpert versusSTAPLE expert were higher than comparator expertIODSC expert for most ROIs. The minimum number of nonexpert segmentations needed for a consensus ROI to achieveIODSC expert acceptability criteria ranged between 2 and 4 for breast, 3 and 5 for sarcoma, 3 and 5 for H&N, 3 and 5 for GYN, and 3 for GI. Conclusions Multiple nonexpert-generated consensus ROIs met or exceeded expert-derived acceptability thresholds. Five nonexperts could potentially generate consensus segmentations for most ROIs with performance approximating experts, suggesting nonexpert segmentations as feasible cost-effective AI inputs.
Collapse
Affiliation(s)
- Diana Lin
- Memorial Sloan Kettering Cancer Center, Department of Radiation Oncology, New York, New York, United States
| | - Kareem A. Wahid
- The University of Texas MD Anderson Cancer Center, Department of Radiation Oncology, Houston, Texas, United States
| | | | - Renjie He
- The University of Texas MD Anderson Cancer Center, Department of Radiation Oncology, Houston, Texas, United States
| | - Mohammed A. Naser
- The University of Texas MD Anderson Cancer Center, Department of Radiation Oncology, Houston, Texas, United States
| | - Simon Duke
- Cambridge University Hospitals, Department of Radiation Oncology, Cambridge, United Kingdom
| | - Michael V. Sherer
- University of California San Diego, Department of Radiation Medicine and Applied Sciences, La Jolla, California, United States
| | - John P. Christodouleas
- The University of Pennsylvania Cancer Center, Department of Radiation Oncology, Philadelphia, Pennsylvania, United States
- Elekta AB, Stockholm, Sweden
| | - Abdallah S. R. Mohamed
- The University of Texas MD Anderson Cancer Center, Department of Radiation Oncology, Houston, Texas, United States
| | - Michael Cislo
- Memorial Sloan Kettering Cancer Center, Department of Radiation Oncology, New York, New York, United States
| | - James D. Murphy
- University of California San Diego, Department of Radiation Medicine and Applied Sciences, La Jolla, California, United States
| | - Clifton D. Fuller
- The University of Texas MD Anderson Cancer Center, Department of Radiation Oncology, Houston, Texas, United States
| | - Erin F. Gillespie
- Memorial Sloan Kettering Cancer Center, Department of Radiation Oncology, New York, New York, United States
- University of Washington Fred Hutchinson Cancer Center, Department of Radiation Oncology, Seattle, Washington, United States
| |
Collapse
|
15
|
Wei Z, Ren J, Korreman SS, Nijkamp J. Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy. Phys Imaging Radiat Oncol 2022; 25:100408. [PMID: 36655215 PMCID: PMC9841279 DOI: 10.1016/j.phro.2022.12.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 12/19/2022] [Accepted: 12/22/2022] [Indexed: 12/26/2022] Open
Abstract
Background and purpose With deep-learning, gross tumour volume (GTV) auto-segmentation has substantially been improved, but still substantial manual corrections are needed. With interactive deep-learning (iDL), manual corrections can be used to update a deep-learning tool while delineating, minimising the input to achieve acceptable segmentations. We present an iDL tool for GTV segmentation that took annotated slices as input and simulated its performance on a head and neck cancer (HNC) dataset. Materials and methods Multimodal image data of 204 HNC patients with clinical tumour and lymph node GTV delineations were used. A baseline convolutional neural network (CNN) was trained (n = 107 training, n = 22 validation) and tested (n = 24). Subsequently, user input was simulated on initial test set by replacing one or more of predicted slices with ground truth delineation, followed by re-training the CNN. The objective was to optimise re-training parameters and simulate slice selection scenarios while limiting annotations to maximally-five slices. The remaining 51 patients were used as an independent test set, where Dice similarity coefficient (DSC), mean surface distance (MSD), and 95% Hausdorff distance (HD95%) were assessed at baseline and after every update. Results Median segmentation accuracy at baseline was DSC = 0.65, MSD = 4.3 mm, HD95% = 17.5 mm. Updating CNN using three slices equally sampled from the craniocaudal axis of the GTV in the first round, followed by two rounds of annotating one extra slice, gave the best results. The accuracy improved to DSC = 0.82, MSD = 1.6 mm, HD95% = 4.8 mm. Every CNN update took 30 s. Conclusions The presented iDL tool achieved substantial segmentation improvement with only five annotated slices.
Collapse
Affiliation(s)
- Zixiang Wei
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Jintao Ren
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Stine Sofia Korreman
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark,Department of Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Jasper Nijkamp
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark,Corresponding author at: Palle Juul Jensensboulevard 25, 8200 Aarhus, Denmark.
| |
Collapse
|
16
|
Target Definition in MR-Guided Adaptive Radiotherapy for Head and Neck Cancer. Cancers (Basel) 2022; 14:cancers14123027. [PMID: 35740691 PMCID: PMC9220977 DOI: 10.3390/cancers14123027] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Revised: 06/14/2022] [Accepted: 06/14/2022] [Indexed: 02/01/2023] Open
Abstract
Simple Summary Adaptive radiotherapy for head and neck cancer has become more routine due to an increase in imaging quality and improvement in radiation techniques. With the availability of faster adaptive workflows, it is possible to adapt more easily to (daily) changes. MRI offers besides great anatomical imaging, also functional information about the tumor and surrounding tissue. The aim of this review is to provide current state of evidence about target definition on MRI for adaptive strategies in the treatment of head and neck cancer. Abstract In recent years, MRI-guided radiotherapy (MRgRT) has taken an increasingly important position in image-guided radiotherapy (IGRT). Magnetic resonance imaging (MRI) offers superior soft tissue contrast in anatomical imaging compared to computed tomography (CT), but also provides functional and dynamic information with selected sequences. Due to these benefits, in current clinical practice, MRI is already used for target delineation and response assessment in patients with head and neck squamous cell carcinoma (HNSCC). Because of the close proximity of target areas and radiosensitive organs at risk (OARs) during HNSCC treatment, MRgRT could provide a more accurate treatment in which OARs receive less radiation dose. With the introduction of several new radiotherapy techniques (i.e., adaptive MRgRT, proton therapy, adaptive cone beam computed tomography (CBCT) RT, (daily) adaptive radiotherapy ensures radiation dose is accurately delivered to the target areas. With the integration of a daily adaptive workflow, interfraction changes have become visible, which allows regular and fast adaptation of target areas. In proton therapy, adaptation is even more important in order to obtain high quality dosimetry, due to its susceptibility for density differences in relation to the range uncertainty of the protons. The question is which adaptations during radiotherapy treatment are oncology safe and at the same time provide better sparing of OARs. For an optimal use of all these new tools there is an urgent need for an update of the target definitions in case of adaptive treatment for HNSCC. This review will provide current state of evidence regarding adaptive target definition using MR during radiotherapy for HNSCC. Additionally, future perspectives for adaptive MR-guided radiotherapy will be discussed.
Collapse
|