1
|
Marquez B, Wooten ZT, Salazar RM, Peterson CB, Fuentes DT, Whitaker TJ, Jhingran A, Pollard-Larkin J, Prajapati S, Beadle B, Cardenas CE, Netherton TJ, Court LE. Analyzing the Relationship between Dose and Geometric Agreement Metrics for Auto-Contouring in Head and Neck Normal Tissues. Diagnostics (Basel) 2024; 14:1632. [PMID: 39125508 PMCID: PMC11311423 DOI: 10.3390/diagnostics14151632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 07/18/2024] [Accepted: 07/19/2024] [Indexed: 08/12/2024] Open
Abstract
This study aimed to determine the relationship between geometric and dosimetric agreement metrics in head and neck (H&N) cancer radiotherapy plans. A total 287 plans were retrospectively analyzed, comparing auto-contoured and clinically used contours using a Dice similarity coefficient (DSC), surface DSC (sDSC), and Hausdorff distance (HD). Organs-at-risk (OARs) with ≥200 cGy dose differences from the clinical contour in terms of Dmax (D0.01cc) and Dmean were further examined against proximity to the planning target volume (PTV). A secondary set of 91 plans from multiple institutions validated these findings. For 4995 contour pairs across 19 OARs, 90% had a DSC, sDSC, and HD of at least 0.75, 0.86, and less than 7.65 mm, respectively. Dosimetrically, the absolute difference between the two contour sets was <200 cGy for 95% of OARs in terms of Dmax and 96% in terms of Dmean. In total, 97% of OARs exhibiting significant dose differences between the clinically edited contour and auto-contour were within 2.5 cm PTV regardless of geometric agreement. There was an approximately linear trend between geometric agreement and identifying at least 200 cGy dose differences, with higher geometric agreement corresponding to a lower fraction of cases being identified. Analysis of the secondary dataset validated these findings. Geometric indices are approximate indicators of contour quality and identify contours exhibiting significant dosimetric discordance. For a small subset of OARs within 2.5 cm of the PTV, geometric agreement metrics can be misleading in terms of contour quality.
Collapse
Affiliation(s)
- Barbara Marquez
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | | | - Ramon M. Salazar
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Christine B. Peterson
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - David T. Fuentes
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - T. J. Whitaker
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
| | - Anuja Jhingran
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Julianne Pollard-Larkin
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Surendra Prajapati
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Beth Beadle
- Department of Radiation Oncology–Radiation Therapy, Stanford University, Stanford, CA 94305, USA;
| | - Carlos E. Cardenas
- Department of Radiation Oncology, The University of Alabama, Birmingham, AL 35294, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
| | - Laurence E. Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (R.M.S.); (T.J.W.); (J.P.-L.); (L.E.C.)
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| |
Collapse
|
2
|
De Kerf G, Claessens M, Raouassi F, Mercier C, Stas D, Ost P, Dirix P, Verellen D. A geometry and dose-volume based performance monitoring of artificial intelligence models in radiotherapy treatment planning for prostate cancer. Phys Imaging Radiat Oncol 2023; 28:100494. [PMID: 37809056 PMCID: PMC10550805 DOI: 10.1016/j.phro.2023.100494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 09/20/2023] [Accepted: 09/20/2023] [Indexed: 10/10/2023] Open
Abstract
Background and Purpose Clinical Artificial Intelligence (AI) implementations lack ground-truth when applied on real-world data. This study investigated how combined geometrical and dose-volume metrics can be used as performance monitoring tools to detect clinically relevant candidates for model retraining. Materials and Methods Fifty patients were analyzed for both AI-segmentation and planning. For AI-segmentation, geometrical (Standard Surface Dice 3 mm and Local Surface Dice 3 mm) and dose-volume based parameters were calculated for two organs (bladder and anorectum) to compare AI output against the clinically corrected structure. A Local Surface Dice was introduced to detect geometrical changes in the vicinity of the target volumes, while an Absolute Dose Difference (ADD) evaluation increased focus on dose-volume related changes. AI-planning performance was evaluated using clinical goal analysis in combination with volume and target overlap metrics. Results The Local Surface Dice reported equal or lower values compared to the Standard Surface Dice (anorectum: (0.93 ± 0.11) vs (0.98 ± 0.04); bladder: (0.97 ± 0.06) vs (0.98 ± 0.04)). The ADD metric showed a difference of (0.9 ± 0.8)Gy for the anorectum D 1 cm 3 . The bladder D 5cm 3 reported a difference of (0.7 ± 1.5)Gy. Mandatory clinical goals were fulfilled in 90 % of the DLP plans. Conclusions Combining dose-volume and geometrical metrics allowed detection of clinically relevant changes, applied to both auto-segmentation and auto-planning output and the Local Surface Dice was more sensitive to local changes compared to the Standard Surface Dice. This monitoring is able to evaluate AI behavior in clinical practice and allows candidate selection for active learning.
Collapse
Affiliation(s)
- Geert De Kerf
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
| | - Michaël Claessens
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Fadoua Raouassi
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
| | - Carole Mercier
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Daan Stas
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium
| | - Piet Ost
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Piet Dirix
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Dirk Verellen
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| |
Collapse
|
3
|
Heilemann G, Buschmann M, Lechner W, Dick V, Eckert F, Heilmann M, Herrmann H, Moll M, Knoth J, Konrad S, Simek IM, Thiele C, Zaharie A, Georg D, Widder J, Trnkova P. Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy. Phys Imaging Radiat Oncol 2023; 28:100515. [PMID: 38111502 PMCID: PMC10726238 DOI: 10.1016/j.phro.2023.100515] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 11/09/2023] [Accepted: 11/09/2023] [Indexed: 12/20/2023] Open
Abstract
Background and purpose Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation. Materials and Methods One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction. Results The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant. Conclusion A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.
Collapse
Affiliation(s)
- Gerd Heilemann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Martin Buschmann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Wolfgang Lechner
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Vincent Dick
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Franziska Eckert
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Martin Heilmann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Harald Herrmann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Matthias Moll
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Johannes Knoth
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Stefan Konrad
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Inga-Malin Simek
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Christopher Thiele
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Alexandru Zaharie
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Dietmar Georg
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Joachim Widder
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Petra Trnkova
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| |
Collapse
|
4
|
Vaassen F, Zegers CML, Hofstede D, Wubbels M, Beurskens H, Verheesen L, Canters R, Looney P, Battye M, Gooding MJ, Compter I, Eekers DBP, van Elmpt W. Geometric and dosimetric analysis of CT- and MR-based automatic contouring for the EPTN contouring atlas in neuro-oncology. Phys Med 2023; 114:103156. [PMID: 37813050 DOI: 10.1016/j.ejmp.2023.103156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 09/21/2023] [Accepted: 09/26/2023] [Indexed: 10/11/2023] Open
Abstract
PURPOSE Atlas-based and deep-learning contouring (DLC) are methods for automatic segmentation of organs-at-risk (OARs). The European Particle Therapy Network (EPTN) published a consensus-based atlas for delineation of OARs in neuro-oncology. In this study, geometric and dosimetric evaluation of automatically-segmented neuro-oncological OARs was performed using CT- and MR-models following the EPTN-contouring atlas. METHODS Image and contouring data from 76 neuro-oncological patients were included. Two atlas-based models (CT-atlas and MR-atlas) and one DLC-model (MR-DLC) were created. Manual contours on registered CT-MR-images were used as ground-truth. Results were analyzed in terms of geometrical (volumetric Dice similarity coefficient (vDSC), surface DSC (sDSC), added path length (APL), and mean slice-wise Hausdorff distance (MSHD)) and dosimetrical accuracy. Distance-to-tumor analysis was performed to analyze to which extent the location of the OAR relative to planning target volume (PTV) has dosimetric impact, using Wilcoxon rank-sum tests. RESULTS CT-atlas outperformed MR-atlas for 22/26 OARs. MR-DLC outperformed MR-atlas for all OARs. Highest median (95 %CI) vDSC and sDSC were found for the brainstem in MR-DLC: 0.92 (0.88-0.95) and 0.84 (0.77-0.89) respectively, as well as lowest MSHD: 0.27 (0.22-0.39)cm. Median dose differences (ΔD) were within ± 1 Gy for 24/26(92 %) OARs for all three models. Distance-to-tumor showed a significant correlation for ΔDmax,0.03cc-parameters when splitting the data in ≤ 4 cm and > 4 cm OAR-distance (p < 0.001). CONCLUSION MR-based DLC and CT-based atlas-contouring enable high-quality segmentation. It was shown that a combination of both CT- and MR-autocontouring models results in the best quality.
Collapse
Affiliation(s)
- Femke Vaassen
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands.
| | - Catharina M L Zegers
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - David Hofstede
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - Mart Wubbels
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - Hilde Beurskens
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - Lindsey Verheesen
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - Richard Canters
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | | | | | | | - Inge Compter
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - Daniëlle B P Eekers
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| | - Wouter van Elmpt
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre(+), Maastricht, the Netherlands
| |
Collapse
|
5
|
Mövik L, Bäck A, Pettersson N. Impact of delineation errors on the estimated organ at risk dose and of dose errors on the normal tissue complication probability model. Med Phys 2023; 50:1879-1892. [PMID: 36693127 DOI: 10.1002/mp.16235] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 12/04/2022] [Accepted: 01/01/2023] [Indexed: 01/25/2023] Open
Abstract
BACKGROUND Normal tissue complication probability (NTCP) models are often based on doses retrieved from delineated volumes. For retrospective dose-response studies focusing on organs that have not been delineated historically, automatic segmentation might be considered. However, automatic segmentation risks generating considerable delineation errors and knowledge regarding how these errors impact the estimated organ dose is important. Furthermore, organ-at-risk (OAR) dose uncertainties cannot be eliminated and might affect the resulting NTCP model. Therefore, it is also of interest to study how OAR dose errors impact the NTCP modeling results. PURPOSE To investigate how random delineation errors of the proximal bronchial tree, heart, and esophagus impact the estimated OAR dose, and to investigate how random errors in the doses used for dose-response modeling affect the estimated NTCPs. METHODS We investigated the impact of random delineation errors on the estimated OAR dose using the treatment plans of 39 patients treated with conventionally fractionated radiation therapy of non-small-cell lung cancer. Study-specific reference structures were defined by manually contouring the proximal bronchial tree, heart and esophagus. For each patient and organ, 120 reshaped structures were created by introducing random shifts and margins to the entire reference structure. The mean and near-maximum dose to the reference and reshaped structures were compared. In a separate investigation, the impact of random dose errors on the NTCP model was studied performing dose-response modeling with study sets containing treatment outcomes and OAR doses with and without introduced errors. Universal patient populations with defined population risks, dose-response relationships and distributions of OAR doses were used as ground truth. From such a universal population, we randomly sampled data sets consisting of OAR dose and treatment outcome into reference populations. Study sets of different sizes were created by repeatedly introducing errors to the OAR doses of each reference population. The NTCP models generated with dose errors were compared to the reference NTCP model of the corresponding reference population. RESULTS A total of 14 040 reshaped structures with random delineation errors were created. The delineation errors resulted in systematic mean dose errors of less than 1% of the prescribed dose (PD). Mean dose differences above 15% of PD and near-maximum doses differences above 25% of PD were observed for 211 and 457 reshaped structures, respectively. Introducing random errors to OAR doses used for dose-response modeling resulted in systematic underestimations of the median NTCP. For all investigated scenarios, the median differences in NTCP were within 0.1 percentage points (p.p.) when comparing different study sizes. CONCLUSIONS Introducing random delineation errors to the proximal bronchial tree, heart and esophagus resulted in mean dose and near-maximum dose differences above 15% and 25% of PD, respectively. We did not observe an association between the dose level and the magnitude of the dose errors. For the scenarios investigated in this study, introducing random errors to OAR doses used for dose-response modeling resulted in systematic underestimations of the median NTCP for reference risks higher than the universal population risk. The median NTCP underestimation was similar for different study sizes, all within 0.1 p.p.
Collapse
Affiliation(s)
- Louise Mövik
- Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Anna Bäck
- Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Therapeutic Radiation Physics, Department of Medical Physics and Biomedical Engineering, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Niclas Pettersson
- Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Therapeutic Radiation Physics, Department of Medical Physics and Biomedical Engineering, Sahlgrenska University Hospital, Gothenburg, Sweden
| |
Collapse
|
6
|
Baroudi H, Brock KK, Cao W, Chen X, Chung C, Court LE, El Basha MD, Farhat M, Gay S, Gronberg MP, Gupta AC, Hernandez S, Huang K, Jaffray DA, Lim R, Marquez B, Nealon K, Netherton TJ, Nguyen CM, Reber B, Rhee DJ, Salazar RM, Shanker MD, Sjogreen C, Woodland M, Yang J, Yu C, Zhao Y. Automated Contouring and Planning in Radiation Therapy: What Is 'Clinically Acceptable'? Diagnostics (Basel) 2023; 13:diagnostics13040667. [PMID: 36832155 PMCID: PMC9955359 DOI: 10.3390/diagnostics13040667] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2022] [Revised: 01/21/2023] [Accepted: 01/30/2023] [Indexed: 02/12/2023] Open
Abstract
Developers and users of artificial-intelligence-based tools for automatic contouring and treatment planning in radiotherapy are expected to assess clinical acceptability of these tools. However, what is 'clinical acceptability'? Quantitative and qualitative approaches have been used to assess this ill-defined concept, all of which have advantages and disadvantages or limitations. The approach chosen may depend on the goal of the study as well as on available resources. In this paper, we discuss various aspects of 'clinical acceptability' and how they can move us toward a standard for defining clinical acceptability of new autocontouring and planning tools.
Collapse
Affiliation(s)
- Hana Baroudi
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Kristy K. Brock
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Wenhua Cao
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Xinru Chen
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Caroline Chung
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Laurence E. Court
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Correspondence:
| | - Mohammad D. El Basha
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Maguy Farhat
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Skylar Gay
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Mary P. Gronberg
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Aashish Chandra Gupta
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Soleil Hernandez
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Kai Huang
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - David A. Jaffray
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Rebecca Lim
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Barbara Marquez
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Kelly Nealon
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Callistus M. Nguyen
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Brandon Reber
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Dong Joo Rhee
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Ramon M. Salazar
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Mihir D. Shanker
- The University of Queensland, Saint Lucia 4072, Australia
- The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Carlos Sjogreen
- Department of Physics, University of Houston, Houston, TX 77004, USA
| | - McKell Woodland
- Department of Imaging Physics, Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- Department of Computer Science, Rice University, Houston, TX 77005, USA
| | - Jinzhong Yang
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Cenji Yu
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| | - Yao Zhao
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX 77030, USA
| |
Collapse
|
7
|
Mackay K, Bernstein D, Glocker B, Kamnitsas K, Taylor A. A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy. Clin Oncol (R Coll Radiol) 2023; 35:354-369. [PMID: 36803407 DOI: 10.1016/j.clon.2023.01.016] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Revised: 12/05/2022] [Accepted: 01/23/2023] [Indexed: 02/01/2023]
Abstract
Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.
Collapse
Affiliation(s)
- K Mackay
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK.
| | - D Bernstein
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK
| | - B Glocker
- Department of Computing, Imperial College London, South Kensington Campus, London, UK
| | - K Kamnitsas
- Department of Computing, Imperial College London, South Kensington Campus, London, UK; Department of Engineering Science, University of Oxford, Oxford, UK
| | - A Taylor
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK
| |
Collapse
|
8
|
Poel R, Rüfenacht E, Ermis E, Müller M, Fix MK, Aebersold DM, Manser P, Reyes M. Impact of random outliers in auto-segmented targets on radiotherapy treatment plans for glioblastoma. Radiat Oncol 2022; 17:170. [PMID: 36273161 PMCID: PMC9587574 DOI: 10.1186/s13014-022-02137-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 09/22/2022] [Indexed: 12/03/2022] Open
Abstract
AIMS To save time and have more consistent contours, fully automatic segmentation of targets and organs at risk (OAR) is a valuable asset in radiotherapy. Though current deep learning (DL) based models are on par with manual contouring, they are not perfect and typical errors, as false positives, occur frequently and unpredictably. While it is possible to solve this for OARs, it is far from straightforward for target structures. In order to tackle this problem, in this study, we analyzed the occurrence and the possible dose effects of automated delineation outliers. METHODS First, a set of controlled experiments on synthetically generated outliers on the CT of a glioblastoma (GBM) patient was performed. We analyzed the dosimetric impact on outliers with different location, shape, absolute size and relative size to the main target, resulting in 61 simulated scenarios. Second, multiple segmentation models where trained on a U-Net network based on 80 training sets consisting of GBM cases with annotated gross tumor volume (GTV) and edema structures. On 20 test cases, 5 different trained models and a majority voting method were used to predict the GTV and edema. The amount of outliers on the predictions were determined, as well as their size and distance from the actual target. RESULTS We found that plans containing outliers result in an increased dose to healthy brain tissue. The extent of the dose effect is dependent on the relative size, location and the distance to the main targets and involved OARs. Generally, the larger the absolute outlier volume and the distance to the target the higher the potential dose effect. For 120 predicted GTV and edema structures, we found 1887 outliers. After construction of the planning treatment volume (PTV), 137 outliers remained with a mean distance to the target of 38.5 ± 5.0 mm and a mean size of 1010.8 ± 95.6 mm3. We also found that majority voting of DL results is capable to reduce outliers. CONCLUSIONS This study shows that there is a severe risk of false positive outliers in current DL predictions of target structures. Additionally, these errors will have an evident detrimental impact on the dose and therefore could affect treatment outcome.
Collapse
Affiliation(s)
- Robert Poel
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 18, 3010 Bern, Switzerland
- ARTORG Center for Biomedical Research, University of Bern, Bern, Switzerland
| | - Elias Rüfenacht
- ARTORG Center for Biomedical Research, University of Bern, Bern, Switzerland
| | - Ekin Ermis
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 18, 3010 Bern, Switzerland
| | - Michael Müller
- ARTORG Center for Biomedical Research, University of Bern, Bern, Switzerland
| | - Michael K. Fix
- Division of Medical Radiation Physics and Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Daniel M. Aebersold
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 18, 3010 Bern, Switzerland
| | - Peter Manser
- Division of Medical Radiation Physics and Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Mauricio Reyes
- ARTORG Center for Biomedical Research, University of Bern, Bern, Switzerland
| |
Collapse
|
9
|
Walls GM, Giacometti V, Apte A, Thor M, McCann C, Hanna GG, O'Connor J, Deasy JO, Hounsell AR, Butterworth KT, Cole AJ, Jain S, McGarry CK. Validation of an established deep learning auto-segmentation tool for cardiac substructures in 4D radiotherapy planning scans. Phys Imaging Radiat Oncol 2022; 23:118-126. [PMID: 35941861 PMCID: PMC9356270 DOI: 10.1016/j.phro.2022.07.003] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 07/20/2022] [Accepted: 07/21/2022] [Indexed: 11/10/2022] Open
Abstract
Cardiotoxicity is a common complication of lung cancer radiotherapy. Segmentation of cardiac substructures is time-consuming and challenging. Deep learning segmentation tools can perform this task in 3D and 4D scans. Performance is high when assessed geometrically, dosimetrically and clinically. Auto-segmentation tools may accelerate clinical workflows and enable research.
Background Emerging data suggest that dose-sparing several key cardiac regions is prognostically beneficial in lung cancer radiotherapy. The cardiac substructures are challenging to contour due to their complex geometry, poor soft tissue definition on computed tomography (CT) and cardiorespiratory motion artefact. A neural network was previously trained to generate the cardiac substructures using three-dimensional radiotherapy planning CT scans (3D-CT). In this study, the performance of that tool on the average intensity projection from four-dimensional (4D) CT scans (4D-AVE), now commonly used in lung radiotherapy, was evaluated. Materials and Methods The 4D-AVE of n=20 patients completing radiotherapy for lung cancer 2015–2020 underwent manual and automated cardiac substructure segmentation. Manual and automated substructures were compared geometrically and dosimetrically. Two senior clinicians also qualitatively assessed the auto-segmentation tool’s output. Results Geometric comparison of the automated and manual segmentations exhibited high levels of similarity across parameters, including volume difference (11.8% overall) and Dice similarity coefficient (0.85 overall), and were consistent with 3D-CT performance. Differences in mean (median 0.2 Gy, range −1.6–0.3 Gy) and maximum (median 0.4 Gy, range −2.2–0.9 Gy) doses to substructures were generally small. Nearly all structures (99.5 %) were deemed to be appropriate for clinical use without further editing. Conclusions Cardiac substructure auto-segmentation using a deep learning-based tool trained on a 3D-CT dataset was feasible on the 4D-AVE scan, meaning this tool is suitable for use on 4D-CT radiotherapy planning scans. Application of this tool would increase the practicality of routine clinical cardiac substructure delineation, and enable further cardiac radiation effects research.
Collapse
|
10
|
González PJ, Simões R, Kiers K, Janssen TM. Explaining the dosimetric impact of contouring errors in head and neck radiotherapy. Biomed Phys Eng Express 2022; 8. [PMID: 35732139 DOI: 10.1088/2057-1976/ac7b4c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 06/22/2022] [Indexed: 11/12/2022]
Abstract
Objective. Auto-contouring of organs at risk (OAR) is becoming more common in radiotherapy. An important issue in clinical decision making is judging the quality of the auto-contours. While recent studies considered contour quality by looking at geometric errors only, this does not capture the dosimetric impact of the errors. In this work, we studied the relationship between geometrical errors, the local dose and the dosimetric impact of the geometrical errors.Approach. For 94 head and neck patients, unmodified atlas-based auto-contours and clinically used delineations of the parotid glands and brainstem were retrieved. VMAT plans were automatically optimized on the auto-contours and evaluated on both contours. We defined the dosimetric impact on evaluation (DIE) as the difference in the dosimetric parameter of interest between the two contours. We developed three linear regression models to predict the DIE using: (1) global geometric metrics, (2) global dosimetric metrics, (3) combined local geometric and dosimetric metrics. For model (3), we next determined the minimal amount of editing information required to produce a reliable prediction. Performance was assessed by the root mean squared error (RMSE) of the predicted DIE using 5-fold cross-validation.Main results. In model (3), the median RMSE of the left parotid was 0.4 Gy using 5% of the largest editing vectors. For the right parotid and brainstem the results were 0.5 Gy using 10% and 0.4 Gy using 1% respectively. The median RMS of the DIE was 0.6 Gy, 0.7 Gy and 0.9 Gy for the left parotid, the right parotid and the brainstem, respectively. Model (3), combining local dosimetric and geometric quantities, outperformed the models that used only geometric or dosimetric information.Significance. We showed that the largest local errors plus the local dose suffice to accurately predict the dosimetric impact, opening the door to automated dosimetric QA of auto-contours.
Collapse
Affiliation(s)
- Patrick J González
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Rita Simões
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Karen Kiers
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Tomas M Janssen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| |
Collapse
|
11
|
Johnston N, De Rycke J, Lievens Y, van Eijkeren M, Aelterman J, Vandersmissen E, Ponte S, Vanderstraeten B. Dose-volume-based evaluation of convolutional neural network-based auto-segmentation of thoracic organs at risk. Phys Imaging Radiat Oncol 2022; 23:109-117. [PMID: 35936797 PMCID: PMC9352974 DOI: 10.1016/j.phro.2022.07.004] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 07/20/2022] [Accepted: 07/21/2022] [Indexed: 12/19/2022] Open
Abstract
Dice score and Hausdorff distance do not correlate with dose-volume-based results. Auto-contours close to the tumor or in entry/exit beams should be checked. Heart and esophagus must be checked for locally advanced non-small cell lung cancer. Bronchi must be checked for peripheral early-stage non-small cell lung cancer. Every treatment plan still passed the clinical goals for the manual organs at risk.
Background and purpose The geometrical accuracy of auto-segmentation using convolutional neural networks (CNNs) has been demonstrated. This study aimed to investigate the dose-volume impact of differences between automatic and manual OARs for locally advanced (LA) and peripherally located early-stage (ES) non-small cell lung cancer (NSCLC). Material and methods A single CNN was created for automatic delineation of the heart, lungs, main left and right bronchus, esophagus, spinal cord and trachea using 55/10/40 patients for training/validation/testing. Dice score coefficient (DSC) and 95th percentile Hausdorff distance (HD95) were used for geometrical analysis. A new treatment plan based on the auto-segmented OARs was created for each test patient using 3D for ES-NSCLC (SBRT, 3–8 fractions) and IMRT for LA-NSCLC (24–35 fractions). The correlation between geometrical metrics and dose-volume differences was investigated. Results The average (±1 SD) DSC and HD95 were 0.82 ± 0.07 and 16.2 ± 22.4 mm, while the average dose-volume differences were 0.5 ± 1.5 Gy (ES) and 1.5 ± 2.8 Gy (LA). The geometrical metrics did not correlate with the observed dose-volume differences (average Pearson for DSC: −0.27 ± 0.18 (ES) and −0.09 ± 0.12 (LA); HD95: 0.1 ± 0.3 mm (ES) and 0.2 ± 0.2 mm (LA)). Conclusions After post-processing, manual adjustments of automatic contours are only needed for clinically relevant OARs situated close to the tumor or within an entry or exit beam e.g., the heart and the esophagus for LA-NSCLC and the bronchi for ES-NSCLC. The lungs do not need to be checked further in detail.
Collapse
Affiliation(s)
- Noémie Johnston
- Centre Hospitalier Universitaire de Liège, Service de Radiothérapie, Liège, Belgium
| | - Jeffrey De Rycke
- Ghent University, Faculty of Medicine and Health Sciences, Department of Human Structure and Repair, Gent, Belgium
| | - Yolande Lievens
- Ghent University, Faculty of Medicine and Health Sciences, Department of Human Structure and Repair, Gent, Belgium
- Ghent University Hospital, Department of Radiotherapy-Oncology, Gent, Belgium
| | - Marc van Eijkeren
- Ghent University, Faculty of Medicine and Health Sciences, Department of Human Structure and Repair, Gent, Belgium
- Ghent University Hospital, Department of Radiotherapy-Oncology, Gent, Belgium
| | - Jan Aelterman
- Ghent University, Department of Physics and Astronomy, Ghent University Centre for X-ray Tomography, Gent, Belgium
- Ghent University, Department TELIN / IMEC, Image Processing Interpretation Group, Gent, Belgium
| | | | - Stephan Ponte
- Centre Hospitalier Universitaire de Liège, Service de Radiothérapie, Liège, Belgium
| | - Barbara Vanderstraeten
- Ghent University, Faculty of Medicine and Health Sciences, Department of Human Structure and Repair, Gent, Belgium
- Ghent University Hospital, Department of Radiotherapy-Oncology, Gent, Belgium
- Corresponding author at: Ghent University Hospital, Department of Radiotherapy-Oncology, RTP Ingang 98, Corneel Heymanslaan 10, B-9000 Gent, Belgium.
| |
Collapse
|
12
|
Chen A, Chen F, Li X, Zhang Y, Chen L, Chen L, Zhu J. A Feasibility Study of Deep Learning-Based Auto-Segmentation Directly Used in VMAT Planning Design and Optimization for Cervical Cancer. Front Oncol 2022; 12:908903. [PMID: 35719942 PMCID: PMC9198405 DOI: 10.3389/fonc.2022.908903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 05/06/2022] [Indexed: 12/02/2022] Open
Abstract
Purpose To investigate the dosimetric impact on target volumes and organs at risk (OARs) when unmodified auto-segmented OAR contours are directly used in the design of treatment plans. Materials and Methods A total of 127 patients with cervical cancer were collected for retrospective analysis, including 105 patients in the training set and 22 patients in the testing set. The 3D U-net architecture was used for model training and auto-segmentation of nine types of organs at risk. The auto-segmented and manually segmented organ contours were used for treatment plan optimization to obtain the AS-VMAT (automatic segmentations VMAT) plan and the MS-VMAT (manual segmentations VMAT) plan, respectively. Geometric accuracy between the manual and predicted contours were evaluated using the Dice similarity coefficient (DSC), mean distance-to-agreement (MDA), and Hausdorff distance (HD). The dose volume histogram (DVH) and the gamma passing rate were used to identify the dose differences between the AS-VMAT plan and the MS-VMAT plan. Results Average DSC, MDA and HD95 across all OARs were 0.82–0.96, 0.45–3.21 mm, and 2.30–17.31 mm on the testing set, respectively. The D99% in the rectum and the Dmean in the spinal cord were 6.04 Gy (P = 0.037) and 0.54 Gy (P = 0.026) higher, respectively, in the AS-VMAT plans than in the MS-VMAT plans. The V20, V30, and V40 in the rectum increased by 1.35% (P = 0.027), 1.73% (P = 0.021), and 1.96% (P = 0.008), respectively, whereas the V10 in the spinal cord increased by 1.93% (P = 0.011). The differences in other dosimetry parameters were not statistically significant. The gamma passing rates in the clinical target volume (CTV) were 92.72% and 98.77%, respectively, using the 2%/2 mm and 3%/3 mm criteria, which satisfied the clinical requirements. Conclusions The dose distributions of target volumes were unaffected when auto-segmented organ contours were used in the design of treatment plans, whereas the impact of automated segmentation on the doses to OARs was complicated. We suggest that the auto-segmented contours of tissues in close proximity to the target volume need to be carefully checked and corrected when necessary.
Collapse
Affiliation(s)
- Along Chen
- Department of Radiation Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
| | - Fei Chen
- School of Biomedical Engineering, Guangzhou Xinhua University, Guangzhou, China
| | - Xiaofang Li
- Department of Radiation Oncology, The Second Affiliated Hospital of Zunyi Medical University, Zunyi, China
| | - Yazhi Zhang
- Department of Oncology and Hematology, The Six People’s Hospital of Huizhou City, Huiyang Hospital Affiliated to Southern Medical University, Huizhou, China
| | - Li Chen
- Department of Radiation Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
| | - Lixin Chen
- Department of Radiation Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
- *Correspondence: Lixin Chen, ; Jinhan Zhu,
| | - Jinhan Zhu
- Department of Radiation Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
- *Correspondence: Lixin Chen, ; Jinhan Zhu,
| |
Collapse
|
13
|
Vaassen F, Boukerroui D, Looney P, Canters R, Verhoeven K, Peeters S, Lubken I, Mannens J, Gooding MJ, van Elmpt W. Real-world analysis of manual editing of deep learning contouring in the thorax region. Phys Imaging Radiat Oncol 2022; 22:104-110. [PMID: 35602549 PMCID: PMC9115320 DOI: 10.1016/j.phro.2022.04.008] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Revised: 04/13/2022] [Accepted: 04/27/2022] [Indexed: 01/18/2023] Open
Abstract
Background and purpose User-adjustments after deep-learning (DL) contouring in radiotherapy were evaluated to get insight in real-world editing during clinical practice. This study assessed the amount, type and spatial regions of editing of auto-contouring for organs-at-risk (OARs) in routine clinical workflow for patients in the thorax region. Materials and methods A total of 350 lung cancer and 362 breast cancer patients, contoured between March 2020 and March 2021 using a commercial DL-contouring method followed by manual adjustments were retrospectively analyzed. Subsampling was performed for some OARs, using an inter-slice gap of 1-3 slices. Commonly-used whole-organ contouring assessment measures were calculated, and all cases were registered to a common reference shape per OAR to identify regions of manual adjustment. Results were expressed as the median, 10th-90th percentile of adjustment and visualized using 3D renderings. Results Per OAR, the median amount of editing was below 1 mm. However, large adjustments were found in some locations for most OARs. In general, enlarging of the auto-contours was needed. Subsampling DL-contours showed less adjustments were made in the interpolated slices compared to simulated no-subsampling for these OARs. Conclusion The real-world performance of automatic DL-contouring software was evaluated and proven useful in clinical practice. Specific regions-of-adjustment were identified per OAR in the thorax region, and separate models were found to be necessary for specific clinical indications different from training data. This analysis showed the need to perform routine clinical analysis especially when procedures or acquisition protocols change to have the best configuration of the workflow.
Collapse
Affiliation(s)
- Femke Vaassen
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | | | | | - Richard Canters
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Karolien Verhoeven
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Stephanie Peeters
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Indra Lubken
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Jolein Mannens
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | | | - Wouter van Elmpt
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| |
Collapse
|
14
|
Lappas G, Wolfs CJA, Staut N, Lieuwes NG, Biemans R, van Hoof SJ, Dubois LJ, Verhaegen F. Automatic contouring of normal tissues with deep learning for preclinical radiation studies. Phys Med Biol 2022; 67. [PMID: 35061600 DOI: 10.1088/1361-6560/ac4da3] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Accepted: 01/21/2022] [Indexed: 02/05/2023]
Abstract
Objective.Delineation of relevant normal tissues is a bottleneck in image-guided precision radiotherapy workflows for small animals. A deep learning (DL) model for automatic contouring using standardized 3D micro cone-beam CT (μCBCT) volumes as input is proposed, to provide a fully automatic, generalizable method for normal tissue contouring in preclinical studies.Approach.A 3D U-net was trained to contour organs in the head (whole brain, left/right brain hemisphere, left/right eye) and thorax (complete lungs, left/right lung, heart, spinal cord, thorax bone) regions. As an important preprocessing step, Hounsfield units (HUs) were converted to mass density (MD) values, to remove the energy dependency of theμCBCT scanner and improve generalizability of the DL model. Model performance was evaluated quantitatively by Dice similarity coefficient (DSC), mean surface distance (MSD), 95th percentile Hausdorff distance (HD95p), and center of mass displacement (ΔCoM). For qualitative assessment, DL-generated contours (for 40 and 80 kV images) were scored (0: unacceptable, manual re-contouring needed - 5: no adjustments needed). An uncertainty analysis using Monte Carlo dropout uncertainty was performed for delineation of the heart.Main results.The proposed DL model and accompanying preprocessing method provide high quality contours, with in general median DSC > 0.85, MSD < 0.25 mm, HD95p < 1 mm and ΔCoM < 0.5 mm. The qualitative assessment showed very few contours needed manual adaptations (40 kV: 20/155 contours, 80 kV: 3/155 contours). The uncertainty of the DL model is small (within 2%).Significance.A DL-based model dedicated to preclinical studies has been developed for multi-organ segmentation in two body sites. For the first time, a method independent of image acquisition parameters has been quantitatively evaluated, resulting in sub-millimeter performance, while qualitative assessment demonstrated the high quality of the DL-generated contours. The uncertainty analysis additionally showed that inherent model variability is low.
Collapse
Affiliation(s)
- Georgios Lappas
- Department of Radiation Oncology (Maastro), GROW-School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Cecile J A Wolfs
- Department of Radiation Oncology (Maastro), GROW-School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, The Netherlands
| | - Nick Staut
- Department of Radiation Oncology (Maastro), GROW-School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, The Netherlands.,SmART Scientific Solutions BV, Maastricht, The Netherlands
| | - Natasja G Lieuwes
- The M-Lab, Department of Precision Medicine, GROW-School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands
| | - Rianne Biemans
- The M-Lab, Department of Precision Medicine, GROW-School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands
| | | | - Ludwig J Dubois
- The M-Lab, Department of Precision Medicine, GROW-School for Oncology and Developmental Biology, Maastricht University, Maastricht, The Netherlands
| | - Frank Verhaegen
- Department of Radiation Oncology (Maastro), GROW-School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, The Netherlands.,SmART Scientific Solutions BV, Maastricht, The Netherlands
| |
Collapse
|