1
|
Erdur AC, Rusche D, Scholz D, Kiechle J, Fischer S, Llorián-Salvador Ó, Buchner JA, Nguyen MQ, Etzel L, Weidner J, Metz MC, Wiestler B, Schnabel J, Rueckert D, Combs SE, Peeken JC. Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives. Strahlenther Onkol 2024:10.1007/s00066-024-02262-2. [PMID: 39105745 DOI: 10.1007/s00066-024-02262-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 06/13/2024] [Indexed: 08/07/2024]
Abstract
The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.
Collapse
Affiliation(s)
- Ayhan Can Erdur
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany.
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany.
| | - Daniel Rusche
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Daniel Scholz
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Johannes Kiechle
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute for Computational Imaging and AI in Medicine, Technical University of Munich, Lichtenberg Str. 2a, 85748, Garching, Bavaria, Germany
- Munich Center for Machine Learning (MCML), Technical University of Munich, Arcisstraße 21, 80333, Munich, Bavaria, Germany
- Konrad Zuse School of Excellence in Reliable AI (relAI), Technical University of Munich, Walther-von-Dyck-Straße 10, 85748, Garching, Bavaria, Germany
| | - Stefan Fischer
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute for Computational Imaging and AI in Medicine, Technical University of Munich, Lichtenberg Str. 2a, 85748, Garching, Bavaria, Germany
- Munich Center for Machine Learning (MCML), Technical University of Munich, Arcisstraße 21, 80333, Munich, Bavaria, Germany
| | - Óscar Llorián-Salvador
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Department for Bioinformatics and Computational Biology - i12, Technical University of Munich, Boltzmannstraße 3, 85748, Garching, Bavaria, Germany
- Institute of Organismic and Molecular Evolution, Johannes Gutenberg University Mainz (JGU), Hüsch-Weg 15, 55128, Mainz, Rhineland-Palatinate, Germany
| | - Josef A Buchner
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Mai Q Nguyen
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Lucas Etzel
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute of Radiation Medicine (IRM), Helmholtz Zentrum, Ingolstädter Landstraße 1, 85764, Oberschleißheim, Bavaria, Germany
| | - Jonas Weidner
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Marie-Christin Metz
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Benedikt Wiestler
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Julia Schnabel
- Institute for Computational Imaging and AI in Medicine, Technical University of Munich, Lichtenberg Str. 2a, 85748, Garching, Bavaria, Germany
- Munich Center for Machine Learning (MCML), Technical University of Munich, Arcisstraße 21, 80333, Munich, Bavaria, Germany
- Konrad Zuse School of Excellence in Reliable AI (relAI), Technical University of Munich, Walther-von-Dyck-Straße 10, 85748, Garching, Bavaria, Germany
- Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Ingolstädter Landstraße 1, 85764, Neuherberg, Bavaria, Germany
- School of Biomedical Engineering & Imaging Sciences, King's College London, Strand, WC2R 2LS, London, London, UK
| | - Daniel Rueckert
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Faculty of Engineering, Department of Computing, Imperial College London, Exhibition Rd, SW7 2BX, London, London, UK
| | - Stephanie E Combs
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute of Radiation Medicine (IRM), Helmholtz Zentrum, Ingolstädter Landstraße 1, 85764, Oberschleißheim, Bavaria, Germany
- Partner Site Munich, German Consortium for Translational Cancer Research (DKTK), Munich, Bavaria, Germany
| | - Jan C Peeken
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute of Radiation Medicine (IRM), Helmholtz Zentrum, Ingolstädter Landstraße 1, 85764, Oberschleißheim, Bavaria, Germany
- Partner Site Munich, German Consortium for Translational Cancer Research (DKTK), Munich, Bavaria, Germany
| |
Collapse
|
2
|
Wahid KA, Kaffey ZY, Farris DP, Humbert-Vidan L, Moreno AC, Rasmussen M, Ren J, Naser MA, Netherton TJ, Korreman S, Balakrishnan G, Fuller CD, Fuentes D, Dohopolski MJ. Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.13.24307226. [PMID: 38798581 PMCID: PMC11118597 DOI: 10.1101/2024.05.13.24307226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Background/purpose The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Zaphanlene Y. Kaffey
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David P. Farris
- Research Medical Library, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Laia Humbert-Vidan
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Amy C. Moreno
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | - Jintao Ren
- Department of Oncology, Aarhus University Hospital, Denmark
| | - Mohamed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Denmark
| | | | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael J. Dohopolski
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, Texas, USA
| |
Collapse
|
3
|
Rong Y, Chen Q, Fu Y, Yang X, Al-Hallaq HA, Wu QJ, Yuan L, Xiao Y, Cai B, Latifi K, Benedict SH, Buchsbaum JC, Qi XS. NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions. Int J Radiat Oncol Biol Phys 2024; 119:261-280. [PMID: 37972715 PMCID: PMC11023777 DOI: 10.1016/j.ijrobp.2023.10.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 09/16/2023] [Accepted: 10/14/2023] [Indexed: 11/19/2023]
Abstract
Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.
Collapse
Affiliation(s)
- Yi Rong
- Mayo Clinic Arizona, Phoenix, AZ
| | - Quan Chen
- City of Hope Comprehensive Cancer Center Duarte, CA
| | - Yabo Fu
- Memorial Sloan Kettering Cancer Center, Commack, NY
| | | | | | | | - Lulin Yuan
- Virginia Commonwealth University, Richmond, VA
| | - Ying Xiao
- University of Pennsylvania/Abramson Cancer Center, Philadelphia, PA
| | - Bin Cai
- The University of Texas Southwestern Medical Center, Dallas, TX
| | | | - Stanley H Benedict
- University of California Davis Comprehensive Cancer Center, Sacramento, CA
| | | | - X Sharon Qi
- University of California Los Angeles, Los Angeles, CA
| |
Collapse
|
4
|
Luan S, Ou-Yang J, Yang X, Wei W, Xue X, Zhu B. A multi-modal vision-language pipeline strategy for contour quality assurance and adaptive optimization. Phys Med Biol 2024; 69:065005. [PMID: 38373347 DOI: 10.1088/1361-6560/ad2a97] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 02/19/2024] [Indexed: 02/21/2024]
Abstract
Objective.Accurate delineation of organs-at-risk (OARs) is a critical step in radiotherapy. The deep learning generated segmentations usually need to be reviewed and corrected by oncologists manually, which is time-consuming and operator-dependent. Therefore, an automated quality assurance (QA) and adaptive optimization correction strategy was proposed to identify and optimize 'incorrect' auto-segmentations.Approach.A total of 586 CT images and labels from nine institutions were used. The OARs included the brainstem, parotid, and mandible. The deep learning generated contours were compared with the manual ground truth delineations. In this study, we proposed a novel contour quality assurance and adaptive optimization (CQA-AO) strategy, which consists of the following three main components: (1) the contour QA module classified the deep learning generated contours as either accepted or unaccepted; (2) the unacceptable contour categories analysis module provided the potential error reasons (five unacceptable category) and locations (attention heatmaps); (3) the adaptive correction of unacceptable contours module integrate vision-language representations and utilize convex optimization algorithms to achieve adaptive correction of 'incorrect' contours.Main results. In the contour QA tasks, the sensitivity (accuracy, precision) of CQA-AO strategy reached 0.940 (0.945, 0.948), 0.962 (0.937, 0.913), and 0.967 (0.962, 0.957) for brainstem, parotid and mandible, respectively. The unacceptable contour category analysis, the(FI,AccI,Fmicro,Fmacro)of CQA-AO strategy reached (0.901, 0.763, 0.862, 0.822), (0.855, 0.737, 0.837, 0.784), and (0.907, 0.762, 0.858, 0.821) for brainstem, parotid and mandible, respectively. After adaptive optimization correction, the DSC values of brainstem, parotid and mandible have been improved by 9.4%, 25.9%, and 13.5%, and Hausdorff distance values decreased by 62%, 70.6%, and 81.6%, respectively.Significance. The proposed CQA-AO strategy, which combines QA of contour and adaptive optimization correction for OARs contouring, demonstrated superior performance compare to conventional methods. This method can be implemented in the clinical contouring procedures and improve the efficiency of delineating and reviewing workflow.
Collapse
Affiliation(s)
- Shunyao Luan
- School of Integrated Circuits, Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, People's Republic of China
| | - Jun Ou-Yang
- School of Integrated Circuits, Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, People's Republic of China
| | - Xiaofei Yang
- School of Integrated Circuits, Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, People's Republic of China
| | - Wei Wei
- Department of Radiation Oncology, Hubei Cancer Hospital, TongJi Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, People's Republic of China
| | - Xudong Xue
- Department of Radiation Oncology, Hubei Cancer Hospital, TongJi Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, People's Republic of China
| | - Benpeng Zhu
- School of Integrated Circuits, Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, People's Republic of China
| |
Collapse
|
5
|
Roberfroid B, Lee JA, Geets X, Sterpin E, Barragán-Montero AM. DIVE-ART: A tool to guide clinicians towards dosimetrically informed volume editions of automatically segmented volumes in adaptive radiation therapy. Radiother Oncol 2024; 192:110108. [PMID: 38272315 DOI: 10.1016/j.radonc.2024.110108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Revised: 01/17/2024] [Accepted: 01/17/2024] [Indexed: 01/27/2024]
Affiliation(s)
- Benjamin Roberfroid
- Université catholique de Louvain - Center of Molecular Imaging, Radiotherapy and Oncology (MIRO), Brussels, Belgium.
| | - John A Lee
- Université catholique de Louvain - Center of Molecular Imaging, Radiotherapy and Oncology (MIRO), Brussels, Belgium
| | - Xavier Geets
- Université catholique de Louvain - Center of Molecular Imaging, Radiotherapy and Oncology (MIRO), Brussels, Belgium; Cliniques universitaires Saint-Luc, Department of Radiation Oncology, Brussels, Belgium
| | - Edmond Sterpin
- Université catholique de Louvain - Center of Molecular Imaging, Radiotherapy and Oncology (MIRO), Brussels, Belgium; KU Leuven - Department of Oncology, Laboratory of Experimental Radiotherapy, Leuven, Belgium; Particle Therapy Interuniversity Center Leuven - PARTICLE, Leuven, Belgium
| | - Ana M Barragán-Montero
- Université catholique de Louvain - Center of Molecular Imaging, Radiotherapy and Oncology (MIRO), Brussels, Belgium
| |
Collapse
|
6
|
Hanna EM, Sargent E, Hua CH, Merchant TE, Ates O. Performance analysis and knowledge-based quality assurance of critical organ auto-segmentation for pediatric craniospinal irradiation. Sci Rep 2024; 14:4251. [PMID: 38378834 DOI: 10.1038/s41598-024-55015-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 02/19/2024] [Indexed: 02/22/2024] Open
Abstract
Craniospinal irradiation (CSI) is a vital therapeutic approach utilized for young patients suffering from central nervous system disorders such as medulloblastoma. The task of accurately outlining the treatment area is particularly time-consuming due to the presence of several sensitive organs at risk (OAR) that can be affected by radiation. This study aimed to assess two different methods for automating the segmentation process: an atlas technique and a deep learning neural network approach. Additionally, a novel method was devised to prospectively evaluate the accuracy of automated segmentation as a knowledge-based quality assurance (QA) tool. Involving a patient cohort of 100, ranging in ages from 2 to 25 years with a median age of 8, this study employed quantitative metrics centered around overlap and distance calculations to determine the most effective approach for practical clinical application. The contours generated by two distinct methods of atlas and neural network were compared to ground truth contours approved by a radiation oncologist, utilizing 13 distinct metrics. Furthermore, an innovative QA tool was conceptualized, designed for forthcoming cases based on the baseline dataset of 100 patient cases. The calculated metrics indicated that, in the majority of cases (60.58%), the neural network method demonstrated a notably higher alignment with the ground truth. Instances where no difference was observed accounted for 31.25%, while utilization of the atlas method represented 8.17%. The QA tool results showed that the two approaches achieved 100% agreement in 39.4% of instances for the atlas method and in 50.6% of instances for the neural network auto-segmentation. The results indicate that the neural network approach showcases superior performance, and its significantly closer physical alignment to ground truth contours in the majority of cases. The metrics derived from overlap and distance measurements have enabled clinicians to discern the optimal choice for practical clinical application.
Collapse
Affiliation(s)
- Emeline M Hanna
- St. Jude Children's Research Hospital, Memphis, TN, 38105, USA
| | - Emma Sargent
- St. Jude Children's Research Hospital, Memphis, TN, 38105, USA
| | - Chia-Ho Hua
- St. Jude Children's Research Hospital, Memphis, TN, 38105, USA
| | | | - Ozgur Ates
- St. Jude Children's Research Hospital, Memphis, TN, 38105, USA.
| |
Collapse
|
7
|
Ocampo J, Heyes G, Dehghani H, Scanlon T, Jolly S, Gibson A. Determination of output factor for CyberKnife using scintillation dosimetry and deep learning. Phys Med Biol 2024; 69:025024. [PMID: 38181420 DOI: 10.1088/1361-6560/ad1b69] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 01/05/2024] [Indexed: 01/07/2024]
Abstract
Objective. Small-field dosimetry is an ongoing challenge in radiotherapy quality assurance (QA) especially for radiosurgery systems such as CyberKnifeTM. The objective of this work is to demonstrate the use of a plastic scintillator imaged with a commercial camera to measure the output factor of a CyberKnife system. The output factor describes the dose on the central axis as a function of collimator size, and is a fundamental part of CyberKnife QA and integral to the data used in the treatment planning system.Approach. A self-contained device consisting of a solid plastic scintillator and a camera was build in a portable Pelicase. Photographs were analysed using classical methods and with convolutional neural networks (CNN) to predict beam parameters which were then compared to measurements.Main results. Initial results using classical image processing to determine standard QA parameters such as percentage depth dose (PDD) were unsuccessful, with 34% of points failing to meet the Gamma criterion (which measures the distance between corresponding points and the relative difference in dose) of 2 mm/2%. However, when images were processed using a CNN trained on simulated data and a green scintillator sheet, 92% of PDD curves agreed with measurements with a microdiamond detector to within 2 mm/2% and 78% to 1%/1 mm. The mean difference between the output factors measured using this system and a microdiamond detector was 1.1%. Confidence in the results was enhanced by using the algorithm to predict the known collimator sizes from the photographs which it was able to do with an accuracy of less than 1 mm.Significance. With refinement, a full output factor curve could be measured in less than an hour, offering a new approach for rapid, convenient small-field dosimetry.
Collapse
Affiliation(s)
- Jeremy Ocampo
- UCL Physics and Astronomy, London, WC1E 6BT, United Kingdom
| | - Geoff Heyes
- Radiotherapy Physics, University Hospitals Birmingham NHS Foundation Trust, Birmingham B15 2TH, United Kingdom
| | - Hamid Dehghani
- School of Computer Science, University of Birmingham, Birmingham B15 2TT, United Kingdom
| | - Tim Scanlon
- UCL Physics and Astronomy, London, WC1E 6BT, United Kingdom
| | - Simon Jolly
- UCL Physics and Astronomy, London, WC1E 6BT, United Kingdom
| | - Adam Gibson
- UCL Medical Physics & Biomedical Engineering, London, WC1E 6BT, United Kingdom
| |
Collapse
|
8
|
Luan S, Wu K, Wu Y, Zhu B, Wei W, Xue X. Accurate and robust auto-segmentation of head and neck organ-at-risks based on a novel CNN fine-tuning workflow. J Appl Clin Med Phys 2024; 25:e14248. [PMID: 38128058 PMCID: PMC10795444 DOI: 10.1002/acm2.14248] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Revised: 12/08/2023] [Accepted: 12/11/2023] [Indexed: 12/23/2023] Open
Abstract
PURPOSE Obvious inconsistencies in auto-segmentations exist among various AI software. In this study, we have developed a novel convolutional neural network (CNN) fine-tuning workflow to achieve precise and robust localized segmentation. METHODS The datasets include Hubei Cancer Hospital dataset, Cetuximab Head and Neck Public Dataset, and Québec Public Dataset. Seven organs-at-risks (OARs), including brain stem, left parotid gland, esophagus, left optic nerve, optic chiasm, mandible, and pharyngeal constrictor, were selected. The auto-segmentation results from four commercial AI software were first compared with the manual delineations. Then a new multi-scale lightweight residual CNN model with an attention module (named as HN-Net) was trained and tested on 40 samples and 10 samples from Hubei Cancer Hospital, respectively. To enhance the network's accuracy and generalization ability, the fine-tuning workflow utilized an uncertainty estimation method for automatic selection of candidate samples of worthiness from Cetuximab Head and Neck Public Dataset for further training. The segmentation performances were evaluated on the Hubei Cancer Hospital dataset and/or the entire Québec Public Dataset. RESULTS A maximum difference of 0.13 and 0.7 mm in average Dice value and Hausdorff distance value for the seven OARs were observed by four AI software. The proposed HN-Net achieved an average Dice value of 0.14 higher than that of the AI software, and it also outperformed other popular CNN models (HN-Net: 0.79, U-Net: 0.78, U-Net++: 0.78, U-Net-Multi-scale: 0.77, AI software: 0.65). Additionally, the HN-Net fine-tuning workflow by using the local datasets and external public datasets further improved the automatic segmentation with the average Dice value by 0.02. CONCLUSION The delineations of commercial AI software need to be carefully reviewed, and localized further training is necessary for clinical practice. The proposed fine-tuning workflow could be feasibly adopted to implement an accurate and robust auto-segmentation model by using local datasets and external public datasets.
Collapse
Affiliation(s)
- Shunyao Luan
- Department of Radiation OncologyHubei Cancer Hospital, Tongji Medical CollegeHuazhong University of Science and TechnologyWuhanChina
- School of Integrated CircuitsLaboratory for OptoelectronicsHuazhong University of Science and TechnologyWuhanChina
| | - Kun Wu
- Department of Radiation OncologyHubei Cancer Hospital, Tongji Medical CollegeHuazhong University of Science and TechnologyWuhanChina
| | - Yuan Wu
- Department of Radiation OncologyHubei Cancer Hospital, Tongji Medical CollegeHuazhong University of Science and TechnologyWuhanChina
| | - Benpeng Zhu
- School of Integrated CircuitsLaboratory for OptoelectronicsHuazhong University of Science and TechnologyWuhanChina
| | - Wei Wei
- Department of Radiation OncologyHubei Cancer Hospital, Tongji Medical CollegeHuazhong University of Science and TechnologyWuhanChina
| | - Xudong Xue
- Department of Radiation OncologyHubei Cancer Hospital, Tongji Medical CollegeHuazhong University of Science and TechnologyWuhanChina
| |
Collapse
|
9
|
Wahid KA, Sahlsten J, Jaskari J, Dohopolski MJ, Kaski K, He R, Glerean E, Kann BH, Mäkitie A, Fuller CD, Naser MA, Fuentes D. Harnessing uncertainty in radiotherapy auto-segmentation quality assurance. Phys Imaging Radiat Oncol 2024; 29:100526. [PMID: 38179210 PMCID: PMC10765294 DOI: 10.1016/j.phro.2023.100526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 12/13/2023] [Indexed: 01/06/2024] Open
Affiliation(s)
- Kareem A. Wahid
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Michael J. Dohopolski
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Benjamin H. Kann
- Artificial Intelligence in Medicine Program, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
| | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mohamed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
10
|
Xu Z, Rauch DE, Mohamed RM, Pashapoor S, Zhou Z, Panthi B, Son JB, Hwang KP, Musall BC, Adrada BE, Candelaria RP, Leung JWT, Le-Petross HTC, Lane DL, Perez F, White J, Clayborn A, Reed B, Chen H, Sun J, Wei P, Thompson A, Korkut A, Huo L, Hunt KK, Litton JK, Valero V, Tripathy D, Yang W, Yam C, Ma J. Deep Learning for Fully Automatic Tumor Segmentation on Serially Acquired Dynamic Contrast-Enhanced MRI Images of Triple-Negative Breast Cancer. Cancers (Basel) 2023; 15:4829. [PMID: 37835523 PMCID: PMC10571741 DOI: 10.3390/cancers15194829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 09/10/2023] [Accepted: 09/22/2023] [Indexed: 10/15/2023] Open
Abstract
Accurate tumor segmentation is required for quantitative image analyses, which are increasingly used for evaluation of tumors. We developed a fully automated and high-performance segmentation model of triple-negative breast cancer using a self-configurable deep learning framework and a large set of dynamic contrast-enhanced MRI images acquired serially over the patients' treatment course. Among all models, the top-performing one that was trained with the images across different time points of a treatment course yielded a Dice similarity coefficient of 93% and a sensitivity of 96% on baseline images. The top-performing model also produced accurate tumor size measurements, which is valuable for practical clinical applications.
Collapse
Affiliation(s)
- Zhan Xu
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - David E. Rauch
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - Rania M. Mohamed
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Sanaz Pashapoor
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Zijian Zhou
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - Bikash Panthi
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - Jong Bum Son
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - Ken-Pin Hwang
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - Benjamin C. Musall
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| | - Beatriz E. Adrada
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Rosalind P. Candelaria
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Jessica W. T. Leung
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Huong T. C. Le-Petross
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Deanna L. Lane
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Frances Perez
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Jason White
- Department of Breast Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Alyson Clayborn
- Department of Breast Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Brandy Reed
- Department of Clinical Research Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Huiqin Chen
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Jia Sun
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Peng Wei
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Alastair Thompson
- Section of Breast Surgery, Baylor College of Medicine, Houston, TX 77030, USA
| | - Anil Korkut
- Department of Bioinformatics & Computational Biology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Lei Huo
- Department of Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Kelly K. Hunt
- Department of Breast Surgical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Jennifer K. Litton
- Department of Breast Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Vicente Valero
- Department of Breast Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Debu Tripathy
- Department of Breast Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Wei Yang
- Department of Breast Imaging, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Clinton Yam
- Department of Breast Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Jingfei Ma
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (Z.X.)
| |
Collapse
|
11
|
Rodríguez Outeiral R, Ferreira Silvério N, González PJ, Schaake EE, Janssen T, van der Heide UA, Simões R. A network score-based metric to optimize the quality assurance of automatic radiotherapy target segmentations. Phys Imaging Radiat Oncol 2023; 28:100500. [PMID: 37869474 PMCID: PMC10587515 DOI: 10.1016/j.phro.2023.100500] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 10/06/2023] [Accepted: 10/09/2023] [Indexed: 10/24/2023] Open
Abstract
Background and purpose Existing methods for quality assurance of the radiotherapy auto-segmentations focus on the correlation between the average model entropy and the Dice Similarity Coefficient (DSC) only. We identified a metric directly derived from the output of the network and correlated it with clinically relevant metrics for contour accuracy. Materials and Methods Magnetic Resonance Imaging auto-segmentations were available for the gross tumor volume for cervical cancer brachytherapy (106 segmentations) and for the clinical target volume for rectal cancer external-beam radiotherapy (77 segmentations). The nnU-Net's output before binarization was taken as a score map. We defined a metric as the mean of the voxels in the score map above a threshold (λ). Comparisons were made with the mean and standard deviation over the score map and with the mean over the entropy map. The DSC, the 95th Hausdorff distance, the mean surface distance (MSD) and the surface DSC were computed for segmentation quality. Correlations between the studied metrics and model quality were assessed with the Pearson correlation coefficient (r). The area under the curve (AUC) was determined for detecting segmentations that require reviewing. Results For both tasks, our metric (λ = 0.30) correlated more strongly with the segmentation quality than the mean over the entropy map (for surface DSC, r > 0.65 vs. r < 0.60). The AUC was above 0.84 for detecting MSD values above 2 mm. Conclusions Our metric correlated strongly with clinically relevant segmentation metrics and detected segmentations that required reviewing, indicating its potential for automatic quality assurance of radiotherapy target auto-segmentations.
Collapse
Affiliation(s)
- Roque Rodríguez Outeiral
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Nicole Ferreira Silvério
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Patrick J. González
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Eva E. Schaake
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Tomas Janssen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Uulke A. van der Heide
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Rita Simões
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| |
Collapse
|
12
|
Duan J, Bernard ME, Castle JR, Feng X, Wang C, Kenamond MC, Chen Q. Contouring quality assurance methodology based on multiple geometric features against deep learning auto-segmentation. Med Phys 2023; 50:2715-2732. [PMID: 36788735 PMCID: PMC10175153 DOI: 10.1002/mp.16299] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 01/06/2023] [Accepted: 01/26/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND Contouring error is one of the top failure modes in radiation treatment. Multiple efforts have been made to develop tools to automatically detect segmentation errors. Deep learning-based auto-segmentation (DLAS) has been used as a baseline for flagging manual segmentation errors, but those efforts are limited to using only one or two contour comparison metrics. PURPOSE The purpose of this research is to develop an improved contouring quality assurance system to identify and flag manual contouring errors. METHODS AND MATERIALS DLAS contours were used as a reference to compare with manually segmented contours. A total of 27 geometric agreement metrics were determined from the comparisons between the two segmentation approaches. Feature selection was performed to optimize the training of a machine learning classification model to identify potential contouring errors. A public dataset with 339 cases was used to train and test the classifier. Four independent classifiers were trained using five-fold cross validation, and the predictions from each classifier were ensembled using soft voting. The trained model was validated on a held-out testing dataset. An additional independent clinical dataset with 60 cases was used to test the generalizability of the model. Model predictions were reviewed by an expert to confirm or reject the findings. RESULTS The proposed machine learning multiple features (ML-MF) approach outperformed traditional nonmachine-learning-based approaches that are based on only one or two geometric agreement metrics. The machine learning model achieved recall (precision) values of 0.842 (0.899), 0.762 (0.762), 0.727 (0.842), and 0.773 (0.773) for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively compared to 0.526 (0.909), 0.619 (0.765), 0.682 (0.882), 0.773 (0.568) for an approach based solely on Dice similarity coefficient values. In the external validation dataset, 66.7, 93.3, 94.1, and 58.8% of flagged cases were confirmed to have contouring errors by an expert for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively. CONCLUSIONS The proposed ML-MF approach, which includes multiple geometric agreement metrics to flag manual contouring errors, demonstrated superior performance in comparison to traditional methods. This method is easy to implement in clinical practice and can help to reduce the significant time and labor costs associated with manual segmentation and review.
Collapse
Affiliation(s)
- Jingwei Duan
- Department of Radiation Medicine, University of Kentucky, Lexington, KY 40506
| | - Mark E. Bernard
- Department of Radiation Medicine, University of Kentucky, Lexington, KY 40506
| | - James R. Castle
- Carina Medical LLC, 145 Graham Ave, A168, Lexington, KY 40506
| | - Xue Feng
- Carina Medical LLC, 145 Graham Ave, A168, Lexington, KY 40506
| | - Chi Wang
- Department of Internal Medicine, University of Kentucky, Lexington, KY 40506
| | - Mark C. Kenamond
- Department of Radiation Medicine, University of Kentucky, Lexington, KY 40506
| | - Quan Chen
- Department of Radiation Medicine, University of Kentucky, Lexington, KY 40506
| |
Collapse
|
13
|
Zhu Z, Wang SH, Zhang YD. A Survey of Convolutional Neural Network in Breast Cancer. COMPUTER MODELING IN ENGINEERING & SCIENCES : CMES 2023; 136:2127-2172. [PMID: 37152661 PMCID: PMC7614504 DOI: 10.32604/cmes.2023.025484] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Accepted: 10/28/2022] [Indexed: 05/09/2023]
Abstract
Problems For people all over the world, cancer is one of the most feared diseases. Cancer is one of the major obstacles to improving life expectancy in countries around the world and one of the biggest causes of death before the age of 70 in 112 countries. Among all kinds of cancers, breast cancer is the most common cancer for women. The data showed that female breast cancer had become one of the most common cancers. Aims A large number of clinical trials have proved that if breast cancer is diagnosed at an early stage, it could give patients more treatment options and improve the treatment effect and survival ability. Based on this situation, there are many diagnostic methods for breast cancer, such as computer-aided diagnosis (CAD). Methods We complete a comprehensive review of the diagnosis of breast cancer based on the convolutional neural network (CNN) after reviewing a sea of recent papers. Firstly, we introduce several different imaging modalities. The structure of CNN is given in the second part. After that, we introduce some public breast cancer data sets. Then, we divide the diagnosis of breast cancer into three different tasks: 1. classification; 2. detection; 3. segmentation. Conclusion Although this diagnosis with CNN has achieved great success, there are still some limitations. (i) There are too few good data sets. A good public breast cancer dataset needs to involve many aspects, such as professional medical knowledge, privacy issues, financial issues, dataset size, and so on. (ii) When the data set is too large, the CNN-based model needs a sea of computation and time to complete the diagnosis. (iii) It is easy to cause overfitting when using small data sets.
Collapse
Affiliation(s)
| | | | - Yu-Dong Zhang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK
| |
Collapse
|
14
|
Claessens M, Oria CS, Brouwer CL, Ziemer BP, Scholey JE, Lin H, Witztum A, Morin O, Naqa IE, Van Elmpt W, Verellen D. Quality Assurance for AI-Based Applications in Radiation Therapy. Semin Radiat Oncol 2022; 32:421-431. [DOI: 10.1016/j.semradonc.2022.06.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
15
|
Sun S, Ren L, Miao Z, Hua L, Wang D, Deng J, Chen J, Liu N, Gong Y. Application of MRI-Based Radiomics in Preoperative Prediction of NF2 Alteration in Intracranial Meningiomas. Front Oncol 2022; 12:879528. [PMID: 36267986 PMCID: PMC9578175 DOI: 10.3389/fonc.2022.879528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2022] [Accepted: 06/06/2022] [Indexed: 11/13/2022] Open
Abstract
PurposeThis study aimed to investigate the feasibility of predicting NF2 mutation status based on the MR radiomic analysis in patients with intracranial meningioma.MethodsThis retrospective study included 105 patients with meningiomas, including 60 NF2-mutant samples and 45 wild-type samples. Radiomic features were extracted from magnetic resonance imaging scans, including T1-weighted, T2-weighted, and contrast T1-weighted images. Student’s t-test and LASSO regression were performed to select the radiomic features. All patients were randomly divided into training and validation cohorts in a 7:3 ratio. Five linear models (RF, SVM, LR, KNN, and xgboost) were trained to predict the NF2 mutational status. Receiver operating characteristic curve and precision-recall analyses were used to evaluate the model performance. Student’s t-tests were then used to compare the posterior probabilities of NF2 mut/loss prediction for patients with different NF2 statuses.ResultsNine features had nonzero coefficients in the LASSO regression model. No significant differences was observed in the clinical features. Nine features showed significant differences in patients with different NF2 statuses. Among all machine learning algorithms, SVM showed the best performance. The area under curve and accuracy of the predictive model were 0.85; the F1-score of the precision-recall curve was 0.80. The model risk was assessed by plotting calibration curves. The p-value for the H-L goodness of fit test was 0.411 (p> 0.05), which indicated that the difference between the obtained model and the perfect model was statistically insignificant. The AUC of our model in external validation was 0.83.ConclusionA combination of radiomic analysis and machine learning showed potential clinical utility in the prediction of preoperative NF2 status. These findings could aid in developing customized neurosurgery plans and meningioma management strategies before postoperative pathology.
Collapse
Affiliation(s)
- Shuchen Sun
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
| | - Leihao Ren
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
| | - Zong Miao
- Department of Neurosurgery, Changhai Hospital, Naval Medical University (Second Military Medical University), Shanghai, China
| | - Lingyang Hua
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
| | - Daijun Wang
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
| | - Jiaojiao Deng
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
| | - Jiawei Chen
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
| | - Ning Liu
- Department of Neurosurgery, Changhai Hospital, Naval Medical University (Second Military Medical University), Shanghai, China
| | - Ye Gong
- Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China
- Institute of Neurosurgery, Fudan University, Shanghai, China
- Shanghai Key Laboratory of Brain Function Restoration and Neural Regeneration, Shanghai, China
- Department of Critical Care Medicine, Huashan Hospital, Fudan University, Shanghai, China
- *Correspondence: Ye Gong,
| |
Collapse
|
16
|
Isaksson LJ, Summers P, Bhalerao A, Gandini S, Raimondi S, Pepa M, Zaffaroni M, Corrao G, Mazzola GC, Rotondi M, Lo Presti G, Haron Z, Alessi S, Pricolo P, Mistretta FA, Luzzago S, Cattani F, Musi G, De Cobelli O, Cremonesi M, Orecchia R, Marvaso G, Petralia G, Jereczek-Fossa BA. Quality assurance for automatically generated contours with additional deep learning. Insights Imaging 2022; 13:137. [PMID: 35976491 PMCID: PMC9385913 DOI: 10.1186/s13244-022-01276-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Accepted: 07/24/2022] [Indexed: 11/10/2022] Open
Abstract
OBJECTIVE Deploying an automatic segmentation model in practice should require rigorous quality assurance (QA) and continuous monitoring of the model's use and performance, particularly in high-stakes scenarios such as healthcare. Currently, however, tools to assist with QA for such models are not available to AI researchers. In this work, we build a deep learning model that estimates the quality of automatically generated contours. METHODS The model was trained to predict the segmentation quality by outputting an estimate of the Dice similarity coefficient given an image contour pair as input. Our dataset contained 60 axial T2-weighted MRI images of prostates with ground truth segmentations along with 80 automatically generated segmentation masks. The model we used was a 3D version of the EfficientDet architecture with a custom regression head. For validation, we used a fivefold cross-validation. To counteract the limitation of the small dataset, we used an extensive data augmentation scheme capable of producing virtually infinite training samples from a single ground truth label mask. In addition, we compared the results against a baseline model that only uses clinical variables for its predictions. RESULTS Our model achieved a mean absolute error of 0.020 ± 0.026 (2.2% mean percentage error) in estimating the Dice score, with a rank correlation of 0.42. Furthermore, the model managed to correctly identify incorrect segmentations (defined in terms of acceptable/unacceptable) 99.6% of the time. CONCLUSION We believe that the trained model can be used alongside automatic segmentation tools to ensure quality and thus allow intervention to prevent undesired segmentation behavior.
Collapse
Affiliation(s)
| | - Paul Summers
- Division of Radiology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Abhir Bhalerao
- Department of Computer Science, University of Warwick, Coventry, Warwick, CV4 7AL, UK
| | - Sara Gandini
- Molecular and Pharmaco-Epidemiology Unit, Department of Experimental Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Sara Raimondi
- Molecular and Pharmaco-Epidemiology Unit, Department of Experimental Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Matteo Pepa
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Mattia Zaffaroni
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Giulia Corrao
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Giovanni Carlo Mazzola
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Marco Rotondi
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Giuliana Lo Presti
- Molecular and Pharmaco-Epidemiology Unit, Department of Experimental Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Zaharudin Haron
- Radiology Department, National Cancer Institute, Putrajaya, Malaysia
| | - Sara Alessi
- Division of Radiology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Paola Pricolo
- Division of Radiology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | | | - Stefano Luzzago
- Division of Urology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Federica Cattani
- Medical Physics Unit, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Gennaro Musi
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,Division of Urology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Ottavio De Cobelli
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,Division of Urology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Marta Cremonesi
- Radiation Research Unit, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Roberto Orecchia
- Scientific Direction, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Giulia Marvaso
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Giuseppe Petralia
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,Precision Imaging and Research Unit, Department of Medical Imaging and Radiation Sciences, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Barbara Alicja Jereczek-Fossa
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| |
Collapse
|
17
|
Yue W, Zhang H, Zhou J, Li G, Tang Z, Sun Z, Cai J, Tian N, Gao S, Dong J, Liu Y, Bai X, Sheng F. Deep learning-based automatic segmentation for size and volumetric measurement of breast cancer on magnetic resonance imaging. Front Oncol 2022; 12:984626. [PMID: 36033453 PMCID: PMC9404224 DOI: 10.3389/fonc.2022.984626] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Accepted: 07/19/2022] [Indexed: 11/30/2022] Open
Abstract
Purpose In clinical work, accurately measuring the volume and the size of breast cancer is significant to develop a treatment plan. However, it is time-consuming, and inter- and intra-observer variations among radiologists exist. The purpose of this study was to assess the performance of a Res-UNet convolutional neural network based on automatic segmentation for size and volumetric measurement of mass enhancement breast cancer on magnetic resonance imaging (MRI). Materials and methods A total of 1,000 female breast cancer patients who underwent preoperative 1.5-T dynamic contrast-enhanced MRI prior to treatment were selected from January 2015 to October 2021 and randomly divided into a training cohort (n = 800) and a testing cohort (n = 200). Compared with the masks named ground truth delineated manually by radiologists, the model performance on segmentation was evaluated with dice similarity coefficient (DSC) and intraclass correlation coefficient (ICC). The performance of tumor (T) stage classification was evaluated with accuracy, sensitivity, and specificity. Results In the test cohort, the DSC of automatic segmentation reached 0.89. Excellent concordance (ICC > 0.95) of the maximal and minimal diameter and good concordance (ICC > 0.80) of volumetric measurement were shown between the model and the radiologists. The trained model took approximately 10–15 s to provide automatic segmentation and classified the T stage with an overall accuracy of 0.93, sensitivity of 0.94, 0.94, and 0.75, and specificity of 0.95, 0.92, and 0.99, respectively, in T1, T2, and T3. Conclusions Our model demonstrated good performance and reliability for automatic segmentation for size and volumetric measurement of breast cancer, which can be time-saving and effective in clinical decision-making.
Collapse
Affiliation(s)
- Wenyi Yue
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
- Chinese PLA General Medical School, Beijing, China
| | - Hongtao Zhang
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Juan Zhou
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Guang Li
- Keya Medical Technology Co., Ltd., Beijing, China
| | - Zhe Tang
- Keya Medical Technology Co., Ltd., Beijing, China
| | - Zeyu Sun
- Keya Medical Technology Co., Ltd., Beijing, China
| | - Jianming Cai
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Ning Tian
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Shen Gao
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Jinghui Dong
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Yuan Liu
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Xu Bai
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Fugeng Sheng
- Department of Radiology, The Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
- *Correspondence: Fugeng Sheng,
| |
Collapse
|
18
|
González PJ, Simões R, Kiers K, Janssen TM. Explaining the dosimetric impact of contouring errors in head and neck radiotherapy. Biomed Phys Eng Express 2022; 8. [PMID: 35732139 DOI: 10.1088/2057-1976/ac7b4c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 06/22/2022] [Indexed: 11/12/2022]
Abstract
Objective. Auto-contouring of organs at risk (OAR) is becoming more common in radiotherapy. An important issue in clinical decision making is judging the quality of the auto-contours. While recent studies considered contour quality by looking at geometric errors only, this does not capture the dosimetric impact of the errors. In this work, we studied the relationship between geometrical errors, the local dose and the dosimetric impact of the geometrical errors.Approach. For 94 head and neck patients, unmodified atlas-based auto-contours and clinically used delineations of the parotid glands and brainstem were retrieved. VMAT plans were automatically optimized on the auto-contours and evaluated on both contours. We defined the dosimetric impact on evaluation (DIE) as the difference in the dosimetric parameter of interest between the two contours. We developed three linear regression models to predict the DIE using: (1) global geometric metrics, (2) global dosimetric metrics, (3) combined local geometric and dosimetric metrics. For model (3), we next determined the minimal amount of editing information required to produce a reliable prediction. Performance was assessed by the root mean squared error (RMSE) of the predicted DIE using 5-fold cross-validation.Main results. In model (3), the median RMSE of the left parotid was 0.4 Gy using 5% of the largest editing vectors. For the right parotid and brainstem the results were 0.5 Gy using 10% and 0.4 Gy using 1% respectively. The median RMS of the DIE was 0.6 Gy, 0.7 Gy and 0.9 Gy for the left parotid, the right parotid and the brainstem, respectively. Model (3), combining local dosimetric and geometric quantities, outperformed the models that used only geometric or dosimetric information.Significance. We showed that the largest local errors plus the local dose suffice to accurately predict the dosimetric impact, opening the door to automated dosimetric QA of auto-contours.
Collapse
Affiliation(s)
- Patrick J González
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Rita Simões
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Karen Kiers
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Tomas M Janssen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| |
Collapse
|
19
|
Claessens M, Vanreusel V, De Kerf G, Mollaert I, Löfman F, Gooding MJ, Brouwer C, Dirix P, Verellen D. Machine learning-based detection of aberrant deep learning segmentations of target and organs at risk for prostate radiotherapy using a secondary segmentation algorithm. Phys Med Biol 2022; 67. [PMID: 35561701 DOI: 10.1088/1361-6560/ac6fad] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Accepted: 05/13/2022] [Indexed: 11/11/2022]
Abstract
Objective.The output of a deep learning (DL) auto-segmentation application should be reviewed, corrected if needed and approved before being used clinically. This verification procedure is labour-intensive, time-consuming and user-dependent, which potentially leads to significant errors with impact on the overall treatment quality. Additionally, when the time needed to correct auto-segmentations approaches the time to delineate target and organs at risk from scratch, the usability of the DL model can be questioned. Therefore, an automated quality assurance framework was developed with the aim to detect in advance aberrant auto-segmentations.Approach. Five organs (prostate, bladder, anorectum, femoral head left and right) were auto-delineated on CT acquisitions for 48 prostate patients by an in-house trained primary DL model. An experienced radiation oncologist assessed the correctness of the model output and categorised the auto-segmentations into two classes whether minor or major adaptations were needed. Subsequently, an independent, secondary DL model was implemented to delineate the same structures as the primary model. Quantitative comparison metrics were calculated using both models' segmentations and used as input features for a machine learning classification model to predict the output quality of the primary model.Main results. For every organ, the approach of independent validation by the secondary model was able to detect primary auto-segmentations that needed major adaptation with high sensitivity (recall = 1) based on the calculated quantitative metrics. The surface DSC and APL were found to be the most indicated parameters in comparison to standard quantitative metrics for the time needed to adapt auto-segmentations.Significance. This proposed method includes a proof of concept for the use of an independent DL segmentation model in combination with a ML classifier to improve time saving during QA of auto-segmentations. The integration of such system into current automatic segmentation pipelines can increase the efficiency of the radiotherapy contouring workflow.
Collapse
Affiliation(s)
- Michaël Claessens
- Department of Radiation Oncology, Iridium Network, Wilrijk (Antwerp), Belgium.,Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Belgium
| | - Verdi Vanreusel
- Department of Radiation Oncology, Iridium Network, Wilrijk (Antwerp), Belgium
| | - Geert De Kerf
- Department of Radiation Oncology, Iridium Network, Wilrijk (Antwerp), Belgium
| | - Isabelle Mollaert
- Department of Radiation Oncology, Iridium Network, Wilrijk (Antwerp), Belgium
| | - Fredrik Löfman
- Department of Machine Learning, RaySearch Laboratories AB, Stockholm, Sweden
| | | | - Charlotte Brouwer
- University of Groningen, University Medical Center Groningen, Department of Radiation Oncology, The Netherlands
| | - Piet Dirix
- Department of Radiation Oncology, Iridium Network, Wilrijk (Antwerp), Belgium.,Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Belgium
| | - Dirk Verellen
- Department of Radiation Oncology, Iridium Network, Wilrijk (Antwerp), Belgium.,Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Belgium
| |
Collapse
|
20
|
Yu P, Wu X, Li J, Mao N, Zhang H, Zheng G, Han X, Dong L, Che K, Wang Q, Li G, Mou Y, Song X. Extrathyroidal Extension Prediction of Papillary Thyroid Cancer With Computed Tomography Based Radiomics Nomogram: A Multicenter Study. Front Endocrinol (Lausanne) 2022; 13:874396. [PMID: 35721715 PMCID: PMC9198261 DOI: 10.3389/fendo.2022.874396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Accepted: 04/27/2022] [Indexed: 12/24/2022] Open
Abstract
OBJECTIVES To develop and validate a Computed Tomography (CT) based radiomics nomogram for preoperative predicting of extrathyroidal extension (ETE) in papillary thyroid cancer (PTC) patients. METHODS A total of 153 patients were randomly assigned to training and internal test sets (7:3). 46 patients were recruited to serve as an external test set. A radiologist with 8 years of experience segmented the images. Radiomics features were extracted from each image and Delta-radiomics features were calculated. Features were selected by using one way analysis of variance and the least absolute shrinkage and selection operator in the training set. K-nearest neighbor, logistic regression, decision tree, linear-support vector machine (linear -SVM), gaussian-SVM, and polynomial-SVM were used to build 6 radiomics models. Next, a radiomics signature score (Rad-score) was constructed by using the linear combination of selected features weighted by their corresponding coefficients. Finally, a nomogram was constructed combining the clinical risk factors with Rad-scores. Receiver operating characteristic (ROC) curve, decision curve analysis (DCA), and calibration curve were performed on the three sets to evaluate the nomogram's performance. RESULTS 4 radiomics features were selected. The six models showed the certain value of radiomics, with area under the curves (AUCs) from 0.642 to 0.701. The nomogram combining the Rad-score and clinical risk factors (radiologists' interpretation) showed good performance (internal test set: AUC 0.750; external test set: AUC 0.797). Calibration curve and DCA demonstrated good performance of the nomogram. CONCLUSION Our radiomics nomogram incorporating the radiomics and radiologists' interpretation has utility in the identification of ETE in PTC patients.
Collapse
Affiliation(s)
- Pengyi Yu
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
| | - Xinxin Wu
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
| | - Jingjing Li
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
| | - Ning Mao
- Big data and Artificial Intelligence Laboratory, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
| | - Haicheng Zhang
- Big data and Artificial Intelligence Laboratory, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
| | - Guibin Zheng
- Department of Thyroid Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
| | - Xiao Han
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
| | - Luchao Dong
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
- Second Clinical Medicine College, Binzhou Medical University, Yantai, Shandong, China
| | - Kaili Che
- Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
| | - Qinglin Wang
- Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
| | - Guan Li
- Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
| | - Yakui Mou
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
- *Correspondence: Xicheng Song, ; Yakui Mou,
| | - Xicheng Song
- Department of Otorhinolaryngology, Head and Neck Surgery, Yantai Yuhuangding Hospital, Qingdao University, Yantai, China
- Shandong Provincial Clinical Research Center for Otorhinolaryngologic Diseases, Yantai, China
- *Correspondence: Xicheng Song, ; Yakui Mou,
| |
Collapse
|
21
|
Yuen AHL, Li AKL, Mak PCY, Leung HL. Implementation of web-based open-source radiotherapy delineation software (WORDS) in organs at risk contouring training for newly qualified radiotherapists: quantitative comparison with conventional one-to-one coaching approach. BMC MEDICAL EDUCATION 2021; 21:564. [PMID: 34749735 PMCID: PMC8573949 DOI: 10.1186/s12909-021-02992-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/04/2021] [Accepted: 10/22/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND Due to the role expansion of radiotherapists in dosimetric aspect, radiotherapists have taken up organs at risk (OARs) contouring work in many clinical settings. However, training of newly qualified radiotherapists in OARs contouring can be time consuming, it may also cause extra burden to experienced radiotherapists. As web-based open-source radiotherapy delineation software (WORDS) has become more readily available, it has provided a free and interactive alternative to conventional one-to-one coaching approach during OARs contouring training. The present study aims to evaluate the effectiveness of WORDS in training OARs contouring skills of newly qualified radiotherapists, compared to those trained by conventional one-to-one coaching approach. METHODS Nine newly qualified radiotherapists (licensed in 2017 - 2018) were enrolled to the conventional one-to-one coaching group (control group), while 11 newly qualified radiotherapists (licensed in 2019 - 2021) were assigned to WORDS training group (measured group). Ten OARs were selected to be contoured in this 3-phases quantitative study. Participants were required to undergo phase 1 OARs contouring in the beginning of the training session. Afterwards, conventional one-to-one training or WORDS training session was provided to participants according to their assigned group. Then the participants did phase 2 and 3 OARs contouring which were separated 1 week apart. Phase 1 - 3 OARs contouring aimed to demonstrate participants' pre-training OARs contouring ability, post-training OARs contouring ability and knowledge retention after one-week interval respectively using either training approach. To prevent bias, the computed tomography dataset for OARs contouring in each phase were different. Variations in the contouring scores for the selected OARs were evaluated between 3 phases using Kruskal-Wallis tests with Dunn tests for pairwise comparisons. Variations in the contouring scores between control and measured group in phase 1 - 3 contouring were analyzed using Wilcoxon signed-rank test. A p-value < 0.05 was considered to be statistically significant. RESULTS In both control group and measured group, significant improvement (p < 0.05) in phase 2 and 3 contouring scores have been observed comparing to phase 1 contouring scores. In comparison of contouring scores between control group and measured group, no significant differences (p > 0.05) were observed in all OARs between both groups. CONCLUSIONS The results in this study have demonstrated that the outcome of OARs contouring training using WORDS is comparable to the conventional training approach. In addition, WORDS can offer flexibility to newly qualified radiotherapists to practice OARs contouring at will, as well as reduce staff training burden of experienced radiotherapists.
Collapse
Affiliation(s)
- Adams Hei Long Yuen
- Oncology Centre, St. Teresa's Hospital, 327 Prince Edward Road, Hong Kong Special Administrative Region, China.
| | - Alex Kai Leung Li
- Oncology Centre, St. Teresa's Hospital, 327 Prince Edward Road, Hong Kong Special Administrative Region, China
| | - Philip Chung Yin Mak
- Oncology Centre, St. Teresa's Hospital, 327 Prince Edward Road, Hong Kong Special Administrative Region, China
| | - Hin Lap Leung
- Oncology Centre, St. Teresa's Hospital, 327 Prince Edward Road, Hong Kong Special Administrative Region, China
| |
Collapse
|
22
|
Min H, Dowling J, Jameson MG, Cloak K, Faustino J, Sidhom M, Martin J, Ebert MA, Haworth A, Chlap P, de Leon J, Berry M, Pryor D, Greer P, Vinod SK, Holloway L. Automatic radiotherapy delineation quality assurance on prostate MRI with deep learning in a multicentre clinical trial. Phys Med Biol 2021; 66. [PMID: 34507305 DOI: 10.1088/1361-6560/ac25d5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Accepted: 09/10/2021] [Indexed: 11/11/2022]
Abstract
Volume delineation quality assurance (QA) is particularly important in clinical trial settings where consistent protocol implementation is required, as outcomes will affect future as well current patients. Currently, where feasible, this is conducted manually, which is time consuming and resource intensive. Although previous studies mostly focused on automating delineation QA on CT, magnetic resonance imaging (MRI) is being increasingly used in radiotherapy treatment. In this work, we propose to perform automatic delineation QA on prostate MRI for both the clinical target volume (CTV) and organs-at-risk (OARs) by using delineations generated by 3D Unet variants as benchmarks for QA. These networks were trained on a small gold standard atlas set and applied on a multicentre radiotherapy clinical trial dataset to generate benchmark delineations. Then, a QA stage was designed to recommend 'pass', 'minor correction' and 'major correction' for each manual delineation in the trial set by thresholding its Dice similarity coefficient to the network generated delineation. Among all 3D Unet variants explored, the Unet with anatomical gates in an AtlasNet architecture performed the best in delineation QA, achieving an area under the receiver operating characteristics curve of 0.97, 0.92, 0.89 and 0.97 for identifying unacceptable (major correction) delineations with a sensitivity of 0.93, 0.73, 0.74 and 0.90 at a specificity of 0.93, 0.86, 0.86 and 0.95 for bladder, prostate CTV, rectum and gel spacer respectively. To the best of our knowledge, this is the first study to propose automated delineation QA for a multicentre radiotherapy clinical trial with treatment planning MRI. The methods proposed in this work can potentially improve the accuracy and consistency of CTV and OAR delineation in radiotherapy treatment planning.
Collapse
Affiliation(s)
- Hang Min
- CSIRO Australian e-Health Research Centre, Herston, Queensland, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,South Western Clinical School, University of New South Wales, Australia
| | - Jason Dowling
- CSIRO Australian e-Health Research Centre, Herston, Queensland, Australia.,South Western Clinical School, University of New South Wales, Australia.,Centre for Medical Radiation Physics, University of Wollongong, New South Wales, Australia.,Institute of Medical Physics, The University of Sydney, New South Wales, Australia.,School of Mathematical and Physical Sciences, University of Newcastle, New South Wales, Australia
| | - Michael G Jameson
- St Vincent's Clinical School, Faculty of Medicine, University of New South Wales, Australia.,GenesisCare, Sydney, New South Wales, Australia
| | - Kirrily Cloak
- Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,South Western Clinical School, University of New South Wales, Australia
| | - Joselle Faustino
- Liverpool and Macarthur Cancer therapy Centres, Liverpool Hospital, New South Wales, Australia
| | - Mark Sidhom
- South Western Clinical School, University of New South Wales, Australia.,Liverpool and Macarthur Cancer therapy Centres, Liverpool Hospital, New South Wales, Australia
| | - Jarad Martin
- Department of Radiation Oncology, Calvary Mater Newcastle, Newcastle, New South Wales, Australia
| | - Martin A Ebert
- Centre for Medical Radiation Physics, University of Wollongong, New South Wales, Australia.,Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, Western Australia, Australia.,School of Physics Mathematics and Computing, University of Western Australia, Perth, Western Australia, Australia
| | - Annette Haworth
- Institute of Medical Physics, The University of Sydney, New South Wales, Australia
| | - Phillip Chlap
- Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,South Western Clinical School, University of New South Wales, Australia.,Liverpool and Macarthur Cancer therapy Centres, Liverpool Hospital, New South Wales, Australia
| | - Jeremiah de Leon
- GenesisCare, Sydney, New South Wales, Australia.,Illawarra Cancer Care Centre, Wollongong, Australia
| | - Megan Berry
- South Western Clinical School, University of New South Wales, Australia.,Liverpool and Macarthur Cancer therapy Centres, Liverpool Hospital, New South Wales, Australia
| | - David Pryor
- Princess Alexandra Hospital, Brisbane, Queensland, Australia
| | - Peter Greer
- School of Mathematical and Physical Sciences, University of Newcastle, New South Wales, Australia.,Department of Radiation Oncology, Calvary Mater Newcastle, Newcastle, New South Wales, Australia
| | - Shalini K Vinod
- Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,South Western Clinical School, University of New South Wales, Australia.,Liverpool and Macarthur Cancer therapy Centres, Liverpool Hospital, New South Wales, Australia
| | - Lois Holloway
- Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,South Western Clinical School, University of New South Wales, Australia.,Centre for Medical Radiation Physics, University of Wollongong, New South Wales, Australia.,Institute of Medical Physics, The University of Sydney, New South Wales, Australia.,Liverpool and Macarthur Cancer therapy Centres, Liverpool Hospital, New South Wales, Australia
| |
Collapse
|
23
|
Robert C, Munoz A, Moreau D, Mazurier J, Sidorski G, Gasnier A, Beldjoudi G, Grégoire V, Deutsch E, Meyer P, Simon L. Clinical implementation of deep-learning based auto-contouring tools-Experience of three French radiotherapy centers. Cancer Radiother 2021; 25:607-616. [PMID: 34389243 DOI: 10.1016/j.canrad.2021.06.023] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Revised: 06/17/2021] [Accepted: 06/18/2021] [Indexed: 12/23/2022]
Abstract
Deep-learning (DL)-based auto-contouring solutions have recently been proposed as a convincing alternative to decrease workload of target volumes and organs-at-risk (OAR) delineation in radiotherapy planning and improve inter-observer consistency. However, there is minimal literature of clinical implementations of such algorithms in a clinical routine. In this paper we first present an update of the state-of-the-art of DL-based solutions. We then summarize recent recommendations proposed by the European society for radiotherapy and oncology (ESTRO) to be followed before any clinical implementation of artificial intelligence-based solutions in clinic. The last section describes the methodology carried out by three French radiation oncology departments to deploy CE-marked commercial solutions. Based on the information collected, a majority of OAR are retained by the centers among those proposed by the manufacturers, validating the usefulness of DL-based models to decrease clinicians' workload. Target volumes, with the exception of lymph node areas in breast, head and neck and pelvic regions, whole breast, breast wall, prostate and seminal vesicles, are not available in the three commercial solutions at this time. No implemented workflows are currently available to continuously improve the models, but these can be adapted/retrained in some solutions during the commissioning phase to best fit local practices. In reported experiences, automatic workflows were implemented to limit human interactions and make the workflow more fluid. Recommendations published by the ESTRO group will be of importance for guiding physicists in the clinical implementation of patient specific and regular quality assurances.
Collapse
Affiliation(s)
- C Robert
- Department of Radiotherapy, Gustave-Roussy, Villejuif, France.
| | - A Munoz
- Department of Radiotherapy, Centre Léon-Bérard, Lyon, France
| | - D Moreau
- Department of Radiotherapy, Hôpital Européen Georges-Pompidou, Paris, France
| | - J Mazurier
- Department of Radiotherapy, Clinique Pasteur-Oncorad, Toulouse, France
| | - G Sidorski
- Department of Radiotherapy, Clinique Pasteur-Oncorad, Toulouse, France
| | - A Gasnier
- Department of Radiotherapy, Gustave-Roussy, Villejuif, France
| | - G Beldjoudi
- Department of Radiotherapy, Centre Léon-Bérard, Lyon, France
| | - V Grégoire
- Department of Radiotherapy, Centre Léon-Bérard, Lyon, France
| | - E Deutsch
- Department of Radiotherapy, Gustave-Roussy, Villejuif, France
| | - P Meyer
- Service d'Oncologie Radiothérapie, Institut de Cancérologie Strasbourg Europe (Icans), Strasbourg, France
| | - L Simon
- Institut Claudius Regaud (ICR), Institut Universitaire du Cancer de Toulouse - Oncopole (IUCT-O), Toulouse, France
| |
Collapse
|
24
|
Nijhuis H, van Rooij W, Gregoire V, Overgaard J, Slotman BJ, Verbakel WF, Dahele M. Investigating the potential of deep learning for patient-specific quality assurance of salivary gland contours using EORTC-1219-DAHANCA-29 clinical trial data. Acta Oncol 2021; 60:575-581. [PMID: 33427555 DOI: 10.1080/0284186x.2020.1863463] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
INTRODUCTION Manual quality assurance (QA) of radiotherapy contours for clinical trials is time and labor intensive and subject to inter-observer variability. Therefore, we investigated whether deep-learning (DL) can provide an automated solution to salivary gland contour QA. MATERIAL AND METHODS DL-models were trained to generate contours for parotid (PG) and submandibular glands (SMG). Sørensen-Dice coefficient (SDC) and Hausdorff distance (HD) were used to assess agreement between DL and clinical contours and thresholds were defined to highlight cases as potentially sub-optimal. 3 types of deliberate errors (expansion, contraction and displacement) were gradually applied to a test set, to confirm that SDC and HD were suitable QA metrics. DL-based QA was performed on 62 patients from the EORTC-1219-DAHANCA-29 trial. All highlighted contours were visually inspected. RESULTS Increasing the magnitude of all 3 types of errors resulted in progressively severe deterioration/increase in average SDC/HD. 19/124 clinical PG contours were highlighted as potentially sub-optimal, of which 5 (26%) were actually deemed clinically sub-optimal. 2/19 non-highlighted contours were false negatives (11%). 15/69 clinical SMG contours were highlighted, with 7 (47%) deemed clinically sub-optimal and 2/15 non-highlighted contours were false negatives (13%). For most incorrectly highlighted contours causes for low agreement could be identified. CONCLUSION Automated DL-based contour QA is feasible but some visual inspection remains essential. The substantial number of false positives were caused by sub-optimal performance of the DL-model. Improvements to the model will increase the extent of automation and reliability, facilitating the adoption of DL-based contour QA in clinical trials and routine practice.
Collapse
Affiliation(s)
- Hanne Nijhuis
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Ward van Rooij
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Vincent Gregoire
- Department of Radiation Oncology, Centre Leon Berard, Lyon, France
| | - Jens Overgaard
- Department of Clinical Medicine – Department of Experimental Clinical Oncology, Aarhus University, Aarhus N, Denmark
| | - Berend J. Slotman
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Wilko F. Verbakel
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Max Dahele
- Department of Radiation Oncology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
25
|
Chung SY, Chang JS, Choi MS, Chang Y, Choi BS, Chun J, Keum KC, Kim JS, Kim YB. Clinical feasibility of deep learning-based auto-segmentation of target volumes and organs-at-risk in breast cancer patients after breast-conserving surgery. Radiat Oncol 2021; 16:44. [PMID: 33632248 PMCID: PMC7905884 DOI: 10.1186/s13014-021-01771-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Accepted: 02/11/2021] [Indexed: 02/07/2023] Open
Abstract
Background In breast cancer patients receiving radiotherapy (RT), accurate target delineation and reduction of radiation doses to the nearby normal organs is important. However, manual clinical target volume (CTV) and organs-at-risk (OARs) segmentation for treatment planning increases physicians’ workload and inter-physician variability considerably. In this study, we evaluated the potential benefits of deep learning-based auto-segmented contours by comparing them to manually delineated contours for breast cancer patients. Methods CTVs for bilateral breasts, regional lymph nodes, and OARs (including the heart, lungs, esophagus, spinal cord, and thyroid) were manually delineated on planning computed tomography scans of 111 breast cancer patients who received breast-conserving surgery. Subsequently, a two-stage convolutional neural network algorithm was used. Quantitative metrics, including the Dice similarity coefficient (DSC) and 95% Hausdorff distance, and qualitative scoring by two panels from 10 institutions were used for analysis. Inter-observer variability and delineation time were assessed; furthermore, dose-volume histograms and dosimetric parameters were also analyzed using another set of patient data. Results The correlation between the auto-segmented and manual contours was acceptable for OARs, with a mean DSC higher than 0.80 for all OARs. In addition, the CTVs showed favorable results, with mean DSCs higher than 0.70 for all breast and regional lymph node CTVs. Furthermore, qualitative subjective scoring showed that the results were acceptable for all CTVs and OARs, with a median score of at least 8 (possible range: 0–10) for (1) the differences between manual and auto-segmented contours and (2) the extent to which auto-segmentation would assist physicians in clinical practice. The differences in dosimetric parameters between the auto-segmented and manual contours were minimal. Conclusions The feasibility of deep learning-based auto-segmentation in breast RT planning was demonstrated. Although deep learning-based auto-segmentation cannot be a substitute for radiation oncologists, it is a useful tool with excellent potential in assisting radiation oncologists in the future. Trial registration Retrospectively registered.
Collapse
Affiliation(s)
- Seung Yeun Chung
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea.,Department of Radiation Oncology, Ajou University School of Medicine, Suwon, Korea
| | - Jee Suk Chang
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea.
| | - Min Seo Choi
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| | | | - Byong Su Choi
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| | - Jaehee Chun
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| | - Ki Chang Keum
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| | - Jin Sung Kim
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea.
| | - Yong Bae Kim
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| |
Collapse
|
26
|
Liu Z, Liu X, Guan H, Zhen H, Sun Y, Chen Q, Chen Y, Wang S, Qiu J. Development and validation of a deep learning algorithm for auto-delineation of clinical target volume and organs at risk in cervical cancer radiotherapy. Radiother Oncol 2020; 153:172-179. [DOI: 10.1016/j.radonc.2020.09.060] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2020] [Revised: 09/28/2020] [Accepted: 09/28/2020] [Indexed: 12/14/2022]
|
27
|
Zadeh Shirazi A, Fornaciari E, McDonnell MD, Yaghoobi M, Cevallos Y, Tello-Oquendo L, Inca D, Gomez GA. The Application of Deep Convolutional Neural Networks to Brain Cancer Images: A Survey. J Pers Med 2020; 10:E224. [PMID: 33198332 PMCID: PMC7711876 DOI: 10.3390/jpm10040224] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Revised: 11/10/2020] [Accepted: 11/10/2020] [Indexed: 12/15/2022] Open
Abstract
In recent years, improved deep learning techniques have been applied to biomedical image processing for the classification and segmentation of different tumors based on magnetic resonance imaging (MRI) and histopathological imaging (H&E) clinical information. Deep Convolutional Neural Networks (DCNNs) architectures include tens to hundreds of processing layers that can extract multiple levels of features in image-based data, which would be otherwise very difficult and time-consuming to be recognized and extracted by experts for classification of tumors into different tumor types, as well as segmentation of tumor images. This article summarizes the latest studies of deep learning techniques applied to three different kinds of brain cancer medical images (histology, magnetic resonance, and computed tomography) and highlights current challenges in the field for the broader applicability of DCNN in personalized brain cancer care by focusing on two main applications of DCNNs: classification and segmentation of brain cancer tumors images.
Collapse
Affiliation(s)
- Amin Zadeh Shirazi
- Centre for Cancer Biology, SA Pathology and the University of South of Australia, Adelaide, SA 5000, Australia;
- Computational Learning Systems Laboratory, UniSA STEM, University of South Australia, Mawson Lakes, SA 5095, Australia;
| | - Eric Fornaciari
- Department of Mathematics of Computation, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA;
| | - Mark D. McDonnell
- Computational Learning Systems Laboratory, UniSA STEM, University of South Australia, Mawson Lakes, SA 5095, Australia;
| | - Mahdi Yaghoobi
- Electrical and Computer Engineering Department, Islamic Azad University, Mashhad Branch, Mashad 917794-8564, Iran;
| | - Yesenia Cevallos
- College of Engineering, Universidad Nacional de Chimborazo, Riobamba 060150, Ecuador; (Y.C.); (L.T.-O.); (D.I.)
| | - Luis Tello-Oquendo
- College of Engineering, Universidad Nacional de Chimborazo, Riobamba 060150, Ecuador; (Y.C.); (L.T.-O.); (D.I.)
| | - Deysi Inca
- College of Engineering, Universidad Nacional de Chimborazo, Riobamba 060150, Ecuador; (Y.C.); (L.T.-O.); (D.I.)
| | - Guillermo A. Gomez
- Centre for Cancer Biology, SA Pathology and the University of South of Australia, Adelaide, SA 5000, Australia;
| |
Collapse
|