1
|
Erdur AC, Rusche D, Scholz D, Kiechle J, Fischer S, Llorián-Salvador Ó, Buchner JA, Nguyen MQ, Etzel L, Weidner J, Metz MC, Wiestler B, Schnabel J, Rueckert D, Combs SE, Peeken JC. Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives. Strahlenther Onkol 2024:10.1007/s00066-024-02262-2. [PMID: 39105745 DOI: 10.1007/s00066-024-02262-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 06/13/2024] [Indexed: 08/07/2024]
Abstract
The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.
Collapse
Affiliation(s)
- Ayhan Can Erdur
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany.
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany.
| | - Daniel Rusche
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Daniel Scholz
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Johannes Kiechle
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute for Computational Imaging and AI in Medicine, Technical University of Munich, Lichtenberg Str. 2a, 85748, Garching, Bavaria, Germany
- Munich Center for Machine Learning (MCML), Technical University of Munich, Arcisstraße 21, 80333, Munich, Bavaria, Germany
- Konrad Zuse School of Excellence in Reliable AI (relAI), Technical University of Munich, Walther-von-Dyck-Straße 10, 85748, Garching, Bavaria, Germany
| | - Stefan Fischer
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute for Computational Imaging and AI in Medicine, Technical University of Munich, Lichtenberg Str. 2a, 85748, Garching, Bavaria, Germany
- Munich Center for Machine Learning (MCML), Technical University of Munich, Arcisstraße 21, 80333, Munich, Bavaria, Germany
| | - Óscar Llorián-Salvador
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Department for Bioinformatics and Computational Biology - i12, Technical University of Munich, Boltzmannstraße 3, 85748, Garching, Bavaria, Germany
- Institute of Organismic and Molecular Evolution, Johannes Gutenberg University Mainz (JGU), Hüsch-Weg 15, 55128, Mainz, Rhineland-Palatinate, Germany
| | - Josef A Buchner
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Mai Q Nguyen
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Lucas Etzel
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute of Radiation Medicine (IRM), Helmholtz Zentrum, Ingolstädter Landstraße 1, 85764, Oberschleißheim, Bavaria, Germany
| | - Jonas Weidner
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Marie-Christin Metz
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Benedikt Wiestler
- Department of Neuroradiology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
| | - Julia Schnabel
- Institute for Computational Imaging and AI in Medicine, Technical University of Munich, Lichtenberg Str. 2a, 85748, Garching, Bavaria, Germany
- Munich Center for Machine Learning (MCML), Technical University of Munich, Arcisstraße 21, 80333, Munich, Bavaria, Germany
- Konrad Zuse School of Excellence in Reliable AI (relAI), Technical University of Munich, Walther-von-Dyck-Straße 10, 85748, Garching, Bavaria, Germany
- Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Ingolstädter Landstraße 1, 85764, Neuherberg, Bavaria, Germany
- School of Biomedical Engineering & Imaging Sciences, King's College London, Strand, WC2R 2LS, London, London, UK
| | - Daniel Rueckert
- Institute for Artificial Intelligence and Informatics in Medicine, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Faculty of Engineering, Department of Computing, Imperial College London, Exhibition Rd, SW7 2BX, London, London, UK
| | - Stephanie E Combs
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute of Radiation Medicine (IRM), Helmholtz Zentrum, Ingolstädter Landstraße 1, 85764, Oberschleißheim, Bavaria, Germany
- Partner Site Munich, German Consortium for Translational Cancer Research (DKTK), Munich, Bavaria, Germany
| | - Jan C Peeken
- Department of Radiation Oncology, TUM School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Ismaninger Str., 81675, Munich, Bavaria, Germany
- Institute of Radiation Medicine (IRM), Helmholtz Zentrum, Ingolstädter Landstraße 1, 85764, Oberschleißheim, Bavaria, Germany
- Partner Site Munich, German Consortium for Translational Cancer Research (DKTK), Munich, Bavaria, Germany
| |
Collapse
|
2
|
Lagedamon V, Leni PE, Gschwind R. Deep learning applied to dose prediction in external radiation therapy: A narrative review. Cancer Radiother 2024; 28:402-414. [PMID: 39138047 DOI: 10.1016/j.canrad.2024.03.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2024] [Revised: 03/28/2024] [Accepted: 03/29/2024] [Indexed: 08/15/2024]
Abstract
Over the last decades, the use of artificial intelligence, machine learning and deep learning in medical fields has skyrocketed. Well known for their results in segmentation, motion management and posttreatment outcome tasks, investigations of machine learning and deep learning models as fast dose calculation or quality assurance tools have been present since 2000. The main motivation for this increasing research and interest in artificial intelligence, machine learning and deep learning is the enhancement of treatment workflows, specifically dosimetry and quality assurance accuracy and time points, which remain important time-consuming aspects of clinical patient management. Since 2014, the evolution of models and architectures for dose calculation has been related to innovations and interest in the theory of information research with pronounced improvements in architecture design. The use of knowledge-based approaches to patient-specific methods has also considerably improved the accuracy of dose predictions. This paper covers the state of all known deep learning architectures and models applied to external radiotherapy with a description of each architecture, followed by a discussion on the performance and future of deep learning predictive models in external radiotherapy.
Collapse
Affiliation(s)
- V Lagedamon
- Laboratoire chronoenvironnement, UMR 6249, université de Franche-Comté, CNRS, 4, place Tharradin, 25200 Montbéliard, France
| | - P-E Leni
- Laboratoire chronoenvironnement, UMR 6249, université de Franche-Comté, CNRS, 4, place Tharradin, 25200 Montbéliard, France.
| | - R Gschwind
- Laboratoire chronoenvironnement, UMR 6249, université de Franche-Comté, CNRS, 4, place Tharradin, 25200 Montbéliard, France
| |
Collapse
|
3
|
Ghadimi DJ, Vahdani AM, Karimi H, Ebrahimi P, Fathi M, Moodi F, Habibzadeh A, Khodadadi Shoushtari F, Valizadeh G, Mobarak Salari H, Saligheh Rad H. Deep Learning-Based Techniques in Glioma Brain Tumor Segmentation Using Multi-Parametric MRI: A Review on Clinical Applications and Future Outlooks. J Magn Reson Imaging 2024. [PMID: 39074952 DOI: 10.1002/jmri.29543] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Revised: 07/07/2024] [Accepted: 07/08/2024] [Indexed: 07/31/2024] Open
Abstract
This comprehensive review explores the role of deep learning (DL) in glioma segmentation using multiparametric magnetic resonance imaging (MRI) data. The study surveys advanced techniques such as multiparametric MRI for capturing the complex nature of gliomas. It delves into the integration of DL with MRI, focusing on convolutional neural networks (CNNs) and their remarkable capabilities in tumor segmentation. Clinical applications of DL-based segmentation are highlighted, including treatment planning, monitoring treatment response, and distinguishing between tumor progression and pseudo-progression. Furthermore, the review examines the evolution of DL-based segmentation studies, from early CNN models to recent advancements such as attention mechanisms and transformer models. Challenges in data quality, gradient vanishing, and model interpretability are discussed. The review concludes with insights into future research directions, emphasizing the importance of addressing tumor heterogeneity, integrating genomic data, and ensuring responsible deployment of DL-driven healthcare technologies. EVIDENCE LEVEL: N/A TECHNICAL EFFICACY: Stage 2.
Collapse
Affiliation(s)
- Delaram J Ghadimi
- School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Amir M Vahdani
- Image Guided Surgery Lab, Research Center for Biomedical Technologies and Robotics, Advanced Medical Technologies and Equipment Institute, Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran
| | - Hanie Karimi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Pouya Ebrahimi
- Cardiovascular Diseases Research Institute, Tehran Heart Center, Tehran University of Medical Sciences, Tehran, Iran
| | - Mobina Fathi
- School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Farzan Moodi
- School of Medicine, Iran University of Medical Sciences, Tehran, Iran
- Quantitative MR Imaging and Spectroscopy Group (QMISG), Tehran University of Medical Sciences, Tehran, Iran
| | - Adrina Habibzadeh
- Student Research Committee, Fasa University of Medical Sciences, Fasa, Iran
| | | | - Gelareh Valizadeh
- Quantitative MR Imaging and Spectroscopy Group (QMISG), Tehran University of Medical Sciences, Tehran, Iran
| | - Hanieh Mobarak Salari
- Quantitative MR Imaging and Spectroscopy Group (QMISG), Tehran University of Medical Sciences, Tehran, Iran
| | - Hamidreza Saligheh Rad
- Quantitative MR Imaging and Spectroscopy Group (QMISG), Tehran University of Medical Sciences, Tehran, Iran
- Department of Medical Physics and Biomedical Engineering, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
4
|
Radici L, Piva C, Casanova Borca V, Cante D, Ferrario S, Paolini M, Cabras L, Petrucci E, Franco P, La Porta MR, Pasquino M. Clinical evaluation of a deep learning CBCT auto-segmentation software for prostate adaptive radiation therapy. Clin Transl Radiat Oncol 2024; 47:100796. [PMID: 38884004 PMCID: PMC11176659 DOI: 10.1016/j.ctro.2024.100796] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 05/09/2024] [Accepted: 05/16/2024] [Indexed: 06/18/2024] Open
Abstract
Purpose Aim of the present study is to characterize a deep learning-based auto-segmentation software (DL) for prostate cone beam computed tomography (CBCT) images and to evaluate its applicability in clinical adaptive radiation therapy routine. Materials and methods Ten patients, who received exclusive radiation therapy with definitive intent on the prostate gland and seminal vesicles, were selected. Femoral heads, bladder, rectum, prostate, and seminal vesicles were retrospectively contoured by four different expert radiation oncologists on patients CBCT, acquired during treatment. Consensus contours (CC) were generated starting from these data and compared with those created by DL with different algorithms, trained on CBCT (DL-CBCT) or computed tomography (DL-CT). Dice similarity coefficient (DSC), centre of mass (COM) shift and volume relative variation (VRV) were chosen as comparison metrics. Since no tolerance limit can be defined, results were also compared with the inter-operator variability (IOV), using the same metrics. Results The best agreement between DL and CC was observed for femoral heads (DSC of 0.96 for both DL-CBCT and DL-CT). Performance worsened for low-contrast soft tissue organs: the worst results were found for seminal vesicles (DSC of 0.70 and 0.59 for DL-CBCT and DL-CT, respectively). The analysis shows that it is appropriate to use algorithms trained on the specific imaging modality. Furthermore, the statistical analysis showed that, for almost all considered structures, there is no significant difference between DL-CBCT and human operator in terms of IOV. Conclusions The accuracy of DL-CBCT is in accordance with CC; its use in clinical practice is justified by the comparison with the inter-operator variability.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Laura Cabras
- Medical Physics Department, ASL TO4 Ivrea, Italy
| | | | - Pierfrancesco Franco
- Department of Translational Sciences (DIMET), University of Eastern Piedmont, Novara, Italy
- Department of Radiation Oncology, 'Maggiore della Carità' University Hospital, Novara, Italy
| | | | | |
Collapse
|
5
|
Finnegan RN, Quinn A, Booth J, Belous G, Hardcastle N, Stewart M, Griffiths B, Carroll S, Thwaites DI. Cardiac substructure delineation in radiation therapy - A state-of-the-art review. J Med Imaging Radiat Oncol 2024. [PMID: 38757728 DOI: 10.1111/1754-9485.13668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 04/29/2024] [Indexed: 05/18/2024]
Abstract
Delineation of cardiac substructures is crucial for a better understanding of radiation-related cardiotoxicities and to facilitate accurate and precise cardiac dose calculation for developing and applying risk models. This review examines recent advancements in cardiac substructure delineation in the radiation therapy (RT) context, aiming to provide a comprehensive overview of the current level of knowledge, challenges and future directions in this evolving field. Imaging used for RT planning presents challenges in reliably visualising cardiac anatomy. Although cardiac atlases and contouring guidelines aid in standardisation and reduction of variability, significant uncertainties remain in defining cardiac anatomy. Coupled with the inherent complexity of the heart, this necessitates auto-contouring for consistent large-scale data analysis and improved efficiency in prospective applications. Auto-contouring models, developed primarily for breast and lung cancer RT, have demonstrated performance comparable to manual contouring, marking a significant milestone in the evolution of cardiac delineation practices. Nevertheless, several key concerns require further investigation. There is an unmet need for expanding cardiac auto-contouring models to encompass a broader range of cancer sites. A shift in focus is needed from ensuring accuracy to enhancing the robustness and accessibility of auto-contouring models. Addressing these challenges is paramount for the integration of cardiac substructure delineation and associated risk models into routine clinical practice, thereby improving the safety of RT for future cancer patients.
Collapse
Affiliation(s)
- Robert N Finnegan
- Northern Sydney Cancer Centre, Royal North Shore Hospital, Sydney, New South Wales, Australia
- Institute of Medical Physics, School of Physics, University of Sydney, Sydney, New South Wales, Australia
| | - Alexandra Quinn
- Northern Sydney Cancer Centre, Royal North Shore Hospital, Sydney, New South Wales, Australia
| | - Jeremy Booth
- Northern Sydney Cancer Centre, Royal North Shore Hospital, Sydney, New South Wales, Australia
- Institute of Medical Physics, School of Physics, University of Sydney, Sydney, New South Wales, Australia
| | - Gregg Belous
- Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation, Brisbane, Queensland, Australia
| | - Nicholas Hardcastle
- Department of Physical Sciences, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
- Sir Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria, Australia
| | - Maegan Stewart
- Northern Sydney Cancer Centre, Royal North Shore Hospital, Sydney, New South Wales, Australia
- School of Health Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia
| | - Brooke Griffiths
- Northern Sydney Cancer Centre, Royal North Shore Hospital, Sydney, New South Wales, Australia
| | - Susan Carroll
- Northern Sydney Cancer Centre, Royal North Shore Hospital, Sydney, New South Wales, Australia
- School of Health Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia
| | - David I Thwaites
- Institute of Medical Physics, School of Physics, University of Sydney, Sydney, New South Wales, Australia
- Radiotherapy Research Group, Leeds Institute of Medical Research, St James's Hospital and University of Leeds, Leeds, UK
| |
Collapse
|
6
|
Zhang L, Liu Z, Zhang L, Wu Z, Yu X, Holmes J, Feng H, Dai H, Li X, Li Q, Wong WW, Vora SA, Zhu D, Liu T, Liu W. Technical Note: Generalizable and Promptable Artificial Intelligence Model to Augment Clinical Delineation in Radiation Oncology. Med Phys 2024; 51:2187-2199. [PMID: 38319676 PMCID: PMC10939804 DOI: 10.1002/mp.16965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 12/29/2023] [Accepted: 01/14/2024] [Indexed: 03/13/2024] Open
Abstract
BACKGROUND Efficient and accurate delineation of organs at risk (OARs) is a critical procedure for treatment planning and dose evaluation. Deep learning-based auto-segmentation of OARs has shown promising results and is increasingly being used in radiation therapy. However, existing deep learning-based auto-segmentation approaches face two challenges in clinical practice: generalizability and human-AI interaction. A generalizable and promptable auto-segmentation model, which segments OARs of multiple disease sites simultaneously and supports on-the-fly human-AI interaction, can significantly enhance the efficiency of radiation therapy treatment planning. PURPOSE Meta's segment anything model (SAM) was proposed as a generalizable and promptable model for next-generation natural image segmentation. We further evaluated the performance of SAM in radiotherapy segmentation. METHODS Computed tomography (CT) images of clinical cases from four disease sites at our institute were collected: prostate, lung, gastrointestinal, and head & neck. For each case, we selected the OARs important in radiotherapy treatment planning. We then compared both the Dice coefficients and Jaccard indices derived from three distinct methods: manual delineation (ground truth), automatic segmentation using SAM's 'segment anything' mode, and automatic segmentation using SAM's 'box prompt' mode that implements manual interaction via live prompts during segmentation. RESULTS Our results indicate that SAM's segment anything mode can achieve clinically acceptable segmentation results in most OARs with Dice scores higher than 0.7. SAM's box prompt mode further improves Dice scores by 0.1∼0.5. Similar results were observed for Jaccard indices. The results show that SAM performs better for prostate and lung, but worse for gastrointestinal and head & neck. When considering the size of organs and the distinctiveness of their boundaries, SAM shows better performance for large organs with distinct boundaries, such as lung and liver, and worse for smaller organs with less distinct boundaries, like parotid and cochlea. CONCLUSIONS Our results demonstrate SAM's robust generalizability with consistent accuracy in automatic segmentation for radiotherapy. Furthermore, the advanced box-prompt method enables the users to augment auto-segmentation interactively and dynamically, leading to patient-specific auto-segmentation in radiation therapy. SAM's generalizability across different disease sites and different modalities makes it feasible to develop a generic auto-segmentation model in radiotherapy.
Collapse
Affiliation(s)
- Lian Zhang
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Zhengliang Liu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Lu Zhang
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
| | - Zihao Wu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Xiaowei Yu
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
| | - Jason Holmes
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Hongying Feng
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Haixing Dai
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Xiang Li
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02115, USA
| | - Quanzheng Li
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02115, USA
| | - William W. Wong
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Sujay A. Vora
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Dajiang Zhu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Tianming Liu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Wei Liu
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| |
Collapse
|
7
|
Ao Y, Shi W, Ji B, Miao Y, He W, Jiang Z. MS-TCNet: An effective Transformer-CNN combined network using multi-scale feature learning for 3D medical image segmentation. Comput Biol Med 2024; 170:108057. [PMID: 38301516 DOI: 10.1016/j.compbiomed.2024.108057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 12/31/2023] [Accepted: 01/26/2024] [Indexed: 02/03/2024]
Abstract
Medical image segmentation is a fundamental research problem in the field of medical image processing. Recently, the Transformer have achieved highly competitive performance in computer vision. Therefore, many methods combining Transformer with convolutional neural networks (CNNs) have emerged for segmenting medical images. However, these methods cannot effectively capture the multi-scale features in medical images, even though texture and contextual information embedded in the multi-scale features are extremely beneficial for segmentation. To alleviate this limitation, we propose a novel Transformer-CNN combined network using multi-scale feature learning for three-dimensional (3D) medical image segmentation, which is called MS-TCNet. The proposed model utilizes a shunted Transformer and CNN to construct an encoder and pyramid decoder, allowing six different scale levels of feature learning. It captures multi-scale features with refinement at each scale level. Additionally, we propose a novel lightweight multi-scale feature fusion (MSFF) module that can fully fuse the different-scale semantic features generated by the pyramid decoder for each segmentation class, resulting in a more accurate segmentation output. We conducted experiments on three widely used 3D medical image segmentation datasets. The experimental results indicated that our method outperformed state-of-the-art medical image segmentation methods, suggesting its effectiveness, robustness, and superiority. Meanwhile, our model has a smaller number of parameters and lower computational complexity than conventional 3D segmentation networks. The results confirmed that the model is capable of effective multi-scale feature learning and that the learned multi-scale features are useful for improving segmentation performance. We open-sourced our code, which can be found at https://github.com/AustinYuAo/MS-TCNet.
Collapse
Affiliation(s)
- Yu Ao
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China
| | - Weili Shi
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Bai Ji
- Department of Hepatobiliary and Pancreatic Surgery, The First Hospital of Jilin University, Changchun, 130061, China
| | - Yu Miao
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Wei He
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Zhengang Jiang
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China.
| |
Collapse
|
8
|
Quinton F, Presles B, Leclerc S, Nodari G, Lopez O, Chevallier O, Pellegrinelli J, Vrigneaud JM, Popoff R, Meriaudeau F, Alberini JL. Navigating the nuances: comparative analysis and hyperparameter optimisation of neural architectures on contrast-enhanced MRI for liver and liver tumour segmentation. Sci Rep 2024; 14:3522. [PMID: 38347017 PMCID: PMC10861452 DOI: 10.1038/s41598-024-53528-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 02/01/2024] [Indexed: 02/15/2024] Open
Abstract
In medical imaging, accurate segmentation is crucial to improving diagnosis, treatment, or both. However, navigating the multitude of available architectures for automatic segmentation can be overwhelming, making it challenging to determine the appropriate type of architecture and tune the most crucial parameters during dataset optimisation. To address this problem, we examined and refined seven distinct architectures for segmenting the liver, as well as liver tumours, with a restricted training collection of 60 3D contrast-enhanced magnetic resonance images (CE-MRI) from the ATLAS dataset. Included in these architectures are convolutional neural networks (CNNs), transformers, and hybrid CNN/transformer architectures. Bayesian search techniques were used for hyperparameter tuning to hasten convergence to the optimal parameter mixes while also minimising the number of trained models. It was unexpected that hybrid models, which typically exhibit superior performance on larger datasets, would exhibit comparable performance to CNNs. The optimisation of parameters contributed to better segmentations, resulting in an average increase of 1.7% and 5.0% in liver and tumour segmentation Dice coefficients, respectively. In conclusion, the findings of this study indicate that hybrid CNN/transformer architectures may serve as a practical substitute for CNNs even in small datasets. This underscores the significance of hyperparameter optimisation.
Collapse
Affiliation(s)
- Felix Quinton
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France.
| | - Benoit Presles
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
| | - Sarah Leclerc
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
| | - Guillaume Nodari
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Olivier Lopez
- Service de Radiologie et Imagerie Medicale Diagnostique et Therapeutique, Centre Hospitalier Universitaire, 21000, Dijon, France
| | - Olivier Chevallier
- Service de Radiologie et Imagerie Medicale Diagnostique et Therapeutique, Centre Hospitalier Universitaire, 21000, Dijon, France
| | - Julie Pellegrinelli
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Jean-Marc Vrigneaud
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Romain Popoff
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Fabrice Meriaudeau
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
| | - Jean-Louis Alberini
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| |
Collapse
|
9
|
Fuchs M, Gonzalez C, Frisch Y, Hahn P, Matthies P, Gruening M, Pinto Dos Santos D, Dratsch T, Kim M, Nensa F, Trenz M, Mukhopadhyay A. Closing the loop for AI-ready radiology. ROFO-FORTSCHR RONTG 2024; 196:154-162. [PMID: 37582385 DOI: 10.1055/a-2124-1958] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/17/2023]
Abstract
BACKGROUND In recent years, AI has made significant advancements in medical diagnosis and prognosis. However, the incorporation of AI into clinical practice is still challenging and under-appreciated. We aim to demonstrate a possible vertical integration approach to close the loop for AI-ready radiology. METHOD This study highlights the importance of two-way communication for AI-assisted radiology. As a key part of the methodology, it demonstrates the integration of AI systems into clinical practice with structured reports and AI visualization, giving more insight into the AI system. By integrating cooperative lifelong learning into the AI system, we ensure the long-term effectiveness of the AI system, while keeping the radiologist in the loop. RESULTS: We demonstrate the use of lifelong learning for AI systems by incorporating AI visualization and structured reports. We evaluate Memory Aware-Synapses and Rehearsal approach and find that both approaches work in practice. Furthermore, we see the advantage of lifelong learning algorithms that do not require the storing or maintaining of samples from previous datasets. CONCLUSION In conclusion, incorporating AI into the clinical routine of radiology requires a two-way communication approach and seamless integration of the AI system, which we achieve with structured reports and visualization of the insight gained by the model. Closing the loop for radiology leads to successful integration, enabling lifelong learning for the AI system, which is crucial for sustainable long-term performance. KEY POINTS · The integration of AI systems into the clinical routine with structured reports and AI visualization.. · Two-way communication between AI and radiologists is necessary to enable AI that keeps the radiologist in the loop.. · Closing the loop enables lifelong learning, which is crucial for long-term, high-performing AI in radiology..
Collapse
Affiliation(s)
| | | | | | | | | | - Maximilian Gruening
- Interorganisational Informationssystems, Georg-August-Universität Göttingen, Goettingen, Germany
| | - Daniel Pinto Dos Santos
- Institute for Diagnostic and Interventional Radiology, Uniklinik Koln, Germany
- Institute for Diagnostic and Interventional Radiology, Universitätsklinikum Frankfurt, Frankfurt am Main, Germany
| | - Thomas Dratsch
- Institute for Diagnostic and Interventional Radiology, Uniklinik Koln, Germany
| | - Moon Kim
- Institute for Diagnostic and Interventional Radiology and Neuroradiology, Universitätsklinikum Essen, Germany
- Institute for Artificial Intelligence in Medicine, Universitätsklinikum Essen, Germany
| | - Felix Nensa
- Institute for Diagnostic and Interventional Radiology and Neuroradiology, Universitätsklinikum Essen, Germany
- Institute for Artificial Intelligence in Medicine, Universitätsklinikum Essen, Germany
| | - Manuel Trenz
- Interorganisational Informationssystems, Georg-August-Universität Göttingen, Goettingen, Germany
| | | |
Collapse
|
10
|
Wu X, Sánchez CA, Lloyd JE, Borgard H, Fels S, Paydarfar JA, Halter RJ. Estimating tongue deformation during laryngoscopy using a hybrid FEM-multibody model and intraoperative tracking - a cadaver study. Comput Methods Biomech Biomed Engin 2024:1-11. [PMID: 38193213 PMCID: PMC11231054 DOI: 10.1080/10255842.2023.2301672] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 12/31/2023] [Indexed: 01/10/2024]
Abstract
Throat tumour margin control remains difficult due to the tight, enclosed space of the oral and throat regions and the tissue deformation resulting from placement of retractors and scopes during surgery. Intraoperative imaging can help with better localization but is hindered by non-image-compatible surgical instruments, cost, and unavailability. We propose a novel method of using instrument tracking and FEM-multibody modelling to simulate soft tissue deformation in the intraoperative setting, without requiring intraoperative imaging, to improve surgical guidance accuracy. We report our first empirical study, based on four trials of a cadaveric head specimen with full neck anatomy, yields a mean TLE of 10.8 ± 5.5 mm, demonstrating methodological feasibility.
Collapse
Affiliation(s)
- Xiaotian Wu
- Gordon Center for Medical Imaging, MA General Hospital and Harvard Medical School, Boston, MA, USA
- Thayer School of Engineering, Dartmouth College, Hanover, NH, USA
| | - C Antonio Sánchez
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada
| | - John E Lloyd
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada
| | - Heather Borgard
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada
| | - Sidney Fels
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada
| | - Joseph A Paydarfar
- Section of Otolaryngology, Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
- Geisel School of Medicine, Dartmouth College, Hanover, NH, USA
| | - Ryan J Halter
- Thayer School of Engineering, Dartmouth College, Hanover, NH, USA
- Geisel School of Medicine, Dartmouth College, Hanover, NH, USA
| |
Collapse
|
11
|
Chen L, Platzer P, Reschl C, Schafasand M, Nachankar A, Lukas Hajdusich C, Kuess P, Stock M, Habraken S, Carlino A. Validation of a deep-learning segmentation model for adult and pediatric head and neck radiotherapy in different patient positions. Phys Imaging Radiat Oncol 2024; 29:100527. [PMID: 38222671 PMCID: PMC10787237 DOI: 10.1016/j.phro.2023.100527] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 12/15/2023] [Accepted: 12/18/2023] [Indexed: 01/16/2024] Open
Abstract
Background and purpose Autocontouring for radiotherapy has the potential to significantly save time and reduce interobserver variability. We aimed to assess the performance of a commercial autocontouring model for head and neck (H&N) patients in eight orientations relevant to particle therapy with fixed beam lines, focusing on validation and implementation for routine clinical use. Materials and methods Autocontouring was performed on sixteen organs at risk (OARs) for 98 adult and pediatric patients with 137 H&N CT scans in eight orientations. A geometric comparison of the autocontours and manual segmentations was performed using the Hausdorff Distance 95th percentile, Dice Similarity Coefficient (DSC) and surface DSC and compared to interobserver variability where available. Additional qualitative scoring and dose-volume-histogram (DVH) parameters analyses were performed for twenty patients in two positions, consisting of scoring on a 0-3 scale based on clinical usability and comparing the mean (Dmean) and near-maximum (D2%) dose, respectively. Results For the geometric analysis, the model performance in head-first-supine straight and hyperextended orientations was in the same range as the interobserver variability. HD95, DSC and surface DSC was heterogeneous in other orientations. No significant geometric differences were found between pediatric and adult autocontours. The qualitative scoring yielded a median score of ≥ 2 for 13/16 OARs while 7/32 DVH parameters were significantly different. Conclusions For head-first-supine straight and hyperextended scans, we found that 13/16 OAR autocontours were suited for use in daily clinical practice and subsequently implemented. Further development is needed for other patient orientations before implementation.
Collapse
Affiliation(s)
- Linda Chen
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
- Erasmus MC Cancer Institute, University Medical Center, Department of Radiotherapy, Rotterdam, the Netherlands
- Delft University of Technology, Faculty of Mechanical, Maritime and Materials Engineering, Delft, the Netherlands
- Leiden University Medical Center, Faculty of Medicine, Leiden, the Netherlands
| | - Patricia Platzer
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
- Fachhochschule Wiener Neustadt, Department MedTech, Wiener Neustadt, Austria
| | - Christian Reschl
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
| | - Mansure Schafasand
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
- Medical University of Vienna, Department of Radiation Oncology, Vienna, Austria
- Karl Landsteiner University of Health Sciences, Department of Oncology, Krems an der Donau, Austria
| | - Ankita Nachankar
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
- ACMIT Gmbh, Department of Medicine, Wiener Neustadt, Austria
| | | | - Peter Kuess
- Medical University of Vienna, Department of Radiation Oncology, Vienna, Austria
| | - Markus Stock
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
- Karl Landsteiner University of Health Sciences, Department of Oncology, Krems an der Donau, Austria
| | - Steven Habraken
- Erasmus MC Cancer Institute, University Medical Center, Department of Radiotherapy, Rotterdam, the Netherlands
- Holland Proton Therapy Center, Department of Medical Physics & Informatics, Delft, the Netherlands
| | - Antonio Carlino
- MedAustron Ion Therapy Center, Department of Medical Physics, Wiener Neustadt, Austria
| |
Collapse
|
12
|
Vivancos Bargalló H, Stick LB, Korreman SS, Kronborg C, Nielsen MM, Borgen AC, Offersen BV, Nørrevang O, Kallehauge JF. Classification of laterality and mastectomy/lumpectomy for breast cancer patients for improved performance of deep learning auto segmentation. Acta Oncol 2023; 62:1546-1550. [PMID: 37584197 DOI: 10.1080/0284186x.2023.2245965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 08/03/2023] [Indexed: 08/17/2023]
Affiliation(s)
- Helena Vivancos Bargalló
- Medical Physics department, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | | | - Stine Sofia Korreman
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Camilla Kronborg
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Mathias M Nielsen
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | | | - Birgitte Vrou Offersen
- Department of Experimental Clinical Oncologyy, Aarhus University Hospital, Aarhus, Denmark
- Department of Experimental Clinical Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Ole Nørrevang
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Jesper F Kallehauge
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| |
Collapse
|
13
|
Weisman AJ, Huff DT, Govindan RM, Chen S, Perk TG. Multi-organ segmentation of CT via convolutional neural network: impact of training setting and scanner manufacturer. Biomed Phys Eng Express 2023; 9:065021. [PMID: 37725928 DOI: 10.1088/2057-1976/acfb06] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 09/19/2023] [Indexed: 09/21/2023]
Abstract
Objective. Automated organ segmentation on CT images can enable the clinical use of advanced quantitative software devices, but model performance sensitivities must be understood before widespread adoption can occur. The goal of this study was to investigate performance differences between Convolutional Neural Networks (CNNs) trained to segment one (single-class) versus multiple (multi-class) organs, and between CNNs trained on scans from a single manufacturer versus multiple manufacturers.Methods. The multi-class CNN was trained on CT images obtained from 455 whole-body PET/CT scans (413 for training, 42 for testing) taken with Siemens, GE, and Phillips PET/CT scanners where 16 organs were segmented. The multi-class CNN was compared to 16 smaller single-class CNNs trained using the same data, but with segmentations of only one organ per model. In addition, CNNs trained on Siemens-only (N = 186) and GE-only (N = 219) scans (manufacturer-specific) were compared with CNNs trained on data from both Siemens and GE scanners (manufacturer-mixed). Segmentation performance was quantified using five performance metrics, including the Dice Similarity Coefficient (DSC).Results. The multi-class CNN performed well compared to previous studies, even in organs usually considered difficult auto-segmentation targets (e.g., pancreas, bowel). Segmentations from the multi-class CNN were significantly superior to those from smaller single-class CNNs in most organs, and the 16 single-class models took, on average, six times longer to segment all 16 organs compared to the single multi-class model. The manufacturer-mixed approach achieved minimally higher performance over the manufacturer-specific approach.Significance. A CNN trained on contours of multiple organs and CT data from multiple manufacturers yielded high-quality segmentations. Such a model is an essential enabler of image processing in a software device that quantifies and analyzes such data to determine a patient's treatment response. To date, this activity of whole organ segmentation has not been adopted due to the intense manual workload and time required.
Collapse
Affiliation(s)
- Amy J Weisman
- AIQ Solutions, Madison, WI, United States of America
| | - Daniel T Huff
- AIQ Solutions, Madison, WI, United States of America
| | | | - Song Chen
- Department of Nuclear Medicine, The First Hospital of China Medical University, Shenyang, Liaoning, People's Republic of China
| | | |
Collapse
|
14
|
Mikalsen SG, Skjøtskift T, Flote VG, Hämäläinen NP, Heydari M, Rydén-Eilertsen K. Extensive clinical testing of Deep Learning Segmentation models for thorax and breast cancer radiotherapy planning. Acta Oncol 2023; 62:1184-1193. [PMID: 37883678 DOI: 10.1080/0284186x.2023.2270152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 10/04/2023] [Indexed: 10/28/2023]
Abstract
BACKGROUND The performance of deep learning segmentation (DLS) models for automatic organ extraction from CT images in the thorax and breast regions was investigated. Furthermore, the readiness and feasibility of integrating DLS into clinical practice were addressed by measuring the potential time savings and dosimetric impact. MATERIAL AND METHODS Thirty patients referred to radiotherapy for breast cancer were prospectively included. A total of 23 clinically relevant left- and right-sided organs were contoured manually on CT images according to ESTRO guidelines. Next, auto-segmentation was executed, and the geometric agreement between the auto-segmented and manually contoured organs was qualitatively assessed applying a scale in the range [0-not acceptable, 3-no corrections]. A quantitative validation was carried out by calculating Dice coefficients (DSC) and the 95% percentile of Hausdorff distances (HD95). The dosimetric impact of optimizing the treatment plans on the uncorrected DLS contours, was investigated from a dose coverage analysis using DVH values of the manually delineated contours as references. RESULTS The qualitative analysis showed that 93% of the DLS generated OAR contours did not need corrections, except for the heart where 67% of the contours needed corrections. The majority of DLS generated CTVs needed corrections, whereas a minority were deemed not acceptable. Still, using the DLS-model for CTV and heart delineation is on average 14 minutes faster. An average DSC=0.91 and H95=9.8 mm were found for the left and right breasts, respectively. Likewise, and average DSC in the range [0.66, 0.76]mm and HD95 in the range [7.04, 12.05]mm were found for the lymph nodes. CONCLUSION The validation showed that the DLS generated OAR contours can be used clinically. Corrections were required to most of the DLS generated CTVs, and therefore warrants more attention before possibly implementing the DLS models clinically.
Collapse
Affiliation(s)
| | | | | | | | - Mojgan Heydari
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
| | | |
Collapse
|
15
|
Groen JA, Crezee J, van Laarhoven HWM, Bijlsma MF, Kok HP. Quantification of tissue property and perfusion uncertainties in hyperthermia treatment planning: Multianalysis using polynomial chaos expansion. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107675. [PMID: 37339535 DOI: 10.1016/j.cmpb.2023.107675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 05/26/2023] [Accepted: 06/08/2023] [Indexed: 06/22/2023]
Abstract
INTRODUCTION Hyperthermia treatment planning (HTP) tools can guide treatment delivery, particularly with locoregional radiative phased array systems. Uncertainties in tissue and perfusion property values presently lead to quantitative inaccuracy of HTP, leading to sub-optimal treatment. Assessment of these uncertainties would allow for better judgement of the reliability of treatment plans and improve their value for treatment guidance. However, systematically investigating the impact of all uncertainties on treatment plans is a complex, high-dimensional problem and too computationally expensive for traditional Monte Carlo approaches. This study aims to systematically quantify the treatment-plan impact of tissue property uncertainties by investigating their individual contribution to, and combined impact on predicted temperature distributions. METHODS A novel Polynomial Chaos Expansion (PCE)-based HTP uncertainty quantification was developed and applied for locoregional hyperthermia of modelled tumours in the pancreatic head, prostate, rectum, and cervix. Patient models were based on the Duke and Ella digital human models. Using Plan2Heat, treatment plans were created to optimise tumour temperature (represented by T90) for treatment using the Alba4D system. For all 25-34 modelled tissues, the impact of tissue property uncertainties was analysed individually i.e., electrical and thermal conductivity, permittivity, density, specific heat capacity and perfusion. Next, combined analyses were performed on the top 30 uncertainties with the largest impact. RESULTS Uncertainties in thermal conductivity and heat capacity were found to have negligible impact on the predicted temperature ( < 1 × 10-10 °C), density and permittivity uncertainties had a small impact (< 0.3 °C). Uncertainties in electrical conductivity and perfusion can lead to large variations in predicted temperature. However, variations in muscle properties result in the largest impact at locations that could limit treatment quality, with a standard deviation up to almost 6 °C (pancreas) and 3.5 °C (prostate) for perfusion and electrical conductivity, respectively. The combined influence of all significant uncertainties leads to large variations with a standard deviation up to 9.0, 3.6, 3.7 and 4.1 °C for the pancreatic, prostate, rectal and cervical cases, respectively. CONCLUSION Uncertainties in tissue and perfusion property values can have a large impact on predicted temperatures from hyperthermia treatment planning. PCE-based analysis helps to identify all major uncertainties, their impact and judge the reliability of treatment plans.
Collapse
Affiliation(s)
- Jort A Groen
- Amsterdam UMC location University of Amsterdam, Radiation Oncology, Meibergdreef 9, Amsterdam, the Netherlands; Cancer Center Amsterdam, Cancer biology and immunology, Amsterdam, the Netherlands; Cancer Center Amsterdam, Treatment and quality of life, Amsterdam, the Netherlands.
| | - Johannes Crezee
- Amsterdam UMC location University of Amsterdam, Radiation Oncology, Meibergdreef 9, Amsterdam, the Netherlands; Cancer Center Amsterdam, Cancer biology and immunology, Amsterdam, the Netherlands; Cancer Center Amsterdam, Treatment and quality of life, Amsterdam, the Netherlands
| | - Hanneke W M van Laarhoven
- Amsterdam UMC location University of Amsterdam, Department of Medical Oncology, Amsterdam, the Netherlands; Cancer Center Amsterdam, Treatment and quality of life, Amsterdam, the Netherlands
| | - Maarten F Bijlsma
- Amsterdam UMC location University of Amsterdam, Center for Experimental and Molecular Medicine, Laboratory for Experimental Oncology and Radiobiology, Meibergdreef 9, Amsterdam, the Netherlands; Cancer Center Amsterdam, Cancer biology and immunology, Amsterdam, the Netherlands; Cancer Center Amsterdam, Imaging and biomarkers, Amsterdam, the Netherlands; Oncode Institute, Amsterdam, the Netherlands
| | - H Petra Kok
- Amsterdam UMC location University of Amsterdam, Radiation Oncology, Meibergdreef 9, Amsterdam, the Netherlands; Cancer Center Amsterdam, Cancer biology and immunology, Amsterdam, the Netherlands; Cancer Center Amsterdam, Treatment and quality of life, Amsterdam, the Netherlands
| |
Collapse
|
16
|
Akiyama R, Goto T, Tameshige T, Sugisaka J, Kuroki K, Sun J, Akita J, Hatakeyama M, Kudoh H, Kenta T, Tonouchi A, Shimahara Y, Sese J, Kutsuna N, Shimizu-Inatsugi R, Shimizu KK. Seasonal pigment fluctuation in diploid and polyploid Arabidopsis revealed by machine learning-based phenotyping method PlantServation. Nat Commun 2023; 14:5792. [PMID: 37737204 PMCID: PMC10517152 DOI: 10.1038/s41467-023-41260-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 08/29/2023] [Indexed: 09/23/2023] Open
Abstract
Long-term field monitoring of leaf pigment content is informative for understanding plant responses to environments distinct from regulated chambers but is impractical by conventional destructive measurements. We developed PlantServation, a method incorporating robust image-acquisition hardware and deep learning-based software that extracts leaf color by detecting plant individuals automatically. As a case study, we applied PlantServation to examine environmental and genotypic effects on the pigment anthocyanin content estimated from leaf color. We processed >4 million images of small individuals of four Arabidopsis species in the field, where the plant shape, color, and background vary over months. Past radiation, coldness, and precipitation significantly affected the anthocyanin content. The synthetic allopolyploid A. kamchatica recapitulated the fluctuations of natural polyploids by integrating diploid responses. The data support a long-standing hypothesis stating that allopolyploids can inherit and combine the traits of progenitors. PlantServation facilitates the study of plant responses to complex environments termed "in natura".
Collapse
Affiliation(s)
- Reiko Akiyama
- Department of Evolutionary Biology and Environmental Studies, University of Zurich, Winterthurerstrasse 190, CH-8057, Zurich, Switzerland
| | - Takao Goto
- Research and Development Division, LPIXEL Inc., Chiyoda-ku, Tokyo, 100-0004, Japan
| | - Toshiaki Tameshige
- Kihara Institute for Biological Research (KIBR), Yokohama City University, 641-12 Maioka, Totsuka-ward, Yokohama, 244-0813, Japan
- Division of Biological Science, Graduate School of Science and Technology, Nara Institute of Science and Technology (NAIST), 8916-5 Takayama-Cho, Ikoma, Nara, 630-0192, Japan
| | - Jiro Sugisaka
- Kihara Institute for Biological Research (KIBR), Yokohama City University, 641-12 Maioka, Totsuka-ward, Yokohama, 244-0813, Japan
- Center for Ecological Research, Kyoto University, Hirano 2-509-3, Otsu, 520-2113, Japan
| | - Ken Kuroki
- Department of Biological Sciences, Graduate School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Jianqiang Sun
- Research Center for Agricultural Information Technology, National Agriculture and Food Research Organization, 3-1-1 Kannondai, Tsukuba, Ibaraki, 305-8517, Japan
| | - Junichi Akita
- Department of Electric and Computer Engineering, Kanazawa University, Kakuma, Kanazawa, 920-1192, Japan
| | - Masaomi Hatakeyama
- Department of Evolutionary Biology and Environmental Studies, University of Zurich, Winterthurerstrasse 190, CH-8057, Zurich, Switzerland
- Functional Genomics Center Zurich, Winterthurerstrasse 190, CH-8057, Zurich, Switzerland
| | - Hiroshi Kudoh
- Center for Ecological Research, Kyoto University, Hirano 2-509-3, Otsu, 520-2113, Japan
| | - Tanaka Kenta
- Sugadaira Research Station, Mountain Science Center, University of Tsukuba, 1278-294 Sugadaira-kogen, Ueda, 386-2204, Japan
| | - Aya Tonouchi
- Research and Development Division, LPIXEL Inc., Chiyoda-ku, Tokyo, 100-0004, Japan
| | - Yuki Shimahara
- Research and Development Division, LPIXEL Inc., Chiyoda-ku, Tokyo, 100-0004, Japan
| | - Jun Sese
- Artificial Intelligence Research Center, AIST, 2-3-26 Aomi, Koto-ku, Tokyo, 135-0064, Japan
- Humanome Lab, Inc., L-HUB 3F, 1-4, Shumomiyabi-cho, Shinjuku, Tokyo, 162-0822, Japan
- AIST-Tokyo Tech RWBC-OIL, 2-12-1 O-okayama, Meguro-ku, Tokyo, 152-8550, Japan
| | - Natsumaro Kutsuna
- Research and Development Division, LPIXEL Inc., Chiyoda-ku, Tokyo, 100-0004, Japan
| | - Rie Shimizu-Inatsugi
- Department of Evolutionary Biology and Environmental Studies, University of Zurich, Winterthurerstrasse 190, CH-8057, Zurich, Switzerland.
| | - Kentaro K Shimizu
- Department of Evolutionary Biology and Environmental Studies, University of Zurich, Winterthurerstrasse 190, CH-8057, Zurich, Switzerland.
- Kihara Institute for Biological Research (KIBR), Yokohama City University, 641-12 Maioka, Totsuka-ward, Yokohama, 244-0813, Japan.
| |
Collapse
|
17
|
Weissmann T, Mansoorian S, May MS, Lettmaier S, Höfler D, Deloch L, Speer S, Balk M, Frey B, Gaipl US, Bert C, Distel LV, Walter F, Belka C, Semrau S, Iro H, Fietkau R, Huang Y, Putz F. Deep Learning and Registration-Based Mapping for Analyzing the Distribution of Nodal Metastases in Head and Neck Cancer Cohorts: Informing Optimal Radiotherapy Target Volume Design. Cancers (Basel) 2023; 15:4620. [PMID: 37760588 PMCID: PMC10526893 DOI: 10.3390/cancers15184620] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Revised: 09/15/2023] [Accepted: 09/15/2023] [Indexed: 09/29/2023] Open
Abstract
We introduce a deep-learning- and a registration-based method for automatically analyzing the spatial distribution of nodal metastases (LNs) in head and neck (H/N) cancer cohorts to inform radiotherapy (RT) target volume design. The two methods are evaluated in a cohort of 193 H/N patients/planning CTs with a total of 449 LNs. In the deep learning method, a previously developed nnU-Net 3D/2D ensemble model is used to autosegment 20 H/N levels, with each LN subsequently being algorithmically assigned to the closest-level autosegmentation. In the nonrigid-registration-based mapping method, LNs are mapped into a calculated template CT representing the cohort-average patient anatomy, and kernel density estimation is employed to estimate the underlying average 3D-LN probability distribution allowing for analysis and visualization without prespecified level definitions. Multireader assessment by three radio-oncologists with majority voting was used to evaluate the deep learning method and obtain the ground-truth distribution. For the mapping technique, the proportion of LNs predicted by the 3D probability distribution for each level was calculated and compared to the deep learning and ground-truth distributions. As determined by a multireader review with majority voting, the deep learning method correctly categorized all 449 LNs to their respective levels. Level 2 showed the highest LN involvement (59.0%). The level involvement predicted by the mapping technique was consistent with the ground-truth distribution (p for difference 0.915). Application of the proposed methods to multicenter cohorts with selected H/N tumor subtypes for informing optimal RT target volume design is promising.
Collapse
Affiliation(s)
- Thomas Weissmann
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Bavarian Cancer Research Center (BZKF), 81377 Munich, Germany; (S.M.); (F.W.); (C.B.)
| | - Sina Mansoorian
- Bavarian Cancer Research Center (BZKF), 81377 Munich, Germany; (S.M.); (F.W.); (C.B.)
- Department of Radiation Oncology, University Hospital, Ludwig Maximilian University of Munich, 81377 Munich, Germany
| | - Matthias Stefan May
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Department of Radiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Sebastian Lettmaier
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Daniel Höfler
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Lisa Deloch
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Translational Radiobiology, Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Stefan Speer
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Matthias Balk
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Department of Otolaryngology, Head and Neck Surgery, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Benjamin Frey
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Translational Radiobiology, Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Udo S. Gaipl
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Translational Radiobiology, Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Christoph Bert
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Luitpold Valentin Distel
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Franziska Walter
- Bavarian Cancer Research Center (BZKF), 81377 Munich, Germany; (S.M.); (F.W.); (C.B.)
- Department of Radiation Oncology, University Hospital, Ludwig Maximilian University of Munich, 81377 Munich, Germany
| | - Claus Belka
- Bavarian Cancer Research Center (BZKF), 81377 Munich, Germany; (S.M.); (F.W.); (C.B.)
- Department of Radiation Oncology, University Hospital, Ludwig Maximilian University of Munich, 81377 Munich, Germany
| | - Sabine Semrau
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Heinrich Iro
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Department of Otolaryngology, Head and Neck Surgery, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Rainer Fietkau
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Bavarian Cancer Research Center (BZKF), 81377 Munich, Germany; (S.M.); (F.W.); (C.B.)
| | - Yixing Huang
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
| | - Florian Putz
- Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91054 Erlangen, Germany; (T.W.); (S.L.); (D.H.); (L.D.); (S.S.); (B.F.); (U.S.G.); (C.B.); (L.V.D.); (S.S.); (R.F.)
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), 91054 Erlangen, Germany; (M.S.M.); (M.B.); (H.I.)
- Bavarian Cancer Research Center (BZKF), 81377 Munich, Germany; (S.M.); (F.W.); (C.B.)
| |
Collapse
|
18
|
Amjad A, Xu J, Thill D, Zhang Y, Ding J, Paulson E, Hall W, Erickson BA, Li XA. Deep learning auto-segmentation on multi-sequence magnetic resonance images for upper abdominal organs. Front Oncol 2023; 13:1209558. [PMID: 37483486 PMCID: PMC10358771 DOI: 10.3389/fonc.2023.1209558] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 06/19/2023] [Indexed: 07/25/2023] Open
Abstract
Introduction Multi-sequence multi-parameter MRIs are often used to define targets and/or organs at risk (OAR) in radiation therapy (RT) planning. Deep learning has so far focused on developing auto-segmentation models based on a single MRI sequence. The purpose of this work is to develop a multi-sequence deep learning based auto-segmentation (mS-DLAS) based on multi-sequence abdominal MRIs. Materials and methods Using a previously developed 3DResUnet network, a mS-DLAS model using 4 T1 and T2 weighted MRI acquired during routine RT simulation for 71 cases with abdominal tumors was trained and tested. Strategies including data pre-processing, Z-normalization approach, and data augmentation were employed. Additional 2 sequence specific T1 weighted (T1-M) and T2 weighted (T2-M) models were trained to evaluate performance of sequence-specific DLAS. Performance of all models was quantitatively evaluated using 6 surface and volumetric accuracy metrics. Results The developed DLAS models were able to generate reasonable contours of 12 upper abdomen organs within 21 seconds for each testing case. The 3D average values of dice similarity coefficient (DSC), mean distance to agreement (MDA mm), 95 percentile Hausdorff distance (HD95% mm), percent volume difference (PVD), surface DSC (sDSC), and relative added path length (rAPL mm/cc) over all organs were 0.87, 1.79, 7.43, -8.95, 0.82, and 12.25, respectively, for mS-DLAS model. Collectively, 71% of the auto-segmented contours by the three models had relatively high quality. Additionally, the obtained mS-DLAS successfully segmented 9 out of 16 MRI sequences that were not used in the model training. Conclusion We have developed an MRI-based mS-DLAS model for auto-segmenting of upper abdominal organs on MRI. Multi-sequence segmentation is desirable in routine clinical practice of RT for accurate organ and target delineation, particularly for abdominal tumors. Our work will act as a stepping stone for acquiring fast and accurate segmentation on multi-contrast MRI and make way for MR only guided radiation therapy.
Collapse
Affiliation(s)
- Asma Amjad
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| | | | - Dan Thill
- Elekta Inc., ST. Charles, MO, United States
| | - Ying Zhang
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - Jie Ding
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - Eric Paulson
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - William Hall
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - Beth A. Erickson
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - X. Allen Li
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, WI, United States
| |
Collapse
|
19
|
Nikulin P, Zschaeck S, Maus J, Cegla P, Lombardo E, Furth C, Kaźmierska J, Rogasch JMM, Holzgreve A, Albert NL, Ferentinos K, Strouthos I, Hajiyianni M, Marschner SN, Belka C, Landry G, Cholewinski W, Kotzerke J, Hofheinz F, van den Hoff J. A convolutional neural network with self-attention for fully automated metabolic tumor volume delineation of head and neck cancer in [Formula: see text]F]FDG PET/CT. Eur J Nucl Med Mol Imaging 2023; 50:2751-2766. [PMID: 37079128 PMCID: PMC10317885 DOI: 10.1007/s00259-023-06197-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 03/14/2023] [Indexed: 04/21/2023]
Abstract
PURPOSE PET-derived metabolic tumor volume (MTV) and total lesion glycolysis of the primary tumor are known to be prognostic of clinical outcome in head and neck cancer (HNC). Including evaluation of lymph node metastases can further increase the prognostic value of PET but accurate manual delineation and classification of all lesions is time-consuming and prone to interobserver variability. Our goal, therefore, was development and evaluation of an automated tool for MTV delineation/classification of primary tumor and lymph node metastases in PET/CT investigations of HNC patients. METHODS Automated lesion delineation was performed with a residual 3D U-Net convolutional neural network (CNN) incorporating a multi-head self-attention block. 698 [Formula: see text]F]FDG PET/CT scans from 3 different sites and 5 public databases were used for network training and testing. An external dataset of 181 [Formula: see text]F]FDG PET/CT scans from 2 additional sites was employed to assess the generalizability of the network. In these data, primary tumor and lymph node (LN) metastases were interactively delineated and labeled by two experienced physicians. Performance of the trained network models was assessed by 5-fold cross-validation in the main dataset and by pooling results from the 5 developed models in the external dataset. The Dice similarity coefficient (DSC) for individual delineation tasks and the primary tumor/metastasis classification accuracy were used as evaluation metrics. Additionally, a survival analysis using univariate Cox regression was performed comparing achieved group separation for manual and automated delineation, respectively. RESULTS In the cross-validation experiment, delineation of all malignant lesions with the trained U-Net models achieves DSC of 0.885, 0.805, and 0.870 for primary tumor, LN metastases, and the union of both, respectively. In external testing, the DSC reaches 0.850, 0.724, and 0.823 for primary tumor, LN metastases, and the union of both, respectively. The voxel classification accuracy was 98.0% and 97.9% in cross-validation and external data, respectively. Univariate Cox analysis in the cross-validation and the external testing reveals that manually and automatically derived total MTVs are both highly prognostic with respect to overall survival, yielding essentially identical hazard ratios (HR) ([Formula: see text]; [Formula: see text] vs. [Formula: see text]; [Formula: see text] in cross-validation and [Formula: see text]; [Formula: see text] vs. [Formula: see text]; [Formula: see text] in external testing). CONCLUSION To the best of our knowledge, this work presents the first CNN model for successful MTV delineation and lesion classification in HNC. In the vast majority of patients, the network performs satisfactory delineation and classification of primary tumor and lymph node metastases and only rarely requires more than minimal manual correction. It is thus able to massively facilitate study data evaluation in large patient groups and also does have clear potential for supervised clinical application.
Collapse
Affiliation(s)
- Pavel Nikulin
- Helmholtz-Zentrum Dresden-Rossendorf, PET Center, Institute of Radiopharmaceutical Cancer Research, Bautzner Landstrasse 400, 01328, Dresden, Germany.
| | - Sebastian Zschaeck
- Department of Radiation Oncology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Jens Maus
- Helmholtz-Zentrum Dresden-Rossendorf, PET Center, Institute of Radiopharmaceutical Cancer Research, Bautzner Landstrasse 400, 01328, Dresden, Germany
| | - Paulina Cegla
- Department of Nuclear Medicine, Greater Poland Cancer Centre, Poznan, Poland
| | - Elia Lombardo
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-University (LMU) Munich, Munich, Germany
| | - Christian Furth
- Department of Nuclear Medicine, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Joanna Kaźmierska
- Electroradiology Department, University of Medical Sciences, Poznan, Poland
- Radiotherapy Department II, Greater Poland Cancer Centre, Poznan, Poland
| | - Julian M M Rogasch
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
- Department of Nuclear Medicine, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Adrien Holzgreve
- Department of Nuclear Medicine, University Hospital, Ludwig-Maximilians-University (LMU) Munich, Munich, Germany
| | - Nathalie L Albert
- Department of Nuclear Medicine, University Hospital, Ludwig-Maximilians-University (LMU) Munich, Munich, Germany
| | - Konstantinos Ferentinos
- Department of Radiation Oncology, German Oncology Center, European University Cyprus, Limassol, Cyprus
| | - Iosif Strouthos
- Department of Radiation Oncology, German Oncology Center, European University Cyprus, Limassol, Cyprus
| | - Marina Hajiyianni
- Department of Radiation Oncology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Sebastian N Marschner
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-University (LMU) Munich, Munich, Germany
| | - Claus Belka
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-University (LMU) Munich, Munich, Germany
- German Cancer Consortium (DKTK), Partner Site Munich, Munich, Germany
| | - Guillaume Landry
- Department of Radiation Oncology, University Hospital, Ludwig-Maximilians-University (LMU) Munich, Munich, Germany
| | - Witold Cholewinski
- Department of Nuclear Medicine, Greater Poland Cancer Centre, Poznan, Poland
- Electroradiology Department, University of Medical Sciences, Poznan, Poland
| | - Jörg Kotzerke
- Department of Nuclear Medicine, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Frank Hofheinz
- Helmholtz-Zentrum Dresden-Rossendorf, PET Center, Institute of Radiopharmaceutical Cancer Research, Bautzner Landstrasse 400, 01328, Dresden, Germany
| | - Jörg van den Hoff
- Helmholtz-Zentrum Dresden-Rossendorf, PET Center, Institute of Radiopharmaceutical Cancer Research, Bautzner Landstrasse 400, 01328, Dresden, Germany
- Department of Nuclear Medicine, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
20
|
Bakx N, van der Sangen M, Theuws J, Bluemink H, Hurkmans C. Comparison of the output of a deep learning segmentation model for locoregional breast cancer radiotherapy trained on 2 different datasets. Tech Innov Patient Support Radiat Oncol 2023; 26:100209. [PMID: 37213441 PMCID: PMC10199413 DOI: 10.1016/j.tipsro.2023.100209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 04/06/2023] [Accepted: 05/09/2023] [Indexed: 05/23/2023] Open
Abstract
Introduction The development of deep learning (DL) models for auto-segmentation is increasing and more models become commercially available. Mostly, commercial models are trained on external data. To study the effect of using a model trained on external data, compared to the same model trained on in-house collected data, the performance of these two DL models was evaluated. Methods The evaluation was performed using in-house collected data of 30 breast cancer patients. Quantitative analysis was performed using Dice similarity coefficient (DSC), surface DSC (sDSC) and 95th percentile of Hausdorff Distance (95% HD). These values were compared with previously reported inter-observer variations (IOV). Results For a number of structures, statistically significant differences were found between the two models. For organs at risk, mean values for DSC ranged from 0.63 to 0.98 and 0.71 to 0.96 for the in-house and external model, respectively. For target volumes, mean DSC values of 0.57 to 0.94 and 0.33 to 0.92 were found. The difference of 95% HD values ranged 0.08 to 3.23 mm between the two models, except for CTVn4 with 9.95 mm. For the external model, both DSC and 95% HD are outside the range of IOV for CTVn4, whereas this is the case for the DSC found for the thyroid of the in-house model. Conclusions Statistically significant differences were found between both models, which were mostly within published inter-observer variations, showing clinical usefulness of both models. Our findings could encourage discussion and revision of existing guidelines, to further decrease inter-observer, but also inter-institute variability.
Collapse
Affiliation(s)
- Nienke Bakx
- Catharina Hospital, Department of Radiation Oncology, 5602ZA Eindhoven, the Netherlands
| | | | - Jacqueline Theuws
- Catharina Hospital, Department of Radiation Oncology, 5602ZA Eindhoven, the Netherlands
| | - Hanneke Bluemink
- Catharina Hospital, Department of Radiation Oncology, 5602ZA Eindhoven, the Netherlands
| | - Coen Hurkmans
- Catharina Hospital, Department of Radiation Oncology, 5602ZA Eindhoven, the Netherlands
- Technical University Eindhoven, Faculties of Physics and Electrical Engineering, 5600MB Eindhoven, the Netherlands
- Corresponding author at: Catharina Hospital, Department of Radiation Oncology, 5602ZA Eindhoven, the Netherlands.
| |
Collapse
|
21
|
Mireștean CC, Iancu RI, Iancu DPT. Simultaneous Integrated Boost (SIB) vs. Sequential Boost in Head and Neck Cancer (HNC) Radiotherapy: A Radiomics-Based Decision Proof of Concept. J Clin Med 2023; 12:jcm12062413. [PMID: 36983413 PMCID: PMC10057404 DOI: 10.3390/jcm12062413] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 03/07/2023] [Accepted: 03/14/2023] [Indexed: 03/30/2023] Open
Abstract
Artificial intelligence (AI) and in particular radiomics has opened new horizons by extracting data from medical imaging that could be used not only to improve diagnostic accuracy, but also to be included in predictive models contributing to treatment stratification of cancer. Head and neck cancers (HNC) are associated with higher recurrence rates, especially in advanced stages of disease. It is considered that approximately 50% of cases will evolve with loco-regional recurrence, even if they will benefit from a current standard treatment consisting of definitive chemo-radiotherapy. Radiotherapy, the cornerstone treatment in locally advanced HNC, could be delivered either by the simultaneous integrated boost (SIB) technique or by the sequential boost technique, the decision often being a subjective one. The principles of radiobiology could be the basis of an optimal decision between the two methods of radiation dose delivery, but the heterogeneity of HNC radio-sensitivity makes this approach difficult. Radiomics has demonstrated the ability to non-invasively predict radio-sensitivity and the risk of relapse in HNC. Tumor heterogeneity evaluated with radiomics, the inclusion of coarseness, entropy and other first order features extracted from gross tumor volume (GTV) in multivariate models could identify pre-treatment cases that will benefit from one of the approaches (SIB or sequential boost radio-chemotherapy) considered the current standard of care for locally advanced HNC. Computer tomography (CT) simulation and daily cone beam CT (CBCT) could be chosen as imaging source for radiomic analysis.
Collapse
Affiliation(s)
- Camil Ciprian Mireștean
- Department of Oncology and Radiotherapy, University of Medicine and Pharmacy Craiova, 200349 Craiova, Romania
- Department of Surgery, Railways Clinical Hospital Iasi, 700506 Iași, Romania
| | - Roxana Irina Iancu
- Oral Pathology Department, Faculty of Dental Medicine, "Gr. T. Popa" University of Medicine and Pharmacy, 700115 Iași, Romania
- Department of Clinical Laboratory, "St. Spiridon" Emergency Universitary Hospital, 700111 Iași, Romania
| | - Dragoș Petru Teodor Iancu
- Oncology and Radiotherapy Department, Faculty of Medicine, "Gr. T. Popa" University of Medicine and Pharmacy, 700115 Iași, Romania
- Department of Radiation Oncology, Regional Institute of Oncology, 700483 Iași, Romania
| |
Collapse
|
22
|
Systematic Review of Tumor Segmentation Strategies for Bone Metastases. Cancers (Basel) 2023; 15:cancers15061750. [PMID: 36980636 PMCID: PMC10046265 DOI: 10.3390/cancers15061750] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 03/09/2023] [Accepted: 03/10/2023] [Indexed: 03/18/2023] Open
Abstract
Purpose: To investigate the segmentation approaches for bone metastases in differentiating benign from malignant bone lesions and characterizing malignant bone lesions. Method: The literature search was conducted in Scopus, PubMed, IEEE and MedLine, and Web of Science electronic databases following the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). A total of 77 original articles, 24 review articles, and 1 comparison paper published between January 2010 and March 2022 were included in the review. Results: The results showed that most studies used neural network-based approaches (58.44%) and CT-based imaging (50.65%) out of 77 original articles. However, the review highlights the lack of a gold standard for tumor boundaries and the need for manual correction of the segmentation output, which largely explains the absence of clinical translation studies. Moreover, only 19 studies (24.67%) specifically mentioned the feasibility of their proposed methods for use in clinical practice. Conclusion: Development of tumor segmentation techniques that combine anatomical information and metabolic activities is encouraging despite not having an optimal tumor segmentation method for all applications or can compensate for all the difficulties built into data limitations.
Collapse
|
23
|
Weissmann T, Huang Y, Fischer S, Roesch J, Mansoorian S, Ayala Gaona H, Gostian AO, Hecht M, Lettmaier S, Deloch L, Frey B, Gaipl US, Distel LV, Maier A, Iro H, Semrau S, Bert C, Fietkau R, Putz F. Deep learning for automatic head and neck lymph node level delineation provides expert-level accuracy. Front Oncol 2023; 13:1115258. [PMID: 36874135 PMCID: PMC9978473 DOI: 10.3389/fonc.2023.1115258] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2022] [Accepted: 01/30/2023] [Indexed: 02/18/2023] Open
Abstract
Background Deep learning-based head and neck lymph node level (HN_LNL) autodelineation is of high relevance to radiotherapy research and clinical treatment planning but still underinvestigated in academic literature. In particular, there is no publicly available open-source solution for large-scale autosegmentation of HN_LNL in the research setting. Methods An expert-delineated cohort of 35 planning CTs was used for training of an nnU-net 3D-fullres/2D-ensemble model for autosegmentation of 20 different HN_LNL. A second cohort acquired at the same institution later in time served as the test set (n = 20). In a completely blinded evaluation, 3 clinical experts rated the quality of deep learning autosegmentations in a head-to-head comparison with expert-created contours. For a subgroup of 10 cases, intraobserver variability was compared to the average deep learning autosegmentation accuracy on the original and recontoured set of expert segmentations. A postprocessing step to adjust craniocaudal boundaries of level autosegmentations to the CT slice plane was introduced and the effect of autocontour consistency with CT slice plane orientation on geometric accuracy and expert rating was investigated. Results Blinded expert ratings for deep learning segmentations and expert-created contours were not significantly different. Deep learning segmentations with slice plane adjustment were rated numerically higher (mean, 81.0 vs. 79.6, p = 0.185) and deep learning segmentations without slice plane adjustment were rated numerically lower (77.2 vs. 79.6, p = 0.167) than manually drawn contours. In a head-to-head comparison, deep learning segmentations with CT slice plane adjustment were rated significantly better than deep learning contours without slice plane adjustment (81.0 vs. 77.2, p = 0.004). Geometric accuracy of deep learning segmentations was not different from intraobserver variability (mean Dice per level, 0.76 vs. 0.77, p = 0.307). Clinical significance of contour consistency with CT slice plane orientation was not represented by geometric accuracy metrics (volumetric Dice, 0.78 vs. 0.78, p = 0.703). Conclusions We show that a nnU-net 3D-fullres/2D-ensemble model can be used for highly accurate autodelineation of HN_LNL using only a limited training dataset that is ideally suited for large-scale standardized autodelineation of HN_LNL in the research setting. Geometric accuracy metrics are only an imperfect surrogate for blinded expert rating.
Collapse
Affiliation(s)
- Thomas Weissmann
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Yixing Huang
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Stefan Fischer
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Johannes Roesch
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Sina Mansoorian
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Horacio Ayala Gaona
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Antoniu-Oreste Gostian
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany.,Department of Otolaryngology, Head and Neck Surgery, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Markus Hecht
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Sebastian Lettmaier
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Lisa Deloch
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany.,Translational Radiobiology, Department of Radiation Oncology, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitätsklinikum Erlangen, Erlangen, Germany
| | - Benjamin Frey
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany.,Translational Radiobiology, Department of Radiation Oncology, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitätsklinikum Erlangen, Erlangen, Germany
| | - Udo S Gaipl
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany.,Translational Radiobiology, Department of Radiation Oncology, Friedrich-Alexander-Universität Erlangen-Nürnberg, Universitätsklinikum Erlangen, Erlangen, Germany
| | - Luitpold Valentin Distel
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Heinrich Iro
- Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany.,Department of Otolaryngology, Head and Neck Surgery, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sabine Semrau
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Christoph Bert
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Rainer Fietkau
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| | - Florian Putz
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.,Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany
| |
Collapse
|
24
|
Hu Q, Li K, Yang C, Wang Y, Huang R, Gu M, Xiao Y, Huang Y, Chen L. The role of artificial intelligence based on PET/CT radiomics in NSCLC: Disease management, opportunities, and challenges. Front Oncol 2023; 13:1133164. [PMID: 36959810 PMCID: PMC10028142 DOI: 10.3389/fonc.2023.1133164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 02/20/2023] [Indexed: 03/09/2023] Open
Abstract
Objectives Lung cancer has been widely characterized through radiomics and artificial intelligence (AI). This review aims to summarize the published studies of AI based on positron emission tomography/computed tomography (PET/CT) radiomics in non-small-cell lung cancer (NSCLC). Materials and methods A comprehensive search of literature published between 2012 and 2022 was conducted on the PubMed database. There were no language or publication status restrictions on the search. About 127 articles in the search results were screened and gradually excluded according to the exclusion criteria. Finally, this review included 39 articles for analysis. Results Classification is conducted according to purposes and several studies were identified at each stage of disease:1) Cancer detection (n=8), 2) histology and stage of cancer (n=11), 3) metastases (n=6), 4) genotype (n=6), 5) treatment outcome and survival (n=8). There is a wide range of heterogeneity among studies due to differences in patient sources, evaluation criteria and workflow of radiomics. On the whole, most models show diagnostic performance comparable to or even better than experts, and the common problems are repeatability and clinical transformability. Conclusion AI-based PET/CT Radiomics play potential roles in NSCLC clinical management. However, there is still a long way to go before being translated into clinical application. Large-scale, multi-center, prospective research is the direction of future efforts, while we need to face the risk of repeatability of radiomics features and the limitation of access to large databases.
Collapse
Affiliation(s)
- Qiuyuan Hu
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Ke Li
- Department of Cancer Biotherapy Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Conghui Yang
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Yue Wang
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Rong Huang
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Mingqiu Gu
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Yuqiang Xiao
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
| | - Yunchao Huang
- Department of Thoracic Surgery I, Key Laboratory of Lung Cancer of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
- *Correspondence: Long Chen, ; Yunchao Huang,
| | - Long Chen
- Department of positron emission tomography/computed tomography (PET/CT) Center, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Cancer Center of Yunnan Province, Kunming, Yunnan, China
- *Correspondence: Long Chen, ; Yunchao Huang,
| |
Collapse
|
25
|
Wei Z, Ren J, Korreman SS, Nijkamp J. Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy. Phys Imaging Radiat Oncol 2022; 25:100408. [PMID: 36655215 PMCID: PMC9841279 DOI: 10.1016/j.phro.2022.12.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 12/19/2022] [Accepted: 12/22/2022] [Indexed: 12/26/2022] Open
Abstract
Background and purpose With deep-learning, gross tumour volume (GTV) auto-segmentation has substantially been improved, but still substantial manual corrections are needed. With interactive deep-learning (iDL), manual corrections can be used to update a deep-learning tool while delineating, minimising the input to achieve acceptable segmentations. We present an iDL tool for GTV segmentation that took annotated slices as input and simulated its performance on a head and neck cancer (HNC) dataset. Materials and methods Multimodal image data of 204 HNC patients with clinical tumour and lymph node GTV delineations were used. A baseline convolutional neural network (CNN) was trained (n = 107 training, n = 22 validation) and tested (n = 24). Subsequently, user input was simulated on initial test set by replacing one or more of predicted slices with ground truth delineation, followed by re-training the CNN. The objective was to optimise re-training parameters and simulate slice selection scenarios while limiting annotations to maximally-five slices. The remaining 51 patients were used as an independent test set, where Dice similarity coefficient (DSC), mean surface distance (MSD), and 95% Hausdorff distance (HD95%) were assessed at baseline and after every update. Results Median segmentation accuracy at baseline was DSC = 0.65, MSD = 4.3 mm, HD95% = 17.5 mm. Updating CNN using three slices equally sampled from the craniocaudal axis of the GTV in the first round, followed by two rounds of annotating one extra slice, gave the best results. The accuracy improved to DSC = 0.82, MSD = 1.6 mm, HD95% = 4.8 mm. Every CNN update took 30 s. Conclusions The presented iDL tool achieved substantial segmentation improvement with only five annotated slices.
Collapse
Affiliation(s)
- Zixiang Wei
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Jintao Ren
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Stine Sofia Korreman
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark,Department of Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Jasper Nijkamp
- Aarhus University, Department of Clinical Medicine, Aarhus, Denmark,Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark,Corresponding author at: Palle Juul Jensensboulevard 25, 8200 Aarhus, Denmark.
| |
Collapse
|
26
|
Roper J, Lin M, Rong Y. Extensive upfront validation and testing are needed prior to the clinical implementation of AI-based auto-segmentation tools. J Appl Clin Med Phys 2022; 24:e13873. [PMID: 36545883 PMCID: PMC9859989 DOI: 10.1002/acm2.13873] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 11/30/2022] [Accepted: 12/01/2022] [Indexed: 12/24/2022] Open
Affiliation(s)
- Justin Roper
- Department of Radiation OncologyWinship Cancer Institute of Emory UniversityAtlantaGeorgiaUSA
| | - Mu‐Han Lin
- Department of Radiation OncologyUniversity of Texas Southwestern Medical CenterDallasTexasUSA
| | - Yi Rong
- Department of Radiation OncologyMayo Clinic HospitalsPhoenixArizonaUSA
| |
Collapse
|
27
|
Gurney-Champion OJ, Landry G, Redalen KR, Thorwarth D. Potential of Deep Learning in Quantitative Magnetic Resonance Imaging for Personalized Radiotherapy. Semin Radiat Oncol 2022; 32:377-388. [DOI: 10.1016/j.semradonc.2022.06.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
28
|
Savjani RR, Lauria M, Bose S, Deng J, Yuan Y, Andrearczyk V. Automated Tumor Segmentation in Radiotherapy. Semin Radiat Oncol 2022; 32:319-329. [DOI: 10.1016/j.semradonc.2022.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
29
|
Taku N, Wahid KA, van Dijk LV, Sahlsten J, Jaskari J, Kaski K, Fuller CD, Naser MA. Auto-detection and segmentation of involved lymph nodes in HPV-associated oropharyngeal cancer using a convolutional deep learning neural network. Clin Transl Radiat Oncol 2022; 36:47-55. [PMID: 35782963 PMCID: PMC9240370 DOI: 10.1016/j.ctro.2022.06.007] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 06/09/2022] [Accepted: 06/16/2022] [Indexed: 11/16/2022] Open
Abstract
Patients diagnosed with HPV-associated OPC are often found to have clinical evidence of lymph node involvement at the time of diagnosis. Manual segmentation of radiographically involved lymph nodes is an integral part of treatment planning for those patients dispositioned to definitive radiotherapy. We present a deep learning, convolutional neural network that can be used to automate the process of lymph node detection and segmentation and with potential to increase the efficiency to the radiotherapy workflow.
Purpose Segmentation of involved lymph nodes on head and neck computed tomography (HN-CT) scans is necessary for the radiotherapy planning of early-stage human papilloma virus (HPV) associated oropharynx cancers (OPC). We aimed to train a deep learning convolutional neural network (DL-CNN) to segment involved lymph nodes on HN-CT scans. Methods Ground-truth segmentation of involved nodes was performed on pre-surgical HN-CT scans for 90 patients who underwent levels II-IV neck dissection for node-positive HPV-OPC (training/validation [n = 70] and testing [n = 20]). A 5-fold cross validation approach was used to train 5 DL-CNN sub-models based on a residual U-net architecture. Validation and testing segmentation masks were compared to ground-truth masks using predetermined metrics. A lymph auto-detection model to discriminate between “node-positive” and “node-negative” HN-CT scans was developed by thresholding segmentation model outputs and evaluated using the area under the receiver operating characteristic curve (AUC). Results In the DL-CNN validation phase, all sub-models yielded segmentation masks with median Dice ≥ 0.90 and median volume similarity score of ≥ 0.95. In the testing phase, the DL-CNN produced consensus segmentation masks with median Dice of 0.92 (IQR, 0.89–0.95), median volume similarity of 0.97 (IQR, 0.94–0.99), and median Hausdorff distance of 4.52 mm (IQR, 1.22–8.38). The detection model achieved an AUC of 0.98. Conclusion The results from this single-institution study demonstrate the successful automation of lymph node segmentation for patients with HPV-OPC using a DL-CNN. Future studies, including validation with an external dataset, are necessary to clarify its role in the larger radiation oncology treatment planning workflow.
Collapse
|
30
|
Field M, I Thwaites D, Carolan M, Delaney GP, Lehmann J, Sykes J, Vinod S, Holloway L. Infrastructure platform for privacy-preserving distributed machine learning development of computer-assisted theragnostics in cancer. J Biomed Inform 2022; 134:104181. [PMID: 36055639 DOI: 10.1016/j.jbi.2022.104181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Revised: 04/29/2022] [Accepted: 08/20/2022] [Indexed: 11/26/2022]
Abstract
INTRODUCTION Emerging evidence suggests that data-driven support tools have found their way into clinical decision-making in a number of areas, including cancer care. Improving them and widening their scope of availability in various differing clinical scenarios, including for prognostic models derived from retrospective data, requires co-ordinated data sharing between clinical centres, secondary analyses of large multi-institutional clinical trial data, or distributed (federated) learning infrastructures. A systematic approach to utilizing routinely collected data across cancer care clinics remains a significant challenge due to privacy, administrative and political barriers. METHODS An information technology infrastructure and web service software was developed and implemented which uses machine learning to construct clinical decision support systems in a privacy-preserving manner across datasets geographically distributed in different hospitals. The infrastructure was deployed in a network of Australian hospitals. A harmonized, international ontology-linked, set of lung cancer databases were built with the routine clinical and imaging data at each centre. The infrastructure was demonstrated with the development of logistic regression models to predict major cardiovascular events following radiation therapy. RESULTS The infrastructure implemented forms the basis of the Australian computer-assisted theragnostics (AusCAT) network for radiation oncology data extraction, reporting and distributed learning. Four radiation oncology departments (across seven hospitals) in New South Wales (NSW) participated in this demonstration study. Infrastructure was deployed at each centre and used to develop a model predicting for cardiovascular admission within a year of receiving curative radiotherapy for non-small cell lung cancer. A total of 10417 lung cancer patients were identified with 802 being eligible for the model. Twenty features were chosen for analysis from the clinical record and linked registries. After selection, 8 features were included and a logistic regression model achieved an area under the receiver operating characteristic (AUROC) curve of 0.70 and C-index of 0.65 on out-of-sample data. CONCLUSION The infrastructure developed was demonstrated to be usable in practice between clinical centres to harmonize routinely collected oncology data and develop models with federated learning. It provides a promising approach to enable further research studies in radiation oncology using real world clinical data.
Collapse
Affiliation(s)
- Matthew Field
- South Western Sydney Clinical Campus, School of Clinical Medicine, University of New South Wales, NSW, Australia; South Western Sydney Cancer Services, NSW Health, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia.
| | - David I Thwaites
- Institute of Medical Physics, School of Physics, University of Sydney, NSW, Australia
| | - Martin Carolan
- Illawarra Cancer Care Centre, Wollongong, NSW, Australia
| | - Geoff P Delaney
- South Western Sydney Clinical Campus, School of Clinical Medicine, University of New South Wales, NSW, Australia; South Western Sydney Cancer Services, NSW Health, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia
| | - Joerg Lehmann
- Institute of Medical Physics, School of Physics, University of Sydney, NSW, Australia; Department of Radiation Oncology, Calvary Mater Newcastle, NSW, Australia
| | - Jonathan Sykes
- Institute of Medical Physics, School of Physics, University of Sydney, NSW, Australia; Blacktown Haematology and Oncology Cancer Care Centre, Blacktown Hospital, Blacktown, NSW, Australia; Crown Princess Mary Cancer Centre, Westmead Hospital, Westmead, NSW, Australia
| | - Shalini Vinod
- South Western Sydney Clinical Campus, School of Clinical Medicine, University of New South Wales, NSW, Australia; South Western Sydney Cancer Services, NSW Health, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia
| | - Lois Holloway
- South Western Sydney Clinical Campus, School of Clinical Medicine, University of New South Wales, NSW, Australia; South Western Sydney Cancer Services, NSW Health, Sydney, NSW, Australia; Ingham Institute for Applied Medical Research, Liverpool, NSW, Australia; Institute of Medical Physics, School of Physics, University of Sydney, NSW, Australia
| |
Collapse
|
31
|
Tryggestad E, Anand A, Beltran C, Brooks J, Cimmiyotti J, Grimaldi N, Hodge T, Hunzeker A, Lucido JJ, Laack NN, Momoh R, Moseley DJ, Patel SH, Ridgway A, Seetamsetty S, Shiraishi S, Undahl L, Foote RL. Scalable radiotherapy data curation infrastructure for deep-learning based autosegmentation of organs-at-risk: A case study in head and neck cancer. Front Oncol 2022; 12:936134. [PMID: 36106100 PMCID: PMC9464982 DOI: 10.3389/fonc.2022.936134] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 08/03/2022] [Indexed: 12/02/2022] Open
Abstract
In this era of patient-centered, outcomes-driven and adaptive radiotherapy, deep learning is now being successfully applied to tackle imaging-related workflow bottlenecks such as autosegmentation and dose planning. These applications typically require supervised learning approaches enabled by relatively large, curated radiotherapy datasets which are highly reflective of the contemporary standard of care. However, little has been previously published describing technical infrastructure, recommendations, methods or standards for radiotherapy dataset curation in a holistic fashion. Our radiation oncology department has recently embarked on a large-scale project in partnership with an external partner to develop deep-learning-based tools to assist with our radiotherapy workflow, beginning with autosegmentation of organs-at-risk. This project will require thousands of carefully curated radiotherapy datasets comprising all body sites we routinely treat with radiotherapy. Given such a large project scope, we have approached the need for dataset curation rigorously, with an aim towards building infrastructure that is compatible with efficiency, automation and scalability. Focusing on our first use-case pertaining to head and neck cancer, we describe our developed infrastructure and novel methods applied to radiotherapy dataset curation, inclusive of personnel and workflow organization, dataset selection, expert organ-at-risk segmentation, quality assurance, patient de-identification, data archival and transfer. Over the course of approximately 13 months, our expert multidisciplinary team generated 490 curated head and neck radiotherapy datasets. This task required approximately 6000 human-expert hours in total (not including planning and infrastructure development time). This infrastructure continues to evolve and will support ongoing and future project efforts.
Collapse
Affiliation(s)
- E. Tryggestad
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
- *Correspondence: E. Tryggestad,
| | - A. Anand
- Department of Radiation Oncology, Mayo Clinic Arizona, Phoenix, AZ, United States
| | - C. Beltran
- Department of Radiation Oncology, Mayo Clinic Florida, Jacksonville, FL, United States
| | - J. Brooks
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - J. Cimmiyotti
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - N. Grimaldi
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - T. Hodge
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - A. Hunzeker
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - J. J. Lucido
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - N. N. Laack
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - R. Momoh
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - D. J. Moseley
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - S. H. Patel
- Department of Radiation Oncology, Mayo Clinic Arizona, Phoenix, AZ, United States
| | - A. Ridgway
- Department of Radiation Oncology, Mayo Clinic Arizona, Phoenix, AZ, United States
| | - S. Seetamsetty
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - S. Shiraishi
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - L. Undahl
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - R. L. Foote
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| |
Collapse
|
32
|
Delpon G, Barateau A, Beneux A, Bessières I, Latorzeff I, Welmant J, Tallet A. [What do we need to deliver "online" adapted radiotherapy treatment plans?]. Cancer Radiother 2022; 26:794-802. [PMID: 36028418 DOI: 10.1016/j.canrad.2022.06.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 06/27/2022] [Accepted: 06/29/2022] [Indexed: 11/17/2022]
Abstract
During the joint SFRO/SFPM session of the 2019 congress, a state of the art of adaptive radiotherapy announced a strong impact in our clinical practice, in particular with the availability of treatment devices coupled to an MRI system. Three years later, it seems relevant to take stock of adaptive radiotherapy in practice, and especially the "online" strategy because it is indeed more and more accessible with recent hardware and software developments, such as coupled accelerators to a three-dimensional imaging device and algorithms based on artificial intelligence. However, the deployment of this promising strategy is complex because it contracts the usual time scale and upsets the usual organizations. So what do we need to deliver adapted treatment plans with an "online" strategy?
Collapse
Affiliation(s)
- G Delpon
- Institut de cancérologie de l'Ouest, Saint-Herblain et IMT Atlantique, Nantes université, CNRS/IN2P3, Subatech, Nantes, France.
| | - A Barateau
- Université Rennes, CLCC Eugène-Marquis, Inserm, LTSI-UMR 1099, Rennes, France
| | - A Beneux
- Hospices Civils de Lyon, Lyon, France
| | - I Bessières
- Centre Georges-François Leclerc, Dijon, France
| | | | - J Welmant
- Institut du cancer de Montpellier, Montpellier, France
| | - A Tallet
- Institut Paoli-Calmettes, Marseille, France
| |
Collapse
|
33
|
Wang X, Fan Y, Zhang N, Li J, Duan Y, Yang B. Performance of Machine Learning for Tissue Outcome Prediction in Acute Ischemic Stroke: A Systematic Review and Meta-Analysis. Front Neurol 2022; 13:910259. [PMID: 35873778 PMCID: PMC9305175 DOI: 10.3389/fneur.2022.910259] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 06/20/2022] [Indexed: 12/03/2022] Open
Abstract
Machine learning (ML) has been proposed for lesion segmentation in acute ischemic stroke (AIS). This study aimed to provide a systematic review and meta-analysis of the overall performance of current ML algorithms for final infarct prediction from baseline imaging. We made a comprehensive literature search on eligible studies developing ML models for core infarcted tissue estimation on admission CT or MRI in AIS patients. Eleven studies meeting the inclusion criteria were included in the quantitative analysis. Study characteristics, model methodology, and predictive performance of the included studies were extracted. A meta-analysis was conducted on the dice similarity coefficient (DSC) score by using a random-effects model to assess the overall predictive performance. Study heterogeneity was assessed by Cochrane Q and Higgins I2 tests. The pooled DSC score of the included ML models was 0.50 (95% CI 0.39–0.61), with high heterogeneity observed across studies (I2 96.5%, p < 0.001). Sensitivity analyses using the one-study removed method showed the adjusted overall DSC score ranged from 0.47 to 0.52. Subgroup analyses indicated that the DL-based models outperformed the conventional ML classifiers with the best performance observed in DL algorithms combined with CT data. Despite the presence of heterogeneity, current ML-based approaches for final infarct prediction showed moderate but promising performance. Before well integrated into clinical stroke workflow, future investigations are suggested to train ML models on large-scale, multi-vendor data, validate on external cohorts and adopt formalized reporting standards for improving model accuracy and robustness.
Collapse
Affiliation(s)
- Xinrui Wang
- Department of Radiology, General Hospital of Northern Theater Command, Shenyang, China
| | - Yiming Fan
- Department of Orthopedics, Chinese PLA General Hospital, Beijing, China
| | - Nan Zhang
- Department of Radiology, General Hospital of Northern Theater Command, Shenyang, China
| | - Jing Li
- Department of Radiology, Changhai Hospital, Shanghai, China
| | - Yang Duan
- Department of Radiology, General Hospital of Northern Theater Command, Shenyang, China
| | - Benqiang Yang
- Department of Radiology, General Hospital of Northern Theater Command, Shenyang, China
- *Correspondence: Benqiang Yang
| |
Collapse
|
34
|
Lempart M, Nilsson MP, Scherman J, Gustafsson CJ, Nilsson M, Alkner S, Engleson J, Adrian G, Munck Af Rosenschöld P, Olsson LE. Pelvic U-Net: multi-label semantic segmentation of pelvic organs at risk for radiation therapy anal cancer patients using a deeply supervised shuffle attention convolutional neural network. Radiat Oncol 2022; 17:114. [PMID: 35765038 PMCID: PMC9238000 DOI: 10.1186/s13014-022-02088-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Accepted: 06/20/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Delineation of organs at risk (OAR) for anal cancer radiation therapy treatment planning is a manual and time-consuming process. Deep learning-based methods can accelerate and partially automate this task. The aim of this study was to develop and evaluate a deep learning model for automated and improved segmentations of OAR in the pelvic region. METHODS A 3D, deeply supervised U-Net architecture with shuffle attention, referred to as Pelvic U-Net, was trained on 143 computed tomography (CT) volumes, to segment OAR in the pelvic region, such as total bone marrow, rectum, bladder, and bowel structures. Model predictions were evaluated on an independent test dataset (n = 15) using the Dice similarity coefficient (DSC), the 95th percentile of the Hausdorff distance (HD95), and the mean surface distance (MSD). In addition, three experienced radiation oncologists rated model predictions on a scale between 1-4 (excellent, good, acceptable, not acceptable). Model performance was also evaluated with respect to segmentation time, by comparing complete manual delineation time against model prediction time without and with manual correction of the predictions. Furthermore, dosimetric implications to treatment plans were evaluated using different dose-volume histogram (DVH) indices. RESULTS Without any manual corrections, mean DSC values of 97%, 87% and 94% were found for total bone marrow, rectum, and bladder. Mean DSC values for bowel cavity, all bowel, small bowel, and large bowel were 95%, 91%, 87% and 81%, respectively. Total bone marrow, bladder, and bowel cavity segmentations derived from our model were rated excellent (89%, 93%, 42%), good (9%, 5%, 42%), or acceptable (2%, 2%, 16%) on average. For almost all the evaluated DVH indices, no significant difference between model predictions and manual delineations was found. Delineation time per patient could be reduced from 40 to 12 min, including manual corrections of model predictions, and to 4 min without corrections. CONCLUSIONS Our Pelvic U-Net led to credible and clinically applicable OAR segmentations and showed improved performance compared to previous studies. Even though manual adjustments were needed for some predicted structures, segmentation time could be reduced by 70% on average. This allows for an accelerated radiation therapy treatment planning workflow for anal cancer patients.
Collapse
Affiliation(s)
- Michael Lempart
- Radiation Physics, Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden. .,Department of Translational Medicine, Medical Radiation Physics, Lund University, Malmö, Sweden.
| | - Martin P Nilsson
- Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden
| | - Jonas Scherman
- Radiation Physics, Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden
| | - Christian Jamtheim Gustafsson
- Radiation Physics, Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Translational Medicine, Medical Radiation Physics, Lund University, Malmö, Sweden
| | - Mikael Nilsson
- Centre for Mathematical Sciences, Lund University, Lund, Sweden
| | - Sara Alkner
- Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Clinical Sciences, Division of Oncology and Pathology, Lund University, Lund, Sweden
| | - Jens Engleson
- Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden
| | - Gabriel Adrian
- Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Clinical Sciences, Division of Oncology and Pathology, Lund University, Lund, Sweden
| | - Per Munck Af Rosenschöld
- Radiation Physics, Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Medical Radiation Physics, Lund University, Lund, Sweden
| | - Lars E Olsson
- Radiation Physics, Department of Hematology, Oncology, and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Translational Medicine, Medical Radiation Physics, Lund University, Malmö, Sweden
| |
Collapse
|
35
|
Hallinan JTPD, Zhu L, Zhang W, Lim DSW, Baskar S, Low XZ, Yeong KY, Teo EC, Kumarakulasinghe NB, Yap QV, Chan YH, Lin S, Tan JH, Kumar N, Vellayappan BA, Ooi BC, Quek ST, Makmur A. Deep Learning Model for Classifying Metastatic Epidural Spinal Cord Compression on MRI. Front Oncol 2022; 12:849447. [PMID: 35600347 PMCID: PMC9114468 DOI: 10.3389/fonc.2022.849447] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Accepted: 03/18/2022] [Indexed: 11/13/2022] Open
Abstract
Background Metastatic epidural spinal cord compression (MESCC) is a devastating complication of advanced cancer. A deep learning (DL) model for automated MESCC classification on MRI could aid earlier diagnosis and referral. Purpose To develop a DL model for automated classification of MESCC on MRI. Materials and Methods Patients with known MESCC diagnosed on MRI between September 2007 and September 2017 were eligible. MRI studies with instrumentation, suboptimal image quality, and non-thoracic regions were excluded. Axial T2-weighted images were utilized. The internal dataset split was 82% and 18% for training/validation and test sets, respectively. External testing was also performed. Internal training/validation data were labeled using the Bilsky MESCC classification by a musculoskeletal radiologist (10-year experience) and a neuroradiologist (5-year experience). These labels were used to train a DL model utilizing a prototypical convolutional neural network. Internal and external test sets were labeled by the musculoskeletal radiologist as the reference standard. For assessment of DL model performance and interobserver variability, test sets were labeled independently by the neuroradiologist (5-year experience), a spine surgeon (5-year experience), and a radiation oncologist (11-year experience). Inter-rater agreement (Gwet’s kappa) and sensitivity/specificity were calculated. Results Overall, 215 MRI spine studies were analyzed [164 patients, mean age = 62 ± 12(SD)] with 177 (82%) for training/validation and 38 (18%) for internal testing. For internal testing, the DL model and specialists all showed almost perfect agreement (kappas = 0.92–0.98, p < 0.001) for dichotomous Bilsky classification (low versus high grade) compared to the reference standard. Similar performance was seen for external testing on a set of 32 MRI spines with the DL model and specialists all showing almost perfect agreement (kappas = 0.94–0.95, p < 0.001) compared to the reference standard. Conclusion A DL model showed comparable agreement to a subspecialist radiologist and clinical specialists for the classification of malignant epidural spinal cord compression and could optimize earlier diagnosis and surgical referral.
Collapse
Affiliation(s)
- James Thomas Patrick Decourcy Hallinan
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore.,Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Lei Zhu
- NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore, Singapore
| | - Wenqiao Zhang
- Department of Computer Science, School of Computing, National University of Singapore, Singapore, Singapore
| | - Desmond Shi Wei Lim
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore.,Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Sangeetha Baskar
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore
| | - Xi Zhen Low
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore.,Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Kuan Yuen Yeong
- Department of Radiology, Ng Teng Fong General Hospital, Singapore, Singapore
| | - Ee Chin Teo
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore
| | | | - Qai Ven Yap
- Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore, Singapore
| | - Yiong Huak Chan
- Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore, Singapore
| | - Shuxun Lin
- Division of Spine Surgery, Department of Orthopaedic Surgery, Ng Teng Fong General Hospital, Singapore, Singapore
| | - Jiong Hao Tan
- University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore, Singapore
| | - Naresh Kumar
- University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore, Singapore
| | - Balamurugan A Vellayappan
- Department of Radiation Oncology, National University Cancer Institute Singapore, National University Hospital, Singapore, Singapore
| | - Beng Chin Ooi
- Department of Computer Science, School of Computing, National University of Singapore, Singapore, Singapore
| | - Swee Tian Quek
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore.,Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Andrew Makmur
- Department of Diagnostic Imaging, National University Hospital, Singapore, Singapore.,Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| |
Collapse
|