1
|
Vivancos Bargalló H, Stick LB, Korreman SS, Kronborg C, Nielsen MM, Borgen AC, Offersen BV, Nørrevang O, Kallehauge JF. Classification of laterality and mastectomy/lumpectomy for breast cancer patients for improved performance of deep learning auto segmentation. Acta Oncol 2023; 62:1546-1550. [PMID: 37584197 DOI: 10.1080/0284186x.2023.2245965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 08/03/2023] [Indexed: 08/17/2023]
Affiliation(s)
- Helena Vivancos Bargalló
- Medical Physics department, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | | | - Stine Sofia Korreman
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Camilla Kronborg
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Mathias M Nielsen
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | | | - Birgitte Vrou Offersen
- Department of Experimental Clinical Oncologyy, Aarhus University Hospital, Aarhus, Denmark
- Department of Experimental Clinical Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Ole Nørrevang
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Jesper F Kallehauge
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| |
Collapse
|
2
|
Pang S, Du A, Orgun MA, Wang Y, Sheng QZ, Wang S, Huang X, Yu Z. Beyond CNNs: Exploiting Further Inherent Symmetries in Medical Image Segmentation. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:6776-6787. [PMID: 36044511 DOI: 10.1109/tcyb.2022.3195447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Automatic tumor or lesion segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because, although the human visual system can detect symmetries in 2-D images effectively, regular CNNs can only exploit translation invariance, overlooking further inherent symmetries existing in medical images, such as rotations and reflections. To solve this problem, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning more precise representations. First, kernel-based equivariant operations are devised on each orientation, which allows it to effectively address the gaps of learning symmetries in existing approaches. Then, to keep segmentation networks globally equivariant, we design distinctive group layers with layer-wise symmetry constraints. Finally, based on our novel framework, extensive experiments conducted on real-world clinical data demonstrate that a group equivariant Res-UNet (called GER-UNet) outperforms its regular CNN-based counterpart and the state-of-the-art segmentation methods in the tasks of hepatic tumor segmentation, COVID-19 lung infection segmentation, and retinal vessel detection. More importantly, the newly built GER-UNet also shows potential in reducing the sample complexity and the redundancy of filters, upgrading current segmentation CNNs, and delineating organs on other medical imaging modalities.
Collapse
|
3
|
Berkley A, Saueressig C, Shukla U, Chowdhury I, Munoz-Gauna A, Shehu O, Singh R, Munbodh R. Clinical capability of modern brain tumor segmentation models. Med Phys 2023; 50:4943-4959. [PMID: 36847185 DOI: 10.1002/mp.16321] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 10/27/2022] [Accepted: 10/27/2022] [Indexed: 03/01/2023] Open
Abstract
PURPOSE State-of-the-art automated segmentation methods achieve exceptionally high performance on the Brain Tumor Segmentation (BraTS) challenge, a dataset of uniformly processed and standardized magnetic resonance generated images (MRIs) of gliomas. However, a reasonable concern is that these models may not fare well on clinical MRIs that do not belong to the specially curated BraTS dataset. Research using the previous generation of deep learning models indicates significant performance loss on cross-institutional predictions. Here, we evaluate the cross-institutional applicability and generalzsability of state-of-the-art deep learning models on new clinical data. METHODS We train a state-of-the-art 3D U-Net model on the conventional BraTS dataset comprising low- and high-grade gliomas. We then evaluate the performance of this model for automatic tumor segmentation of brain tumors on in-house clinical data. This dataset contains MRIs of different tumor types, resolutions, and standardization than those found in the BraTS dataset. Ground truth segmentations to validate the automated segmentation for in-house clinical data were obtained from expert radiation oncologists. RESULTS We report average Dice scores of 0.764, 0.648, and 0.61 for the whole tumor, tumor core, and enhancing tumor, respectively, in the clinical MRIs. These means are higher than numbers reported previously on same institution and cross-institution datasets of different origin using different methods. There is no statistically significant difference when comparing the dice scores to the inter-annotation variability between two expert clinical radiation oncologists. Although performance on the clinical data is lower than on the BraTS data, these numbers indicate that models trained on the BraTS dataset have impressive segmentation performance on previously unseen images obtained at a separate clinical institution. These images differ in the imaging resolutions, standardization pipelines, and tumor types from the BraTS data. CONCLUSIONS State-of-the-art deep learning models demonstrate promising performance on cross-institutional predictions. They considerably improve on previous models and can transfer knowledge to new types of brain tumors without additional modeling.
Collapse
Affiliation(s)
- Adam Berkley
- Department of Computer Science, Brown University, Providence, Rhode Island, USA
| | - Camillo Saueressig
- Department of Computer Science, Brown University, Providence, Rhode Island, USA
| | - Utkarsh Shukla
- Department of Radiation Oncology, Rhode Island Hospital, Providence, Rhode Island, USA
- The Department of Radiation Oncology at Tufts Medical Center, Boston, Massachusetts, USA
- Department of Radiation Oncology, Warren Alpert Medical School of Brown University, Providence, Rhode Island, USA
| | - Imran Chowdhury
- Department of Radiation Oncology, Rhode Island Hospital, Providence, Rhode Island, USA
- The Department of Radiation Oncology at Tufts Medical Center, Boston, Massachusetts, USA
- Department of Radiation Oncology, Warren Alpert Medical School of Brown University, Providence, Rhode Island, USA
| | - Anthony Munoz-Gauna
- Department of Radiation Oncology, Warren Alpert Medical School of Brown University, Providence, Rhode Island, USA
| | - Olalekan Shehu
- Department of Physics, University of Rhode Island, Kingston, Rhode Island, USA
| | - Ritambhara Singh
- Department of Computer Science, Brown University, Providence, Rhode Island, USA
- Center for Computational Molecular Biology, Brown University, Providence, Rhode Island, USA
| | - Reshma Munbodh
- Department of Radiation Oncology, Warren Alpert Medical School of Brown University, Providence, Rhode Island, USA
- Department of Radiation Oncology, Columbia University Irving Medical Center, New York, New York, USA
| |
Collapse
|
4
|
Bakx N, Rijkaart D, van der Sangen M, Theuws J, van der Toorn PP, Verrijssen AS, van der Leer J, Mutsaers J, van Nunen T, Reinders M, Schuengel I, Smits J, Hagelaar E, van Gruijthuijsen D, Bluemink H, Hurkmans C. Clinical evaluation of a deep learning segmentation model including manual adjustments afterwards for locally advanced breast cancer. Tech Innov Patient Support Radiat Oncol 2023; 26:100211. [PMID: 37229460 PMCID: PMC10205480 DOI: 10.1016/j.tipsro.2023.100211] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 04/23/2023] [Accepted: 05/09/2023] [Indexed: 05/27/2023] Open
Abstract
Introduction Deep learning (DL) models are increasingly developed for auto-segmentation in radiotherapy. Qualitative analysis is of great importance for clinical implementation, next to quantitative. This study evaluates a DL segmentation model for left- and right-sided locally advanced breast cancer both quantitatively and qualitatively. Methods For each side a DL model was trained, including primary breast CTV (CTVp), lymph node levels 1-4, heart, lungs, humeral head, thyroid and esophagus. For evaluation, both automatic segmentation, including correction of contours when needed, and manual delineation was performed and both processes were timed. Quantitative scoring with dice-similarity coefficient (DSC), 95% Hausdorff Distance (95%HD) and surface DSC (sDSC) was used to compare both the automatic (not-corrected) and corrected contours with the manual contours. Qualitative scoring was performed by five radiotherapy technologists and five radiation oncologists using a 3-point Likert scale. Results Time reduction was achieved using auto-segmentation in 95% of the cases, including correction. The time reduction (mean ± std) was 42.4% ± 26.5% and 58.5% ± 19.1% for OARs and CTVs, respectively, corresponding to an absolute mean reduction (hh:mm:ss) of 00:08:51 and 00:25:38. Good quantitative results were achieved before correction, e.g. mean DSC for the right-sided CTVp was 0.92 ± 0.06, whereas correction statistically significantly improved this contour by only 0.02 ± 0.05, respectively. In 92% of the cases, auto-contours were scored as clinically acceptable, with or without corrections. Conclusions A DL segmentation model was trained and was shown to be a time-efficient way to generate clinically acceptable contours for locally advanced breast cancer.
Collapse
Affiliation(s)
- Nienke Bakx
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Dorien Rijkaart
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | | | - Jacqueline Theuws
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | | | - An-Sofie Verrijssen
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Jorien van der Leer
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Joline Mutsaers
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Thérèse van Nunen
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Marjon Reinders
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Inge Schuengel
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Julia Smits
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Els Hagelaar
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | | | - Hanneke Bluemink
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Coen Hurkmans
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
- Technical University Eindhoven, Faculties of Physics and Electrical Engineering, Eindhoven, the Netherlands
| |
Collapse
|
5
|
Kazemimoghadam M, Yang Z, Chen M, Rahimi A, Kim N, Alluri P, Nwachukwu C, Lu W, Gu X. A deep learning approach for automatic delineation of clinical target volume in stereotactic partial breast irradiation (S-PBI). Phys Med Biol 2023; 68:10.1088/1361-6560/accf5e. [PMID: 37084739 PMCID: PMC10325028 DOI: 10.1088/1361-6560/accf5e] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Accepted: 04/21/2023] [Indexed: 04/23/2023]
Abstract
Accurate and efficient delineation of the clinical target volume (CTV) is of utmost significance in post-operative breast cancer radiotherapy. However, CTV delineation is challenging as the exact extent of microscopic disease encompassed by CTV is not visualizable in radiological images and remains uncertain. We proposed to mimic physicians' contouring practice for CTV segmentation in stereotactic partial breast irradiation (S-PBI) where CTV is derived from tumor bed volume (TBV) via a margin expansion followed by correcting the extensions for anatomical barriers of tumor invasion (e.g. skin, chest wall). We proposed a deep-learning model, where CT images and the corresponding TBV masks formed a multi-channel input for a 3D U-Net based architecture. The design guided the model to encode the location-related image features and directed the network to focus on TBV to initiate CTV segmentation. Gradient weighted class activation map (Grad-CAM) visualizations of the model predictions revealed that the extension rules and geometric/anatomical boundaries were learnt during model training to assist the network to limit the expansion to a certain distance from the chest wall and the skin. We retrospectively collected 175 prone CT images from 35 post-operative breast cancer patients who received 5-fraction partial breast irradiation regimen on GammaPod. The 35 patients were randomly split into training (25), validation (5) and test (5) sets. Our model achieved mean (standard deviation) of 0.94 (±0.02), 2.46 (±0.5) mm, and 0.53 (±0.14) mm for Dice similarity coefficient, 95th percentile Hausdorff distance, and average symmetric surface distance respectively on the test set. The results are promising for improving the efficiency and accuracy of CTV delineation during on-line treatment planning procedure.
Collapse
Affiliation(s)
- Mahdieh Kazemimoghadam
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Zi Yang
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Mingli Chen
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Asal Rahimi
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Nathan Kim
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Prasanna Alluri
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Chika Nwachukwu
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Weiguo Lu
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Xuejun Gu
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
- Department of Radiation Oncology, Stanford University, Stanford, CA 94305
| |
Collapse
|
6
|
ME-CCNN: Multi-encoded images and a cascade convolutional neural network for breast tumor segmentation and recognition. Artif Intell Rev 2023. [DOI: 10.1007/s10462-023-10426-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/21/2023]
|
7
|
Hu Z, Wang B, Pan X, Cao D, Gao A, Yang X, Chen Y, Lin Z. Using deep learning to distinguish malignant from benign parotid tumors on plain computed tomography images. Front Oncol 2022; 12:919088. [PMID: 35978811 PMCID: PMC9376440 DOI: 10.3389/fonc.2022.919088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Accepted: 06/28/2022] [Indexed: 11/13/2022] Open
Abstract
Objectives Evaluating the diagnostic efficiency of deep-learning models to distinguish malignant from benign parotid tumors on plain computed tomography (CT) images. Materials and methods The CT images of 283 patients with parotid tumors were enrolled and analyzed retrospectively. Of them, 150 were benign and 133 were malignant according to pathology results. A total of 917 regions of interest of parotid tumors were cropped (456 benign and 461 malignant). Three deep-learning networks (ResNet50, VGG16_bn, and DenseNet169) were used for diagnosis (approximately 3:1 for training and testing). The diagnostic efficiencies (accuracy, sensitivity, specificity, and area under the curve [AUC]) of three networks were calculated and compared based on the 917 images. To simulate the process of human diagnosis, a voting model was developed at the end of the networks and the 283 tumors were classified as benign or malignant. Meanwhile, 917 tumor images were classified by two radiologists (A and B) and original CT images were classified by radiologist B. The diagnostic efficiencies of the three deep-learning network models (after voting) and the two radiologists were calculated. Results For the 917 CT images, ResNet50 presented high accuracy and sensitivity for diagnosing malignant parotid tumors; the accuracy, sensitivity, specificity, and AUC were 90.8%, 91.3%, 90.4%, and 0.96, respectively. For the 283 tumors, the accuracy, sensitivity, and specificity of ResNet50 (after voting) were 92.3%, 93.5% and 91.2%, respectively. Conclusion ResNet50 presented high sensitivity in distinguishing malignant from benign parotid tumors on plain CT images; this made it a promising auxiliary diagnostic method to screen malignant parotid tumors.
Collapse
Affiliation(s)
- Ziyang Hu
- Department of Dentomaxillofacial Radiology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Baixin Wang
- School of Electronic Science and Engineering, Nanjing University, Nanjing, China
| | - Xiao Pan
- Department of Dentomaxillofacial Radiology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Dantong Cao
- Department of Dentomaxillofacial Radiology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Antian Gao
- Department of Dentomaxillofacial Radiology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
| | - Xudong Yang
- Department of Oral and Maxillofacial Surgery, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
- *Correspondence: Zitong Lin, ; Ying Chen, ; Xudong Yang,
| | - Ying Chen
- School of Electronic Science and Engineering, Nanjing University, Nanjing, China
- *Correspondence: Zitong Lin, ; Ying Chen, ; Xudong Yang,
| | - Zitong Lin
- Department of Dentomaxillofacial Radiology, Nanjing Stomatological Hospital, Medical School of Nanjing University, Nanjing, China
- *Correspondence: Zitong Lin, ; Ying Chen, ; Xudong Yang,
| |
Collapse
|
8
|
Shusharina N, Söderberg J, Lidberg D, Niyazi M, Shih HA, Bortfeld T. Accounting for uncertainties in the position of anatomical barriers used to define the clinical target volume. Phys Med Biol 2021; 66. [PMID: 34171846 DOI: 10.1088/1361-6560/ac0ea3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Accepted: 06/25/2021] [Indexed: 11/11/2022]
Abstract
The definition of the clinical target volume (CTV) is becoming the weakest link in the radiotherapy chain. CTV definition consensus guidelines include the geometric expansion beyond the visible gross tumor volume, while avoiding anatomical barriers. In a previous publication we described how to implement these consensus guidelines using deep learning and graph search techniques in a computerized CTV auto-delineation process. In this paper we address the remaining problem of how to deal with uncertainties in positions of the anatomical barriers. The objective was to develop an algorithm that implements the consensus guidelines on considering barrier uncertainties. Our approach is to perform multiple expansions using the fast marching method with barriers in place or removed at different stages of the expansion. We validate the algorithm in a computational phantom and compare manually generated with automated CTV contours, both taking barrier uncertainties into account.
Collapse
Affiliation(s)
- Nadya Shusharina
- Division of Radiation Biophysics, Department of Radiation Oncology, Massachusetts General Hospital and Harvard Medical School, Boston, United States of America
| | | | | | - Maximilian Niyazi
- Department of Radiation Oncology, University Hospital, LMU Munich, Munich, Germany.,German Cancer Consortium (DKTK), Partner Site Munich, Munich, Germany
| | - Helen A Shih
- Department of Radiation Oncology, Massachusetts General Hospital and Harvard Medical School, Boston, United States of America
| | - Thomas Bortfeld
- Division of Radiation Biophysics, Department of Radiation Oncology, Massachusetts General Hospital and Harvard Medical School, Boston, United States of America
| |
Collapse
|