1
|
Huang L, Ruan S, Xing Y, Feng M. A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods. Med Image Anal 2024; 97:103223. [PMID: 38861770 DOI: 10.1016/j.media.2024.103223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Revised: 03/16/2024] [Accepted: 05/27/2024] [Indexed: 06/13/2024]
Abstract
The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the results. In this review, we offer a comprehensive overview of the prevailing methods proposed to quantify the uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models.
Collapse
Affiliation(s)
- Ling Huang
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Su Ruan
- Quantif, LITIS, University of Rouen Normandy, France.
| | - Yucheng Xing
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Mengling Feng
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore; Institute of Data Science, National University of Singapore, Singapore
| |
Collapse
|
2
|
Familiar AM, Fathi Kazerooni A, Vossough A, Ware JB, Bagheri S, Khalili N, Anderson H, Haldar D, Storm PB, Resnick AC, Kann BH, Aboian M, Kline C, Weller M, Huang RY, Chang SM, Fangusaro JR, Hoffman LM, Mueller S, Prados M, Nabavizadeh A. Towards consistency in pediatric brain tumor measurements: Challenges, solutions, and the role of artificial intelligence-based segmentation. Neuro Oncol 2024; 26:1557-1571. [PMID: 38769022 PMCID: PMC11376457 DOI: 10.1093/neuonc/noae093] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Indexed: 05/22/2024] Open
Abstract
MR imaging is central to the assessment of tumor burden and changes over time in neuro-oncology. Several response assessment guidelines have been set forth by the Response Assessment in Pediatric Neuro-Oncology (RAPNO) working groups in different tumor histologies; however, the visual delineation of tumor components using MRIs is not always straightforward, and complexities not currently addressed by these criteria can introduce inter- and intra-observer variability in manual assessments. Differentiation of non-enhancing tumors from peritumoral edema, mild enhancement from absence of enhancement, and various cystic components can be challenging; particularly given a lack of sufficient and uniform imaging protocols in clinical practice. Automated tumor segmentation with artificial intelligence (AI) may be able to provide more objective delineations, but rely on accurate and consistent training data created manually (ground truth). Herein, this paper reviews existing challenges and potential solutions to identifying and defining subregions of pediatric brain tumors (PBTs) that are not explicitly addressed by current guidelines. The goal is to assert the importance of defining and adopting criteria for addressing these challenges, as it will be critical to achieving standardized tumor measurements and reproducible response assessment in PBTs, ultimately leading to more precise outcome metrics and accurate comparisons among clinical studies.
Collapse
Affiliation(s)
- Ariana M Familiar
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Department of Neurosurgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Anahita Fathi Kazerooni
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Department of Neurosurgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- AI2D Center for AI and Data Science for Integrated Diagnostics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Arastoo Vossough
- Division of Radiology, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Jeffrey B Ware
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Sina Bagheri
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Nastaran Khalili
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Department of Neurosurgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Hannah Anderson
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Debanjan Haldar
- Department of Neurosurgery, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Phillip B Storm
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Department of Neurosurgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Adam C Resnick
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
- Department of Neurosurgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Benjamin H Kann
- Department of Radiation Oncology, Dana-Farber Cancer Institute | Brigham and Women's Hospital | Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Mariam Aboian
- Division of Radiology, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Cassie Kline
- Division of Oncology, Department of Pediatrics, Children's Hospital of Philadelphia, University of Pennsylvania, Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Michael Weller
- Department of Neurology, University Hospital and University of Zurich, Zurich, Switzerland
| | - Raymond Y Huang
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Susan M Chang
- Division of Neuro-Oncology, Department of Neurosurgery, University of California, San Francisco, California, USA
| | - Jason R Fangusaro
- The Aflac Cancer Center, Children's Healthcare of Atlanta and the Emory University School of Medicine, Atlanta, Georgia, USA
| | - Lindsey M Hoffman
- Division of Hematology/Oncology, Phoenix Children's Hospital, Phoenix, Arizona, USA
| | - Sabine Mueller
- Department of Neurology, Neurosurgery and Pediatrics, University of California, San Francisco, California, USA
| | - Michael Prados
- Department of Neurosurgery and Pediatrics, University of California, San Francisco, California, USA
| | - Ali Nabavizadeh
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Center for Data-Driven Discovery in Biomedicine (D3b), Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| |
Collapse
|
3
|
Sahlsten J, Jaskari J, Wahid KA, Ahmed S, Glerean E, He R, Kann BH, Mäkitie A, Fuller CD, Naser MA, Kaski K. Application of simultaneous uncertainty quantification and segmentation for oropharyngeal cancer use-case with Bayesian deep learning. COMMUNICATIONS MEDICINE 2024; 4:110. [PMID: 38851837 PMCID: PMC11162474 DOI: 10.1038/s43856-024-00528-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 05/16/2024] [Indexed: 06/10/2024] Open
Abstract
BACKGROUND Radiotherapy is a core treatment modality for oropharyngeal cancer (OPC), where the primary gross tumor volume (GTVp) is manually segmented with high interobserver variability. This calls for reliable and trustworthy automated tools in clinician workflow. Therefore, accurate uncertainty quantification and its downstream utilization is critical. METHODS Here we propose uncertainty-aware deep learning for OPC GTVp segmentation, and illustrate the utility of uncertainty in multiple applications. We examine two Bayesian deep learning (BDL) models and eight uncertainty measures, and utilize a large multi-institute dataset of 292 PET/CT scans to systematically analyze our approach. RESULTS We show that our uncertainty-based approach accurately predicts the quality of the deep learning segmentation in 86.6% of cases, identifies low performance cases for semi-automated correction, and visualizes regions of the scans where the segmentations likely fail. CONCLUSIONS Our BDL-based analysis provides a first-step towards more widespread implementation of uncertainty quantification in OPC GTVp segmentation.
Collapse
Affiliation(s)
- Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Sara Ahmed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
- Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland.
| |
Collapse
|
4
|
Li L, Yu J, Li Y, Wei J, Fan R, Wu D, Ye Y. Multi-sequence generative adversarial network: better generation for enhanced magnetic resonance imaging images. Front Comput Neurosci 2024; 18:1365238. [PMID: 38841427 PMCID: PMC11151883 DOI: 10.3389/fncom.2024.1365238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Accepted: 03/27/2024] [Indexed: 06/07/2024] Open
Abstract
Introduction MRI is one of the commonly used diagnostic methods in clinical practice, especially in brain diseases. There are many sequences in MRI, but T1CE images can only be obtained by using contrast agents. Many patients (such as cancer patients) must undergo alignment of multiple MRI sequences for diagnosis, especially the contrast-enhanced magnetic resonance sequence. However, some patients such as pregnant women, children, etc. find it difficult to use contrast agents to obtain enhanced sequences, and contrast agents have many adverse reactions, which can pose a significant risk. With the continuous development of deep learning, the emergence of generative adversarial networks makes it possible to extract features from one type of image to generate another type of image. Methods We propose a generative adversarial network model with multimodal inputs and end-to-end decoding based on the pix2pix model. For the pix2pix model, we used four evaluation metrics: NMSE, RMSE, SSIM, and PNSR to assess the effectiveness of our generated model. Results Through statistical analysis, we compared our proposed new model with pix2pix and found significant differences between the two. Our model outperformed pix2pix, with higher SSIM and PNSR, lower NMSE and RMSE. We also found that the input of T1W images and T2W images had better effects than other combinations, providing new ideas for subsequent work on generating magnetic resonance enhancement sequence images. By using our model, it is possible to generate magnetic resonance enhanced sequence images based on magnetic resonance non-enhanced sequence images. Discussion This has significant implications as it can greatly reduce the use of contrast agents to protect populations such as pregnant women and children who are contraindicated for contrast agents. Additionally, contrast agents are relatively expensive, and this generation method may bring about substantial economic benefits.
Collapse
Affiliation(s)
- Leizi Li
- South China Normal University-Panyu Central Hospital Joint Laboratory of Basic and Translational Medical Research, Guangzhou Panyu Central Hospital, Guangzhou, China
- Guangzhou Key Laboratory of Subtropical Biodiversity and Biomonitoring and Guangdong Provincial Engineering Technology Research Center for Drug and Food Biological Resources Processing and Comprehensive Utilization, School of Life Sciences, South China Normal University, Guangzhou, China
| | - Jingchun Yu
- Guangzhou Key Laboratory of Subtropical Biodiversity and Biomonitoring and Guangdong Provincial Engineering Technology Research Center for Drug and Food Biological Resources Processing and Comprehensive Utilization, School of Life Sciences, South China Normal University, Guangzhou, China
| | - Yijin Li
- Guangzhou Key Laboratory of Subtropical Biodiversity and Biomonitoring and Guangdong Provincial Engineering Technology Research Center for Drug and Food Biological Resources Processing and Comprehensive Utilization, School of Life Sciences, South China Normal University, Guangzhou, China
| | - Jinbo Wei
- South China Normal University-Panyu Central Hospital Joint Laboratory of Basic and Translational Medical Research, Guangzhou Panyu Central Hospital, Guangzhou, China
| | - Ruifang Fan
- Guangzhou Key Laboratory of Subtropical Biodiversity and Biomonitoring and Guangdong Provincial Engineering Technology Research Center for Drug and Food Biological Resources Processing and Comprehensive Utilization, School of Life Sciences, South China Normal University, Guangzhou, China
| | - Dieen Wu
- South China Normal University-Panyu Central Hospital Joint Laboratory of Basic and Translational Medical Research, Guangzhou Panyu Central Hospital, Guangzhou, China
| | - Yufeng Ye
- South China Normal University-Panyu Central Hospital Joint Laboratory of Basic and Translational Medical Research, Guangzhou Panyu Central Hospital, Guangzhou, China
- Medical Imaging Institute of Panyu, Guangzhou, China
| |
Collapse
|
5
|
Zhang G, He Z, Zhang Y, Li Z, Wu L. SC-Net: Symmetrical conical network for colorectal pathology image segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 248:108119. [PMID: 38520785 DOI: 10.1016/j.cmpb.2024.108119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Revised: 02/25/2024] [Accepted: 03/04/2024] [Indexed: 03/25/2024]
Abstract
BACKGROUND AND OBJECTIVE Image segmentation of histopathology of colorectal cancer is a core task of computer aided medical image diagnosis system. Existing convolutional neural networks generally extract multi-scale information in linear flow structures by inserting multi-branch modules, which is difficult to extract heterogeneous semantic information under multi-level and different receptive field and tough to establish context dependency among different receptive field features. METHODS To address these issues, we propose a symmetric spiral progressive feature fusion encoder-decoder network called the Symmetric Conical Network (SC-Net). First, we design a Multi-scale Feature Extraction Block (MFEB) matching with the Symmetric Conical Network to obtain multi-branch heterogeneous semantic information under different receptive fields, so as to enrich the diversity of extracted feature information. The encoder is composed of MFEB through spiral and multi-branch arrangement to enhance context dependence between different information flow. Secondly, the information loss of contour, color and others in high-level semantic information through causally stacking MFEB, the Feature Mapping Layer (FML) is designed to map low-level features to high-level semantic features along the down-sampling branch and solve the problem of insufficient global feature extraction in deep levels. RESULTS The SC-Net was evaluated on our self-constructed colorectal cancer dataset, a publicly available breast cancer dataset and a polyp dataset. The results revealed that the mDice of segmentation reached 0.8611, 0.7259 and 0.7144. We compare our model with the state-of-art semantic segmentation UNet++, PSPNet, Attention U-Net, R2U-Net and other advanced segmentation networks. The experimental results demonstrate that we achieve the most advanced performance. CONCLUSIONS The results indicate that the proposed SC-Net excels in segmenting H&E stained pathology images, effectively preserving morphological features and spatial information even in scenarios with weak texture, poor contrast, and variations in appearance.
Collapse
Affiliation(s)
- Gang Zhang
- Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China.
| | - Zifen He
- Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China.
| | - Yinhui Zhang
- Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China.
| | - Zhenhui Li
- Yunnan Cancer Hospital, Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Center, Kunming 650118, China.
| | - Lin Wu
- Yunnan Cancer Hospital, Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Center, Kunming 650118, China.
| |
Collapse
|
6
|
Liu H, Gao W, Yang L, Wu D, Zhao D, Chen K, Liu J, Ye Y, Xu RX, Sun M. Semantic uncertainty Guided Cross-Transformer for enhanced macular edema segmentation in OCT images. Comput Biol Med 2024; 174:108458. [PMID: 38631114 DOI: 10.1016/j.compbiomed.2024.108458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 03/03/2024] [Accepted: 04/07/2024] [Indexed: 04/19/2024]
Abstract
Macular edema, a prevalent ocular complication observed in various retinal diseases, can lead to significant vision loss or blindness, necessitating accurate and timely diagnosis. Despite the potential of deep learning for segmentation of macular edema, challenges persist in accurately identifying lesion boundaries, especially in low-contrast and noisy regions, and in distinguishing between Inner Retinal Fluid (IRF), Sub-Retinal Fluid (SRF), and Pigment Epithelial Detachment (PED) lesions. To address these challenges, we present a novel approach, termed Semantic Uncertainty Guided Cross-Transformer Network (SuGCTNet), for the simultaneous segmentation of multi-class macular edema. Our proposed method comprises two key components, the semantic uncertainty guided attention module (SuGAM) and the Cross-Transformer module (CTM). The SuGAM module utilizes semantic uncertainty to allocate additional attention to regions with semantic ambiguity, improves the segmentation performance of these challenging areas. On the other hand, the CTM module capitalizes on both uncertainty information and multi-scale image features to enhance the overall continuity of the segmentation process, effectively minimizing feature confusion among different lesion types. Rigorous evaluation on public datasets and various OCT imaging device data demonstrates the superior performance of our proposed method compared to state-of-the-art approaches, highlighting its potential as a valuable tool for improving the accuracy and reproducibility of macular edema segmentation in clinical settings, and ultimately aiding in the early detection and diagnosis of macular edema-related diseases and associated retinal conditions.
Collapse
Affiliation(s)
- Hui Liu
- Department of Precision Machinery and Instruments, University of Science and Technology of China, Hefei, Anhui, 230026, PR China; School of Biomedical Engineering, Division of Life and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, PR China; Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, PR China
| | - Wenteng Gao
- Department of Precision Machinery and Instruments, University of Science and Technology of China, Hefei, Anhui, 230026, PR China
| | - Lei Yang
- Department of Precision Machinery and Instruments, University of Science and Technology of China, Hefei, Anhui, 230026, PR China
| | - Di Wu
- School of Biomedical Engineering, Division of Life and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, PR China
| | - Dehan Zhao
- Department of Precision Machinery and Instruments, University of Science and Technology of China, Hefei, Anhui, 230026, PR China
| | - Kun Chen
- Department of Precision Machinery and Instruments, University of Science and Technology of China, Hefei, Anhui, 230026, PR China
| | - Jicheng Liu
- School of Biomedical Engineering, Division of Life and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, PR China; Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, PR China
| | - Yu Ye
- Nanjing Research Institute of Electronics Technology, Nanjing, Jiangsu, 210039, PR China
| | - Ronald X Xu
- School of Biomedical Engineering, Division of Life and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, PR China; Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, PR China.
| | - Mingzhai Sun
- School of Biomedical Engineering, Division of Life and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, PR China; Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, PR China.
| |
Collapse
|
7
|
Peng L, Chen B, Yu E, Lin Y, Lin J, Zheng D, Fu Y, Chen Z, Zheng H, Zhan Z, Chen Y. The application value of LAVA-flex sequences in enhanced MRI scans of nasopharyngeal carcinoma: comparison with T1WI-IDEAL. Front Oncol 2024; 14:1320280. [PMID: 38420018 PMCID: PMC10899686 DOI: 10.3389/fonc.2024.1320280] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 01/16/2024] [Indexed: 03/02/2024] Open
Abstract
Introduction Magnetic resonance imaging (MRI) staging scans are critical for the diagnosis and treatment of patients with nasopharyngeal cancer (NPC). We aimed to evaluate the application value of LAVA-Flex and T1WI-IDEAL sequences in MRI staging scans. Methods Eighty-four newly diagnosed NPC patients underwent both LAVA-Flex and T1WI-IDEAL sequences during MRI examinations. Two radiologists independently scored the acquisitions of image quality, fat suppression quality, artifacts, vascular and nerve display. The obtained scores were compared using the Wilcoxon signed rank test. According to the signal intensity (SI) measurements, the uniformity of fat suppression, contrast between tumor lesions and subcutaneous fat tissue, and signal-to-noise ratio (SNR) were compared by the paired t-test. Results Compared to the T1WI-IDEAL sequence, LAVA-Flex exhibited fewer artifacts (P<0.05), better visualization of nerves and vessels (P<0.05), and performed superior in the fat contrast ratio of the primary lesion and metastatic lymph nodes (0.80 vs. 0.52, 0.81 vs. 0.56, separately, P<0.001). There was no statistically significant difference in overall image quality, tumor signal-to-noise ratio (SNR), muscle SNR, and the detection rate of lesions between the two sequences (P>0.05). T1WI-IDEAL was superior to LAVA-Flex in the evaluation of fat suppression uniformity (P<0.05). Discussion LAVA-Flex sequence provides satisfactory image quality and better visualization of nerves and vessels for NPC with shorter scanning times.
Collapse
Affiliation(s)
- Li Peng
- Department of Radiology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Bijuan Chen
- Department of Radiation Oncology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Erhan Yu
- Department of Neurology, Fujian Medical University Union Hospital, Fuzhou, Fujian, China
| | - Yifei Lin
- Department of Radiology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Jiahao Lin
- Department of Radiology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Dechun Zheng
- Department of Radiology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Yu Fu
- School of Basic Medical Sciences of Fujian Medical University, Fuzhou, Fujian, China
| | - Zhipeng Chen
- School of Basic Medical Sciences of Fujian Medical University, Fuzhou, Fujian, China
| | - Hanchen Zheng
- Department of Medical Oncology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Zhouwei Zhan
- Department of Medical Oncology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| | - Yunbin Chen
- Department of Radiology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian, China
| |
Collapse
|
8
|
Wahid KA, Sahlsten J, Jaskari J, Dohopolski MJ, Kaski K, He R, Glerean E, Kann BH, Mäkitie A, Fuller CD, Naser MA, Fuentes D. Harnessing uncertainty in radiotherapy auto-segmentation quality assurance. Phys Imaging Radiat Oncol 2024; 29:100526. [PMID: 38179210 PMCID: PMC10765294 DOI: 10.1016/j.phro.2023.100526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 12/13/2023] [Indexed: 01/06/2024] Open
Affiliation(s)
- Kareem A. Wahid
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Michael J. Dohopolski
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Benjamin H. Kann
- Artificial Intelligence in Medicine Program, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
| | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mohamed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
9
|
Huang YY, Deng YS, Liu Y, Qiang MY, Qiu WZ, Xia WX, Jing BZ, Feng CY, Chen HH, Cao X, Zhou JY, Huang HY, Zhan ZJ, Deng Y, Tang LQ, Mai HQ, Sun Y, Xie CM, Guo X, Ke LR, Lv X, Li CF. A deep learning-based semiautomated workflow for triaging follow-up MR scans in treated nasopharyngeal carcinoma. iScience 2023; 26:108347. [PMID: 38125021 PMCID: PMC10730347 DOI: 10.1016/j.isci.2023.108347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 08/08/2023] [Accepted: 10/24/2023] [Indexed: 12/23/2023] Open
Abstract
It is imperative to optimally utilize virtues and obviate defects of fully automated analysis and expert knowledge in new paradigms of healthcare. We present a deep learning-based semiautomated workflow (RAINMAN) with 12,809 follow-up scans among 2,172 patients with treated nasopharyngeal carcinoma from three centers (ChiCTR.org.cn, Chi-CTR2200056595). A boost of diagnostic performance and reduced workload was observed in RAINMAN compared with the original manual interpretations (internal vs. external: sensitivity, 2.5% [p = 0.500] vs. 3.2% [p = 0.031]; specificity, 2.9% [p < 0.001] vs. 0.3% [p = 0.302]; workload reduction, 79.3% vs. 76.2%). The workflow also yielded a triaging performance of 83.6%, with increases of 1.5% in sensitivity (p = 1.000) and 0.6%-1.3% (all p < 0.05) in specificity compared to three radiologists in the reader study. The semiautomated workflow shows its unique superiority in reducing radiologist's workload by eliminating negative scans while retaining the diagnostic performance of radiologists.
Collapse
Affiliation(s)
- Ying-Ying Huang
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Radiology, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Yi-Shu Deng
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Information, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
- School of Electronics and Information Technology, Sun Yat-sen University, Guangzhou 510006, China
| | - Yang Liu
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Meng-Yun Qiang
- Department of Radiation Oncology, Cancer Hospital of The University of Chinese Academy of Sciences, Hangzhou 310005, China
| | - Wen-Ze Qiu
- Department of Radiation Oncology, The Affiliated Cancer Hospital of Guangzhou Medical University, Guangzhou 510095, China
| | - Wei-Xiong Xia
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Bing-Zhong Jing
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Information, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Chen-Yang Feng
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Information, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Hao-Hua Chen
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Information, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Xun Cao
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Critical Care Medicine, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Jia-Yu Zhou
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Hao-Yang Huang
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Ze-Jiang Zhan
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Ying Deng
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Lin-Quan Tang
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Hai-Qiang Mai
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Ying Sun
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Chuan-Miao Xie
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Radiology, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Xiang Guo
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Liang-Ru Ke
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Radiology, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Xing Lv
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Nasopharyngeal Carcinoma, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| | - Chao-Feng Li
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou 510060, China
- Department of Information, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
| |
Collapse
|
10
|
Xu S, Duan L, Zhang Y, Zhang Z, Sun T, Tian L. Graph- and transformer-guided boundary aware network for medical image segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107849. [PMID: 37837887 DOI: 10.1016/j.cmpb.2023.107849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 09/29/2023] [Accepted: 10/06/2023] [Indexed: 10/16/2023]
Abstract
BACKGROUND AND OBJECTIVE Despite the considerable progress achieved by U-Net-based models, medical image segmentation remains a challenging task due to complex backgrounds, irrelevant noises, and ambiguous boundaries. In this study, we present a novel approach called U-shaped Graph- and Transformer-guided Boundary Aware Network (GTBA-Net) to tackle these challenges. METHODS GTBA-Net uses the pre-trained ResNet34 as its basic structure, and involves Global Feature Aggregation (GFA) modules for target localization, Graph-based Dynamic Feature Fusion (GDFF) modules for effective noise suppression, and Uncertainty-based Boundary Refinement (UBR) modules for accurate delineation of ambiguous boundaries. The GFA modules employ an efficient self-attention mechanism to facilitate coarse target localization amidst complex backgrounds, without introducing additional computational complexity. The GDFF modules leverage graph attention mechanism to aggregate information hidden among high- and low-level features, effectively suppressing target-irrelevant noises while preserving valuable spatial details. The UBR modules introduce an uncertainty quantification strategy and auxiliary loss to guide the model's focus towards target regions and uncertain "ridges", gradually mitigating boundary uncertainty and ultimately achieving accurate boundary delineation. RESULTS Comparative experiments on five datasets encompassing diverse modalities (including X-ray, CT, endoscopic procedures, and ultrasound) demonstrate that the proposed GTBA-Net outperforms existing methods in various challenging scenarios. Subsequent ablation studies further demonstrate the efficacy of the GFA, GDFF, and UBR modules in target localization, noise suppression, and ambiguous boundary delineation, respectively. CONCLUSIONS GTBA-Net exhibits substantial potential for extensive application in the field of medical image segmentation, particularly in scenarios involving complex backgrounds, target-irrelevant noises, or ambiguous boundaries.
Collapse
Affiliation(s)
- Shanshan Xu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China; Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing 100044, China
| | - Lianhong Duan
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, China; Senior Department of Orthopedics, The Fourth Medical Center of PLA General Hospital, Beijing, China
| | - Yang Zhang
- Senior Department of Orthopedics, The Fourth Medical Center of PLA General Hospital, Beijing, China
| | - Zhicheng Zhang
- Senior Department of Orthopedics, The Fourth Medical Center of PLA General Hospital, Beijing, China
| | - Tiansheng Sun
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, China; Senior Department of Orthopedics, The Fourth Medical Center of PLA General Hospital, Beijing, China.
| | - Lixia Tian
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China.
| |
Collapse
|
11
|
Lan X, Chen H, Jin W. DRI-Net: segmentation of polyp in colonoscopy images using dense residual-inception network. Front Physiol 2023; 14:1290820. [PMID: 37954444 PMCID: PMC10634602 DOI: 10.3389/fphys.2023.1290820] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 10/04/2023] [Indexed: 11/14/2023] Open
Abstract
Colorectal cancer is a common malignant tumor in the gastrointestinal tract, which usually evolves from adenomatous polyps. However, due to the similarity in color between polyps and their surrounding tissues in colonoscopy images, and their diversity in size, shape, and texture, intelligent diagnosis still remains great challenges. For this reason, we present a novel dense residual-inception network (DRI-Net) which utilizes U-Net as the backbone. Firstly, in order to increase the width of the network, a modified residual-inception block is designed to replace the traditional convolutional, thereby improving its capacity and expressiveness. Moreover, the dense connection scheme is adopted to increase the network depth so that more complex feature inputs can be fitted. Finally, an improved down-sampling module is built to reduce the loss of image feature information. For fair comparison, we validated all method on the Kvasir-SEG dataset using three popular evaluation metrics. Experimental results consistently illustrates that the values of DRI-Net on IoU, Mcc and Dice attain 77.72%, 85.94% and 86.51%, which were 1.41%, 0.66% and 0.75% higher than the suboptimal model. Similarly, through ablation studies, it also demonstrated the effectiveness of our approach in colorectal semantic segmentation.
Collapse
Affiliation(s)
| | - Honghuan Chen
- College of Internet of Things Technology, Hangzhou Polytechnic, Hangzhou, China
| | | |
Collapse
|
12
|
Islam M, Seenivasan L, Sharan SP, Viekash VK, Gupta B, Glocker B, Ren H. Paced-curriculum distillation with prediction and label uncertainty for image segmentation. Int J Comput Assist Radiol Surg 2023; 18:1875-1883. [PMID: 36862365 DOI: 10.1007/s11548-023-02847-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2022] [Accepted: 01/31/2023] [Indexed: 03/03/2023]
Abstract
PURPOSE In curriculum learning, the idea is to train on easier samples first and gradually increase the difficulty, while in self-paced learning, a pacing function defines the speed to adapt the training progress. While both methods heavily rely on the ability to score the difficulty of data samples, an optimal scoring function is still under exploration. METHODOLOGY Distillation is a knowledge transfer approach where a teacher network guides a student network by feeding a sequence of random samples. We argue that guiding student networks with an efficient curriculum strategy can improve model generalization and robustness. For this purpose, we design an uncertainty-based paced curriculum learning in self-distillation for medical image segmentation. We fuse the prediction uncertainty and annotation boundary uncertainty to develop a novel paced-curriculum distillation (P-CD). We utilize the teacher model to obtain prediction uncertainty and spatially varying label smoothing with Gaussian kernel to generate segmentation boundary uncertainty from the annotation. We also investigate the robustness of our method by applying various types and severity of image perturbation and corruption. RESULTS The proposed technique is validated on two medical datasets of breast ultrasound image segmentation and robot-assisted surgical scene segmentation and achieved significantly better performance in terms of segmentation and robustness. CONCLUSION P-CD improves the performance and obtains better generalization and robustness over the dataset shift. While curriculum learning requires extensive tuning of hyper-parameters for pacing function, the level of performance improvement suppresses this limitation.
Collapse
Affiliation(s)
- Mobarakol Islam
- Department of Computing, Imperial College London, London, UK
| | - Lalithkumar Seenivasan
- Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore
| | - S P Sharan
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tiruchirappalli, India
| | - V K Viekash
- Department of Instrumentation and Control Engineering, National Institute of Technology Tiruchirappalli, Tiruchirappalli, India
| | - Bhavesh Gupta
- Department of Mechanical Engineering, Indian Institute of Technology, Delhi, India
| | - Ben Glocker
- Department of Computing, Imperial College London, London, UK
| | - Hongliang Ren
- Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore.
- Department of Electrical Engineering and Shun Hing Institute of Advanced Engineering, The Chinese University of Hong Kong, Hong Kong, China.
| |
Collapse
|
13
|
Wang J, Peng Y. MHL-Net: A Multistage Hierarchical Learning Network for Head and Neck Multiorgan Segmentation. IEEE J Biomed Health Inform 2023; 27:4074-4085. [PMID: 37171918 DOI: 10.1109/jbhi.2023.3275746] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
Accurate segmentation of head and neck organs at risk is crucial in radiotherapy. However, the existing methods suffer from incomplete feature mining, insufficient information utilization, and difficulty in simultaneously improving the performance of small and large organ segmentation. In this paper, a multistage hierarchical learning network is designed to fully extract multidimensional features, combined with anatomical prior information and imaging features, using multistage subnetworks to improve the segmentation performance. First, multilevel subnetworks are constructed for primary segmentation, localization, and fine segmentation by dividing organs into two levels-large and small. Different networks both have their own learning focuses and feature reuse and information sharing among each other, which comprehensively improved the segmentation performance of all organs. Second, an anatomical prior probability map and a boundary contour attention mechanism are developed to address the problem of complex anatomical shapes. Prior information and boundary contour features effectively assist in detecting and segmenting special shapes. Finally, a multidimensional combination attention mechanism is proposed to analyze axial, coronal, and sagittal information, capture spatial and channel features, and maximize the use of structural information and semantic features of 3D medical images. Experimental results on several datasets showed that our method was competitive with state-of-the-art methods and improved the segmentation results for multiscale organs.
Collapse
|
14
|
Zeng Y, Zeng P, Shen S, Liang W, Li J, Zhao Z, Zhang K, Shen C. DCTR U-Net: automatic segmentation algorithm for medical images of nasopharyngeal cancer in the context of deep learning. Front Oncol 2023; 13:1190075. [PMID: 37546396 PMCID: PMC10402756 DOI: 10.3389/fonc.2023.1190075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 05/30/2023] [Indexed: 08/08/2023] Open
Abstract
Nasopharyngeal carcinoma (NPC) is a malignant tumor that occurs in the wall of the nasopharyngeal cavity and is prevalent in Southern China, Southeast Asia, North Africa, and the Middle East. According to studies, NPC is one of the most common malignant tumors in Hainan, China, and it has the highest incidence rate among otorhinolaryngological malignancies. We proposed a new deep learning network model to improve the segmentation accuracy of the target region of nasopharyngeal cancer. Our model is based on the U-Net-based network, to which we add Dilated Convolution Module, Transformer Module, and Residual Module. The new deep learning network model can effectively solve the problem of restricted convolutional fields of perception and achieve global and local multi-scale feature fusion. In our experiments, the proposed network was trained and validated using 10-fold cross-validation based on the records of 300 clinical patients. The results of our network were evaluated using the dice similarity coefficient (DSC) and the average symmetric surface distance (ASSD). The DSC and ASSD values are 0.852 and 0.544 mm, respectively. With the effective combination of the Dilated Convolution Module, Transformer Module, and Residual Module, we significantly improved the segmentation performance of the target region of the NPC.
Collapse
Affiliation(s)
- Yan Zeng
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
- ChinaPersonnel Department, Hainan Medical University, Haikou, China
| | - PengHui Zeng
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
| | - ShaoDong Shen
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Wei Liang
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Jun Li
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Zhe Zhao
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Kun Zhang
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
- School of Information Science and Technology, Hainan Normal University, Haikou, China
| | - Chong Shen
- State Key Laboratory of Marine Resource Utilization in South China Sea, School of Information and Communication Engineering, Hainan University, Haikou, China
| |
Collapse
|
15
|
Li Z, Fang J, Qiu R, Gong H, Zhang W, Li L, Jiang J. CDA-Net: A contrastive deep adversarial model for prostate cancer segmentation in MRI images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104622] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
|
16
|
Li Z, Zhang N, Gong H, Qiu R, Zhang W. MFA-Net: Multiple Feature Association Network for medical image segmentation. Comput Biol Med 2023; 158:106834. [PMID: 37003067 DOI: 10.1016/j.compbiomed.2023.106834] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 03/01/2023] [Accepted: 03/26/2023] [Indexed: 03/30/2023]
Abstract
Medical image segmentation plays a crucial role in computer-aided diagnosis. However, due to the large variability of medical images, accurate segmentation is a highly challenging task. In this paper, we present a novel medical image segmentation network named the Multiple Feature Association Network (MFA-Net), which is based on deep learning techniques. The MFA-Net utilizes an encoder-decoder architecture with skip connections as its backbone network, and a parallelly dilated convolutions arrangement (PDCA) module is integrated between the encoder and the decoder to capture more representative deep features. Furthermore, a multi-scale feature restructuring module (MFRM) is introduced to restructure and fuse the deep features of the encoder. To enhance global attention perception, the proposed global attention stacking (GAS) modules are cascaded on the decoder. The proposed MFA-Net leverages novel global attention mechanisms to improve the segmentation performance at different feature scales. We evaluated our MFA-Net on four segmentation tasks, including lesions in intestinal polyp, liver tumor, prostate cancer, and skin lesion. Our experimental results and ablation study demonstrate that the proposed MFA-Net outperforms state-of-the-art methods in terms of global positioning and local edge recognition.
Collapse
Affiliation(s)
- Zhixun Li
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| | - Nan Zhang
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| | - Huiling Gong
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China.
| | - Ruiyun Qiu
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| | - Wei Zhang
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| |
Collapse
|
17
|
Sahlsten J, Jaskari J, Wahid KA, Ahmed S, Glerean E, He R, Kann BH, Mäkitie A, Fuller CD, Naser MA, Kaski K. Application of simultaneous uncertainty quantification for image segmentation with probabilistic deep learning: Performance benchmarking of oropharyngeal cancer target delineation as a use-case. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.02.20.23286188. [PMID: 36865296 PMCID: PMC9980236 DOI: 10.1101/2023.02.20.23286188] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/26/2023]
Abstract
Background Oropharyngeal cancer (OPC) is a widespread disease, with radiotherapy being a core treatment modality. Manual segmentation of the primary gross tumor volume (GTVp) is currently employed for OPC radiotherapy planning, but is subject to significant interobserver variability. Deep learning (DL) approaches have shown promise in automating GTVp segmentation, but comparative (auto)confidence metrics of these models predictions has not been well-explored. Quantifying instance-specific DL model uncertainty is crucial to improving clinician trust and facilitating broad clinical implementation. Therefore, in this study, probabilistic DL models for GTVp auto-segmentation were developed using large-scale PET/CT datasets, and various uncertainty auto-estimation methods were systematically investigated and benchmarked. Methods We utilized the publicly available 2021 HECKTOR Challenge training dataset with 224 co-registered PET/CT scans of OPC patients with corresponding GTVp segmentations as a development set. A separate set of 67 co-registered PET/CT scans of OPC patients with corresponding GTVp segmentations was used for external validation. Two approximate Bayesian deep learning methods, the MC Dropout Ensemble and Deep Ensemble, both with five submodels, were evaluated for GTVp segmentation and uncertainty performance. The segmentation performance was evaluated using the volumetric Dice similarity coefficient (DSC), mean surface distance (MSD), and Hausdorff distance at 95% (95HD). The uncertainty was evaluated using four measures from literature: coefficient of variation (CV), structure expected entropy, structure predictive entropy, and structure mutual information, and additionally with our novel Dice-risk measure. The utility of uncertainty information was evaluated with the accuracy of uncertainty-based segmentation performance prediction using the Accuracy vs Uncertainty (AvU) metric, and by examining the linear correlation between uncertainty estimates and DSC. In addition, batch-based and instance-based referral processes were examined, where the patients with high uncertainty were rejected from the set. In the batch referral process, the area under the referral curve with DSC (R-DSC AUC) was used for evaluation, whereas in the instance referral process, the DSC at various uncertainty thresholds were examined. Results Both models behaved similarly in terms of the segmentation performance and uncertainty estimation. Specifically, the MC Dropout Ensemble had 0.776 DSC, 1.703 mm MSD, and 5.385 mm 95HD. The Deep Ensemble had 0.767 DSC, 1.717 mm MSD, and 5.477 mm 95HD. The uncertainty measure with the highest DSC correlation was structure predictive entropy with correlation coefficients of 0.699 and 0.692 for the MC Dropout Ensemble and the Deep Ensemble, respectively. The highest AvU value was 0.866 for both models. The best performing uncertainty measure for both models was the CV which had R-DSC AUC of 0.783 and 0.782 for the MC Dropout Ensemble and Deep Ensemble, respectively. With referring patients based on uncertainty thresholds from 0.85 validation DSC for all uncertainty measures, on average the DSC improved from the full dataset by 4.7% and 5.0% while referring 21.8% and 22% patients for MC Dropout Ensemble and Deep Ensemble, respectively. Conclusion We found that many of the investigated methods provide overall similar but distinct utility in terms of predicting segmentation quality and referral performance. These findings are a critical first-step towards more widespread implementation of uncertainty quantification in OPC GTVp segmentation.
Collapse
Affiliation(s)
- Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Sara Ahmed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| |
Collapse
|
18
|
Rajaraman S, Yang F, Zamzmi G, Xue Z, Antani S. Assessing the Impact of Image Resolution on Deep Learning for TB Lesion Segmentation on Frontal Chest X-rays. Diagnostics (Basel) 2023; 13:747. [PMID: 36832235 PMCID: PMC9955202 DOI: 10.3390/diagnostics13040747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Revised: 02/10/2023] [Accepted: 02/15/2023] [Indexed: 02/18/2023] Open
Abstract
Deep learning (DL) models are state-of-the-art in segmenting anatomical and disease regions of interest (ROIs) in medical images. Particularly, a large number of DL-based techniques have been reported using chest X-rays (CXRs). However, these models are reportedly trained on reduced image resolutions for reasons related to the lack of computational resources. Literature is sparse in discussing the optimal image resolution to train these models for segmenting the tuberculosis (TB)-consistent lesions in CXRs. In this study, we investigated the performance variations with an Inception-V3 UNet model using various image resolutions with/without lung ROI cropping and aspect ratio adjustments and identified the optimal image resolution through extensive empirical evaluations to improve TB-consistent lesion segmentation performance. We used the Shenzhen CXR dataset for the study, which includes 326 normal patients and 336 TB patients. We proposed a combinatorial approach consisting of storing model snapshots, optimizing segmentation threshold and test-time augmentation (TTA), and averaging the snapshot predictions, to further improve performance with the optimal resolution. Our experimental results demonstrate that higher image resolutions are not always necessary; however, identifying the optimal image resolution is critical to achieving superior performance.
Collapse
Affiliation(s)
- Sivaramakrishnan Rajaraman
- Computational Health Research Branch, National Library of Medicine, National Institutes of Health, Bethesda, MD 20894, USA
| | | | | | | | | |
Collapse
|
19
|
Peng Y, Yu D, Guo Y. MShNet: Multi-scale feature combined with h-network for medical image segmentation. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
20
|
Li F, Li W, Gao X, Liu R, Xiao B. DCNet: Diversity convolutional network for ventricle segmentation on short-axis cardiac magnetic resonance images. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.110033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
21
|
GEMA-An Automatic Segmentation Method for Real-Time Analysis of Mammalian Cell Growth in Microfluidic Devices. J Imaging 2022; 8:jimaging8100281. [PMID: 36286375 PMCID: PMC9605644 DOI: 10.3390/jimaging8100281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 09/15/2022] [Accepted: 09/20/2022] [Indexed: 01/24/2023] Open
Abstract
Nowadays, image analysis has a relevant role in most scientific and research areas. This process is used to extract and understand information from images to obtain a model, knowledge, and rules in the decision process. In the case of biological areas, images are acquired to describe the behavior of a biological agent in time such as cells using a mathematical and computational approach to generate a system with automatic control. In this paper, MCF7 cells are used to model their growth and death when they have been injected with a drug. These mammalian cells allow understanding of behavior, gene expression, and drug resistance to breast cancer. For this, an automatic segmentation method called GEMA is presented to analyze the apoptosis and confluence stages of culture by measuring the increase or decrease of the image area occupied by cells in microfluidic devices. In vitro, the biological experiments can be analyzed through a sequence of images taken at specific intervals of time. To automate the image segmentation, the proposed algorithm is based on a Gabor filter, a coefficient of variation (CV), and linear regression. This allows the processing of images in real time during the evolution of biological experiments. Moreover, GEMA has been compared with another three representative methods such as gold standard (manual segmentation), morphological gradient, and a semi-automatic algorithm using FIJI. The experiments show promising results, due to the proposed algorithm achieving an accuracy above 90% and a lower computation time because it requires on average 1 s to process each image. This makes it suitable for image-based real-time automatization of biological lab-on-a-chip experiments.
Collapse
|
22
|
Guo X, Ye C, Yang Y, Zhang L, Liang L, Lu S, Lv H, Guo C, Ma T. Ensemble learning via supervision augmentation for white matter hyperintensity segmentation. Front Neurosci 2022; 16:946343. [PMID: 36188477 PMCID: PMC9521379 DOI: 10.3389/fnins.2022.946343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Accepted: 08/15/2022] [Indexed: 11/13/2022] Open
Abstract
Since the ambiguous boundary of the lesion and inter-observer variability, white matter hyperintensity segmentation annotations are inherently noisy and uncertain. On the other hand, the high capacity of deep neural networks (DNN) enables them to overfit labels with noise and uncertainty, which may lead to biased models with weak generalization ability. This challenge has been addressed by leveraging multiple annotations per image. However, multiple annotations are often not available in a real-world scenario. To mitigate the issue, this paper proposes a supervision augmentation method (SA) and combines it with ensemble learning (SA-EN) to improve the generalization ability of the model. SA can obtain diverse supervision information by estimating the uncertainty of annotation in a real-world scenario that per image have only one ambiguous annotation. Then different base learners in EN are trained with diverse supervision information. The experimental results on two white matter hyperintensity segmentation datasets demonstrate that SA-EN gets the optimal accuracy compared with other state-of-the-art ensemble methods. SA-EN is more effective on small datasets, which is more suitable for medical image segmentation with few annotations. A quantitative study is presented to show the effect of ensemble size and the effectiveness of the ensemble model. Furthermore, SA-EN can capture two types of uncertainty, aleatoric uncertainty modeled in SA and epistemic uncertainty modeled in EN.
Collapse
Affiliation(s)
- Xutao Guo
- School of Electronics and Information Engineering, Harbin Institute of Technology, Shenzhen, China
- Peng Cheng Laboratory, Shenzhen, China
| | - Chenfei Ye
- International Research Institute for Artifcial Intelligence, Harbin Institute of Technology, Shenzhen, China
| | - Yanwu Yang
- School of Electronics and Information Engineering, Harbin Institute of Technology, Shenzhen, China
- Peng Cheng Laboratory, Shenzhen, China
| | - Li Zhang
- Department of Radiology, The First Hospital of Jilin University, Changchun, China
| | - Li Liang
- School of Electronics and Information Engineering, Harbin Institute of Technology, Shenzhen, China
| | - Shang Lu
- School of Electronics and Information Engineering, Harbin Institute of Technology, Shenzhen, China
| | - Haiyan Lv
- Mindsgo Life Science Company, Shenzhen, China
| | - Chunjie Guo
- Department of Radiology, The First Hospital of Jilin University, Changchun, China
- *Correspondence: Chunjie Guo
| | - Ting Ma
- School of Electronics and Information Engineering, Harbin Institute of Technology, Shenzhen, China
- Peng Cheng Laboratory, Shenzhen, China
- International Research Institute for Artifcial Intelligence, Harbin Institute of Technology, Shenzhen, China
- Ting Ma
| |
Collapse
|
23
|
Zhan B, Zhou L, Li Z, Wu X, Pu Y, Zhou J, Wang Y, Shen D. D2FE-GAN: Decoupled dual feature extraction based GAN for MRI image synthesis. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
24
|
A Systematic Evaluation of Ensemble Learning Methods for Fine-Grained Semantic Segmentation of Tuberculosis-Consistent Lesions in Chest Radiographs. Bioengineering (Basel) 2022; 9:bioengineering9090413. [PMID: 36134959 PMCID: PMC9495849 DOI: 10.3390/bioengineering9090413] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 08/22/2022] [Accepted: 08/22/2022] [Indexed: 11/24/2022] Open
Abstract
Automated segmentation of tuberculosis (TB)-consistent lesions in chest X-rays (CXRs) using deep learning (DL) methods can help reduce radiologist effort, supplement clinical decision-making, and potentially result in improved patient treatment. The majority of works in the literature discuss training automatic segmentation models using coarse bounding box annotations. However, the granularity of the bounding box annotation could result in the inclusion of a considerable fraction of false positives and negatives at the pixel level that may adversely impact overall semantic segmentation performance. This study evaluates the benefits of using fine-grained annotations of TB-consistent lesions toward training the variants of U-Net models and constructing their ensembles for semantically segmenting TB-consistent lesions in both original and bone-suppressed frontal CXRs. The segmentation performance is evaluated using several ensemble methods such as bitwise- AND, bitwise-OR, bitwise-MAX, and stacking. Extensive empirical evaluations showcased that the stacking ensemble demonstrated superior segmentation performance (Dice score: 0.5743, 95% confidence interval: (0.4055, 0.7431)) compared to the individual constituent models and other ensemble methods. To the best of our knowledge, this is the first study to apply ensemble learning to improve fine-grained TB-consistent lesion segmentation performance.
Collapse
|
25
|
Chen Y, Han G, Lin T, Liu X. CAFS: An Attention-Based Co-Segmentation Semi-Supervised Method for Nasopharyngeal Carcinoma Segmentation. SENSORS (BASEL, SWITZERLAND) 2022; 22:5053. [PMID: 35808548 PMCID: PMC9269783 DOI: 10.3390/s22135053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/28/2022] [Revised: 06/25/2022] [Accepted: 06/30/2022] [Indexed: 02/06/2023]
Abstract
Accurate segmentation of nasopharyngeal carcinoma is essential to its treatment effect. However, there are several challenges in existing deep learning-based segmentation methods. First, the acquisition of labeled data are challenging. Second, the nasopharyngeal carcinoma is similar to the surrounding tissues. Third, the shape of nasopharyngeal carcinoma is complex. These challenges make the segmentation of nasopharyngeal carcinoma difficult. This paper proposes a novel semi-supervised method named CAFS for automatic segmentation of nasopharyngeal carcinoma. CAFS addresses the above challenges through three mechanisms: the teacher-student cooperative segmentation mechanism, the attention mechanism, and the feedback mechanism. CAFS can use only a small amount of labeled nasopharyngeal carcinoma data to segment the cancer region accurately. The average DSC value of CAFS is 0.8723 on the nasopharyngeal carcinoma segmentation task. Moreover, CAFS has outperformed the state-of-the-art nasopharyngeal carcinoma segmentation methods in the comparison experiment. Among the compared state-of-the-art methods, CAFS achieved the highest values of DSC, Jaccard, and precision. In particular, the DSC value of CAFS is 7.42% higher than the highest DSC value in the state-of-the-art methods.
Collapse
Affiliation(s)
- Yitong Chen
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen 518107, China; (Y.C.); (G.H.); (T.L.)
| | - Guanghui Han
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen 518107, China; (Y.C.); (G.H.); (T.L.)
- School of Information Engineering, North China University of Water Resources and Electric Power, Zhengzhou 450046, China
| | - Tianyu Lin
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen 518107, China; (Y.C.); (G.H.); (T.L.)
| | - Xiujian Liu
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen 518107, China; (Y.C.); (G.H.); (T.L.)
| |
Collapse
|
26
|
Kuang X, Cheung JPY, Wong KYK, Lam WY, Lam CH, Choy RW, Cheng CP, Wu H, Yang C, Wang K, Li Y, Zhang T. Spine-GFlow: A hybrid learning framework for robust multi-tissue segmentation in lumbar MRI without manual annotation. Comput Med Imaging Graph 2022; 99:102091. [PMID: 35803034 DOI: 10.1016/j.compmedimag.2022.102091] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 05/30/2022] [Accepted: 06/13/2022] [Indexed: 10/18/2022]
Abstract
Most learning-based magnetic resonance image (MRI) segmentation methods rely on the manual annotation to provide supervision, which is extremely tedious, especially when multiple anatomical structures are required. In this work, we aim to develop a hybrid framework named Spine-GFlow that combines the image features learned by a CNN model and anatomical priors for multi-tissue segmentation in a sagittal lumbar MRI. Our framework does not require any manual annotation and is robust against image feature variation caused by different image settings and/or underlying pathology. Our contributions include: 1) a rule-based method that automatically generates the weak annotation (initial seed area), 2) a novel proposal generation method that integrates the multi-scale image features and anatomical prior, 3) a comprehensive loss for CNN training that optimizes the pixel classification and feature distribution simultaneously. Our Spine-GFlow has been validated on 2 independent datasets: HKDDC (containing images obtained from 3 different machines) and IVDM3Seg. The segmentation results of vertebral bodies (VB), intervertebral discs (IVD), and spinal canal (SC) are evaluated quantitatively using intersection over union (IoU) and the Dice coefficient. Results show that our method, without requiring manual annotation, has achieved a segmentation performance comparable to a model trained with full supervision (mean Dice 0.914 vs 0.916).
Collapse
Affiliation(s)
- Xihe Kuang
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
| | - Jason Pui Yin Cheung
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
| | - Kwan-Yee K Wong
- Department of Computer Science, Faculty of Engineering, University of Hong Kong, Hong Kong, China
| | - Wai Yi Lam
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
| | - Chak Hei Lam
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
| | - Richard W Choy
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
| | | | - Honghan Wu
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
| | - Cao Yang
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China
| | - Kun Wang
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China
| | - Yang Li
- School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China.
| | - Teng Zhang
- Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China.
| |
Collapse
|
27
|
Wang K, Wang Y, Zhan B, Yang Y, Zu C, Wu X, Zhou J, Nie D, Zhou L. An Efficient Semi-Supervised Framework with Multi-Task and Curriculum Learning for Medical Image Segmentation. Int J Neural Syst 2022; 32:2250043. [DOI: 10.1142/s0129065722500435] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
28
|
Rajaraman S, Zamzmi G, Yang F, Xue Z, Jaeger S, Antani SK. Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays. Biomedicines 2022; 10:1323. [PMID: 35740345 PMCID: PMC9220007 DOI: 10.3390/biomedicines10061323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 05/30/2022] [Accepted: 06/03/2022] [Indexed: 12/10/2022] Open
Abstract
Deep learning (DL) methods have demonstrated superior performance in medical image segmentation tasks. However, selecting a loss function that conforms to the data characteristics is critical for optimal performance. Further, the direct use of traditional DL models does not provide a measure of uncertainty in predictions. Even high-quality automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust. In this study, we aim to investigate the benefits of (i) selecting an appropriate loss function and (ii) quantifying uncertainty in predictions using a VGG16-based-U-Net model with the Monto-Carlo (MCD) Dropout method for segmenting Tuberculosis (TB)-consistent findings in frontal chest X-rays (CXRs). We determine an optimal uncertainty threshold based on several uncertainty-related metrics. This threshold is used to select and refer highly uncertain cases to an expert. Experimental results demonstrate that (i) the model trained with a modified Focal Tversky loss function delivered superior segmentation performance (mean average precision (mAP): 0.5710, 95% confidence interval (CI): (0.4021,0.7399)), (ii) the model with 30 MC forward passes during inference further improved and stabilized performance (mAP: 0.5721, 95% CI: (0.4032,0.7410), and (iii) an uncertainty threshold of 0.7 is observed to be optimal to refer highly uncertain cases.
Collapse
Affiliation(s)
- Sivaramakrishnan Rajaraman
- National Library of Medicine, National Institutes of Health, Bethesda, MD 20892, USA; (G.Z.); (F.Y.); (Z.X.); (S.J.); (S.K.A.)
| | | | | | | | | | | |
Collapse
|
29
|
Semi-supervised Medical Image Segmentation via a Tripled-uncertainty Guided Mean Teacher Model with Contrastive Learning. Med Image Anal 2022; 79:102447. [DOI: 10.1016/j.media.2022.102447] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Revised: 03/16/2022] [Accepted: 04/01/2022] [Indexed: 11/18/2022]
|
30
|
Explainable attention guided adversarial deep network for 3D radiotherapy dose distribution prediction. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.108324] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
31
|
Mask-Free Radiotherapy Dose Prediction via Multi-Task Learning. 2022 IEEE 19TH INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING (ISBI) 2022. [DOI: 10.1109/isbi52829.2022.9761505] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
|
32
|
Zeng J, Cao C, Peng X, Xiao J, Zu C, Wu X, Zhou J, Wang Y. Two-Phase Progressive Deep Transfer Learning for Cervical Cancer Dose Map Prediction. 2022 IEEE 19TH INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING (ISBI) 2022. [DOI: 10.1109/isbi52829.2022.9761628] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Affiliation(s)
- Jie Zeng
- Sichuan University,School of Computer Science,China
| | | | - Xingchen Peng
- Sichuan University,Cancer Center, West China Hospital,Department of Biotherapy,China
| | - Jianghong Xiao
- Sichuan University,Cancer Center, West China Hospital,Department of Radiation Oncology,China
| | - Chen Zu
- JD.com,Department of Risk Controlling Research,China
| | - Xi Wu
- Chengdu University of Information Technology,School of Computer Science,China
| | - Jiliu Zhou
- Sichuan University,School of Computer Science,China
| | - Yan Wang
- Sichuan University,School of Computer Science,China
| |
Collapse
|
33
|
Sun Y, Yang H, Zhou J, Wang Y. ISSMF: Integrated semantic and spatial information of multi-level features for automatic segmentation in prenatal ultrasound images. Artif Intell Med 2022; 125:102254. [DOI: 10.1016/j.artmed.2022.102254] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2020] [Revised: 12/27/2021] [Accepted: 02/05/2022] [Indexed: 11/02/2022]
|