1
|
Zhao J, Li S. Evidence modeling for reliability learning and interpretable decision-making under multi-modality medical image segmentation. Comput Med Imaging Graph 2024; 116:102422. [PMID: 39116707 DOI: 10.1016/j.compmedimag.2024.102422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Revised: 06/04/2024] [Accepted: 07/30/2024] [Indexed: 08/10/2024]
Abstract
Reliability learning and interpretable decision-making are crucial for multi-modality medical image segmentation. Although many works have attempted multi-modality medical image segmentation, they rarely explore how much reliability is provided by each modality for segmentation. Moreover, the existing approach of decision-making such as the softmax function lacks the interpretability for multi-modality fusion. In this study, we proposed a novel approach named contextual discounted evidential network (CDE-Net) for reliability learning and interpretable decision-making under multi-modality medical image segmentation. Specifically, the CDE-Net first models the semantic evidence by uncertainty measurement using the proposed evidential decision-making module. Then, it leverages the contextual discounted fusion layer to learn the reliability provided by each modality. Finally, a multi-level loss function is deployed for the optimization of evidence modeling and reliability learning. Moreover, this study elaborates on the framework interpretability by discussing the consistency between pixel attribution maps and the learned reliability coefficients. Extensive experiments are conducted on both multi-modality brain and liver datasets. The CDE-Net gains high performance with an average Dice score of 0.914 for brain tumor segmentation and 0.913 for liver tumor segmentation, which proves CDE-Net has great potential to facilitate the interpretation of artificial intelligence-based multi-modality medical image fusion.
Collapse
Affiliation(s)
- Jianfeng Zhao
- School of Biomedical Engineering, Western University, London, ON, Canada
| | - Shuo Li
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA; Department of Computer and Data Science, Case Western Reserve University, Cleveland, OH, USA.
| |
Collapse
|
2
|
Wei H, Zheng T, Zhang X, Zheng C, Jiang D, Wu Y, Lee JM, Bashir MR, Lerner E, Liu R, Wu B, Guo H, Chen Y, Yang T, Gong X, Jiang H, Song B. Deep learning-based 3D quantitative total tumor burden predicts early recurrence of BCLC A and B HCC after resection. Eur Radiol 2024:10.1007/s00330-024-10941-y. [PMID: 39028376 DOI: 10.1007/s00330-024-10941-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 05/15/2024] [Accepted: 06/16/2024] [Indexed: 07/20/2024]
Abstract
OBJECTIVES This study aimed to evaluate the potential of deep learning (DL)-assisted automated three-dimensional quantitative tumor burden at MRI to predict postoperative early recurrence (ER) of hepatocellular carcinoma (HCC). MATERIALS AND METHODS This was a single-center retrospective study enrolling patients who underwent resection for BCLC A and B HCC and preoperative contrast-enhanced MRI. Quantitative total tumor volume (cm3) and total tumor burden (TTB, %) were obtained using a DL automated segmentation tool. Radiologists' visual assessment was used to ensure the quality control of automated segmentation. The prognostic value of clinicopathological variables and tumor burden-related parameters for ER was determined by Cox regression analyses. RESULTS A total of 592 patients were included, with 525 and 67 patients assigned to BCLC A and B, respectively (2-year ER rate: 30.0% vs. 45.3%; hazard ratio (HR) = 1.8; p = 0.007). TTB was the most important predictor of ER (HR = 2.2; p < 0.001). Using 6.84% as the threshold of TTB, two ER risk strata were obtained in overall (p < 0.001), BCLC A (p < 0.001), and BCLC B (p = 0.027) patients, respectively. The BCLC B low-TTB patients had a similar risk for ER to BCLC A patients and thus were reassigned to a BCLC An stage; whilst the BCLC B high-TTB patients remained in a BCLC Bn stage. The 2-year ER rate was 30.5% for BCLC An patients vs. 58.1% for BCLC Bn patients (HR = 2.8; p < 0.001). CONCLUSIONS TTB determined by DL-based automated segmentation at MRI was a predictive biomarker for postoperative ER and facilitated refined subcategorization of patients within BCLC stages A and B. CLINICAL RELEVANCE STATEMENT Total tumor burden derived by deep learning-based automated segmentation at MRI may serve as an imaging biomarker for predicting early recurrence, thereby improving subclassification of Barcelona Clinic Liver Cancer A and B hepatocellular carcinoma patients after hepatectomy. KEY POINTS Total tumor burden (TTB) is important for Barcelona Clinic Liver Cancer (BCLC) staging, but is heterogenous. TTB derived by deep learning-based automated segmentation was predictive of postoperative early recurrence. Incorporating TTB into the BCLC algorithm resulted in successful subcategorization of BCLC A and B patients.
Collapse
Affiliation(s)
- Hong Wei
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
- Department of Radiology, Seoul National University Hospital, Seoul, 03080, Republic of Korea
| | - Tianying Zheng
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | | | - Chao Zheng
- Shukun Technology Co., Ltd, Beijing, 100102, China
| | - Difei Jiang
- Shukun Technology Co., Ltd, Beijing, 100102, China
| | - Yuanan Wu
- Big Data Research Center, University of Electronic Science and Technology of China, Chengdu, Sichuan, 610000, China
| | - Jeong Min Lee
- Department of Radiology, Seoul National University Hospital, Seoul, 03080, Republic of Korea
- Department of Radiology, Seoul National University College of Medicine, Seoul, 03080, Republic of Korea
| | - Mustafa R Bashir
- Department of Radiology, Duke University Medical Center, Durham, NC, 27710, USA
- Center for Advanced Magnetic Resonance in Medicine, Duke University Medical Center, Durham, NC, 27705, USA
- Division of Gastroenterology, Department of Medicine, Duke University Medical Center, Durham, NC, 27710, USA
| | - Emily Lerner
- Department of Radiology, Duke University Medical Center, Durham, NC, 27710, USA
| | - Rongbo Liu
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | - Botong Wu
- Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100102, China
| | - Hua Guo
- Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100102, China
| | - Yidi Chen
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | - Ting Yang
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | - Xiaoling Gong
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | - Hanyu Jiang
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China.
| | - Bin Song
- Department of Radiology, Functional, and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China.
- Department of Radiology, Sanya People's Hospital, Sanya, Hainan, 572000, China.
| |
Collapse
|
3
|
Gao L, Wang W, Meng X, Zhang S, Xu J, Ju S, Wang YC. TPA: Two-stage progressive attention segmentation framework for hepatocellular carcinoma on multi-modality MRI. Med Phys 2024; 51:4936-4947. [PMID: 38306473 DOI: 10.1002/mp.16968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Revised: 01/04/2024] [Accepted: 01/21/2024] [Indexed: 02/04/2024] Open
Abstract
BACKGROUND Dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) plays a crucial role in the diagnosis and measurement of hepatocellular carcinoma (HCC). The multi-modality information contained in the multi-phase images of DCE-MRI is important for improving segmentation. However, this remains a challenging task due to the heterogeneity of HCC, which may cause one HCC lesion to have varied imaging appearance in each phase of DCE-MRI. In particular, some phases exhibit inconsistent sizes and boundaries will result in a lack of correlation between modalities, and it may pose inaccurate segmentation results. PURPOSE We aim to design a multi-modality segmentation model that can learn meaningful inter-phase correlation for achieving HCC segmentation. METHODS In this study, we propose a two-stage progressive attention segmentation framework (TPA) for HCC based on the transformer and the decision-making process of radiologists. Specifically, the first stage aims to fuse features from multi-phase images to identify HCC and provide localization region. In the second stage, a multi-modality attention transformer module (MAT) is designed to focus on the features that can represent the actual size. RESULTS We conduct training, validation, and test in a single-center dataset (386 cases), followed by external test on a batch of multi-center datasets (83 cases). Furthermore, we analyze a subgroup of data with weak inter-phase correlation in the test set. The proposed model achieves Dice coefficient of 0.822 and 0.772 in the internal and external test sets, respectively, and 0.829, 0.791 in the subgroup. The experimental results demonstrate that our model outperforms state-of-the-art models, particularly within subgroup. CONCLUSIONS The proposed TPA provides best segmentation results, and utilizing clinical prior knowledge for network design is practical and feasible.
Collapse
Affiliation(s)
- Lei Gao
- Institute for AI in Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China
| | - Weilang Wang
- Department of Radiology, Zhongda Hospital, Jiangsu Key Laboratory of Molecular and Functional Imaging, School of Medicine, Southeast University, Nanjing, China
| | - Xiangpan Meng
- Department of Radiology, Zhongda Hospital, Jiangsu Key Laboratory of Molecular and Functional Imaging, School of Medicine, Southeast University, Nanjing, China
| | - Shuhang Zhang
- Department of Radiology, Zhongda Hospital, Jiangsu Key Laboratory of Molecular and Functional Imaging, School of Medicine, Southeast University, Nanjing, China
| | - Jun Xu
- Institute for AI in Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China
| | - Shenghong Ju
- Institute for AI in Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China
- Department of Radiology, Zhongda Hospital, Jiangsu Key Laboratory of Molecular and Functional Imaging, School of Medicine, Southeast University, Nanjing, China
| | - Yuan-Cheng Wang
- Institute for AI in Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China
- Department of Radiology, Zhongda Hospital, Jiangsu Key Laboratory of Molecular and Functional Imaging, School of Medicine, Southeast University, Nanjing, China
| |
Collapse
|
4
|
Meng K, Gong G, Liu R, Du S, Yin Y. Advances in gross tumor target volume determination in radiotherapy for patients with hepatocellular carcinoma. Front Oncol 2024; 14:1346407. [PMID: 38841160 PMCID: PMC11150548 DOI: 10.3389/fonc.2024.1346407] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 05/06/2024] [Indexed: 06/07/2024] Open
Abstract
Hepatocellular Carcinoma (HCC) is one of the most common malignant neoplasms. With the advancement of technology, the precision of radiotherapy (RT) for HCC has considerably increased, and it is an indispensable modality in the comprehensive management of HCC. Some RT techniques increase the radiation dose to HCC, which decreases the radiation dose delivered to the surrounding normal liver tissue. This approach significantly improves the efficacy of HCC treatment and reduces the incidence of Radiation-induced Liver Disease (RILD). Clear imaging and precise determination of the Gross Target Volume (GTV) are prerequisites of precise RT of HCC. The main hindrances in determining the HCC GTV include indistinct tumor boundaries on imaging and the impact on respiratory motion. The integration of multimodal imaging, four-dimensional imaging, and artificial intelligence (AI) techniques can help overcome challenges for HCC GTV. In this article, the advancements in medical imaging and precise determination for HCC GTV have been reviewed, providing a framework for the precise RT of HCC.
Collapse
Affiliation(s)
- Kangning Meng
- Department of Graduate, Shandong First Medical University, Shandong Academy of Medical Sciences, Jinan, China
- Department of Radiation Physics, Shandong First Medical University Affiliated Cancer Hospital, Shandong Cancer Hospital and Institute (Shandong Cancer Hospital), Jinan, China
| | - Guanzhong Gong
- Department of Radiation Physics, Shandong First Medical University Affiliated Cancer Hospital, Shandong Cancer Hospital and Institute (Shandong Cancer Hospital), Jinan, China
| | - Rui Liu
- Department of Graduate, Shandong First Medical University, Shandong Academy of Medical Sciences, Jinan, China
- Department of Radiation Physics, Shandong First Medical University Affiliated Cancer Hospital, Shandong Cancer Hospital and Institute (Shandong Cancer Hospital), Jinan, China
| | - Shanshan Du
- Department of Radiation Physics, Shandong First Medical University Affiliated Cancer Hospital, Shandong Cancer Hospital and Institute (Shandong Cancer Hospital), Jinan, China
| | - Yong Yin
- Department of Radiation Physics, Shandong First Medical University Affiliated Cancer Hospital, Shandong Cancer Hospital and Institute (Shandong Cancer Hospital), Jinan, China
| |
Collapse
|
5
|
Wen M, Shcherbakov P, Xu Y, Li J, Hu Y, Zhou Q, Liang H, Yuan L, Zhang X. A temporal enhanced semi-supervised training framework for needle segmentation in 3D ultrasound images. Phys Med Biol 2024; 69:115023. [PMID: 38684166 DOI: 10.1088/1361-6560/ad450b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 04/29/2024] [Indexed: 05/02/2024]
Abstract
Objective.Automated biopsy needle segmentation in 3D ultrasound images can be used for biopsy navigation, but it is quite challenging due to the low ultrasound image resolution and interference similar to the needle appearance. For 3D medical image segmentation, such deep learning networks as convolutional neural network and transformer have been investigated. However, these segmentation methods require numerous labeled data for training, have difficulty in meeting the real-time segmentation requirement and involve high memory consumption.Approach.In this paper, we have proposed the temporal information-based semi-supervised training framework for fast and accurate needle segmentation. Firstly, a novel circle transformer module based on the static and dynamic features has been designed after the encoders for extracting and fusing the temporal information. Then, the consistency constraints of the outputs before and after combining temporal information are proposed to provide the semi-supervision for the unlabeled volume. Finally, the model is trained using the loss function which combines the cross-entropy and Dice similarity coefficient (DSC) based segmentation loss with mean square error based consistency loss. The trained model with the single ultrasound volume input is applied to realize the needle segmentation in ultrasound volume.Main results.Experimental results on three needle ultrasound datasets acquired during the beagle biopsy show that our approach is superior to the most competitive mainstream temporal segmentation model and semi-supervised method by providing higher DSC (77.1% versus 76.5%), smaller needle tip position (1.28 mm versus 1.87 mm) and length (1.78 mm versus 2.19 mm) errors on the kidney dataset as well as DSC (78.5% versus 76.9%), needle tip position (0.86 mm versus 1.12 mm) and length (1.01 mm versus 1.26 mm) errors on the prostate dataset.Significance.The proposed method can significantly enhance needle segmentation accuracy by training with sequential images at no additional cost. This enhancement may further improve the effectiveness of biopsy navigation systems.
Collapse
Affiliation(s)
- Mingwei Wen
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
| | - Pavel Shcherbakov
- Institute for Control Science, Russian Academy of Sciences, 65, Profsoyuznaya str., Moscow 117997, Russia
| | - Yang Xu
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
- Hubei Medical Devices Quality Supervision and Test Institute, Wuhan, 430075, People's Republic of China
| | - Jing Li
- Hubei Medical Devices Quality Supervision and Test Institute, Wuhan, 430075, People's Republic of China
| | - Yi Hu
- Hubei Medical Devices Quality Supervision and Test Institute, Wuhan, 430075, People's Republic of China
| | - Quan Zhou
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
| | - Huageng Liang
- Department of Urology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, No 13, Hangkong Road, Wuhan 430022, People's Republic of China
| | - Li Yuan
- Department of Ultrasound imaging, Wuhan Children's Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, People's Republic of China
| | - Xuming Zhang
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
| |
Collapse
|
6
|
Wei H, Zheng T, Zhang X, Wu Y, Chen Y, Zheng C, Jiang D, Wu B, Guo H, Jiang H, Song B. MRI radiomics based on deep learning automated segmentation to predict early recurrence of hepatocellular carcinoma. Insights Imaging 2024; 15:120. [PMID: 38763975 PMCID: PMC11102894 DOI: 10.1186/s13244-024-01679-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2024] [Accepted: 03/23/2024] [Indexed: 05/21/2024] Open
Abstract
OBJECTIVES To investigate the utility of deep learning (DL) automated segmentation-based MRI radiomic features and clinical-radiological characteristics in predicting early recurrence after curative resection of single hepatocellular carcinoma (HCC). METHODS This single-center, retrospective study included consecutive patients with surgically proven HCC who underwent contrast-enhanced MRI before curative hepatectomy from December 2009 to December 2021. Using 3D U-net-based DL algorithms, automated segmentation of the liver and HCC was performed on six MRI sequences. Radiomic features were extracted from the tumor, tumor border extensions (5 mm, 10 mm, and 20 mm), and the liver. A hybrid model incorporating the optimal radiomic signature and preoperative clinical-radiological characteristics was constructed via Cox regression analyses for early recurrence. Model discrimination was characterized with C-index and time-dependent area under the receiver operating curve (tdAUC) and compared with the widely-adopted BCLC and CNLC staging systems. RESULTS Four hundred and thirty-four patients (median age, 52.0 years; 376 men) were included. Among all radiomic signatures, HCC with 5 mm tumor border extension and liver showed the optimal predictive performance (training set C-index, 0.696). By incorporating this radiomic signature, rim arterial phase hyperenhancement (APHE), and incomplete tumor "capsule," a hybrid model demonstrated a validation set C-index of 0.706 and superior 2-year tdAUC (0.743) than both the BCLC (0.550; p < 0.001) and CNLC (0.635; p = 0.032) systems. This model stratified patients into two prognostically distinct risk strata (both datasets p < 0.001). CONCLUSION A preoperative imaging model incorporating the DL automated segmentation-based radiomic signature with rim APHE and incomplete tumor "capsule" accurately predicted early postsurgical recurrence of a single HCC. CRITICAL RELEVANCE STATEMENT The DL automated segmentation-based MRI radiomic model with rim APHE and incomplete tumor "capsule" hold the potential to facilitate individualized risk estimation of postsurgical early recurrence in a single HCC. KEY POINTS A hybrid model integrating MRI radiomic signature was constructed for early recurrence prediction of HCC. The hybrid model demonstrated superior 2-year AUC than the BCLC and CNLC systems. The model categorized the low-risk HCC group carried longer RFS.
Collapse
Affiliation(s)
- Hong Wei
- Department of Radiology, Functional and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | - Tianying Zheng
- Department of Radiology, Functional and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | | | - Yuanan Wu
- Big Data Research Center, University of Electronic Science and Technology of China, Chengdu, Sichuan, 610000, China
| | - Yidi Chen
- Department of Radiology, Functional and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China
| | - Chao Zheng
- Shukun Technology Co., Ltd, Beijing, 100102, China
| | - Difei Jiang
- Shukun Technology Co., Ltd, Beijing, 100102, China
| | - Botong Wu
- Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100102, China
| | - Hua Guo
- Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100102, China
| | - Hanyu Jiang
- Department of Radiology, Functional and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China.
| | - Bin Song
- Department of Radiology, Functional and Molecular Imaging Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, China.
- Department of Radiology, Sanya People's Hospital, Sanya, Hainan, 572000, China.
| |
Collapse
|
7
|
Maleki Varnosfaderani S, Forouzanfar M. The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century. Bioengineering (Basel) 2024; 11:337. [PMID: 38671759 PMCID: PMC11047988 DOI: 10.3390/bioengineering11040337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2024] [Revised: 03/25/2024] [Accepted: 03/26/2024] [Indexed: 04/28/2024] Open
Abstract
As healthcare systems around the world face challenges such as escalating costs, limited access, and growing demand for personalized care, artificial intelligence (AI) is emerging as a key force for transformation. This review is motivated by the urgent need to harness AI's potential to mitigate these issues and aims to critically assess AI's integration in different healthcare domains. We explore how AI empowers clinical decision-making, optimizes hospital operation and management, refines medical image analysis, and revolutionizes patient care and monitoring through AI-powered wearables. Through several case studies, we review how AI has transformed specific healthcare domains and discuss the remaining challenges and possible solutions. Additionally, we will discuss methodologies for assessing AI healthcare solutions, ethical challenges of AI deployment, and the importance of data privacy and bias mitigation for responsible technology use. By presenting a critical assessment of AI's transformative potential, this review equips researchers with a deeper understanding of AI's current and future impact on healthcare. It encourages an interdisciplinary dialogue between researchers, clinicians, and technologists to navigate the complexities of AI implementation, fostering the development of AI-driven solutions that prioritize ethical standards, equity, and a patient-centered approach.
Collapse
Affiliation(s)
| | - Mohamad Forouzanfar
- Département de Génie des Systèmes, École de Technologie Supérieure (ÉTS), Université du Québec, Montréal, QC H3C 1K3, Canada
- Centre de Recherche de L’institut Universitaire de Gériatrie de Montréal (CRIUGM), Montréal, QC H3W 1W5, Canada
| |
Collapse
|
8
|
Luo X, Li P, Chen H, Zhou K, Piao S, Yang L, Hu B, Geng D. Automatic segmentation of hepatocellular carcinoma on dynamic contrast-enhanced MRI based on deep learning. Phys Med Biol 2024; 69:065008. [PMID: 38330492 DOI: 10.1088/1361-6560/ad2790] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 02/08/2024] [Indexed: 02/10/2024]
Abstract
Objective. Precise hepatocellular carcinoma (HCC) detection is crucial for clinical management. While studies focus on computed tomography-based automatic algorithms, there is a rareness of research on automatic detection based on dynamic contrast enhanced (DCE) magnetic resonance imaging. This study is to develop an automatic detection and segmentation deep learning model for HCC using DCE.Approach: DCE images acquired from 2016 to 2021 were retrospectively collected. Then, 382 patients (301 male; 81 female) with 466 lesions pathologically confirmed were included and divided into an 80% training-validation set and a 20% independent test set. For external validation, 51 patients (42 male; 9 female) in another hospital from 2018 to 2021 were included. The U-net architecture was modified to accommodate multi-phasic DCE input. The model was trained with the training-validation set using five-fold cross-validation, and furtherly evaluated with the independent test set using comprehensive metrics for segmentation and detection performance. The proposed automatic segmentation model consisted of five main steps: phase registration, automatic liver region extraction using a pre-trained model, automatic HCC lesion segmentation using the multi-phasic deep learning model, ensemble of five-fold predictions, and post-processing using connected component analysis to enhance the performance to refine predictions and eliminate false positives.Main results. The proposed model achieved a mean dice similarity coefficient (DSC) of 0.81 ± 0.11, a sensitivity of 94.41 ± 15.50%, a precision of 94.19 ± 17.32%, and 0.14 ± 0.48 false positive lesions per patient in the independent test set. The model detected 88% (80/91) HCC lesions in the condition of DSC > 0.5, and the DSC per tumor was 0.80 ± 0.13. In the external set, the model detected 92% (58/62) lesions with 0.12 ± 0.33 false positives per patient, and the DSC per tumor was 0.75 ± 0.10.Significance.This study developed an automatic detection and segmentation deep learning model for HCC using DCE, which yielded promising post-processed results in accurately identifying and delineating HCC lesions.
Collapse
Affiliation(s)
- Xiao Luo
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Peiwen Li
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Hongyi Chen
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Kun Zhou
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Sirong Piao
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic China
| | - Liqin Yang
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Shanghai Engineering Research Center of Intelligent Imaging for Critical Brain Diseases, Shanghai, People's Republic China
- Institute of Functional and Molecular Medical Imaging, Fudan University, Shanghai, People's Republic of China
| | - Bin Hu
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Daoying Geng
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Shanghai Engineering Research Center of Intelligent Imaging for Critical Brain Diseases, Shanghai, People's Republic China
- Institute of Functional and Molecular Medical Imaging, Fudan University, Shanghai, People's Republic of China
| |
Collapse
|
9
|
Xia T, Zhao B, Li B, Lei Y, Song Y, Wang Y, Tang T, Ju S. MRI-Based Radiomics and Deep Learning in Biological Characteristics and Prognosis of Hepatocellular Carcinoma: Opportunities and Challenges. J Magn Reson Imaging 2024; 59:767-783. [PMID: 37647155 DOI: 10.1002/jmri.28982] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Revised: 08/15/2023] [Accepted: 08/16/2023] [Indexed: 09/01/2023] Open
Abstract
Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer-related death worldwide. HCC exhibits strong inter-tumor heterogeneity, with different biological characteristics closely associated with prognosis. In addition, patients with HCC often distribute at different stages and require diverse treatment options at each stage. Due to the variability in tumor sensitivity to different therapies, determining the optimal treatment approach can be challenging for clinicians prior to treatment. Artificial intelligence (AI) technology, including radiomics and deep learning approaches, has emerged as a unique opportunity to improve the spectrum of HCC clinical care by predicting biological characteristics and prognosis in the medical imaging field. The radiomics approach utilizes handcrafted features derived from specific mathematical formulas to construct various machine-learning models for medical applications. In terms of the deep learning approach, convolutional neural network models are developed to achieve high classification performance based on automatic feature extraction from images. Magnetic resonance imaging offers the advantage of superior tissue resolution and functional information. This comprehensive evaluation plays a vital role in the accurate assessment and effective treatment planning for HCC patients. Recent studies have applied radiomics and deep learning approaches to develop AI-enabled models to improve accuracy in predicting biological characteristics and prognosis, such as microvascular invasion and tumor recurrence. Although AI-enabled models have demonstrated promising potential in HCC with biological characteristics and prognosis prediction with high performance, one of the biggest challenges, interpretability, has hindered their implementation in clinical practice. In the future, continued research is needed to improve the interpretability of AI-enabled models, including aspects such as domain knowledge, novel algorithms, and multi-dimension data sources. Overcoming these challenges would allow AI-enabled models to significantly impact the care provided to HCC patients, ultimately leading to their deployment for clinical use. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 2.
Collapse
Affiliation(s)
- Tianyi Xia
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Ben Zhao
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Binrong Li
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Ying Lei
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Yang Song
- MR Scientific Marketing, Siemens Healthineers Ltd., Shanghai, China
| | - Yuancheng Wang
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Tianyu Tang
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Shenghong Ju
- Jiangsu Key Laboratory of Molecular and Functional Imaging, Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| |
Collapse
|
10
|
Ye Y, Zhang N, Wu D, Huang B, Cai X, Ruan X, Chen L, Huang K, Li ZP, Wu PM, Jiang J, Dan G, Peng Z. Deep Learning Combined with Radiologist's Intervention Achieves Accurate Segmentation of Hepatocellular Carcinoma in Dual-Phase Magnetic Resonance Images. BIOMED RESEARCH INTERNATIONAL 2024; 2024:9267554. [PMID: 38464681 PMCID: PMC10923620 DOI: 10.1155/2024/9267554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 12/20/2023] [Accepted: 02/08/2024] [Indexed: 03/12/2024]
Abstract
Purpose Segmentation of hepatocellular carcinoma (HCC) is crucial; however, manual segmentation is subjective and time-consuming. Accurate and automatic lesion contouring for HCC is desirable in clinical practice. In response to this need, our study introduced a segmentation approach for HCC combining deep convolutional neural networks (DCNNs) and radiologist intervention in magnetic resonance imaging (MRI). We sought to design a segmentation method with a deep learning method that automatically segments using manual location information for moderately experienced radiologists. In addition, we verified the viability of this method to assist radiologists in accurate and fast lesion segmentation. Method In our study, we developed a semiautomatic approach for segmenting HCC using DCNN in conjunction with radiologist intervention in dual-phase gadolinium-ethoxybenzyl-diethylenetriamine penta-acetic acid- (Gd-EOB-DTPA-) enhanced MRI. We developed a DCNN and deep fusion network (DFN) trained on full-size images, namely, DCNN-F and DFN-F. Furthermore, DFN was applied to the image blocks containing tumor lesions that were roughly contoured by a radiologist with 10 years of experience in abdominal MRI, and this method was named DFN-R. Another radiologist with five years of experience (moderate experience) performed tumor lesion contouring for comparison with our proposed methods. The ground truth image was contoured by an experienced radiologist and reviewed by an independent experienced radiologist. Results The mean DSC of DCNN-F, DFN-F, and DFN-R was 0.69 ± 0.20 (median, 0.72), 0.74 ± 0.21 (median, 0.77), and 0.83 ± 0.13 (median, 0.88), respectively. The mean DSC of the segmentation by the radiologist with moderate experience was 0.79 ± 0.11 (median, 0.83), which was lower than the performance of DFN-R. Conclusions Deep learning using dual-phase MRI shows great potential for HCC lesion segmentation. The radiologist-aided semiautomated method (DFN-R) achieved improved performance compared to manual contouring by the radiologist with moderate experience, although the difference was not statistically significant.
Collapse
Affiliation(s)
- Yufeng Ye
- The First Clinical College of Jinan University, Guangzhou, China
- Department of Radiology, Panyu Central Hospital, Guangzhou, China
| | - Naiwen Zhang
- Medical AI Lab, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
| | - Dasheng Wu
- Medical AI Lab, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
| | - Bingsheng Huang
- Department of Radiology, Panyu Central Hospital, Guangzhou, China
- Medical AI Lab, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
- Shenzhen University Clinical Research Center for Neurological Diseases, Shenzhen, Guangdong, China
| | - Xun Cai
- Medical AI Lab, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
| | - Xiaolei Ruan
- Jiuquan Satellite Launch Center, Lanzhou, Gansu, China
| | - Liangliang Chen
- Medical AI Lab, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
| | - Kun Huang
- Department of Radiology, First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- Department of Radiology, Guizhou Provincial People's Hospital, Guiyang, China
| | - Zi-Ping Li
- Department of Radiology, First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Po-Man Wu
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Hong Kong, China
| | - Jinzhao Jiang
- Department of Radiology, Shenzhen University General Hospital, Shenzhen, China
| | - Guo Dan
- School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
| | - Zhenpeng Peng
- Department of Radiology, First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
11
|
Quinton F, Presles B, Leclerc S, Nodari G, Lopez O, Chevallier O, Pellegrinelli J, Vrigneaud JM, Popoff R, Meriaudeau F, Alberini JL. Navigating the nuances: comparative analysis and hyperparameter optimisation of neural architectures on contrast-enhanced MRI for liver and liver tumour segmentation. Sci Rep 2024; 14:3522. [PMID: 38347017 PMCID: PMC10861452 DOI: 10.1038/s41598-024-53528-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 02/01/2024] [Indexed: 02/15/2024] Open
Abstract
In medical imaging, accurate segmentation is crucial to improving diagnosis, treatment, or both. However, navigating the multitude of available architectures for automatic segmentation can be overwhelming, making it challenging to determine the appropriate type of architecture and tune the most crucial parameters during dataset optimisation. To address this problem, we examined and refined seven distinct architectures for segmenting the liver, as well as liver tumours, with a restricted training collection of 60 3D contrast-enhanced magnetic resonance images (CE-MRI) from the ATLAS dataset. Included in these architectures are convolutional neural networks (CNNs), transformers, and hybrid CNN/transformer architectures. Bayesian search techniques were used for hyperparameter tuning to hasten convergence to the optimal parameter mixes while also minimising the number of trained models. It was unexpected that hybrid models, which typically exhibit superior performance on larger datasets, would exhibit comparable performance to CNNs. The optimisation of parameters contributed to better segmentations, resulting in an average increase of 1.7% and 5.0% in liver and tumour segmentation Dice coefficients, respectively. In conclusion, the findings of this study indicate that hybrid CNN/transformer architectures may serve as a practical substitute for CNNs even in small datasets. This underscores the significance of hyperparameter optimisation.
Collapse
Affiliation(s)
- Felix Quinton
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France.
| | - Benoit Presles
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
| | - Sarah Leclerc
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
| | - Guillaume Nodari
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Olivier Lopez
- Service de Radiologie et Imagerie Medicale Diagnostique et Therapeutique, Centre Hospitalier Universitaire, 21000, Dijon, France
| | - Olivier Chevallier
- Service de Radiologie et Imagerie Medicale Diagnostique et Therapeutique, Centre Hospitalier Universitaire, 21000, Dijon, France
| | - Julie Pellegrinelli
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Jean-Marc Vrigneaud
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Romain Popoff
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| | - Fabrice Meriaudeau
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
| | - Jean-Louis Alberini
- Institut de Chimie Moléculaire de l'Université de Bourgogne, ICMUB UMR CNRS 6302, Université Bourgogne, 21000, Dijon, France
- Service de Médecine Nucléaire, Centre Georges-François Leclerc, 21000, Dijon, France
| |
Collapse
|
12
|
Yang Z, Yang X, Cao Y, Shao Q, Tang D, Peng Z, Di S, Zhao Y, Li S. Deep learning based automatic internal gross target volume delineation from 4D-CT of hepatocellular carcinoma patients. J Appl Clin Med Phys 2024; 25:e14211. [PMID: 37992226 PMCID: PMC10795452 DOI: 10.1002/acm2.14211] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 10/07/2023] [Accepted: 11/02/2023] [Indexed: 11/24/2023] Open
Abstract
BACKGROUND The location and morphology of the liver are significantly affected by respiratory motion. Therefore, delineating the gross target volume (GTV) based on 4D medical images is more accurate than regular 3D-CT with contrast. However, the 4D method is also more time-consuming and laborious. This study proposes a deep learning (DL) framework based on 4D-CT that can achieve automatic delineation of internal GTV. METHODS The proposed network consists of two encoding paths, one for feature extraction of adjacent slices (spatial slices) in a specific 3D-CT sequence, and one for feature extraction of slices at the same location in three adjacent phase 3D-CT sequences (temporal slices), a feature fusion module based on an attention mechanism was proposed for fusing the temporal and spatial features. Twenty-six patients' 4D-CT, each consisting of 10 respiratory phases, were used as the dataset. The Hausdorff distance (HD95), Dice similarity coefficient (DSC), and volume difference (VD) between the manual and predicted tumor contour were computed to evaluate the model's segmentation accuracy. RESULTS The predicted GTVs and IGTVs were compared quantitatively and visually with the ground truth. For the test dataset, the proposed method achieved a mean DSC of 0.869 ± 0.089 and an HD95 of 5.14 ± 3.34 mm for all GTVs, with under-segmented GTVs on some CT slices being compensated by GTVs on other slices, resulting in better agreement between the predicted IGTVs and the ground truth, with a mean DSC of 0.882 ± 0.085 and an HD95 of 4.88 ± 2.84 mm. The best GTV results were generally observed at the end-inspiration stage. CONCLUSIONS Our proposed DL framework for tumor segmentation on 4D-CT datasets shows promise for fully automated delineation in the future. The promising results of this work provide impetus for its integration into the 4DCT treatment planning workflow to improve hepatocellular carcinoma radiotherapy.
Collapse
Affiliation(s)
- Zhen Yang
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
- School of AutomationCentral South UniversityChangshaChina
| | - Xiaoyu Yang
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
- School of AutomationCentral South UniversityChangshaChina
| | - Ying Cao
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
| | - Qigang Shao
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
| | - Du Tang
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
| | - Zhao Peng
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
| | - Shuanhu Di
- School of AutomationCentral South UniversityChangshaChina
| | - Yuqian Zhao
- School of AutomationCentral South UniversityChangshaChina
| | - Shuzhou Li
- Department of OncologyNational Clinical Research Center for Geriatric DisordersXiangya HospitalCentral South UniversityChangshaChina
| |
Collapse
|
13
|
Horkaew P, Chansangrat J, Keeratibharat N, Le DC. Recent advances in computerized imaging and its vital roles in liver disease diagnosis, preoperative planning, and interventional liver surgery: A review. World J Gastrointest Surg 2023; 15:2382-2397. [PMID: 38111769 PMCID: PMC10725533 DOI: 10.4240/wjgs.v15.i11.2382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 08/30/2023] [Accepted: 09/27/2023] [Indexed: 11/26/2023] Open
Abstract
The earliest and most accurate detection of the pathological manifestations of hepatic diseases ensures effective treatments and thus positive prognostic outcomes. In clinical settings, screening and determining the extent of a pathology are prominent factors in preparing remedial agents and administering appropriate therapeutic procedures. Moreover, in a patient undergoing liver resection, a realistic preoperative simulation of the subject-specific anatomy and physiology also plays a vital part in conducting initial assessments, making surgical decisions during the procedure, and anticipating postoperative results. Conventionally, various medical imaging modalities, e.g., computed tomography, magnetic resonance imaging, and positron emission tomography, have been employed to assist in these tasks. In fact, several standardized procedures, such as lesion detection and liver segmentation, are also incorporated into prominent commercial software packages. Thus far, most integrated software as a medical device typically involves tedious interactions from the physician, such as manual delineation and empirical adjustments, as per a given patient. With the rapid progress in digital health approaches, especially medical image analysis, a wide range of computer algorithms have been proposed to facilitate those procedures. They include pattern recognition of a liver, its periphery, and lesion, as well as pre- and postoperative simulations. Prior to clinical adoption, however, software must conform to regulatory requirements set by the governing agency, for instance, valid clinical association and analytical and clinical validation. Therefore, this paper provides a detailed account and discussion of the state-of-the-art methods for liver image analyses, visualization, and simulation in the literature. Emphasis is placed upon their concepts, algorithmic classifications, merits, limitations, clinical considerations, and future research trends.
Collapse
Affiliation(s)
- Paramate Horkaew
- School of Computer Engineering, Suranaree University of Technology, Nakhon Ratchasima 30000, Thailand
| | - Jirapa Chansangrat
- School of Radiology, Institute of Medicine, Suranaree University of Technology, Nakhon Ratchasima 30000, Thailand
| | - Nattawut Keeratibharat
- School of Surgery, Institute of Medicine, Suranaree University of Technology, Nakhon Ratchasima 30000, Thailand
| | - Doan Cong Le
- Faculty of Information Technology, An Giang University, Vietnam National University (Ho Chi Minh City), An Giang 90000, Vietnam
| |
Collapse
|
14
|
Khoshkhabar M, Meshgini S, Afrouzian R, Danishvar S. Automatic Liver Tumor Segmentation from CT Images Using Graph Convolutional Network. SENSORS (BASEL, SWITZERLAND) 2023; 23:7561. [PMID: 37688038 PMCID: PMC10490641 DOI: 10.3390/s23177561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Revised: 08/13/2023] [Accepted: 08/16/2023] [Indexed: 09/10/2023]
Abstract
Segmenting the liver and liver tumors in computed tomography (CT) images is an important step toward quantifiable biomarkers for a computer-aided decision-making system and precise medical diagnosis. Radiologists and specialized physicians use CT images to diagnose and classify liver organs and tumors. Because these organs have similar characteristics in form, texture, and light intensity values, other internal organs such as the heart, spleen, stomach, and kidneys confuse visual recognition of the liver and tumor division. Furthermore, visual identification of liver tumors is time-consuming, complicated, and error-prone, and incorrect diagnosis and segmentation can hurt the patient's life. Many automatic and semi-automatic methods based on machine learning algorithms have recently been suggested for liver organ recognition and tumor segmentation. However, there are still difficulties due to poor recognition precision and speed and a lack of dependability. This paper presents a novel deep learning-based technique for segmenting liver tumors and identifying liver organs in computed tomography maps. Based on the LiTS17 database, the suggested technique comprises four Chebyshev graph convolution layers and a fully connected layer that can accurately segment the liver and liver tumors. Thus, the accuracy, Dice coefficient, mean IoU, sensitivity, precision, and recall obtained based on the proposed method according to the LiTS17 dataset are around 99.1%, 91.1%, 90.8%, 99.4%, 99.4%, and 91.2%, respectively. In addition, the effectiveness of the proposed method was evaluated in a noisy environment, and the proposed network could withstand a wide range of environmental signal-to-noise ratios (SNRs). Thus, at SNR = -4 dB, the accuracy of the proposed method for liver organ segmentation remained around 90%. The proposed model has obtained satisfactory and favorable results compared to previous research. According to the positive results, the proposed model is expected to be used to assist radiologists and specialist doctors in the near future.
Collapse
Affiliation(s)
- Maryam Khoshkhabar
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz 51666-16471, Iran
| | - Saeed Meshgini
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz 51666-16471, Iran
| | - Reza Afrouzian
- Miyaneh Faculty of Engineering, University of Tabriz, Miyaneh 51666-16471, Iran
| | - Sebelan Danishvar
- College of Engineering, Design, and Physical Sciences, Brunel University London, Uxbridge UB8 3PH, UK
| |
Collapse
|
15
|
Zhang S, Miao Y, Chen J, Zhang X, Han L, Ran D, Huang Z, Pei N, Liu H, An C. Twist-Net: A multi-modality transfer learning network with the hybrid bilateral encoder for hypopharyngeal cancer segmentation. Comput Biol Med 2023; 154:106555. [PMID: 36701967 DOI: 10.1016/j.compbiomed.2023.106555] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 12/31/2022] [Accepted: 01/11/2023] [Indexed: 01/15/2023]
Abstract
Hypopharyngeal cancer (HPC) is a rare disease. Therefore, it is a challenge to automatically segment HPC tumors and metastatic lymph nodes (HPC risk areas) from medical images with the small-scale dataset. Combining low-level details and high-level semantics from feature maps in different scales can improve the accuracy of segmentation. Herein, we propose a Multi-Modality Transfer Learning Network with Hybrid Bilateral Encoder (Twist-Net) for Hypopharyngeal Cancer Segmentation. Specifically, we propose a Bilateral Transition (BT) block and a Bilateral Gather (BG) block to twist (fuse) high-level semantic feature maps and low-level detailed feature maps. We design a block with multi-receptive field extraction capabilities, M Block, to capture multi-scale information. To avoid overfitting caused by the small scale of the dataset, we propose a transfer learning method that can transfer priors experience from large computer vision datasets to multi-modality medical imaging datasets. Compared with other methods, our method outperforms other methods on HPC dataset, achieving the highest Dice of 82.98%. Our method is also superior to other methods on two public medical segmentation datasets, i.e., the CHASE_DB1 dataset and BraTS2018 dataset. On these two datasets, the Dice of our method is 79.83% and 84.87%, respectively. The code is available at: https://github.com/zhongqiu1245/TwistNet.
Collapse
Affiliation(s)
- Shuo Zhang
- Beijing University of Technology, Beijing, China
| | - Yang Miao
- Beijing University of Technology, Beijing, China; Beijing Key Laboratory of Advanced Manufacturing Technology, Beijing, China
| | - Jun Chen
- Beijing Engineering Research Center of Pediatric Surgery, Engineering and Transformation Center, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
| | - Xiwei Zhang
- Department of Head and Neck Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Lei Han
- Beijing University of Posts and Telecommunications, Beijing, China.
| | - Dongsheng Ran
- Beijing University of Posts and Telecommunications, Beijing, China
| | - Zehao Huang
- Department of Head and Neck Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ning Pei
- Beijing Institute of Technology, Beijing, China
| | - Haibin Liu
- Beijing University of Technology, Beijing, China.
| | - Changming An
- Department of Head and Neck Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
| |
Collapse
|
16
|
Wei J, Jiang H, Zhou Y, Tian J, Furtado FS, Catalano OA. Radiomics: A radiological evidence-based artificial intelligence technique to facilitate personalized precision medicine in hepatocellular carcinoma. Dig Liver Dis 2023:S1590-8658(22)00863-5. [PMID: 36641292 DOI: 10.1016/j.dld.2022.12.015] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 12/15/2022] [Accepted: 12/19/2022] [Indexed: 01/16/2023]
Abstract
The high postoperative recurrence rates in hepatocellular carcinoma (HCC) remain a major hurdle in its management. Appropriate staging and treatment selection may alleviate the extent of fatal recurrence. However, effective methods to preoperatively evaluate pathophysiologic and molecular characteristics of HCC are lacking. Imaging plays a central role in HCC diagnosis and stratification due to the non-invasive diagnostic criteria. Vast and crucial information is hidden within image data. Other than providing a morphological sketch for lesion diagnosis, imaging could provide new insights to describe the pathophysiological and genetic landscape of HCC. Radiomics aims to facilitate diagnosis and prognosis of HCC using artificial intelligence techniques to harness the immense information contained in medical images. Radiomics produces a set of archetypal and robust imaging features that are correlated to key pathological or molecular biomarkers to preoperatively risk-stratify HCC patients. Inferred with outcome data, comprehensive combination of radiomic, clinical and/or multi-omics data could also improve direct prediction of response to treatment and prognosis. The evolution of radiomics is changing our understanding of personalized precision medicine in HCC management. Herein, we review the key techniques and clinical applications in HCC radiomics and discuss current limitations and future opportunities to improve clinical decision making.
Collapse
Affiliation(s)
- Jingwei Wei
- Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, PR. China; Beijing Key Laboratory of Molecular Imaging, Beijing 100190, PR. China.
| | - Hanyu Jiang
- Department of Radiology, West China Hospital, Sichuan University, Chengdu, Sichuan, 610041, PR. China
| | - Yu Zhou
- Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, PR. China; Beijing Key Laboratory of Molecular Imaging, Beijing 100190, PR. China; School of Life Science and Technology, Xidian University, Xi'an, PR. China
| | - Jie Tian
- Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, PR. China; Beijing Key Laboratory of Molecular Imaging, Beijing 100190, PR. China; Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine, Beihang University, Beijing, 100191, PR. China; Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Xi'an, Shaanxi, 710126, PR. China.
| | - Felipe S Furtado
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, United States; Harvard Medical School, 25 Shattuck St, Boston, MA 02115, United States
| | - Onofrio A Catalano
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, United States; Harvard Medical School, 25 Shattuck St, Boston, MA 02115, United States.
| |
Collapse
|
17
|
Joint model- and immunohistochemistry-driven few-shot learning scheme for breast cancer segmentation on 4D DCE-MRI. APPL INTELL 2022. [DOI: 10.1007/s10489-022-04272-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
18
|
Meng K, Cattani P, Villecco F. Brain Tumor Segmentation Based on Bendlet Transform and Improved Chan-Vese Model. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1199. [PMID: 36141085 PMCID: PMC9497468 DOI: 10.3390/e24091199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 08/21/2022] [Accepted: 08/24/2022] [Indexed: 06/16/2023]
Abstract
Automated segmentation of brain tumors is a difficult procedure due to the variability and blurred boundary of the lesions. In this study, we propose an automated model based on Bendlet transform and improved Chan-Vese (CV) model for brain tumor segmentation. Since the Bendlet system is based on the principle of sparse approximation, Bendlet transform is applied to describe the images and map images to the feature space and, thereby, first obtain the feature set. This can help in effectively exploring the mapping relationship between brain lesions and normal tissues, and achieving multi-scale and multi-directional registration. Secondly, the SSIM region detection method is proposed to preliminarily locate the tumor region from three aspects of brightness, structure, and contrast. Finally, the CV model is solved by the Hermite-Shannon-Cosine wavelet homotopy method, and the boundary of the tumor region is more accurately delineated by the wavelet transform coefficient. We randomly selected some cross-sectional images to verify the effectiveness of the proposed algorithm and compared with CV, Ostu, K-FCM, and region growing segmentation methods. The experimental results showed that the proposed algorithm had higher segmentation accuracy and better stability.
Collapse
Affiliation(s)
- Kexin Meng
- College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
| | - Piercarlo Cattani
- Department of Computer, Control and Management Engineering, University of Rome “La Sapienza”, Via Ariosto 25, 00185 Roma, Italy
| | - Francesco Villecco
- Department of Industrial Engineering, University of Salerno, via Giovanni Paolo II 132, 84084 Fisciano, Italy
| |
Collapse
|