1
|
Zhao X, Lai L, Li Y, Zhou X, Cheng X, Chen Y, Huang H, Guo J, Wang G. A lightweight bladder tumor segmentation method based on attention mechanism. Med Biol Eng Comput 2024; 62:1519-1534. [PMID: 38308022 DOI: 10.1007/s11517-024-03018-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Accepted: 01/05/2024] [Indexed: 02/04/2024]
Abstract
In the endoscopic images of bladder, accurate segmentation of different grade bladder tumor from blurred boundary regions and highly variable shapes is of great significance for doctors' diagnosis and patients' later treatment. We propose a nested attentional feature fusion segmentation network (NAFF-Net) based on the encoder-decoder structure formed by the combination of weighted pyramid pooling module (WPPM) and nested attentional feature fusion (NAFF). Among them, WPPM applies the cascade of atrous convolution to enhance the overall perceptual field while introducing adaptive weights to optimize multi-scale feature extraction, NAFF integrates deep semantic information into shallow feature maps, effectively focusing on edge and detail information in bladder tumor images. Additionally, a weighted mixed loss function is constructed to alleviate the impact of imbalance between positive and negative sample distribution on segmentation accuracy. Experiments illustrate the proposed NAFF-Net achieves better segmentation results compared to other mainstream models, with a MIoU of 84.05%, MPrecision of 91.52%, MRecall of 90.81%, and F1-score of 91.16%, and also achieves good results on the public datasets Kvasir-SEG and CVC-ClinicDB. Compared to other models, NAFF-Net has a smaller number of parameters, which is a significant advantage in model deployment.
Collapse
Affiliation(s)
- Xiushun Zhao
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China
| | - Libing Lai
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Yunjiao Li
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China
| | - Xiaochen Zhou
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Xiaofeng Cheng
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Yujun Chen
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Haohui Huang
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China
| | - Jing Guo
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China.
| | - Gongxian Wang
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China.
| |
Collapse
|
2
|
Yu J, Cai L, Chen C, Zou Y, Xiao Y, Fu X, Wang L, Yang X, Liu P, Lu Q, Sun X, Shao Q. A novel predict method for muscular invasion of bladder cancer based on 3D mp-MRI feature fusion. Phys Med Biol 2024; 69:055011. [PMID: 38306973 DOI: 10.1088/1361-6560/ad25c7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 02/01/2024] [Indexed: 02/04/2024]
Abstract
Objective. To assist urologist and radiologist in the preoperative diagnosis of non-muscle-invasive bladder cancer (NMIBC) and muscle-invasive bladder cancer (MIBC), we proposed a combination models strategy (CMS) utilizing multiparametric magnetic resonance imaging.Approach. The CMS includes three components: image registration, image segmentation, and multisequence feature fusion. To ensure spatial structure consistency of T2-weighted imaging (T2WI), diffusion-weighted imaging (DWI), and dynamic contrast-enhanced imaging (DCE), a registration network based on patch sampling normalized mutual information was proposed to register DWI and DCE to T2WI. Moreover, to remove redundant information around the bladder, we employed a segmentation network to obtain the bladder and tumor regions from T2WI. Using the coordinate mapping from T2WI, we extracted these regions from DWI and DCE and integrated them into a three-branch dual-channel input. Finally, to fully fuse low-level and high-level features of T2WI, DWI, and DCE, we proposed a distributed multilayer fusion model for preoperative MIBC prediction with five-fold cross-validation.Main results. The study included 436 patients, of which 404 were for the internal cohort and 32 for external cohort. The MIBC was confirmed by pathological examination. In the internal cohort, the area under the curve, accuracy, sensitivity, and specificity achieved by our method were 0.928, 0.869, 0.753, and 0.929, respectively. For the urologist and radiologist, Vesical Imaging-Reporting and Data System score >3 was employed to determine MIBC. The urologist demonstrated an accuracy, sensitivity, and specificity of 0.842, 0.737, and 0.895, respectively, while the radiologist achieved 0.871, 0.803, and 0.906, respectively. In the external cohort, the accuracy of our method was 0.831, which was higher than that of the urologist (0.781) and the radiologist (0.813).Significance. Our proposed method achieved better diagnostic performance than urologist and was comparable to senior radiologist. These results indicate that CMS can effectively assist junior urologists and radiologists in diagnosing preoperative MIBC.
Collapse
Affiliation(s)
- Jie Yu
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of China
| | - Lingkai Cai
- Department of Urology, the First Affiliated Hospital with Nanjing Medical University, Nanjing, People's Republic of China
| | - Chunxiao Chen
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of China
| | - Yuan Zou
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of China
| | - Yueyue Xiao
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of China
| | - Xue Fu
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of China
| | - Liang Wang
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of China
| | - Xiao Yang
- Department of Urology, the First Affiliated Hospital with Nanjing Medical University, Nanjing, People's Republic of China
| | - Peikun Liu
- Department of Urology, the First Affiliated Hospital with Nanjing Medical University, Nanjing, People's Republic of China
| | - Qiang Lu
- Department of Urology, the First Affiliated Hospital with Nanjing Medical University, Nanjing, People's Republic of China
| | - Xueying Sun
- Department of Radiology, the First Affiliated Hospital with Nanjing Medical University, Nanjing, People's Republic of China
| | - Qiang Shao
- Department of Urology, the Affiliated Suzhou Hospital of Nanjing Medical University, People's Republic of China
| |
Collapse
|
3
|
Akin O, Lema-Dopico A, Paudyal R, Konar AS, Chenevert TL, Malyarenko D, Hadjiiski L, Al-Ahmadie H, Goh AC, Bochner B, Rosenberg J, Schwartz LH, Shukla-Dave A. Multiparametric MRI in Era of Artificial Intelligence for Bladder Cancer Therapies. Cancers (Basel) 2023; 15:5468. [PMID: 38001728 PMCID: PMC10670574 DOI: 10.3390/cancers15225468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Revised: 10/27/2023] [Accepted: 10/30/2023] [Indexed: 11/26/2023] Open
Abstract
This review focuses on the principles, applications, and performance of mpMRI for bladder imaging. Quantitative imaging biomarkers (QIBs) derived from mpMRI are increasingly used in oncological applications, including tumor staging, prognosis, and assessment of treatment response. To standardize mpMRI acquisition and interpretation, an expert panel developed the Vesical Imaging-Reporting and Data System (VI-RADS). Many studies confirm the standardization and high degree of inter-reader agreement to discriminate muscle invasiveness in bladder cancer, supporting VI-RADS implementation in routine clinical practice. The standard MRI sequences for VI-RADS scoring are anatomical imaging, including T2w images, and physiological imaging with diffusion-weighted MRI (DW-MRI) and dynamic contrast-enhanced MRI (DCE-MRI). Physiological QIBs derived from analysis of DW- and DCE-MRI data and radiomic image features extracted from mpMRI images play an important role in bladder cancer. The current development of AI tools for analyzing mpMRI data and their potential impact on bladder imaging are surveyed. AI architectures are often implemented based on convolutional neural networks (CNNs), focusing on narrow/specific tasks. The application of AI can substantially impact bladder imaging clinical workflows; for example, manual tumor segmentation, which demands high time commitment and has inter-reader variability, can be replaced by an autosegmentation tool. The use of mpMRI and AI is projected to drive the field toward the personalized management of bladder cancer patients.
Collapse
Affiliation(s)
- Oguz Akin
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
| | - Alfonso Lema-Dopico
- Department of Medical Physics, Memorial Sloan Kettering Cancer, New York, NY 10065, USA
| | - Ramesh Paudyal
- Department of Medical Physics, Memorial Sloan Kettering Cancer, New York, NY 10065, USA
| | | | | | - Dariya Malyarenko
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Lubomir Hadjiiski
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Hikmat Al-Ahmadie
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
| | - Alvin C. Goh
- Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
| | - Bernard Bochner
- Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
| | - Jonathan Rosenberg
- Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
| | - Lawrence H. Schwartz
- Department of Medical Physics, Memorial Sloan Kettering Cancer, New York, NY 10065, USA
| | - Amita Shukla-Dave
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
- Department of Medical Physics, Memorial Sloan Kettering Cancer, New York, NY 10065, USA
| |
Collapse
|
4
|
Development of Deep Learning with RDA U-Net Network for Bladder Cancer Segmentation. Cancers (Basel) 2023; 15:cancers15041343. [PMID: 36831685 PMCID: PMC9954660 DOI: 10.3390/cancers15041343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 02/15/2023] [Accepted: 02/17/2023] [Indexed: 02/23/2023] Open
Abstract
In today's high-order health examination, imaging examination accounts for a large proportion. Computed tomography (CT), which can detect the whole body, uses X-rays to penetrate the human body to obtain images. Its presentation is a high-resolution black-and-white image composed of gray scales. It is expected to assist doctors in making judgments through deep learning based on the image recognition technology of artificial intelligence. It used CT images to identify the bladder and lesions and then segmented them in the images. The images can achieve high accuracy without using a developer. In this study, the U-Net neural network, commonly used in the medical field, was used to extend the encoder position in combination with the ResBlock in ResNet and the Dense Block in DenseNet, so that the training could maintain the training parameters while reducing the overall identification operation time. The decoder could be used in combination with Attention Gates to suppress the irrelevant areas of the image while paying attention to significant features. Combined with the above algorithm, we proposed a Residual-Dense Attention (RDA) U-Net model, which was used to identify organs and lesions from CT images of abdominal scans. The accuracy (ACC) of using this model for the bladder and its lesions was 96% and 93%, respectively. The values of Intersection over Union (IoU) were 0.9505 and 0.8024, respectively. Average Hausdorff distance (AVGDIST) was as low as 0.02 and 0.12, respectively, and the overall training time was reduced by up to 44% compared with other convolution neural networks.
Collapse
|
5
|
Zou Y, Cai L, Chen C, Shao Q, Fu X, Yu J, Wang L, Chen Z, Yang X, Yuan B, Liu P, Lu Q. Multi-task deep learning based on T2-Weighted Images for predicting Muscular-Invasive Bladder Cancer. Comput Biol Med 2022; 151:106219. [PMID: 36343408 DOI: 10.1016/j.compbiomed.2022.106219] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2022] [Revised: 09/29/2022] [Accepted: 10/15/2022] [Indexed: 12/27/2022]
Abstract
BACKGROUND An accurate preoperative assessment of Non-Muscle-Invasive Bladder Cancer (NMIBC) and Muscle-Invasive Bladder Cancer (MIBC) in Bladder Cancer (BCa) can help the urologist make diagnostic decisions. Considering the absence of multiparametric MRI for contrast medium allergy and economic reasons, this study aims to develop a deep learning method based on T2-Weighted (T2WI) images alone for predicting NMIBC and MIBC. METHOD We propose a Multi-task BCa Muscular Invasion Prediction (MBMIP) model to discriminate MIBC from NMIBC. The three-channel-input including the original T2WI image, segmented bladder, and the region of interest can help the MBMIP model locate the bladder and pay more attention to the surrounding information of the tumor. Inception V3 is used as the feature extraction module, which uses multiple branches to extract high-level features with different degrees of abstraction. In addition, based on the idea of multi-task learning, a reconstruction block for T2WI images is also introduced to assist the backbone classification network to improve the classification performance. RESULTS The entire data consist of retrospective data (390 cases), prospective data (39 cases), and multi-center data (39 cases). In the retrospective test, the accuracy, sensitivity, and specificity of the MBMIP model are 0.911, 0.889, and 0.920 respectively, while those of the prospective test are 0.923, 1.000, and 0.885. And in the muti-center test, the MBMIP model yields accuracy, sensitivity, and specificity of 0.846, 0.667, and 0.879. CONCLUSION The MBMIP model could achieve a satisfactory prediction result in discriminating between NMIBC and MIBC, which may aid urologists in preoperative decision-making for BCa patients.
Collapse
Affiliation(s)
- Yuan Zou
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Lingkai Cai
- Department of Urology, the First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Chunxiao Chen
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China.
| | - Qiang Shao
- Department of Urology, the Affiliated Suzhou Hospital of Nanjing Medical University, Nanjing, China.
| | - Xue Fu
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Jie Yu
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Liang Wang
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Zhiying Chen
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Xiao Yang
- Department of Urology, the First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Baorui Yuan
- Department of Urology, the First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Peikun Liu
- Department of Urology, the First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Qiang Lu
- Department of Urology, the First Affiliated Hospital of Nanjing Medical University, Nanjing, China.
| |
Collapse
|
6
|
Gong T, Han H, Tan Z, Ning Z, Qiao H, Yu M, Zhao X, Tang X, Liu G, Shang F, Liu S. Segmentation and differentiation of periventricular and deep white matter hyperintensities in 2D T2-FLAIR MRI based on a cascade U-net. Front Neurol 2022; 13:1021477. [DOI: 10.3389/fneur.2022.1021477] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 10/27/2022] [Indexed: 11/18/2022] Open
Abstract
BackgroundWhite matter hyperintensities (WMHs) are a subtype of cerebral small vessel disease and can be divided into periventricular WMHs (pvWMHs) and deep WMHs (dWMHs). pvWMHs and dWMHs were proved to be determined by different etiologies. This study aimed to develop a 2D Cascade U-net (Cascade U) for the segmentation and differentiation of pvWMHs and dWMHs on 2D T2-FLAIR images.MethodsA total of 253 subjects were recruited in the present study. All subjects underwent 2D T2-FLAIR scan on a 3.0 Tesla MR scanner. Both contours of pvWMHs and dWMHs were manually delineated by the observers and considered as the gold standard. Fazekas scale was used to evaluate the burdens of pvWMHs and dWMHs, respectively. Cascade U consisted of a segmentation U-net and a differentiation U-net and was trained with a combined loss function. The performance of Cascade U was compared with two other U-net models (Pipeline U and Separate U). Dice similarity coefficient (DSC), Matthews correlation coefficient (MCC), precision, and recall were used to evaluate the performances of all models. The linear correlations between WMHs volume (WMHV) measured by all models and the gold standard were also conducted.ResultsCompared with other models, Cascade U exhibited a better performance on WMHs segmentation and pvWMHs identification. Cascade U achieved DSC values of 0.605 ± 0.135, 0.517 ± 0.263, and 0.510 ± 0.241 and MCC values of 0.617 ± 0.122, 0.526 ± 0.263, and 0.522 ± 0.243 on the segmentation of total WMHs, pvWMHs, and dWMHs, respectively. Cascade U exhibited strong correlations with the gold standard on measuring WMHV (R2 = 0.954, p < 0.001), pvWMHV (R2 = 0.933, p < 0.001), and dWMHV (R2 = 0.918, p < 0.001). A significant correlation was found on lesion volume between Cascade U and gold standard (r > 0.510, p < 0.001).ConclusionCascade U showed competitive results in segmentation and differentiation of pvWMHs and dWMHs on 2D T2-FLAIR images, indicating potential feasibility in precisely evaluating the burdens of WMHs.
Collapse
|
7
|
Li M, Jiang Z, Shen W, Liu H. Deep learning in bladder cancer imaging: A review. Front Oncol 2022; 12:930917. [PMID: 36338676 PMCID: PMC9631317 DOI: 10.3389/fonc.2022.930917] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 09/30/2022] [Indexed: 11/13/2022] Open
Abstract
Deep learning (DL) is a rapidly developing field in machine learning (ML). The concept of deep learning originates from research on artificial neural networks and is an upgrade of traditional neural networks. It has achieved great success in various domains and has shown potential in solving medical problems, particularly when using medical images. Bladder cancer (BCa) is the tenth most common cancer in the world. Imaging, as a safe, noninvasive, and relatively inexpensive technique, is a powerful tool to aid in the diagnosis and treatment of bladder cancer. In this review, we provide an overview of the latest progress in the application of deep learning to the imaging assessment of bladder cancer. First, we review the current deep learning approaches used for bladder segmentation. We then provide examples of how deep learning helps in the diagnosis, staging, and treatment management of bladder cancer using medical images. Finally, we summarize the current limitations of deep learning and provide suggestions for future improvements.
Collapse
Affiliation(s)
- Mingyang Li
- Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zekun Jiang
- Ministry of Education (MoE) Key Lab of Artificial Intelligence, Artificial Intelligence (AI) Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Wei Shen
- Ministry of Education (MoE) Key Lab of Artificial Intelligence, Artificial Intelligence (AI) Institute, Shanghai Jiao Tong University, Shanghai, China
- *Correspondence: Haitao Liu, ; Wei Shen,
| | - Haitao Liu
- Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- *Correspondence: Haitao Liu, ; Wei Shen,
| |
Collapse
|