1
|
Li X, Wang J, Wei H, Cong J, Sun H, Wang P, Wei B. MH2AFormer: An Efficient Multiscale Hierarchical Hybrid Attention With a Transformer for Bladder Wall and Tumor Segmentation. IEEE J Biomed Health Inform 2024; 28:4772-4784. [PMID: 38713566 DOI: 10.1109/jbhi.2024.3397698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/09/2024]
Abstract
Achieving accurate bladder wall and tumor segmentation from MRI is critical for diagnosing and treating bladder cancer. However, automated segmentation remains challenging due to factors such as comparable density distributions, intricate tumor morphologies, and unclear boundaries. Considering the attributes of bladder MRI images, we propose an efficient multiscale hierarchical hybrid attention with a transformer (MH2AFormer) for bladder cancer and wall segmentation. Specifically, a multiscale hybrid attention and transformer (MHAT) module in the encoder is designed to adaptively extract and aggregate multiscale hybrid feature representations from the input image. In the decoder stage, we devise a multiscale hybrid attention (MHA) module to generate high-quality segmentation results from multiscale hybrid features. Combining these modules enhances the feature representation and guides the model to focus on tumor and wall regions, which helps to solve bladder image segmentation challenges. Moreover, MHAT utilizes the Fast Fourier Transformer with a large kernel (e.g., 224 × 224) to model global feature relationships while reducing computational complexity in the encoding stage. The model performance was evaluated on two datasets. As a result, the model achieves relatively best results regarding the intersection over union (IoU) and dice similarity coefficient (DSC) on both datasets (Dataset A: IoU = 80.26%, DSC = 88.20%; Dataset B: IoU = 89.74%, DSC = 94.48%). These advantageous outcomes substantiate the practical utility of our approach, highlighting its potential to alleviate the workload of radiologists when applied in clinical settings.
Collapse
|
2
|
Li M, Jiang Z, Shen W, Liu H. Deep learning in bladder cancer imaging: A review. Front Oncol 2022; 12:930917. [PMID: 36338676 PMCID: PMC9631317 DOI: 10.3389/fonc.2022.930917] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 09/30/2022] [Indexed: 11/13/2022] Open
Abstract
Deep learning (DL) is a rapidly developing field in machine learning (ML). The concept of deep learning originates from research on artificial neural networks and is an upgrade of traditional neural networks. It has achieved great success in various domains and has shown potential in solving medical problems, particularly when using medical images. Bladder cancer (BCa) is the tenth most common cancer in the world. Imaging, as a safe, noninvasive, and relatively inexpensive technique, is a powerful tool to aid in the diagnosis and treatment of bladder cancer. In this review, we provide an overview of the latest progress in the application of deep learning to the imaging assessment of bladder cancer. First, we review the current deep learning approaches used for bladder segmentation. We then provide examples of how deep learning helps in the diagnosis, staging, and treatment management of bladder cancer using medical images. Finally, we summarize the current limitations of deep learning and provide suggestions for future improvements.
Collapse
Affiliation(s)
- Mingyang Li
- Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zekun Jiang
- Ministry of Education (MoE) Key Lab of Artificial Intelligence, Artificial Intelligence (AI) Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Wei Shen
- Ministry of Education (MoE) Key Lab of Artificial Intelligence, Artificial Intelligence (AI) Institute, Shanghai Jiao Tong University, Shanghai, China
- *Correspondence: Haitao Liu, ; Wei Shen,
| | - Haitao Liu
- Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- *Correspondence: Haitao Liu, ; Wei Shen,
| |
Collapse
|
3
|
Xu X, Wang H, Guo Y, Zhang X, Li B, Du P, Liu Y, Lu H. Study Progress of Noninvasive Imaging and Radiomics for Decoding the Phenotypes and Recurrence Risk of Bladder Cancer. Front Oncol 2021; 11:704039. [PMID: 34336691 PMCID: PMC8321511 DOI: 10.3389/fonc.2021.704039] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2021] [Accepted: 06/30/2021] [Indexed: 12/24/2022] Open
Abstract
Urinary bladder cancer (BCa) is a highly prevalent disease among aged males. Precise diagnosis of tumor phenotypes and recurrence risk is of vital importance in the clinical management of BCa. Although imaging modalities such as CT and multiparametric MRI have played an essential role in the noninvasive diagnosis and prognosis of BCa, radiomics has also shown great potential in the precise diagnosis of BCa and preoperative prediction of the recurrence risk. Radiomics-empowered image interpretation can amplify the differences in tumor heterogeneity between different phenotypes, i.e., high-grade vs. low-grade, early-stage vs. advanced-stage, and nonmuscle-invasive vs. muscle-invasive. With a multimodal radiomics strategy, the recurrence risk of BCa can be preoperatively predicted, providing critical information for the clinical decision making. We thus reviewed the rapid progress in the field of medical imaging empowered by the radiomics for decoding the phenotype and recurrence risk of BCa during the past 20 years, summarizing the entire pipeline of the radiomics strategy for the definition of BCa phenotype and recurrence risk including region of interest definition, radiomics feature extraction, tumor phenotype prediction and recurrence risk stratification. We particularly focus on current pitfalls, challenges and opportunities to promote massive clinical applications of radiomics pipeline in the near future.
Collapse
Affiliation(s)
- Xiaopan Xu
- School of Biomedical Engineering, Air Force Medical University, Xi’an, China
| | - Huanjun Wang
- Department of Radiology, The First Affiliated Hospital, Sun Yat-Sen University, Guangzhou, China
| | - Yan Guo
- Department of Radiology, The First Affiliated Hospital, Sun Yat-Sen University, Guangzhou, China
| | - Xi Zhang
- School of Biomedical Engineering, Air Force Medical University, Xi’an, China
| | - Baojuan Li
- School of Biomedical Engineering, Air Force Medical University, Xi’an, China
| | - Peng Du
- School of Biomedical Engineering, Air Force Medical University, Xi’an, China
| | - Yang Liu
- School of Biomedical Engineering, Air Force Medical University, Xi’an, China
| | - Hongbing Lu
- School of Biomedical Engineering, Air Force Medical University, Xi’an, China
| |
Collapse
|
4
|
Miyake M, Maesaka F, Marugami N, Miyamoto T, Nakai Y, Ohnishi S, Gotoh D, Owari T, Hori S, Morizawa Y, Itami Y, Inoue T, Anai S, Torimoto K, Fujii T, Shimada K, Tanaka N, Fujimoto K. A Potential Application of Dynamic Contrast-Enhanced Magnetic Resonance Imaging Combined with Photodynamic Diagnosis for the Detection of Bladder Carcinoma in Situ: Toward the Future 'MRI-PDD Fusion TURBT'. Diagnostics (Basel) 2019; 9:diagnostics9030112. [PMID: 31487881 PMCID: PMC6787687 DOI: 10.3390/diagnostics9030112] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2019] [Revised: 08/30/2019] [Accepted: 09/02/2019] [Indexed: 11/16/2022] Open
Abstract
The detection of carcinoma in situ (CIS) is essential for the management of high-risk non-muscle invasive bladder cancers. Here, we focused on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) combined with photodynamic diagnosis (PDD) for the detection of CIS. A total of 45 patients undergoing pre-surgical DCE-MRI and PDD-assisted endoscopic surgery accompanied by biopsies of the eight segmentations were analyzed. Immunohistochemical analysis of the biopsies revealed hypervascularity of CIS lesions, a cause of strong submucosal contrast-enhancement. It was found that 56 (16.2%) of 344 biopsies had pathologically proven CIS. In the DCE-MRI, the overall sensitivity and specificity for detecting CIS were 48.2% and 81.9%, respectively. We set out two different combinations of PDD and DCE-MRI for detecting CIS. Combination 1 was positive when either the PDD or DCE-MRI were test-positive. Combination 2 was positive only when both PDD and DCE-MRI were test-positive. The overall sensitivity of combinations 1 and 2 were 75.0% and 37.5%, respectively (McNemar test, vs PDD alone; p = 0.041 and p < 0.001, respectively). However, the specificity was 74.0% and 91.7%, respectively (vs PDD alone; both p < 0.001). Our future goal is to establish 'MRI-PDD fusion transurethral resction of the bladder tumor (TURBT), which could be an effective therapeutic and diagnostic approach in the clinical management of high-risk disease.
Collapse
Affiliation(s)
- Makito Miyake
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan.
| | - Fumisato Maesaka
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Nagaaki Marugami
- Department of Radiology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Tatsuki Miyamoto
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Yasushi Nakai
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Sayuri Ohnishi
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Daisuke Gotoh
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Takuya Owari
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Shunta Hori
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Yosuke Morizawa
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Yoshitaka Itami
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Takeshi Inoue
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Satoshi Anai
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Kazumasa Torimoto
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Tomomi Fujii
- Department of Diagnostic Pathology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Keiji Shimada
- Department of Pathology, Nara City Hospital, 1-50-1 Higashi kidera-cho, Nara, Nara 630-8305, Japan
| | - Nobumichi Tanaka
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| | - Kiyohide Fujimoto
- Department of Urology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara 634-8522, Japan
| |
Collapse
|
5
|
Dolz J, Xu X, Rony J, Yuan J, Liu Y, Granger E, Desrosiers C, Zhang X, Ben Ayed I, Lu H. Multiregion segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks. Med Phys 2018; 45:5482-5493. [DOI: 10.1002/mp.13240] [Citation(s) in RCA: 45] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2018] [Revised: 10/03/2018] [Accepted: 10/05/2018] [Indexed: 12/15/2022] Open
Affiliation(s)
- Jose Dolz
- Laboratory for Imagery, Vision and Artificial Intelligence (LIVIA) École de technologie supérieure Montréal QCH3C 1K3Canada
| | - Xiaopan Xu
- School of Biomedical Engineering Fourth Military Medical University Xi’an Shaanxi710032China
| | - Jérôme Rony
- Laboratory for Imagery, Vision and Artificial Intelligence (LIVIA) École de technologie supérieure Montréal QCH3C 1K3Canada
| | - Jing Yuan
- School of Mathematics and Statistics Xidian University Xi’an 710071China
| | - Yang Liu
- School of Biomedical Engineering Fourth Military Medical University Xi’an Shaanxi710032China
| | - Eric Granger
- Laboratory for Imagery, Vision and Artificial Intelligence (LIVIA) École de technologie supérieure Montréal QCH3C 1K3Canada
| | - Christian Desrosiers
- Laboratory for Imagery, Vision and Artificial Intelligence (LIVIA) École de technologie supérieure Montréal QCH3C 1K3Canada
| | - Xi Zhang
- School of Biomedical Engineering Fourth Military Medical University Xi’an Shaanxi710032China
| | - Ismail Ben Ayed
- Laboratory for Imagery, Vision and Artificial Intelligence (LIVIA) École de technologie supérieure Montréal QCH3C 1K3Canada
| | - Hongbing Lu
- School of Biomedical Engineering Fourth Military Medical University Xi’an Shaanxi710032China
| |
Collapse
|
6
|
Wu K, Garnier C, Alirezaie J, Dillenseger JL. Adaptation and evaluation of the multiple organs OSD for T2 MRI prostate segmentation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2015; 2014:4687-90. [PMID: 25571038 DOI: 10.1109/embc.2014.6944670] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
This paper deals with the adaptation, the tuning and the evaluation of the multiple organs Optimal Surface Detection (OSD) algorithm for the T2 MRI prostate segmentation. This algorithm is initialized by first surface approximations of the prostate (obtained after a model adjustment), the bladder (obtained automatically) and the rectum (interactive geometrical model). These three organs are then segmented together in a multiple organs OSD scheme which proposes a competition between the gray level characteristics and some topological and anatomical information of these three organs. This method has been evaluated on the MICCAI Grand Challenge: Prostate MR Image Segmentation (PROMISE) 2012 training dataset.
Collapse
|
7
|
Shevchenko N, Fallert JA, Stepp H, Sahli H, Karl A, Lueth TC. A high resolution bladder wall map: feasibility study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2012; 2012:5761-5764. [PMID: 23367238 DOI: 10.1109/embc.2012.6347303] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
The aim of this work is to provide the surgeon-urologist with a system for automatic 2D and 3D-reconstruction of the bladder wall to help him within the treatment of bladder cancer as well as planning and documentation of the interventions. Within this small pilot-framework a fast feasibility study was made to clear if it is generally possible to build a bladder wall model using a special endoscope with an embedded laser-based distance measurement, an optical navigation system and modern image stitching techniques. Some experiments with a realistic bladder phantom have shown that this initial concept is generally acceptable and can be used with some extensions to build a system which can provide an automatic bladder wall reconstruction in real time to be used within a surgical intervention.
Collapse
Affiliation(s)
- Nikita Shevchenko
- Technical University of Munich, Department of Micro-Technology and Medical Device Technology, Garching, Germany.
| | | | | | | | | | | |
Collapse
|