1
|
Pace DF, Contreras HTM, Romanowicz J, Ghelani S, Rahaman I, Zhang Y, Gao P, Jubair MI, Yeh T, Golland P, Geva T, Ghelani S, Powell AJ, Moghari MH. HVSMR-2.0: A 3D cardiovascular MR dataset for whole-heart segmentation in congenital heart disease. Sci Data 2024; 11:721. [PMID: 38956063 PMCID: PMC11219801 DOI: 10.1038/s41597-024-03469-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 06/04/2024] [Indexed: 07/04/2024] Open
Abstract
Patients with congenital heart disease often have cardiac anatomy that deviates significantly from normal, frequently requiring multiple heart surgeries. Image segmentation from a preoperative cardiovascular magnetic resonance (CMR) scan would enable creation of patient-specific 3D surface models of the heart, which have potential to improve surgical planning, enable surgical simulation, and allow automatic computation of quantitative metrics of heart function. However, there is no publicly available CMR dataset for whole-heart segmentation in patients with congenital heart disease. Here, we release the HVSMR-2.0 dataset, comprising 60 CMR scans alongside manual segmentation masks of the 4 cardiac chambers and 4 great vessels. The images showcase a wide range of heart defects and prior surgical interventions. The dataset also includes masks of required and optional extents of the great vessels, enabling fairer comparisons across algorithms. Detailed diagnoses for each subject are also provided. By releasing HVSMR-2.0, we aim to encourage development of robust segmentation algorithms and clinically relevant tools for congenital heart disease.
Collapse
Affiliation(s)
- Danielle F Pace
- A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA.
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA.
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
| | - Hannah T M Contreras
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
- Pediatric Surgical Research Laboratories, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jennifer Romanowicz
- Department of Pediatrics, Section of Cardiology, Children's Hospital Colorado, Aurora, CO, USA
| | - Shruti Ghelani
- Department of Computer Science, University of Massachusetts Boston, Boston, MA, USA
| | - Imon Rahaman
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Yue Zhang
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
- Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Department of Biochemistry and Molecular Genetics, Northwestern University, Chicago, IL, USA
- School of Medicine, Indiana University, Indianapolis, IN, USA
| | - Patricia Gao
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | | | - Tom Yeh
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, USA
- Department of Psychology, Ewha Womans University, Seoul, South Korea
| | - Polina Golland
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Tal Geva
- Department of Cardiology, Boston Children's Hospital, Boston, MA, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| | - Sunil Ghelani
- Department of Cardiology, Boston Children's Hospital, Boston, MA, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| | - Andrew J Powell
- Department of Cardiology, Boston Children's Hospital, Boston, MA, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| | - Mehdi Hedjazi Moghari
- Department of Cardiology, Boston Children's Hospital, Boston, MA, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
- School of Medicine, The University of Colorado, Aurora, CO, USA
- Department of Radiology, Children's Hospital Colorado, Aurora, CO, USA
| |
Collapse
|
2
|
Lim ZW, Li J, Wong D, Chung J, Toh A, Lee JL, Lam C, Balakrishnan M, Chia A, Chua J, Girard M, Hoang QV, Chong R, Wong CW, Saw SM, Schmetterer L, Brennan N, Ang M. Comparison of manual and artificial intelligence-automated choroidal thickness segmentation of optical coherence tomography imaging in myopic adults. EYE AND VISION (LONDON, ENGLAND) 2024; 11:21. [PMID: 38831465 PMCID: PMC11145894 DOI: 10.1186/s40662-024-00385-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 04/17/2024] [Indexed: 06/05/2024]
Abstract
BACKGROUND Myopia affects 1.4 billion individuals worldwide. Notably, there is increasing evidence that choroidal thickness plays an important role in myopia and risk of developing myopia-related conditions. With the advancements in artificial intelligence (AI), choroidal thickness segmentation can now be automated, offering inherent advantages such as better repeatability, reduced grader variability, and less reliance for manpower. Hence, we aimed to evaluate the agreement between AI-automated and manual segmented measurements of subfoveal choroidal thickness (SFCT) using two swept-source optical coherence tomography (OCT) systems. METHODS Subjects aged ≥ 16 years, with myopia of ≥ 0.50 diopters in both eyes, were recruited from the Prospective Myopia Cohort Study in Singapore (PROMYSE). OCT scans were acquired using Triton DRI-OCT and PLEX Elite 9000. OCT images were segmented both automatically with an established SA-Net architecture and manually using a standard technique with adjudication by two independent graders. SFCT was subsequently determined based on the segmentation. The Bland-Altman plot and intraclass correlation coefficient (ICC) were used to evaluate the agreement. RESULTS A total of 229 subjects (456 eyes) with mean [± standard deviation (SD)] age of 34.1 (10.4) years were included. The overall SFCT (mean ± SD) based on manual segmentation was 216.9 ± 82.7 µm with Triton DRI-OCT and 239.3 ± 84.3 µm with PLEX Elite 9000. ICC values demonstrated excellent agreement between AI-automated and manual segmented SFCT measurements (PLEX Elite 9000: ICC = 0.937, 95% CI: 0.922 to 0.949, P < 0.001; Triton DRI-OCT: ICC = 0.887, 95% CI: 0.608 to 0.950, P < 0.001). For PLEX Elite 9000, manual segmented measurements were generally thicker when compared to AI-automated segmented measurements, with a fixed bias of 6.3 µm (95% CI: 3.8 to 8.9, P < 0.001) and proportional bias of 0.120 (P < 0.001). On the other hand, manual segmented measurements were comparatively thinner than AI-automated segmented measurements for Triton DRI-OCT, with a fixed bias of - 26.7 µm (95% CI: - 29.7 to - 23.7, P < 0.001) and proportional bias of - 0.090 (P < 0.001). CONCLUSION We observed an excellent agreement in choroidal segmentation measurements when comparing manual with AI-automated techniques, using images from two SS-OCT systems. Given its edge over manual segmentation, automated segmentation may potentially emerge as the primary method of choroidal thickness measurement in the future.
Collapse
Affiliation(s)
- Zhi Wei Lim
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Jonathan Li
- Department of Ophthalmology, University of California, San Francisco, CA, USA
| | - Damon Wong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Ophthalmology and Visual Sciences Department, Duke-NUS Medical School, Singapore, Singapore
- SERI-NTU Advanced Ocular Engineering (STANCE), Singapore Eye Research Institute and Nanyang Technological University, Singapore, Singapore
- Center for Medical Physics and Biomedical Engineering, Medical University Vienna, Vienna, Austria
| | - Joey Chung
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Angeline Toh
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Jia Ling Lee
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Crystal Lam
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Maithily Balakrishnan
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Audrey Chia
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
| | - Jacqueline Chua
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Ophthalmology and Visual Sciences Department, Duke-NUS Medical School, Singapore, Singapore
- SERI-NTU Advanced Ocular Engineering (STANCE), Singapore Eye Research Institute and Nanyang Technological University, Singapore, Singapore
| | - Michael Girard
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
- Institute of Molecular and Clinical Ophthalmology, Basel, Switzerland
| | - Quan V Hoang
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
- Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- Department of Ophthalmology, Edward S. Harkness Eye Institute, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA
| | - Rachel Chong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
| | - Chee Wai Wong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Seang Mei Saw
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
| | - Leopold Schmetterer
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- SERI-NTU Advanced Ocular Engineering (STANCE), Singapore Eye Research Institute and Nanyang Technological University, Singapore, Singapore
- Center for Medical Physics and Biomedical Engineering, Medical University Vienna, Vienna, Austria
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
- Institute of Molecular and Clinical Ophthalmology, Basel, Switzerland
- School of Chemistry, Chemical Engineering and Biotechnology, Nanyang Technological University, Singapore, Singapore
- Department of Clinical Pharmacology, Medical University Vienna, Vienna, Austria
| | | | - Marcus Ang
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.
- Ophthalmology and Visual Sciences Department, Duke-NUS Medical School, Singapore, Singapore.
| |
Collapse
|
3
|
Yeom YS, Braunstein L, Morton LM, Bolton KL, Choi JW, Choi HY, Greenstein N, Lee C. A novel method for rapid estimation of active bone marrow dose for radiotherapy patients in epidemiological studies. Med Phys 2024; 51:4472-4481. [PMID: 38734989 DOI: 10.1002/mp.17118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Revised: 03/21/2024] [Accepted: 04/18/2024] [Indexed: 05/13/2024] Open
Abstract
BACKGROUND In a dedicated effort to improve the assessment of clonal hematopoiesis (CH) and study leukemia risk following radiotherapy, we are developing a large-scale cohort study among cancer patients who received radiation. To that end, it will be critical to analyze dosimetric parameters of red bone marrow (ABM) exposure in relation to CH and its progression to myeloid neoplasms, requiring reconstruction method for ABM doses of a large-scale patients rapidly and accurately. PURPOSE To support a large-scale cohort study on the assessment of clonal hematopoiesis and leukemia risk following radiotherapy, we present a new method for the rapid reconstruction of ABM doses of radiotherapy among cancer patients. METHODS The key idea of the presented method is to segment patient bones rapidly and automatically by matching a whole-body computational human phantom, in which the skeletal system is divided into 34 bone sites, to patient CT images via 3D skeletal registration. The automatic approach was used to segment site-specific bones for 40 radiotherapy patients. Also, we segmented the bones manually. The bones segmented both manually and automatically were then combined with the patient dose matrix calculated by the treatment planning system (TPS) to derive patient ABM dose. We evaluated the performance of the automatic method in geometric and dosimetric accuracy by comparison with the manual approach. RESULTS The pelvis showed the best geometric performance [volume overlap fraction (VOF): 52% (mean) with 23% (σ) and average distance (AD): 0.8 cm (mean) with 0.5 cm (σ)]. The pelvis also showed the best dosimetry performance [absorbed dose difference (ADD): 0.7 Gy (mean) with 1.0 Gy (σ)]. Some bones showed unsatisfactory performances such as the cervical vertebrae [ADD: 5.2 Gy (mean) with 10.8 Gy (σ)]. This impact on the total ABM dose, however, was not significant. An excellent agreement for the total ABM dose was indeed observed [ADD: 0.4 Gy (mean) with 0.4 Gy (σ)]. The computation time required for dose calculation using our method was robust (about one minute per patient). CONCLUSIONS We confirmed that our method estimates ABM doses across treatment sites accurately, while providing high computational efficiency. The method will be used to reconstruct patient-specific ABM doses for dose-response assessment in a large cohort study. The method can also be applied to prospective dose calculation within a clinical TPS to support clinical decision making at the point of care.
Collapse
Affiliation(s)
- Yeon Soo Yeom
- Department of Radiation Convergence Engineering, Yonsei University, Wonju, Gangwon, Republic of Korea
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, Maryland, USA
| | - Lior Braunstein
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, New York, USA
| | - Lindsay M Morton
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, Maryland, USA
| | - Kelly L Bolton
- Department of Medicine, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Ji Won Choi
- Department of Radiation Convergence Engineering, Yonsei University, Wonju, Gangwon, Republic of Korea
| | - Hyeong Yun Choi
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, Maryland, USA
| | | | - Choonsik Lee
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, Maryland, USA
| |
Collapse
|
4
|
Liu H, Yang J, Jiang C, He S, Fu Y, Zhang S, Hu X, Fang J, Ji W. S2DA-Net: Spatial and spectral-learning double-branch aggregation network for liver tumor segmentation in CT images. Comput Biol Med 2024; 174:108400. [PMID: 38613888 DOI: 10.1016/j.compbiomed.2024.108400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 03/10/2024] [Accepted: 04/01/2024] [Indexed: 04/15/2024]
Abstract
Accurate liver tumor segmentation is crucial for aiding radiologists in hepatocellular carcinoma evaluation and surgical planning. While convolutional neural networks (CNNs) have been successful in medical image segmentation, they face challenges in capturing long-term dependencies among pixels. On the other hand, Transformer-based models demand a high number of parameters and involve significant computational costs. To address these issues, we propose the Spatial and Spectral-learning Double-branched Aggregation Network (S2DA-Net) for liver tumor segmentation. S2DA-Net consists of a double-branched encoder and a decoder with a Group Multi-Head Cross-Attention Aggregation (GMCA) module, Two branches in the encoder consist of a Fourier Spectral-learning Multi-scale Fusion (FSMF) branch and a Multi-axis Aggregation Hadamard Attention (MAHA) branch. The FSMF branch employs a Fourier-based network to learn amplitude and phase information, capturing richer features and detailed information without introducing an excessive number of parameters. The FSMF branch utilizes a Fourier-based network to capture amplitude and phase information, enriching features without introducing excessive parameters. The MAHA branch incorporates spatial information, enhancing discriminative features while minimizing computational costs. In the decoding path, a GMCA module extracts local information and establishes long-term dependencies, improving localization capabilities by amalgamating features from diverse branches. Experimental results on the public LiTS2017 liver tumor datasets show that the proposed segmentation model achieves significant improvements compared to the state-of-the-art methods, obtaining dice per case (DPC) 69.4 % and global dice (DG) 80.0 % for liver tumor segmentation on the LiTS2017 dataset. Meanwhile, the pre-trained model based on the LiTS2017 datasets obtain, DPC 73.4 % and an DG 82.2 % on the 3DIRCADb dataset.
Collapse
Affiliation(s)
- Huaxiang Liu
- Department Radiology of Taizhou Hospital, Zhejiang University, Taizhou, 318000, Zhejiang, China; Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China; Key Laboratory of Evidence-based Radiology of Taizhou, Taizhou, 317000, Zhejiang, China
| | - Jie Yang
- School of Geophysics and Measurement and Control Technology, East China University of Technology, Nanchang, 330013, China
| | - Chao Jiang
- School of Geophysics and Measurement and Control Technology, East China University of Technology, Nanchang, 330013, China
| | - Sailing He
- Department Radiology of Taizhou Hospital, Zhejiang University, Taizhou, 318000, Zhejiang, China
| | - Youyao Fu
- Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China
| | - Shiqing Zhang
- Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China
| | - Xudong Hu
- Key Laboratory of Evidence-based Radiology of Taizhou, Taizhou, 317000, Zhejiang, China
| | - Jiangxiong Fang
- Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China.
| | - Wenbin Ji
- Department Radiology of Taizhou Hospital, Zhejiang University, Taizhou, 318000, Zhejiang, China; Key Laboratory of Evidence-based Radiology of Taizhou, Taizhou, 317000, Zhejiang, China.
| |
Collapse
|
5
|
Chen C, Chen Y, Li X, Ning H, Xiao R. Linear semantic transformation for semi-supervised medical image segmentation. Comput Biol Med 2024; 173:108331. [PMID: 38522252 DOI: 10.1016/j.compbiomed.2024.108331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 02/29/2024] [Accepted: 03/17/2024] [Indexed: 03/26/2024]
Abstract
Medical image segmentation is a focus research and foundation in developing intelligent medical systems. Recently, deep learning for medical image segmentation has become a standard process and succeeded significantly, promoting the development of reconstruction, and surgical planning of disease diagnosis. However, semantic learning is often inefficient owing to the lack of supervision of feature maps, resulting in that high-quality segmentation models always rely on numerous and accurate data annotations. Learning robust semantic representation in latent spaces remains a challenge. In this paper, we propose a novel semi-supervised learning framework to learn vital attributes in medical images, which constructs generalized representation from diverse semantics to realize medical image segmentation. We first build a self-supervised learning part that achieves context recovery by reconstructing space and intensity of medical images, which conduct semantic representation for feature maps. Subsequently, we combine semantic-rich feature maps and utilize simple linear semantic transformation to convert them into image segmentation. The proposed framework was tested using five medical segmentation datasets. Quantitative assessments indicate the highest scores of our method on IXI (73.78%), ScaF (47.50%), COVID-19-Seg (50.72%), PC-Seg (65.06%), and Brain-MR (72.63%) datasets. Finally, we compared our method with the latest semi-supervised learning methods and obtained 77.15% and 75.22% DSC values, respectively, ranking first on two representative datasets. The experimental results not only proved that the proposed linear semantic transformation was effectively applied to medical image segmentation, but also presented its simplicity and ease-of-use to pursue robust segmentation in semi-supervised learning. Our code is now open at: https://github.com/QingYunA/Linear-Semantic-Transformation-for-Semi-Supervised-Medical-Image-Segmentation.
Collapse
Affiliation(s)
- Cheng Chen
- School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China
| | - Yunqing Chen
- School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China
| | - Xiaoheng Li
- School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China
| | - Huansheng Ning
- School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China
| | - Ruoxiu Xiao
- School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China; Shunde Innovation School, University of Science and Technology Beijing, Foshan, 100024, China.
| |
Collapse
|
6
|
Zhang Z, Han J, Ji W, Lou H, Li Z, Hu Y, Wang M, Qi B, Liu S. Improved deep learning for automatic localisation and segmentation of rectal cancer on T2-weighted MRI. J Med Radiat Sci 2024. [PMID: 38654675 DOI: 10.1002/jmrs.794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Accepted: 04/09/2024] [Indexed: 04/26/2024] Open
Abstract
INTRODUCTION The automatic segmentation approaches of rectal cancer from magnetic resonance imaging (MRI) are very valuable to relieve physicians from heavy workloads and enhance working efficiency. This study aimed to compare the segmentation accuracy of a proposed model with the other three models and the inter-observer consistency. METHODS A total of 65 patients with rectal cancer who underwent MRI examination were enrolled in our cohort and were randomly divided into a training cohort (n = 45) and a validation cohort (n = 20). Two experienced radiologists independently segmented rectal cancer lesions. A novel segmentation model (AttSEResUNet) was trained on T2WI based on ResUNet and attention mechanisms. The segmentation performance of the AttSEResUNet, U-Net, ResUNet and U-Net with Attention Gate (AttUNet) was compared, using Dice similarity coefficient (DSC), Hausdorff distance (HD), mean distance to agreement (MDA) and Jaccard index. The segmentation variability of automatic segmentation models and inter-observer was also evaluated. RESULTS The AttSEResUNet with post-processing showed perfect lesion recognition rate (100%) and false recognition rate (0), and its evaluation metrics outperformed other three models for two independent readers (observer 1: DSC = 0.839 ± 0.112, HD = 9.55 ± 6.68, MDA = 0.556 ± 0.722, Jaccard index = 0.736 ± 0.150; observer 2: DSC = 0.856 ± 0.099, HD = 11.0 ± 10.1, MDA = 0.789 ± 1.07, Jaccard index = 0.673 ± 0.130). The segmentation performance of AttSEResUNet was comparable and similar to manual variability (DSC = 0.857 ± 0.115, HD = 10.0 ± 10.0, MDA = 0.704 ± 1.17, Jaccard index = 0.666 ± 0.139). CONCLUSION Comparing with other three models, the proposed AttSEResUNet model was demonstrated as a more accurate model for contouring the rectal tumours in axial T2WI images, whose variability was similar to that of inter-observer.
Collapse
Affiliation(s)
- Zaixian Zhang
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Junqi Han
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Weina Ji
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Henan Lou
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Zhiming Li
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Yabin Hu
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Mingjia Wang
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao, China
| | - Baozhu Qi
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao, China
| | - Shunli Liu
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, China
| |
Collapse
|
7
|
Zhang K, Yang X, Cui Y, Zhao J, Li D. Imaging segmentation mechanism for rectal tumors using improved U-Net. BMC Med Imaging 2024; 24:95. [PMID: 38654162 DOI: 10.1186/s12880-024-01269-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Accepted: 04/05/2024] [Indexed: 04/25/2024] Open
Abstract
OBJECTIVE In radiation therapy, cancerous region segmentation in magnetic resonance images (MRI) is a critical step. For rectal cancer, the automatic segmentation of rectal tumors from an MRI is a great challenge. There are two main shortcomings in existing deep learning-based methods that lead to incorrect segmentation: 1) there are many organs surrounding the rectum, and the shape of some organs is similar to that of rectal tumors; 2) high-level features extracted by conventional neural networks often do not contain enough high-resolution information. Therefore, an improved U-Net segmentation network based on attention mechanisms is proposed to replace the traditional U-Net network. METHODS The overall framework of the proposed method is based on traditional U-Net. A ResNeSt module was added to extract the overall features, and a shape module was added after the encoder layer. We then combined the outputs of the shape module and the decoder to obtain the results. Moreover, the model used different types of attention mechanisms, so that the network learned information to improve segmentation accuracy. RESULTS We validated the effectiveness of the proposed method using 3773 2D MRI datasets from 304 patients. The results showed that the proposed method achieved 0.987, 0.946, 0.897, and 0.899 for Dice, MPA, MioU, and FWIoU, respectively; these values are significantly better than those of other existing methods. CONCLUSION Due to time savings, the proposed method can help radiologists segment rectal tumors effectively and enable them to focus on patients whose cancerous regions are difficult for the network to segment. SIGNIFICANCE The proposed method can help doctors segment rectal tumors, thereby ensuring good diagnostic quality and accuracy.
Collapse
Affiliation(s)
- Kenan Zhang
- College of Electronic Information and Optical Engineering, Taiyuan University of Technology, Taiyuan, 030024, China
- Key Laboratory of Big Data Fusion Analysis and Application of Shanxi Province, Taiyuan, 030024, China
| | - Xiaotang Yang
- Department of Radiology, Shanxi Cancer Hospital, Shanxi Medical University, Taiyuan, 030013, China.
| | - Yanfen Cui
- Department of Radiology, Shanxi Cancer Hospital, Shanxi Medical University, Taiyuan, 030013, China
| | - Jumin Zhao
- College of Electronic Information and Optical Engineering, Taiyuan University of Technology, Taiyuan, 030024, China
- Key Laboratory of Big Data Fusion Analysis and Application of Shanxi Province, Taiyuan, 030024, China
- Intelligent Perception Engineering Technology Center of Shanxi, Taiyuan, 030024, China
| | - Dengao Li
- College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, Taiyuan, 030024, China.
- Key Laboratory of Big Data Fusion Analysis and Application of Shanxi Province, Taiyuan, 030024, China.
- Intelligent Perception Engineering Technology Center of Shanxi, Taiyuan, 030024, China.
| |
Collapse
|
8
|
Song Z, Wu H, Chen W, Slowik A. Improving automatic segmentation of liver tumor images using a deep learning model. Heliyon 2024; 10:e28538. [PMID: 38571625 PMCID: PMC10988037 DOI: 10.1016/j.heliyon.2024.e28538] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 03/17/2024] [Accepted: 03/20/2024] [Indexed: 04/05/2024] Open
Abstract
Liver tumors are one of the most aggressive malignancies in the human body. Computer-aided technology and liver interventional surgery are effective in the prediction, identification and management of liver neoplasms. One of the important processes is to accurately grasp the morphological structure of the liver and liver blood vessels. However, accurate identification and segmentation of hepatic blood vessels in CT images poses a formidable challenge. Manually locating and segmenting liver vessels in CT images is time-consuming and impractical. There is an imperative clinical requirement for a precise and effective algorithm to segment liver vessels. In response to this demand, the current paper advocates a liver vessel segmentation approach that employs an enhanced 3D fully convolutional neural network V-Net. The network model improves the basic network structure according to the characteristics of liver vessels. First, a pyramidal convolution block is introduced between the encoder and decoder of the network to improve the network localization ability. Then, multi-resolution deep supervision is introduced in the network, resulting in more robust segmentation. Finally, by fusing feature maps of different resolutions, the overall segmentation result is predicted. Evaluation experiments on public datasets demonstrate that our improved scheme can increase the segmentation ability of existing network models for liver vessels. Compared with the existing work, the experimental outcomes demonstrate that the technique presented in this manuscript has attained superior performance on the Dice Coefficient index, which can promote the treatment of liver tumors.
Collapse
Affiliation(s)
- Zhendong Song
- School of Mechanical and Electrical Engineering, Shenzhen Polytechnic University, Shenzhen, 518055, China
| | - Huiming Wu
- School of Mechanical and Electrical Engineering, Shenzhen Polytechnic University, Shenzhen, 518055, China
| | - Wei Chen
- School of Mechanical and Electrical Engineering, Shenzhen Polytechnic University, Shenzhen, 518055, China
| | - Adam Slowik
- Koszalin University of Technology, Koszalin, Poland
| |
Collapse
|
9
|
Zheng S, Sun Q, Ye X, Li W, Yu L, Yang C. Multi-scale adversarial learning with difficult region supervision learning models for primary tumor segmentation. Phys Med Biol 2024; 69:085009. [PMID: 38471170 DOI: 10.1088/1361-6560/ad3321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 03/12/2024] [Indexed: 03/14/2024]
Abstract
Objective.Recently, deep learning techniques have found extensive application in accurate and automated segmentation of tumor regions. However, owing to the variety of tumor shapes, complex types, and unpredictability of spatial distribution, tumor segmentation still faces major challenges. Taking cues from the deep supervision and adversarial learning, we have devised a cascade-based methodology incorporating multi-scale adversarial learning and difficult-region supervision learning in this study to tackle these challenges.Approach.Overall, the method adheres to a coarse-to-fine strategy, first roughly locating the target region, and then refining the target object with multi-stage cascaded binary segmentation which converts complex multi-class segmentation problems into multiple simpler binary segmentation problems. In addition, a multi-scale adversarial learning difficult supervised UNet (MSALDS-UNet) is proposed as our model for fine-segmentation, which applies multiple discriminators along the decoding path of the segmentation network to implement multi-scale adversarial learning, thereby enhancing the accuracy of network segmentation. Meanwhile, in MSALDS-UNet, we introduce a difficult region supervision loss to effectively utilize structural information for segmenting difficult-to-distinguish areas, such as blurry boundary areas.Main results.A thorough validation of three independent public databases (KiTS21, MSD's Brain and Pancreas datasets) shows that our model achieves satisfactory results for tumor segmentation in terms of key evaluation metrics including dice similarity coefficient, Jaccard similarity coefficient, and HD95.Significance.This paper introduces a cascade approach that combines multi-scale adversarial learning and difficult supervision to achieve precise tumor segmentation. It confirms that the combination can improve the segmentation performance, especially for small objects (our codes are publicly availabled onhttps://zhengshenhai.github.io/).
Collapse
Affiliation(s)
- Shenhai Zheng
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, People's Republic of China
- Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, People's Republic of China
| | - Qiuyu Sun
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, People's Republic of China
| | - Xin Ye
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, People's Republic of China
| | - Weisheng Li
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, People's Republic of China
- Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, People's Republic of China
| | - Lei Yu
- Emergency Department, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, People's Republic of China
| | - Chaohui Yang
- Nanpeng Artificial Intelligence Research Institute, Chongqing, People's Republic of China
| |
Collapse
|
10
|
Zhan F, Wang W, Chen Q, Guo Y, He L, Wang L. Three-Direction Fusion for Accurate Volumetric Liver and Tumor Segmentation. IEEE J Biomed Health Inform 2024; 28:2175-2186. [PMID: 38109246 DOI: 10.1109/jbhi.2023.3344392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2023]
Abstract
Biomedical image segmentation of organs, tissues and lesions has gained increasing attention in clinical treatment planning and navigation, which involves the exploration of two-dimensional (2D) and three-dimensional (3D) contexts in the biomedical image. Compared to 2D methods, 3D methods pay more attention to inter-slice correlations, which offer additional spatial information for image segmentation. An organ or tumor has a 3D structure that can be observed from three directions. Previous studies focus only on the vertical axis, limiting the understanding of the relationship between a tumor and its surrounding tissues. Important information can also be obtained from sagittal and coronal axes. Therefore, spatial information of organs and tumors can be obtained from three directions, i.e. the sagittal, coronal and vertical axes, to understand better the invasion depth of tumor and its relationship with the surrounding tissues. Moreover, the edges of organs and tumors in biomedical image may be blurred. To address these problems, we propose a three-direction fusion volumetric segmentation (TFVS) model for segmenting 3D biomedical images from three perspectives in sagittal, coronal and transverse planes, respectively. We use the dataset of the liver task provided by the Medical Segmentation Decathlon challenge to train our model. The TFVS method demonstrates a competitive performance on the 3D-IRCADB dataset. In addition, the t-test and Wilcoxon signed-rank test are also performed to show the statistical significance of the improvement by the proposed method as compared with the baseline methods. The proposed method is expected to be beneficial in guiding and facilitating clinical diagnosis and treatment.
Collapse
|
11
|
Ilesanmi AE, Ilesanmi TO, Ajayi BO. Reviewing 3D convolutional neural network approaches for medical image segmentation. Heliyon 2024; 10:e27398. [PMID: 38496891 PMCID: PMC10944240 DOI: 10.1016/j.heliyon.2024.e27398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 02/27/2024] [Accepted: 02/28/2024] [Indexed: 03/19/2024] Open
Abstract
Background Convolutional neural networks (CNNs) assume pivotal roles in aiding clinicians in diagnosis and treatment decisions. The rapid evolution of imaging technology has established three-dimensional (3D) CNNs as a formidable framework for delineating organs and anomalies in medical images. The prominence of 3D CNN frameworks is steadily growing within medical image segmentation and classification. Thus, our proposition entails a comprehensive review, encapsulating diverse 3D CNN algorithms for the segmentation of medical image anomalies and organs. Methods This study systematically presents an exhaustive review of recent 3D CNN methodologies. Rigorous screening of abstracts and titles were carried out to establish their relevance. Research papers disseminated across academic repositories were meticulously chosen, analyzed, and appraised against specific criteria. Insights into the realm of anomalies and organ segmentation were derived, encompassing details such as network architecture and achieved accuracies. Results This paper offers an all-encompassing analysis, unveiling the prevailing trends in 3D CNN segmentation. In-depth elucidations encompass essential insights, constraints, observations, and avenues for future exploration. A discerning examination indicates the preponderance of the encoder-decoder network in segmentation tasks. The encoder-decoder framework affords a coherent methodology for the segmentation of medical images. Conclusion The findings of this study are poised to find application in clinical diagnosis and therapeutic interventions. Despite inherent limitations, CNN algorithms showcase commendable accuracy levels, solidifying their potential in medical image segmentation and classification endeavors.
Collapse
Affiliation(s)
- Ademola E. Ilesanmi
- University of Pennsylvania, 3710 Hamilton Walk, 6th Floor, Philadelphia, PA, 19104, United States
| | | | - Babatunde O. Ajayi
- National Astronomical Research Institute of Thailand, Chiang Mai 50180, Thailand
| |
Collapse
|
12
|
Bommanapally V, Abeyrathna D, Chundi P, Subramaniam M. Super resolution-based methodology for self-supervised segmentation of microscopy images. Front Microbiol 2024; 15:1255850. [PMID: 38533330 PMCID: PMC10963421 DOI: 10.3389/fmicb.2024.1255850] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Accepted: 02/15/2024] [Indexed: 03/28/2024] Open
Abstract
Data-driven Artificial Intelligence (AI)/Machine learning (ML) image analysis approaches have gained a lot of momentum in analyzing microscopy images in bioengineering, biotechnology, and medicine. The success of these approaches crucially relies on the availability of high-quality microscopy images, which is often a challenge due to the diverse experimental conditions and modes under which these images are obtained. In this study, we propose the use of recent ML-based image super-resolution (SR) techniques for improving the image quality of microscopy images, incorporating them into multiple ML-based image analysis tasks, and describing a comprehensive study, investigating the impact of SR techniques on the segmentation of microscopy images. The impacts of four Generative Adversarial Network (GAN)- and transformer-based SR techniques on microscopy image quality are measured using three well-established quality metrics. These SR techniques are incorporated into multiple deep network pipelines using supervised, contrastive, and non-contrastive self-supervised methods to semantically segment microscopy images from multiple datasets. Our results show that the image quality of microscopy images has a direct influence on the ML model performance and that both supervised and self-supervised network pipelines using SR images perform better by 2%-6% in comparison to baselines, not using SR. Based on our experiments, we also establish that the image quality improvement threshold range [20-64] for the complemented Perception-based Image Quality Evaluator(PIQE) metric can be used as a pre-condition by domain experts to incorporate SR techniques to significantly improve segmentation performance. A plug-and-play software platform developed to integrate SR techniques with various deep networks using supervised and self-supervised learning methods is also presented.
Collapse
Affiliation(s)
- Vidya Bommanapally
- Department of Computer Science, University of Nebraska, Omaha, NE, United States
| | | | | | | |
Collapse
|
13
|
Ao Y, Shi W, Ji B, Miao Y, He W, Jiang Z. MS-TCNet: An effective Transformer-CNN combined network using multi-scale feature learning for 3D medical image segmentation. Comput Biol Med 2024; 170:108057. [PMID: 38301516 DOI: 10.1016/j.compbiomed.2024.108057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 12/31/2023] [Accepted: 01/26/2024] [Indexed: 02/03/2024]
Abstract
Medical image segmentation is a fundamental research problem in the field of medical image processing. Recently, the Transformer have achieved highly competitive performance in computer vision. Therefore, many methods combining Transformer with convolutional neural networks (CNNs) have emerged for segmenting medical images. However, these methods cannot effectively capture the multi-scale features in medical images, even though texture and contextual information embedded in the multi-scale features are extremely beneficial for segmentation. To alleviate this limitation, we propose a novel Transformer-CNN combined network using multi-scale feature learning for three-dimensional (3D) medical image segmentation, which is called MS-TCNet. The proposed model utilizes a shunted Transformer and CNN to construct an encoder and pyramid decoder, allowing six different scale levels of feature learning. It captures multi-scale features with refinement at each scale level. Additionally, we propose a novel lightweight multi-scale feature fusion (MSFF) module that can fully fuse the different-scale semantic features generated by the pyramid decoder for each segmentation class, resulting in a more accurate segmentation output. We conducted experiments on three widely used 3D medical image segmentation datasets. The experimental results indicated that our method outperformed state-of-the-art medical image segmentation methods, suggesting its effectiveness, robustness, and superiority. Meanwhile, our model has a smaller number of parameters and lower computational complexity than conventional 3D segmentation networks. The results confirmed that the model is capable of effective multi-scale feature learning and that the learned multi-scale features are useful for improving segmentation performance. We open-sourced our code, which can be found at https://github.com/AustinYuAo/MS-TCNet.
Collapse
Affiliation(s)
- Yu Ao
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China
| | - Weili Shi
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Bai Ji
- Department of Hepatobiliary and Pancreatic Surgery, The First Hospital of Jilin University, Changchun, 130061, China
| | - Yu Miao
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Wei He
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Zhengang Jiang
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China.
| |
Collapse
|
14
|
Yang C, Zhang H, Chi D, Li Y, Xiao Q, Bai Y, Li Z, Li H, Li H. Contour attention network for cerebrovascular segmentation from TOF-MRA volumetric images. Med Phys 2024; 51:2020-2031. [PMID: 37672343 DOI: 10.1002/mp.16720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 06/25/2023] [Accepted: 07/20/2023] [Indexed: 09/08/2023] Open
Abstract
BACKGROUND Cerebrovascular segmentation is a crucial step in the computer-assisted diagnosis of cerebrovascular pathologies. However, accurate extraction of cerebral vessels from time-of-flight magnetic resonance angiography (TOF-MRA) data is still challenging due to the complex topology and slender shape. PURPOSE The existing deep learning-based approaches pay more attention to the skeleton and ignore the contour, which limits the segmentation performance of the cerebrovascular structure. We aim to weight the contour of brain vessels in shallow features when concatenating with deep features. It helps to obtain more accurate cerebrovascular details and narrows the semantic gap between multilevel features. METHODS This work proposes a novel framework for priority extraction of contours in cerebrovascular structures. We first design a neighborhood-based algorithm to generate the ground truth of the cerebrovascular contour from original annotations, which can introduce useful shape information for the segmentation network. Moreover, We propose an encoder-dual decoder-based contour attention network (CA-Net), which consists of the dilated asymmetry convolution block (DACB) and the Contour Attention Module (CAM). The ancillary decoder uses the DACB to obtain cerebrovascular contour features under the supervision of contour annotations. The CAM transforms these features into a spatial attention map to increase the weight of the contour voxels in main decoder to better restored the vessel contour details. RESULTS The CA-Net is thoroughly validated using two publicly available datasets, and the experimental results demonstrate that our network outperforms the competitors for cerebrovascular segmentation. We achieved the average dice similarity coefficient (D S C $DSC$ ) of 68.15 and 99.92% in natural and synthetic datasets. Our method segments cerebrovascular structures with better completeness. CONCLUSIONS We propose a new framework containing contour annotation generation and cerebrovascular segmentation network that better captures the tiny vessels and improve vessel connectivity.
Collapse
Affiliation(s)
- Chaozhi Yang
- College of Computer Science and Technology, China University of Petroleum (EastChina), Qingdao, China
| | | | - Dianwei Chi
- School of Artificial Intelligence, Yantai Institute of Technology, Yantai, China
| | - Yachuan Li
- College of Computer Science and Technology, China University of Petroleum (EastChina), Qingdao, China
| | - Qian Xiao
- College of Computer Science and Technology, China University of Petroleum (EastChina), Qingdao, China
| | - Yun Bai
- College of Computer Science and Technology, China University of Petroleum (EastChina), Qingdao, China
| | - Zongmin Li
- College of Computer Science and Technology, China University of Petroleum (EastChina), Qingdao, China
- Shengli College of China University of Petroleum, Dongying, China
| | - Hongyi Li
- Beijing Hospital, National Center of Gerontology, Institute of Geriatric Medicine, Chinese Academy of Medical Science, Beijing, China
| | - Hua Li
- Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
15
|
Chen S, Xie F, Chen S, Liu S, Li H, Gong Q, Ruan G, Liu L, Chen H. TdDS-UNet: top-down deeply supervised U-Net for the delineation of 3D colorectal cancer. Phys Med Biol 2024; 69:055018. [PMID: 38306960 DOI: 10.1088/1361-6560/ad25c5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 02/01/2024] [Indexed: 02/04/2024]
Abstract
Automatically delineating colorectal cancers with fuzzy boundaries from 3D images is a challenging task, but the problem of fuzzy boundary delineation in existing deep learning-based methods have not been investigated in depth. Here, an encoder-decoder-based U-shaped network (U-Net) based on top-down deep supervision (TdDS) was designed to accurately and automatically delineate the fuzzy boundaries of colorectal cancer. TdDS refines the semantic targets of the upper and lower stages by mapping ground truths that are more consistent with the stage properties than upsampling deep supervision. This stage-specific approach can guide the model to learn a coarse-to-fine delineation process and improve the delineation accuracy of fuzzy boundaries by gradually shrinking the boundaries. Experimental results showed that TdDS is more customizable and plays a role similar to the attentional mechanism, and it can further improve the capability of the model to delineate colorectal cancer contours. A total of 103, 12, and 29 3D pelvic magnetic resonance imaging volumes were used for training, validation, and testing, respectively. The comparative results indicate that the proposed method exhibits the best comprehensive performance, with a dice similarity coefficient (DSC) of 0.805 ± 0.053 and a hausdorff distance (HD) of 9.28 ± 5.14 voxels. In the delineation performance analysis section also showed that 44.49% of the delineation results are satisfactory and do not require revisions. This study can provide new technical support for the delineation of 3D colorectal cancer. Our method is open source, and the code is available athttps://github.com/odindis/TdDS/tree/main.
Collapse
Affiliation(s)
- Shuchao Chen
- School of Life & Environmental Science, Guilin University of Electronic Technology, Guilin 541004, People's Republic of China
| | - Fei Xie
- State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510060, People's Republic of China
| | - Shenghuan Chen
- Department of Radiology, The Sixth Affiliated Hospital of Guangzhou Medical university, Qingyuan People's Hospital, Qingyuan, People's Republic of China
| | - Shanshan Liu
- School of Life & Environmental Science, Guilin University of Electronic Technology, Guilin 541004, People's Republic of China
| | - Haojiang Li
- State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510060, People's Republic of China
| | - Qiong Gong
- School of Life & Environmental Science, Guilin University of Electronic Technology, Guilin 541004, People's Republic of China
| | - Guangying Ruan
- State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510060, People's Republic of China
| | - Lizhi Liu
- State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510060, People's Republic of China
| | - Hongbo Chen
- School of Life & Environmental Science, Guilin University of Electronic Technology, Guilin 541004, People's Republic of China
- Guangxi Human Physiological Information NonInvasive Detection Engineering Technology Research Center, Guilin 541004, People's Republic of China
- Guangxi Colleges and Universities Key Laboratory of Biomedical Sensors and Intelligent Instruments, Guilin 541004, People's Republic of China
- Guangxi Key Laboratory of Metabolic Reprogramming and Intelligent Medical Engineering for Chronic Diseases, Guilin, People's Republic of China
| |
Collapse
|
16
|
Tsai YL, Yu PC, Nien HH, Lu TP. Time variation of high-risk groups for liver function deteriorations within fluctuating long-term liver function after hepatic radiotherapy in patients with hepatocellular carcinoma. Eur J Med Res 2024; 29:104. [PMID: 38326881 PMCID: PMC10848403 DOI: 10.1186/s40001-024-01692-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 01/20/2024] [Indexed: 02/09/2024] Open
Abstract
PURPOSE The purpose of this study is to find essential risk factors associated with liver function (LF) deteriorations within fluctuating long-term LF and their time-varying effects in patients with hepatocellular carcinoma (HCC) receiving hepatic radiotherapy and to identify high-risk groups for adverse LF deteriorations and their changes over time in facilitating the prevention of hepatic decompensation and the improvement of survival. MATERIALS AND METHODS A total of 133 HCC patients treated by hepatic radiotherapy were enrolled. A study design was conducted to convert posttreatment long-term LF with fluctuating levels over time to recurrent LF events using defined upgrades in a grading scale. The hazard ratios (HR) of pretreatment biochemical, demographic, clinical, and dosimetric factors in developing posttreatment LF events were estimated using the Cox model. Methodologies of the counting process approach, robust variance estimation, goodness-of-fit testing based on the Schoenfeld residuals, and time-dependent covariates in survival analysis were employed to handle the correlation within subjects and evaluate the time-varying effects during long-term follow-up. RESULTS Baseline LF score before radiotherapy and gender were significant factors. Initial HR in developing LF events was 1.17 (95% CI 1.11-1.23; P < 0.001) for each increase of baseline LF score and kept almost constant over time (HR, 1.00; 95% CI 1.00-1.01; P = 0.065). However, no difference was observed regarding initial hazards for gender (HR, 1.00; 95% CI 0.64-1.56; P = 0.994), but the hazard for women got higher monthly over time compared with men (HR, 1.04; 95% CI 1.01-1.07; P = 0.006). CONCLUSIONS High-risk groups for adverse LF deteriorations after hepatic radiotherapy may change over time. Patients with poor baseline LF are vulnerable from the beginning. Women require prevention strategies and careful monitoring for deteriorations at a later stage.
Collapse
Affiliation(s)
- Yu-Lun Tsai
- Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan
- Department of Radiation Oncology, Cathay General Hospital, Taipei, Taiwan
| | - Pei-Chieh Yu
- Department of Radiation Oncology, Cathay General Hospital, Taipei, Taiwan
- School of Medicine, China Medical University, Taichung, Taiwan
| | - Hsin-Hua Nien
- Department of Radiation Oncology, Cathay General Hospital, Taipei, Taiwan
- School of Medicine, Fu Jen Catholic University, New Taipei City, Taiwan
- Institute of Biomedical Engineering, College of Electrical and Computer Engineering, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Tzu-Pin Lu
- Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan.
- Bioinformatics and Biostatistics Core, Center of Genomic and Precision Medicine, National Taiwan University, Taipei, Taiwan.
- Institute of Health Data Analytics and Statistics, College of Public Health, National Taiwan University, Taipei, Taiwan.
| |
Collapse
|
17
|
Chen X, Ma N, Xu T, Xu C. Deep learning-based tooth segmentation methods in medical imaging: A review. Proc Inst Mech Eng H 2024; 238:115-131. [PMID: 38314788 DOI: 10.1177/09544119231217603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2024]
Abstract
Deep learning approaches for tooth segmentation employ convolutional neural networks (CNNs) or Transformers to derive tooth feature maps from extensive training datasets. Tooth segmentation serves as a critical prerequisite for clinical dental analysis and surgical procedures, enabling dentists to comprehensively assess oral conditions and subsequently diagnose pathologies. Over the past decade, deep learning has experienced significant advancements, with researchers introducing efficient models such as U-Net, Mask R-CNN, and Segmentation Transformer (SETR). Building upon these frameworks, scholars have proposed numerous enhancement and optimization modules to attain superior tooth segmentation performance. This paper discusses the deep learning methods of tooth segmentation on dental panoramic radiographs (DPRs), cone-beam computed tomography (CBCT) images, intro oral scan (IOS) models, and others. Finally, we outline performance-enhancing techniques and suggest potential avenues for ongoing research. Numerous challenges remain, including data annotation and model generalization limitations. This paper offers insights for future tooth segmentation studies, potentially facilitating broader clinical adoption.
Collapse
Affiliation(s)
- Xiaokang Chen
- Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing, China
| | - Nan Ma
- Faculty of Information and Technology, Beijing University of Technology, Beijing, China
- Engineering Research Center of Intelligence Perception and Autonomous Control, Ministry of Education, Beijing University of Technology, Beijing, China
| | - Tongkai Xu
- Department of General Dentistry II, Peking University School and Hospital of Stomatology, Beijing, China
| | - Cheng Xu
- Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing, China
| |
Collapse
|
18
|
Cobb R, Cook GJR, Reader AJ. Deep Learned Segmentations of Inflammation for Novel ⁹⁹ mTc-maraciclatide Imaging of Rheumatoid Arthritis. Diagnostics (Basel) 2023; 13:3298. [PMID: 37958194 PMCID: PMC10647206 DOI: 10.3390/diagnostics13213298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 10/04/2023] [Accepted: 10/10/2023] [Indexed: 11/15/2023] Open
Abstract
Rheumatoid arthritis (RA) is an autoimmune disease that causes joint pain, stiffness, and erosion. Power Doppler ultrasound and MRI are imaging modalities used in detecting and monitoring the disease, but they have limitations. ⁹⁹mTc-maraciclatide gamma camera imaging is a novel technique that can detect joint inflammation at all sites in a single examination and has been shown to correlate with power Doppler ultrasound. In this work, we investigate if machine learning models can be used to automatically segment regions of normal, low, and highly inflamed tissue from 192 ⁹⁹mTc-maraciclatide scans of the hands and wrists from 48 patients. Two models were trained: a thresholding model that learns lower and upper threshold values and a neural-network-based nnU-Net model that uses a convolutional neural network (CNN). The nnU-Net model showed 0.94 ± 0.01, 0.51 ± 0.14, and 0.76 ± 0.16 modified Dice scores for segmenting the normal, low, and highly inflamed tissue, respectively, when compared to clinical segmented labels. This outperforms the thresholding model, which achieved modified Dice scores of 0.92 ± 0.01, 0.14 ± 0.07, and 0.35 ± 0.21, respectively. This is an important first step in developing artificial intelligence (AI) tools to assist clinicians' workflow in the use of this new radiopharmaceutical.
Collapse
Affiliation(s)
- Robert Cobb
- Department of Biomedical Engineering, School of Biomedical Engineering and Imaging Sciences, King’s College London, London WC2R 2LS, UK;
| | - Gary J. R. Cook
- Department of Cancer Imaging, School of Biomedical Engineering and Imaging Sciences, King’s College London, London WC2R 2LS, UK;
- King’s College London and Guy’s and St Thomas’ PET Centre, King’s College London, London WC2R 2LS, UK
| | - Andrew J. Reader
- Department of Biomedical Engineering, School of Biomedical Engineering and Imaging Sciences, King’s College London, London WC2R 2LS, UK;
| |
Collapse
|
19
|
Radiya K, Joakimsen HL, Mikalsen KØ, Aahlin EK, Lindsetmo RO, Mortensen KE. Performance and clinical applicability of machine learning in liver computed tomography imaging: a systematic review. Eur Radiol 2023; 33:6689-6717. [PMID: 37171491 PMCID: PMC10511359 DOI: 10.1007/s00330-023-09609-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 02/02/2023] [Accepted: 02/06/2023] [Indexed: 05/13/2023]
Abstract
OBJECTIVES Machine learning (ML) for medical imaging is emerging for several organs and image modalities. Our objectives were to provide clinicians with an overview of this field by answering the following questions: (1) How is ML applied in liver computed tomography (CT) imaging? (2) How well do ML systems perform in liver CT imaging? (3) What are the clinical applications of ML in liver CT imaging? METHODS A systematic review was carried out according to the guidelines from the PRISMA-P statement. The search string focused on studies containing content relating to artificial intelligence, liver, and computed tomography. RESULTS One hundred ninety-one studies were included in the study. ML was applied to CT liver imaging by image analysis without clinicians' intervention in majority of studies while in newer studies the fusion of ML method with clinical intervention have been identified. Several were documented to perform very accurately on reliable but small data. Most models identified were deep learning-based, mainly using convolutional neural networks. Potentially many clinical applications of ML to CT liver imaging have been identified through our review including liver and its lesion segmentation and classification, segmentation of vascular structure inside the liver, fibrosis and cirrhosis staging, metastasis prediction, and evaluation of chemotherapy. CONCLUSION Several studies attempted to provide transparent result of the model. To make the model convenient for a clinical application, prospective clinical validation studies are in urgent call. Computer scientists and engineers should seek to cooperate with health professionals to ensure this. KEY POINTS • ML shows great potential for CT liver image tasks such as pixel-wise segmentation and classification of liver and liver lesions, fibrosis staging, metastasis prediction, and retrieval of relevant liver lesions from similar cases of other patients. • Despite presenting the result is not standardized, many studies have attempted to provide transparent results to interpret the machine learning method performance in the literature. • Prospective studies are in urgent call for clinical validation of ML method, preferably carried out by cooperation between clinicians and computer scientists.
Collapse
Affiliation(s)
- Keyur Radiya
- Department of Gastroenterological Surgery at University Hospital of North Norway (UNN), Tromso, Norway.
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway.
| | - Henrik Lykke Joakimsen
- Institute of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
- Centre for Clinical Artificial Intelligence (SPKI), University Hospital of North Norway, Tromso, Norway
| | - Karl Øyvind Mikalsen
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
- Centre for Clinical Artificial Intelligence (SPKI), University Hospital of North Norway, Tromso, Norway
- UiT Machine Learning Group, Department of Physics and Technology, UiT the Arctic University of Norway, Tromso, Norway
| | - Eirik Kjus Aahlin
- Department of Gastroenterological Surgery at University Hospital of North Norway (UNN), Tromso, Norway
| | - Rolv-Ole Lindsetmo
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
- Head Clinic of Surgery, Oncology and Women Health, University Hospital of North Norway, Tromso, Norway
| | - Kim Erlend Mortensen
- Department of Gastroenterological Surgery at University Hospital of North Norway (UNN), Tromso, Norway
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
| |
Collapse
|
20
|
Weng X, Song F, Tang M, Wang K, Zhang Y, Miao Y, Chan LWC, Lei P, Hu Z, Yang F. MDM-U-Net: A novel network for renal cancer structure segmentation. Comput Med Imaging Graph 2023; 109:102301. [PMID: 37738774 DOI: 10.1016/j.compmedimag.2023.102301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 07/27/2023] [Accepted: 09/08/2023] [Indexed: 09/24/2023]
Abstract
Accurate segmentation of the renal cancer structure, including the kidney, renal tumors, veins, and arteries, has great clinical significance, which can assist clinicians in diagnosing and treating renal cancer. For accurate segmentation of the renal cancer structure in contrast-enhanced computed tomography (CT) images, we proposed a novel encoder-decoder structure segmentation network named MDM-U-Net comprising a multi-scale anisotropic convolution block, dual activation attention block, and multi-scale deep supervision mechanism. The multi-scale anisotropic convolution block was used to improve the feature extraction ability of the network, the dual activation attention block as a channel-wise mechanism was used to guide the network to exploit important information, and the multi-scale deep supervision mechanism was used to supervise the layers of the decoder part for improving segmentation performance. In this study, we developed a feasible and generalizable MDM-U-Net model for renal cancer structure segmentation, trained the model from the public KiPA22 dataset, and tested it on the KiPA22 dataset and an in-house dataset. For the KiPA22 dataset, our method ranked first in renal cancer structure segmentation, achieving state-of-the-art (SOTA) performance in terms of 6 of 12 evaluation metrics (3 metrics per structure). For the in-house dataset, our method achieves SOTA performance in terms of 9 of 12 evaluation metrics (3 metrics per structure), demonstrating its superiority and generalization ability over the compared networks in renal structure segmentation from contrast-enhanced CT scans.
Collapse
Affiliation(s)
- Xin Weng
- School of Biology & Engineering (School of Modern Industry for Health and Medicine), Guizhou Medical University, Guiyang, Guizhou, China
| | - Fasong Song
- Department of Radiology, The Affiliated Hospital of Guizhou Medical University, Guiyang, Guizhou, China
| | - Maowen Tang
- Department of Radiology, The Affiliated Hospital of Guizhou Medical University, Guiyang, Guizhou, China
| | - Kansui Wang
- Department of Radiology, The First Affiliated Hospital of Guizhou University of Traditional Chinese Medicine, Guiyang, Guizhou, China
| | - Yusui Zhang
- Department of Radiology, The First Affiliated Hospital of Guizhou University of Traditional Chinese Medicine, Guiyang, Guizhou, China
| | - Yuehong Miao
- School of Biology & Engineering (School of Modern Industry for Health and Medicine), Guizhou Medical University, Guiyang, Guizhou, China
| | - Lawrence Wing-Chi Chan
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Pinggui Lei
- Department of Radiology, The Affiliated Hospital of Guizhou Medical University, Guiyang, Guizhou, China; Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong SAR, China.
| | - Zuquan Hu
- School of Biology & Engineering (School of Modern Industry for Health and Medicine), Guizhou Medical University, Guiyang, Guizhou, China; Immune Cells and Antibody Engineering Research Center in University of Guizhou Province, Key Laboratory of Biology and Medical Engineering, Guizhou Medical University, Guiyang, Guizhou, China.
| | - Fan Yang
- School of Biology & Engineering (School of Modern Industry for Health and Medicine), Guizhou Medical University, Guiyang, Guizhou, China.
| |
Collapse
|
21
|
Hu H, Pan N, Frangi AF. Fully Automatic initialization and segmentation of left and right ventricles for large-scale cardiac MRI using a deeply supervised network and 3D-ASM. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107679. [PMID: 37364366 DOI: 10.1016/j.cmpb.2023.107679] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Revised: 06/12/2023] [Accepted: 06/13/2023] [Indexed: 06/28/2023]
Abstract
BACKGROUND AND OBJECTIVE The sheer volume of data generated by population imaging studies is unparalleled by current capabilities to extract objective and quantitative cardiac phenotypes; subjective and time-consuming manual image analysis remains the gold standard. Automated image analytics to compute quantitative imaging biomarkers of cardiac function are desperately needed. Data volumes and their variability pose a challenge to most state-of-the-art methods for endo- and epicardial contours, which lack robustness when applied to very large datasets. Our aim is to develop an analysis pipeline for the automatic quantification of cardiac function from cine magnetic resonance imaging data. METHOD This work adopt 4,638 cardiac MRI cases coming from UK Biobank with ground truth available for left and RV contours. A hybrid and robust algorithm is proposed to improve the accuracy of automatic left and right ventricle segmentation by harnessing the localization accuracy of deep learning and the morphological accuracy of 3D-ASM (three-dimensional active shape models). The contributions of this paper are three-fold. First, a fully automatic method is proposed for left and right ventricle initialization and cardiac MRI segmentation by taking full advantage of spatiotemporal constraint. Second, a deeply supervised network is introduced to train and segment the heart. Third, the 3D-ASM image search procedure is improved by combining image intensity models with convolutional neural network (CNN) derived distance maps improving endo- and epicardial edge localization. RESULTS The proposed architecture outperformed the state of the art for cardiac MRI segmentation from UK Biobank. The statistics of RV landmarks detection errors for Triscuspid valve and RV apex are 4.17 mm and 5.58 mm separately. The overlap metric, mean contour distance, Hausdorff distance and cardiac functional parameters are calculated for the LV (Left Ventricle) and RV (Right Ventricle) contour segmentation. Bland-Altman analysis for clinical parameters shows that the results from our automated image analysis pipelines are in good agreement with results from expert manual analysis. CONCLUSIONS Our hybrid scheme combines deep learning and statistical shape modeling for automatic segmentation of the LV/RV from cardiac MRI datasets is effective and robust and can compute cardiac functional indexes from population imaging.
Collapse
Affiliation(s)
- Huaifei Hu
- College of Biomedical Engineering, South-Central Minzu University, Wuhan 430074, China; Hubei Key Laboratory of Medical Information Analysis and Tumor Diagnosis & Treatment, Wuhan 430074, China; Key Laboratory of Cognitive Science, State Ethnic Affairs Commission, Wuhan 430074, China
| | - Ning Pan
- College of Biomedical Engineering, South-Central Minzu University, Wuhan 430074, China; Hubei Key Laboratory of Medical Information Analysis and Tumor Diagnosis & Treatment, Wuhan 430074, China; Key Laboratory of Cognitive Science, State Ethnic Affairs Commission, Wuhan 430074, China.
| | - Alejandro F Frangi
- Centre for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB), School of Computing, University of Leeds, Leeds, UK; Biomedical Imaging Sciences Department, Leeds Institute for Cardiovascular and Metabolic Medicine (LICAMM), School of Medicine, University of Leeds, Leeds, UK; Medical Imaging Research Centre, Cardiovascular Sciences Department, KU Leuven, Leuven, Belgium; Medical Imaging Research Centre, Electrical Engineering Department, KU Leuven, Leuven, Belgium.
| |
Collapse
|
22
|
Wang R, Zhou Q, Zheng G. EDRL: Entropy-guided disentangled representation learning for unsupervised domain adaptation in semantic segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107729. [PMID: 37531690 DOI: 10.1016/j.cmpb.2023.107729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Revised: 07/15/2023] [Accepted: 07/19/2023] [Indexed: 08/04/2023]
Abstract
BACKGROUND AND OBJECTIVE Deep learning-based approaches are excellent at learning from large amounts of data, but can be poor at generalizing the learned knowledge to testing datasets with domain shift, i.e., when there exists distribution discrepancy between the training dataset (source domain) and the testing dataset (target domain). In this paper, we investigate unsupervised domain adaptation (UDA) techniques to train a cross-domain segmentation method which is robust to domain shift, eliminating the requirement of any annotations on the target domain. METHODS To this end, we propose an Entropy-guided Disentangled Representation Learning, referred as EDRL, for UDA in semantic segmentation. Concretely, we synergistically integrate image alignment via disentangled representation learning with feature alignment via entropy-based adversarial learning into one network, which is trained end-to-end. We additionally introduce a dynamic feature selection mechanism via soft gating, which helps to further enhance the task-specific feature alignment. We validate the proposed method on two publicly available datasets: the CT-MR dataset and the multi-sequence cardiac MR (MS-CMR) dataset. RESULTS On both datasets, our method achieved better results than the state-of-the-art (SOTA) methods. Specifically, on the CT-MR dataset, our method achieved an average DSC of 84.8% when taking CT as the source domain and MR as the target domain, and an average DSC of 84.0% when taking MR as the source domain and CT as the target domain. CONCLUSIONS Results from comprehensive experiments demonstrate the efficacy of the proposed EDRL model for cross-domain medical image segmentation.
Collapse
Affiliation(s)
- Runze Wang
- Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, No. 800, Dongchuan Road, Shanghai, 200240, China
| | - Qin Zhou
- Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, No. 800, Dongchuan Road, Shanghai, 200240, China
| | - Guoyan Zheng
- Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, No. 800, Dongchuan Road, Shanghai, 200240, China.
| |
Collapse
|
23
|
Li D, Peng Y, Sun J, Guo Y. A task-unified network with transformer and spatial-temporal convolution for left ventricular quantification. Sci Rep 2023; 13:13529. [PMID: 37598235 PMCID: PMC10439898 DOI: 10.1038/s41598-023-40841-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 08/17/2023] [Indexed: 08/21/2023] Open
Abstract
Quantification of the cardiac function is vital for diagnosing and curing the cardiovascular diseases. Left ventricular function measurement is the most commonly used measure to evaluate the function of cardiac in clinical practice, how to improve the accuracy of left ventricular quantitative assessment results has always been the subject of research by medical researchers. Although considerable efforts have been put forward to measure the left ventricle (LV) automatically using deep learning methods, the accurate quantification is yet a challenge work as a result of the changeable anatomy structure of heart in the systolic diastolic cycle. Besides, most methods used direct regression method which lacks of visual based analysis. In this work, a deep learning segmentation and regression task-unified network with transformer and spatial-temporal convolution is proposed to segment and quantify the LV simultaneously. The segmentation module leverages a U-Net like 3D Transformer model to predict the contour of three anatomy structures, while the regression module learns spatial-temporal representations from the original images and the reconstruct feature map from segmentation path to estimate the finally desired quantification metrics. Furthermore, we employ a joint task loss function to train the two module networks. Our framework is evaluated on the MICCAI 2017 Left Ventricle Full Quantification Challenge dataset. The results of experiments demonstrate the effectiveness of our framework, which achieves competitive cardiac quantification metric results and at the same time produces visualized segmentation results that are conducive to later analysis.
Collapse
Affiliation(s)
- Dapeng Li
- Shandong University of Science and Technology, Qingdao, China
| | - Yanjun Peng
- Shandong University of Science and Technology, Qingdao, China.
- Shandong Province Key Laboratory of Wisdom Mining Information Technology, Qingdao, China.
| | - Jindong Sun
- Shandong University of Science and Technology, Qingdao, China
| | - Yanfei Guo
- Shandong University of Science and Technology, Qingdao, China
| |
Collapse
|
24
|
Liu JTC, Glaser AK, Poudel C, Vaughan JC. Nondestructive 3D Pathology with Light-Sheet Fluorescence Microscopy for Translational Research and Clinical Assays. ANNUAL REVIEW OF ANALYTICAL CHEMISTRY (PALO ALTO, CALIF.) 2023; 16:231-252. [PMID: 36854208 DOI: 10.1146/annurev-anchem-091222-092734] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In recent years, there has been a revived appreciation for the importance of spatial context and morphological phenotypes for both understanding disease progression and guiding treatment decisions. Compared with conventional 2D histopathology, which is the current gold standard of medical diagnostics, nondestructive 3D pathology offers researchers and clinicians the ability to visualize orders of magnitude more tissue within their natural volumetric context. This has been enabled by rapid advances in tissue-preparation methods, high-throughput 3D microscopy instrumentation, and computational tools for processing these massive feature-rich data sets. Here, we provide a brief overview of many of these technical advances along with remaining challenges to be overcome. We also speculate on the future of 3D pathology as applied in translational investigations, preclinical drug development, and clinical decision-support assays.
Collapse
Affiliation(s)
- Jonathan T C Liu
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA;
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Adam K Glaser
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA;
- Allen Institute for Neural Dynamics, Seattle, Washington, USA
| | - Chetan Poudel
- Department of Chemistry, University of Washington, Seattle, Washington, USA
| | - Joshua C Vaughan
- Department of Chemistry, University of Washington, Seattle, Washington, USA
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington, USA
| |
Collapse
|
25
|
He M, Cao Y, Chi C, Yang X, Ramin R, Wang S, Yang G, Mukhtorov O, Zhang L, Kazantsev A, Enikeev M, Hu K. Research progress on deep learning in magnetic resonance imaging-based diagnosis and treatment of prostate cancer: a review on the current status and perspectives. Front Oncol 2023; 13:1189370. [PMID: 37546423 PMCID: PMC10400334 DOI: 10.3389/fonc.2023.1189370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2023] [Accepted: 05/30/2023] [Indexed: 08/08/2023] Open
Abstract
Multiparametric magnetic resonance imaging (mpMRI) has emerged as a first-line screening and diagnostic tool for prostate cancer, aiding in treatment selection and noninvasive radiotherapy guidance. However, the manual interpretation of MRI data is challenging and time-consuming, which may impact sensitivity and specificity. With recent technological advances, artificial intelligence (AI) in the form of computer-aided diagnosis (CAD) based on MRI data has been applied to prostate cancer diagnosis and treatment. Among AI techniques, deep learning involving convolutional neural networks contributes to detection, segmentation, scoring, grading, and prognostic evaluation of prostate cancer. CAD systems have automatic operation, rapid processing, and accuracy, incorporating multiple sequences of multiparametric MRI data of the prostate gland into the deep learning model. Thus, they have become a research direction of great interest, especially in smart healthcare. This review highlights the current progress of deep learning technology in MRI-based diagnosis and treatment of prostate cancer. The key elements of deep learning-based MRI image processing in CAD systems and radiotherapy of prostate cancer are briefly described, making it understandable not only for radiologists but also for general physicians without specialized imaging interpretation training. Deep learning technology enables lesion identification, detection, and segmentation, grading and scoring of prostate cancer, and prediction of postoperative recurrence and prognostic outcomes. The diagnostic accuracy of deep learning can be improved by optimizing models and algorithms, expanding medical database resources, and combining multi-omics data and comprehensive analysis of various morphological data. Deep learning has the potential to become the key diagnostic method in prostate cancer diagnosis and treatment in the future.
Collapse
Affiliation(s)
- Mingze He
- Institute for Urology and Reproductive Health, I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Yu Cao
- I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Changliang Chi
- Department of Urology, The First Hospital of Jilin University (Lequn Branch), Changchun, Jilin, China
| | - Xinyi Yang
- I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Rzayev Ramin
- Department of Radiology, The Second University Clinic, I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Shuowen Wang
- I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Guodong Yang
- I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Otabek Mukhtorov
- Regional State Budgetary Health Care Institution, Kostroma Regional Clinical Hospital named after Korolev E.I. Avenue Mira, Kostroma, Russia
| | - Liqun Zhang
- School of Biomedical Engineering, Faculty of Medicine, Dalian University of Technology, Dalian, Liaoning, China
| | - Anton Kazantsev
- Regional State Budgetary Health Care Institution, Kostroma Regional Clinical Hospital named after Korolev E.I. Avenue Mira, Kostroma, Russia
| | - Mikhail Enikeev
- Institute for Urology and Reproductive Health, I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia
| | - Kebang Hu
- Department of Urology, The First Hospital of Jilin University (Lequn Branch), Changchun, Jilin, China
| |
Collapse
|
26
|
Li K, Qian Z, Han Y, Chang EIC, Wei B, Lai M, Liao J, Fan Y, Xu Y. Weakly supervised histopathology image segmentation with self-attention. Med Image Anal 2023; 86:102791. [PMID: 36933385 DOI: 10.1016/j.media.2023.102791] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 01/09/2023] [Accepted: 02/24/2023] [Indexed: 03/13/2023]
Abstract
Accurate segmentation in histopathology images at pixel-level plays a critical role in the digital pathology workflow. The development of weakly supervised methods for histopathology image segmentation liberates pathologists from time-consuming and labor-intensive works, opening up possibilities of further automated quantitative analysis of whole-slide histopathology images. As an effective subgroup of weakly supervised methods, multiple instance learning (MIL) has achieved great success in histopathology images. In this paper, we specially treat pixels as instances so that the histopathology image segmentation task is transformed into an instance prediction task in MIL. However, the lack of relations between instances in MIL limits the further improvement of segmentation performance. Therefore, we propose a novel weakly supervised method called SA-MIL for pixel-level segmentation in histopathology images. SA-MIL introduces a self-attention mechanism into the MIL framework, which captures global correlation among all instances. In addition, we use deep supervision to make the best use of information from limited annotations in the weakly supervised method. Our approach makes up for the shortcoming that instances are independent of each other in MIL by aggregating global contextual information. We demonstrate state-of-the-art results compared to other weakly supervised methods on two histopathology image datasets. It is evident that our approach has generalization ability for the high performance on both tissue and cell histopathology datasets. There is potential in our approach for various applications in medical images.
Collapse
Affiliation(s)
- Kailu Li
- School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China.
| | - Ziniu Qian
- School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China.
| | - Yingnan Han
- School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China.
| | | | | | - Maode Lai
- Department of Pathology, School of Medicine, Zhejiang University, Hangzhou 310027, China.
| | - Jing Liao
- Department of Computer Science, City University of Hong Kong, 999077, Hong Kong SAR, China.
| | - Yubo Fan
- School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China.
| | - Yan Xu
- School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China; Microsoft Research, Beijing 100080, China.
| |
Collapse
|
27
|
Piri R, Hamakan Y, Vang A, Edenbrandt L, Larsson M, Enqvist O, Gerke O, Høilund-Carlsen PF. Common carotid segmentation in 18 F-sodium fluoride PET/CT scans: Head-to-head comparison of artificial intelligence-based and manual method. Clin Physiol Funct Imaging 2023; 43:71-77. [PMID: 36331059 PMCID: PMC10100011 DOI: 10.1111/cpf.12793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2022] [Revised: 10/06/2022] [Accepted: 10/14/2022] [Indexed: 11/06/2022]
Abstract
BACKGROUND Carotid atherosclerosis is a major cause of stroke, traditionally diagnosed late. Positron emission tomography/computed tomography (PET/CT) with 18 F-sodium fluoride (NaF) detects arterial wall micro-calcification long before macro-calcification becomes detectable by ultrasound, CT or magnetic resonance imaging. However, manual PET/CT processing is time-consuming and requires experience. We compared a convolutional neural network (CNN) approach with manual segmentation of the common carotids. METHODS Segmentation in NaF-PET/CT scans of 29 healthy volunteers and 20 angina pectoris patients were compared for segmented volume (Vol) and mean, maximal, and total standardized uptake values (SUVmean, SUVmax, and SUVtotal). SUVmean was the average of SUVmeans within the VOI, SUVmax the highest SUV in all voxels in the VOI, and SUVtotal the SUVmean multiplied by the Vol of the VOI. Intra and Interobserver variability with manual segmentation was examined in 25 randomly selected scans. RESULTS Bias for Vol, SUVmean, SUVmax, and SUVtotal were 1.33 ± 2.06, -0.01 ± 0.05, 0.09 ± 0.48, and 1.18 ± 1.99 in the left and 1.89 ± 1.5, -0.07 ± 0.12, 0.05 ± 0.47, and 1.61 ± 1.47, respectively, in the right common carotid artery. Manual segmentation lasted typically 20 min versus 1 min with the CNN-based approach. Mean Vol deviation at repeat manual segmentation was 14% and 27% in left and right common carotids. CONCLUSIONS CNN-based segmentation was much faster and provided SUVmean values virtually identical to manually obtained ones, suggesting CNN-based analysis as a promising substitute of slow and cumbersome manual processing.
Collapse
Affiliation(s)
- Reza Piri
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark.,Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| | - Yaran Hamakan
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark
| | - Ask Vang
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark
| | - Lars Edenbrandt
- Department of Molecular and Clinical Medicine, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.,Department of Clinical Physiology, Region Västra Götaland, Sahlgrenska University Hospital, Gothenburg, Sweden
| | | | - Olof Enqvist
- Eigenvision AB, Malmö, Sweden.,Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Oke Gerke
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark.,Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| | - Poul Flemming Høilund-Carlsen
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark.,Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
28
|
Qu T, Li X, Wang X, Deng W, Mao L, He M, Li X, Wang Y, Liu Z, Zhang L, Jin Z, Xue H, Yu Y. Transformer guided progressive fusion network for 3D pancreas and pancreatic mass segmentation. Med Image Anal 2023; 86:102801. [PMID: 37028237 DOI: 10.1016/j.media.2023.102801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 12/21/2022] [Accepted: 03/22/2023] [Indexed: 04/03/2023]
Abstract
Pancreatic masses are diverse in type, often making their clinical management challenging. This study aims to address the task of various types of pancreatic mass segmentation and detection while accurately segmenting the pancreas. Although convolution operation performs well at extracting local details, it experiences difficulty capturing global representations. To alleviate this limitation, we propose a transformer guided progressive fusion network (TGPFN) that utilizes the global representation captured by the transformer to supplement long-range dependencies lost by convolution operations at different resolutions. TGPFN is built on a branch-integrated network structure, where the convolutional neural network and transformer branches first perform separate feature extraction in the encoder, and then the local and global features are progressively fused in the decoder. To effectively integrate the information of the two branches, we design a transformer guidance flow to ensure feature consistency, and present a cross-network attention module to capture the channel dependencies. Extensive experiments with nnUNet (3D) show that TGPFN improves the mass segmentation (Dice: 73.93% vs. 69.40%) and detection accuracy (detection rate: 91.71% vs. 84.97%) on 416 private CTs, and also obtains performance improvements of mass segmentation (Dice: 43.86% vs. 42.07%) and detection (detection rate: 83.33% vs. 71.74%) on 419 public CTs.
Collapse
|
29
|
Large-Kernel Attention for 3D Medical Image Segmentation. Cognit Comput 2023. [DOI: 10.1007/s12559-023-10126-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/03/2023]
Abstract
AbstractAutomated segmentation of multiple organs and tumors from 3D medical images such as magnetic resonance imaging (MRI) and computed tomography (CT) scans using deep learning methods can aid in diagnosing and treating cancer. However, organs often overlap and are complexly connected, characterized by extensive anatomical variation and low contrast. In addition, the diversity of tumor shape, location, and appearance, coupled with the dominance of background voxels, makes accurate 3D medical image segmentation difficult. In this paper, a novel 3D large-kernel (LK) attention module is proposed to address these problems to achieve accurate multi-organ segmentation and tumor segmentation. The advantages of biologically inspired self-attention and convolution are combined in the proposed LK attention module, including local contextual information, long-range dependencies, and channel adaptation. The module also decomposes the LK convolution to optimize the computational cost and can be easily incorporated into CNNs such as U-Net. Comprehensive ablation experiments demonstrated the feasibility of convolutional decomposition and explored the most efficient and effective network design. Among them, the best Mid-type 3D LK attention-based U-Net network was evaluated on CT-ORG and BraTS 2020 datasets, achieving state-of-the-art segmentation performance when compared to avant-garde CNN and Transformer-based methods for medical image segmentation. The performance improvement due to the proposed 3D LK attention module was statistically validated.
Collapse
|
30
|
Rehman A, Usman M, Shahid A, Latif S, Qadir J. Selective Deeply Supervised Multi-Scale Attention Network for Brain Tumor Segmentation. SENSORS (BASEL, SWITZERLAND) 2023; 23:2346. [PMID: 36850942 PMCID: PMC9964702 DOI: 10.3390/s23042346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 02/15/2023] [Accepted: 02/17/2023] [Indexed: 06/18/2023]
Abstract
Brain tumors are among the deadliest forms of cancer, characterized by abnormal proliferation of brain cells. While early identification of brain tumors can greatly aid in their therapy, the process of manual segmentation performed by expert doctors, which is often time-consuming, tedious, and prone to human error, can act as a bottleneck in the diagnostic process. This motivates the development of automated algorithms for brain tumor segmentation. However, accurately segmenting the enhanced and core tumor regions is complicated due to high levels of inter- and intra-tumor heterogeneity in terms of texture, morphology, and shape. This study proposes a fully automatic method called the selective deeply supervised multi-scale attention network (SDS-MSA-Net) for segmenting brain tumor regions using a multi-scale attention network with novel selective deep supervision (SDS) mechanisms for training. The method utilizes a 3D input composed of five consecutive slices, in addition to a 2D slice, to maintain sequential information. The proposed multi-scale architecture includes two encoding units to extract meaningful global and local features from the 3D and 2D inputs, respectively. These coarse features are then passed through attention units to filter out redundant information by assigning lower weights. The refined features are fed into a decoder block, which upscales the features at various levels while learning patterns relevant to all tumor regions. The SDS block is introduced to immediately upscale features from intermediate layers of the decoder, with the aim of producing segmentations of the whole, enhanced, and core tumor regions. The proposed framework was evaluated on the BraTS2020 dataset and showed improved performance in brain tumor region segmentation, particularly in the segmentation of the core and enhancing tumor regions, demonstrating the effectiveness of the proposed approach. Our code is publicly available.
Collapse
Affiliation(s)
- Azka Rehman
- Center for Artificial Intelligence in Medicine and Imaging, HealthHub Co., Ltd., Seoul 06524, Republic of Korea
| | - Muhammad Usman
- Department of Computer Science and Engineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Republic of Korea
| | - Abdullah Shahid
- Center for Artificial Intelligence in Medicine and Imaging, HealthHub Co., Ltd., Seoul 06524, Republic of Korea
| | - Siddique Latif
- Faculty of Health, Engineering and Sciences, University of Southern Queensland, Springfield 4300, Australia
| | - Junaid Qadir
- Department of Computer Science and Engineering, College of Engineering, Qatar University, Doha 2713, Qatar
| |
Collapse
|
31
|
Afriyie Y, Weyori BA, Opoku AA. A scaling up approach: a research agenda for medical imaging analysis with applications in deep learning. J EXP THEOR ARTIF IN 2023. [DOI: 10.1080/0952813x.2023.2165721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Affiliation(s)
- Yaw Afriyie
- Department of Computer Science and Informatics, University of Energy and Natural Resources, School of Sciences, Sunyani, Ghana
- Department of Computer Science, Faculty of Information and Communication Technology, SD Dombo University of Business and Integrated Development Studies, Wa, Ghana
| | - Benjamin A. Weyori
- Department of Computer Science and Informatics, University of Energy and Natural Resources, School of Sciences, Sunyani, Ghana
| | - Alex A. Opoku
- Department of Mathematics & Statistics, University of Energy and Natural Resources, School of Sciences, Sunyani, Ghana
| |
Collapse
|
32
|
Peng Y, Liu Y, Shen G, Chen Z, Chen M, Miao J, Zhao C, Deng J, Qi Z, Deng X. Improved accuracy of auto-segmentation of organs at risk in radiotherapy planning for nasopharyngeal carcinoma based on fully convolutional neural network deep learning. Oral Oncol 2023; 136:106261. [PMID: 36446186 DOI: 10.1016/j.oraloncology.2022.106261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 11/13/2022] [Accepted: 11/19/2022] [Indexed: 11/27/2022]
Abstract
OBJECTIVE We examined a modified encoder-decoder architecture-based fully convolutional neural network, OrganNet, for simultaneous auto-segmentation of 24 organs at risk (OARs) in the head and neck, followed by validation tests and evaluation of clinical application. MATERIALS AND METHODS Computed tomography (CT) images from 310 radiotherapy plans were used as the experimental data set, of which 260 and 50 were used as the training and test sets, respectively. An improved U-Net architecture was established by introducing a batch normalization layer, residual squeeze-and-excitation layer, and unique organ-specific loss function for deep learning training. The performance of the trained network model was evaluated by comparing the manual-delineation and the STAPLE contour of 10 physicians from different centers. RESULTS Our model achieved good segmentation in all 24 OARs in nasopharyngeal cancer radiotherapy plan CT images, with an average Dice similarity coefficient of 83.75%. Specifically, the mean Dice coefficients in large-volume organs (brainstem, spinal cord, left/right parotid glands, left/right temporal lobes, and left/right mandibles) were 84.97% - 95.00%, and in small-volume organs (pituitary, lens, optic nerve, and optic chiasma) were 55.46% - 91.56%. respectively. Using the STAPLE contours as standard contour, the OrganNet achieved comparable or better DICE in organ segmentation then that of the manual-delineation as well. CONCLUSION The established OrganNet enables simultaneous automatic segmentation of multiple targets on CT images of the head and neck radiotherapy plans, effectively improves the accuracy of U-Net based segmentation for OARs, especially for small-volume organs.
Collapse
Affiliation(s)
- Yinglin Peng
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China; School of Biomedical Engineering, Sun Yat-sen University, Guangzhou, China
| | - Yimei Liu
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China
| | - Guanzhu Shen
- Department of Radiation Oncology, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Zijie Chen
- Shenying Medical Technology (Shenzhen) Co., Ltd., Shenzhen, Guangdong, China
| | - Meining Chen
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China
| | - Jingjing Miao
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China
| | - Chong Zhao
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China
| | - Jincheng Deng
- Shenying Medical Technology (Shenzhen) Co., Ltd., Shenzhen, Guangdong, China
| | - Zhenyu Qi
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China.
| | - Xiaowu Deng
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China.
| |
Collapse
|
33
|
Wang J, Zhang X, Guo L, Shi C, Tamura S. Multi-scale attention and deep supervision-based 3D UNet for automatic liver segmentation from CT. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:1297-1316. [PMID: 36650812 DOI: 10.3934/mbe.2023059] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
BACKGROUND Automatic liver segmentation is a prerequisite for hepatoma treatment; however, the low accuracy and stability hinder its clinical application. To alleviate this limitation, we deeply mine the context information of different scales and combine it with deep supervision to improve the accuracy of liver segmentation in this paper. METHODS We proposed a new network called MAD-UNet for automatic liver segmentation from CT. It is grounded in the 3D UNet and leverages multi-scale attention and deep supervision mechanisms. In the encoder, the downsampling pooling in 3D UNet is replaced by convolution to alleviate the loss of feature information. Meanwhile, the residual module is introduced to avoid gradient vanishment. Besides, we use the long-short skip connections (LSSC) to replace the ordinary skip connections to preserve more edge detail. In the decoder, the features of different scales are aggregated, and the attention module is employed to capture the spatial context information. Moreover, we utilized the deep supervision mechanism to improve the learning ability on deep and shallow information. RESULTS We evaluated the proposed method on three public datasets, including, LiTS17, SLiver07, and 3DIRCADb, and obtained Dice scores of 0.9727, 0.9752, and 0.9691 for liver segmentation, respectively, which outperform the other state-of-the-art (SOTA) methods. CONCLUSIONS Both qualitative and quantitative experimental results demonstrate that the proposed method can make full use of the feature information of different stages while enhancing spatial data's learning ability, thereby achieving high liver segmentation accuracy. Thus, it proved to be a promising tool for automatic liver segmentation in clinical assistance.
Collapse
Affiliation(s)
- Jinke Wang
- Department of Software Engineering, Harbin University of Science and Technology, Rongcheng 264300, China
- School of Automation, Harbin University of Science and Technology, Harbin 150080, China
| | - Xiangyang Zhang
- School of Automation, Harbin University of Science and Technology, Harbin 150080, China
| | - Liang Guo
- School of Automation, Harbin University of Science and Technology, Harbin 150080, China
| | - Changfa Shi
- Mobile E-business Collaborative Innovation Center of Hunan Province, Hunan University of Technology and Business, Changsha 410205, China
| | | |
Collapse
|
34
|
Extension-contraction transformation network for pancreas segmentation in abdominal CT scans. Comput Biol Med 2023; 152:106410. [PMID: 36516578 DOI: 10.1016/j.compbiomed.2022.106410] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Revised: 11/08/2022] [Accepted: 12/03/2022] [Indexed: 12/12/2022]
Abstract
Accurate and automatic pancreas segmentation from abdominal computed tomography (CT) scans is crucial for the diagnosis and prognosis of pancreatic diseases. However, the pancreas accounts for a relatively small portion of the scan and presents high anatomical variability and low contrast, making traditional automated segmentation methods fail to generate satisfactory results. In this paper, we propose an extension-contraction transformation network (ECTN) and deploy it into a cascaded two-stage segmentation framework for accurate pancreas segmenting. This model can enhance the perception of 3D context by distinguishing and exploiting the extension and contraction transformation of the pancreas between slices. It consists of an encoder, a segmentation decoder, and an extension-contraction (EC) decoder. The EC decoder is responsible for predicting the inter-slice extension and contraction transformation of the pancreas by feeding the extension and contraction information generated by the segmentation decoder; meanwhile, its output is combined with the output of the segmentation decoder to reconstruct and refine the segmentation results. Quantitative evaluation is performed on NIH Pancreas Segmentation (Pancreas-CT) dataset using 4-fold cross-validation. We obtained average Precision of 86.59±6.14% , Recall of 85.11±5.96%, Dice similarity coefficient (DSC) of 85.58±3.98%. and Jaccard Index (JI) of 74.99±5.86%. The performance of our method outperforms several baseline and state-of-the-art methods.
Collapse
|
35
|
Wang J, Zhang L, Zhang Y. Mixture 2D Convolutions for 3D Medical Image Segmentation. Int J Neural Syst 2023; 33:2250059. [PMID: 36328969 DOI: 10.1142/s0129065722500599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
Three-dimensional (3D) medical image segmentation plays a crucial role in medical care applications. Although various two-dimensional (2D) and 3D neural network models have been applied to 3D medical image segmentation and achieved impressive results, a trade-off remains between efficiency and accuracy. To address this issue, a novel mixture convolutional network (MixConvNet) is proposed, in which traditional 2D/3D convolutional blocks are replaced with novel MixConv blocks. In the MixConv block, 3D convolution is decomposed into a mixture of 2D convolutions from different views. Therefore, the MixConv block fully utilizes the advantages of 2D convolution and maintains the learning ability of 3D convolution. It acts as 3D convolutions and thus can process volumetric input directly and learn intra-slice features, which are absent in the traditional 2D convolutional block. By contrast, the proposed MixConv block only contains 2D convolutions; hence, it has significantly fewer trainable parameters and less computation budget than a block containing 3D convolutions. Furthermore, the proposed MixConvNet is pre-trained with small input patches and fine-tuned with large input patches to improve segmentation performance further. In experiments on the Decathlon Heart dataset and Sliver07 dataset, the proposed MixConvNet outperformed the state-of-the-art methods such as UNet3D, VNet, and nnUnet.
Collapse
Affiliation(s)
- Jianyong Wang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, Sichuan, P. R. China
| | - Lei Zhang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, Sichuan, P. R. China
| | - Yi Zhang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, Sichuan, P. R. China
| |
Collapse
|
36
|
Zhao G, Liang K, Pan C, Zhang F, Wu X, Hu X, Yu Y. Graph Convolution Based Cross-Network Multiscale Feature Fusion for Deep Vessel Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:183-195. [PMID: 36112564 DOI: 10.1109/tmi.2022.3207093] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Vessel segmentation is widely used to help with vascular disease diagnosis. Vessels reconstructed using existing methods are often not sufficiently accurate to meet clinical use standards. This is because 3D vessel structures are highly complicated and exhibit unique characteristics, including sparsity and anisotropy. In this paper, we propose a novel hybrid deep neural network for vessel segmentation. Our network consists of two cascaded subnetworks performing initial and refined segmentation respectively. The second subnetwork further has two tightly coupled components, a traditional CNN-based U-Net and a graph U-Net. Cross-network multi-scale feature fusion is performed between these two U-shaped networks to effectively support high-quality vessel segmentation. The entire cascaded network can be trained from end to end. The graph in the second subnetwork is constructed according to a vessel probability map as well as appearance and semantic similarities in the original CT volume. To tackle the challenges caused by the sparsity and anisotropy of vessels, a higher percentage of graph nodes are distributed in areas that potentially contain vessels while a higher percentage of edges follow the orientation of potential nearby vessels. Extensive experiments demonstrate our deep network achieves state-of-the-art 3D vessel segmentation performance on multiple public and in-house datasets.
Collapse
|
37
|
Meng Y, Du Z, Zhao C, Dong M, Pienta D, Tang J, Zhou W. Automatic extraction of coronary arteries using deep learning in invasive coronary angiograms. Technol Health Care 2023; 31:2303-2317. [PMID: 37545276 DOI: 10.3233/thc-230278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
BACKGROUND Accurate extraction of coronary arteries from invasive coronary angiography (ICA) images is essential for the diagnosis and risk stratification of coronary artery disease (CAD). OBJECTIVE In this study, a novel deep learning (DL) method is proposed for automatically extracting coronary arteries from ICA images. METHODS A convolutional neural network (CNN) was developed with full-scale skip connections and full-scale deep supervisions. The encoder architecture was based on the residual and inception modules to obtain multi-scale features from multiple convolutional layers with different window shapes. Transfer learning was utilized to improve both the initial performance and learning efficiency. A hybrid loss function was employed to further optimize the segmentation model. RESULTS The model was tested on a data set of 616 ICAs obtained from 210 patients, composed of 437 images for training, 49 images for validation, and 130 images for testing. The segmentation model achieved a Dice score of 0.8942, a sensitivity of 0.8735, a specificity of 0.9954, and a Hausdorff distance of 6.0794 mm; it could predict arteries for a single ICA frame in 0.2114 seconds. CONCLUSIONS The results showed that our model outperformed the state-of-the-art deep-learning models. Our new method has great potential for clinical use.
Collapse
Affiliation(s)
- Yinghui Meng
- School of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou, Henan, China
| | - Zhenglong Du
- School of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou, Henan, China
| | - Chen Zhao
- Department of Applied Computing, Michigan Technological University, Houghton, MI, USA
| | - Minghao Dong
- School of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou, Henan, China
| | - Drew Pienta
- Department of Mechanical Engineering-Engineering Mechanics, Michigan Technological University, Houghton, MI, USA
| | - Jinshan Tang
- Department of Health Administration and Policy, College of Health and Human Services, George Mason University, Fairfax, VA, USA
| | - Weihua Zhou
- Department of Applied Computing, Michigan Technological University, Houghton, MI, USA
- Center for Biocomputing and Digital Health, Institute of Computing and Cybersystems, Michigan Technological University, Houghton, MI, USA
- Health Research Institute, Michigan Technological University, Houghton, MI, USA
| |
Collapse
|
38
|
Feng H, Fu Z, Wang Y, Zhang P, Lai H, Zhao J. Automatic segmentation of thrombosed aortic dissection in post-operative CT-angiography images. Med Phys 2022. [PMID: 36542417 DOI: 10.1002/mp.16169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 11/02/2022] [Accepted: 11/23/2022] [Indexed: 12/24/2022] Open
Abstract
PURPOSE The thrombus in the false lumen (FL) of aortic dissection (AD) patients is a meaningful indicator to determine aortic remodeling but difficult to measure in clinic. In this study, a novel segmentation strategy based on deep learning was proposed to automatically extract the thrombus in the FL in post-operative computed tomography angiography (CTA) images of AD patients, which provided an efficient and convenient segmentation method with high accuracy. METHODS A two-step segmentation strategy was proposed. Each step contained a convolutional neural network (CNN) to segment the aorta and the thrombus, respectively. In the first step, a CNN was used to obtain the binary segmentation mask of the whole aorta. In the second step, another CNN was introduced to segment the thrombus. The results of the first step were used as additional input to the second step to highlight the aorta in the complex background. Moreover, skip connection attention refinement (SAR) modules were designed and added in the second step to improve the segmentation accuracy of the thrombus details by efficiently using the low-level features. RESULTS The proposed method provided accurate thrombus segmentation results (0.903 ± 0.062 in dice score, 0.828 ± 0.092 in Jaccard index, and 2.209 ± 2.945 in 95% Hausdorff distance), which showed improvement compared to the methods without prior information (0.846 ± 0.085 in dice score) and the method without SAR (0.899 ± 0.060 in dice score). Moreover, the proposed method achieved 0.967 ± 0.029 and 0.948 ± 0.041 in dice score of true lumen (TL) and patent FL (PFL) segmentation, respectively, indicating the excellence of the proposed method in the segmentation task of the overall aorta. CONCLUSIONS A novel CNN-based segmentation framework was proposed to automatically obtain thrombus segmentation for thrombosed AD in post-operative CTA images, which provided a useful tool for further application of thrombus-related indicators in clinical and research application.
Collapse
Affiliation(s)
- Hanying Feng
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Zheng Fu
- Department of Cardiovascular Surgery, Zhongshan Hospital Fudan University, Shanghai, People's Republic of China
| | - Yulin Wang
- Department of Cardiovascular Surgery, Zhongshan Hospital Fudan University, Shanghai, People's Republic of China
| | - Puming Zhang
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Hao Lai
- Department of Cardiovascular Surgery, Zhongshan Hospital Fudan University, Shanghai, People's Republic of China
| | - Jun Zhao
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
39
|
Huang Z, Guo Y, Zhang N, Huang X, Decazes P, Becker S, Ruan S. Multi-scale feature similarity-based weakly supervised lymphoma segmentation in PET/CT images. Comput Biol Med 2022; 151:106230. [PMID: 36306574 DOI: 10.1016/j.compbiomed.2022.106230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 09/28/2022] [Accepted: 10/16/2022] [Indexed: 12/27/2022]
Abstract
Accurate lymphoma segmentation in PET/CT images is important for evaluating Diffuse Large B-Cell Lymphoma (DLBCL) prognosis. As systemic multiple lymphomas, DLBCL lesions vary in number and size for different patients, which makes DLBCL labeling labor-intensive and time-consuming. To reduce the reliance on accurately labeled datasets, a weakly supervised deep learning method based on multi-scale feature similarity is proposed for automatic lymphoma segmentation. Weak labeling was performed by randomly dawning a small and salient lymphoma volume for the patient without accurate labels. A 3D V-Net is used as the backbone of the segmentation network and image features extracted in different convolutional layers are fused with the Atrous Spatial Pyramid Pooling (ASPP) module to generate multi-scale feature representations of input images. By imposing multi-scale feature consistency constraints on the predicted tumor regions as well as the labeled tumor regions, weakly labeled data can also be effectively used for network training. The cosine similarity, which has strong generalization, is exploited here to measure feature distances. The proposed method is evaluated with a PET/CT dataset of 147 lymphoma patients. Experimental results show that when using data, half of which have accurate labels and the other half have weak labels, the proposed method performed similarly to a fully supervised segmentation network and achieved an average Dice Similarity Coefficient (DSC) of 71.47%. The proposed method is able to reduce the requirement for expert annotations in deep learning-based lymphoma segmentation.
Collapse
Affiliation(s)
- Zhengshan Huang
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, Tianjin, China
| | - Yu Guo
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, Tianjin, China.
| | - Ning Zhang
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, Tianjin, China
| | - Xian Huang
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, Tianjin, China
| | - Pierre Decazes
- LITIS, University of Rouen Normandy, Rouen, France; Department of Nuclear Medicine, Henri Becquerel Cancer Centre, Rouen, France
| | - Stephanie Becker
- LITIS, University of Rouen Normandy, Rouen, France; Department of Nuclear Medicine, Henri Becquerel Cancer Centre, Rouen, France
| | - Su Ruan
- LITIS, University of Rouen Normandy, Rouen, France
| |
Collapse
|
40
|
Chen Y, Jin D, Guo B, Bai X. Attention-Assisted Adversarial Model for Cerebrovascular Segmentation in 3D TOF-MRA Volumes. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3520-3532. [PMID: 35759584 DOI: 10.1109/tmi.2022.3186731] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Cerebrovascular segmentation in time-of-flight magnetic resonance angiography (TOF-MRA) volumes is essential for a variety of diagnostic and analytical applications. However, accurate cerebrovascular segmentation in 3D TOF-MRA is faced with multiple issues, including vast variations in cerebrovascular morphology and intensity, noisy background, and severe class imbalance between foreground cerebral vessels and background. In this work, a 3D adversarial network model called A-SegAN is proposed to segment cerebral vessels in TOF-MRA volumes. The proposed model is composed of a segmentation network A-SegS to predict segmentation maps, and a critic network A-SegC to discriminate predictions from ground truth. Based on this model, the aforementioned issues are addressed by the prevailing visual attention mechanism. First, A-SegS is incorporated with feature-attention blocks to filter out discriminative feature maps, though the cerebrovascular has varied appearances. Second, a hard-example-attention loss is exploited to boost the training of A-SegS on hard samples. Further, A-SegC is combined with an input-attention layer to attach importance to foreground cerebrovascular class. The proposed methods were evaluated on a self-constructed voxel-wise annotated cerebrovascular TOF-MRA segmentation dataset, and experimental results indicate that A-SegAN achieves competitive or better cerebrovascular segmentation results compared to other deep learning methods, effectively alleviating the above issues.
Collapse
|
41
|
Zhang Y, Yang J, Liu Y, Tian J, Wang S, Zhong C, Shi Z, Zhang Y, He Z. Decoupled pyramid correlation network for liver tumor segmentation from CT images. Med Phys 2022; 49:7207-7221. [PMID: 35620834 DOI: 10.1002/mp.15723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Revised: 03/24/2022] [Accepted: 04/16/2022] [Indexed: 12/13/2022] Open
Abstract
PURPOSE Automated liver tumor segmentation from computed tomography (CT) images is a necessary prerequisite in the interventions of hepatic abnormalities and surgery planning. However, accurate liver tumor segmentation remains challenging due to the large variability of tumor sizes and inhomogeneous texture. Recent advances based on fully convolutional network (FCN) for medical image segmentation drew on the success of learning discriminative pyramid features. In this paper, we propose a decoupled pyramid correlation network (DPC-Net) that exploits attention mechanisms to fully leverage both low- and high-level features embedded in FCN to segment liver tumor. METHODS We first design a powerful pyramid feature encoder (PFE) to extract multilevel features from input images. Then we decouple the characteristics of features concerning spatial dimension (i.e., height, width, depth) and semantic dimension (i.e., channel). On top of that, we present two types of attention modules, spatial correlation (SpaCor) and semantic correlation (SemCor) modules, to recursively measure the correlation of multilevel features. The former selectively emphasizes global semantic information in low-level features with the guidance of high-level ones. The latter adaptively enhance spatial details in high-level features with the guidance of low-level ones. RESULTS We evaluate the DPC-Net on MICCAI 2017 LiTS Liver Tumor Segmentation (LiTS) challenge data set. Dice similarity coefficient (DSC) and average symmetric surface distance (ASSD) are employed for evaluation. The proposed method obtains a DSC of 76.4% and an ASSD of 0.838 mm for liver tumor segmentation, outperforming the state-of-the-art methods. It also achieves a competitive result with a DSC of 96.0% and an ASSD of 1.636 mm for liver segmentation. CONCLUSIONS The experimental results show promising performance of DPC-Net for liver and tumor segmentation from CT images. Furthermore, the proposed SemCor and SpaCor can effectively model the multilevel correlation from both semantic and spatial dimensions. The proposed attention modules are lightweight and can be easily extended to other multilevel methods in an end-to-end manner.
Collapse
Affiliation(s)
- Yao Zhang
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China.,Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Jiawei Yang
- Electrical and Computer Engineering, University of California, Los Angeles, California, USA
| | - Yang Liu
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China.,Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China
| | | | - Siyun Wang
- Dornsife College of Letters, Arts and Sciences, University of Southern California, Los Angeles, California, USA
| | | | | | | | | |
Collapse
|
42
|
Zhang S, Yang G, Qian J, Zhu X, Li J, Li P, He Y, Xu Y, Shao P, Wang Z. A novel 3D deep learning model to automatically demonstrate renal artery segmentation and its validation in nephron-sparing surgery. Front Oncol 2022; 12:997911. [PMID: 36313655 PMCID: PMC9614169 DOI: 10.3389/fonc.2022.997911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 09/28/2022] [Indexed: 12/02/2022] Open
Abstract
Purpose Nephron-sparing surgery (NSS) is a mainstream treatment for localized renal tumors. Segmental renal artery clamping (SRAC) is commonly used in NSS. Automatic and precise segmentations of renal artery trees are required to improve the workflow of SRAC in NSS. In this study, we developed a tridimensional kidney perfusion (TKP) model based on deep learning technique to automatically demonstrate renal artery segmentation, and verified the precision and feasibility during laparoscopic partial nephrectomy (PN). Methods The TKP model was established based on convolutional neural network (CNN), and the precision was validated in porcine models. From April 2018 to January 2020, TKP model was applied in laparoscopic PN in 131 patients with T1a tumors. Demographics, perioperative variables, and data from the TKP models were assessed. Indocyanine green (ICG) with near-infrared fluorescence (NIRF) imaging was applied after clamping and dice coefficient was used to evaluate the precision of the model. Results The precision of the TKP model was validated in porcine models with the mean dice coefficient of 0.82. Laparoscopic PN was successfully performed in all cases with segmental renal artery clamping (SRAC) under TKP model’s guidance. The mean operation time was 100.8 min; the median estimated blood loss was 110 ml. The ischemic regions recorded in NIRF imaging were highly consistent with the perfusion regions in the TKP models (mean dice coefficient = 0.81). Multivariate analysis revealed that the feeding lobar artery number was strongly correlated with tumor size and contact surface area; the supplying segmental arteries number correlated with tumor size. Conclusions Using the CNN technique, the TKP model is developed to automatically present the renal artery trees and precisely delineate the perfusion regions of different segmental arteries. The guidance of the TKP model is feasible and effective in nephron-sparing surgery.
Collapse
Affiliation(s)
- Shaobo Zhang
- Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Guanyu Yang
- Key Laboratory of Computer Network and Information Integration, Southeast University, Ministry of Education, Nanjing, China
| | - Jian Qian
- Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Xiaomei Zhu
- Department of Radiology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Jie Li
- Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Pu Li
- Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Yuting He
- Key Laboratory of Computer Network and Information Integration, Southeast University, Ministry of Education, Nanjing, China
| | - Yi Xu
- Department of Radiology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Pengfei Shao
- Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
- *Correspondence: Pengfei Shao,
| | - Zengjun Wang
- Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| |
Collapse
|
43
|
Dillman JR, Somasundaram E, Brady SL, He L. Current and emerging artificial intelligence applications for pediatric abdominal imaging. Pediatr Radiol 2022; 52:2139-2148. [PMID: 33844048 DOI: 10.1007/s00247-021-05057-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Revised: 01/25/2021] [Accepted: 03/16/2021] [Indexed: 12/12/2022]
Abstract
Artificial intelligence (AI) uses computers to mimic cognitive functions of the human brain, allowing inferences to be made from generally large datasets. Traditional machine learning (e.g., decision tree analysis, support vector machines) and deep learning (e.g., convolutional neural networks) are two commonly employed AI approaches both outside and within the field of medicine. Such techniques can be used to evaluate medical images for the purposes of automated detection and segmentation, classification tasks (including diagnosis, lesion or tissue characterization, and prediction), and image reconstruction. In this review article we highlight recent literature describing current and emerging AI methods applied to abdominal imaging (e.g., CT, MRI and US) and suggest potential future applications of AI in the pediatric population.
Collapse
Affiliation(s)
- Jonathan R Dillman
- Department of Radiology, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave., Cincinnati, OH, 45229, USA. .,Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA.
| | - Elan Somasundaram
- Department of Radiology, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave., Cincinnati, OH, 45229, USA.,Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Samuel L Brady
- Department of Radiology, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave., Cincinnati, OH, 45229, USA.,Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Lili He
- Department of Radiology, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave., Cincinnati, OH, 45229, USA.,Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| |
Collapse
|
44
|
Piri R, Edenbrandt L, Larsson M, Enqvist O, Skovrup S, Iversen KK, Saboury B, Alavi A, Gerke O, Høilund-Carlsen PF. "Global" cardiac atherosclerotic burden assessed by artificial intelligence-based versus manual segmentation in 18F-sodium fluoride PET/CT scans: Head-to-head comparison. J Nucl Cardiol 2022; 29:2531-2539. [PMID: 34386861 DOI: 10.1007/s12350-021-02758-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Accepted: 07/13/2021] [Indexed: 01/06/2023]
Abstract
BACKGROUND Artificial intelligence (AI) is known to provide effective means to accelerate and facilitate clinical and research processes. So in this study it was aimed to compare a AI-based method for cardiac segmentation in positron emission tomography/computed tomography (PET/CT) scans with manual segmentation to assess global cardiac atherosclerosis burden. METHODS A trained convolutional neural network (CNN) was used for cardiac segmentation in 18F-sodium fluoride PET/CT scans of 29 healthy volunteers and 20 angina pectoris patients and compared with manual segmentation. Parameters for segmented volume (Vol) and mean, maximal, and total standardized uptake values (SUVmean, SUVmax, SUVtotal) were analyzed by Bland-Altman Limits of Agreement. Repeatability with AI-based assessment of the same scans is 100%. Repeatability (same conditions, same operator) and reproducibility (same conditions, two different operators) of manual segmentation was examined by re-segmentation in 25 randomly selected scans. RESULTS Mean (± SD) values with manual vs. CNN-based segmentation were Vol 617.65 ± 154.99 mL vs 625.26 ± 153.55 mL (P = .21), SUVmean 0.69 ± 0.15 vs 0.69 ± 0.15 (P = .26), SUVmax 2.68 ± 0.86 vs 2.77 ± 1.05 (P = .34), and SUVtotal 425.51 ± 138.93 vs 427.91 ± 132.68 (P = .62). Limits of agreement were - 89.42 to 74.2, - 0.02 to 0.02, - 1.52 to 1.32, and - 68.02 to 63.21, respectively. Manual segmentation lasted typically 30 minutes vs about one minute with the CNN-based approach. The maximal deviation at manual re-segmentation was for the four parameters 0% to 0.5% with the same and 0% to 1% with different operators. CONCLUSION The CNN-based method was faster and provided values for Vol, SUVmean, SUVmax, and SUVtotal comparable to the manually obtained ones. This AI-based segmentation approach appears to offer a more reproducible and much faster substitute for slow and cumbersome manual segmentation of the heart.
Collapse
Affiliation(s)
- Reza Piri
- Department of Nuclear Medicine, Odense University Hospital, 5000, Odense C, Denmark.
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark.
| | - Lars Edenbrandt
- Department of Molecular and Clinical Medicine, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Department of Clinical Physiology, Sahlgrenska University Hospital, Region Västra Götaland, Gothenburg, Sweden
| | | | - Olof Enqvist
- Eigenvision AB, Malmö, Sweden
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Sofie Skovrup
- Department of Nuclear Medicine, Odense University Hospital, 5000, Odense C, Denmark
| | - Kasper Karmark Iversen
- Department of Cardiology, Herlev and Gentofte Hospital, Copenhagen, Denmark
- Department of Emergency Medicine, Herlev and Gentofte Hospital, Copenhagen, Denmark
| | - Babak Saboury
- Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
- Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, Baltimore, MD, USA
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health (NIH), Bethesda, MD, USA
| | - Abass Alavi
- Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Oke Gerke
- Department of Nuclear Medicine, Odense University Hospital, 5000, Odense C, Denmark
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| | - Poul Flemming Høilund-Carlsen
- Department of Nuclear Medicine, Odense University Hospital, 5000, Odense C, Denmark
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
45
|
Boutillon A, Borotikar B, Burdin V, Conze PH. Multi-structure bone segmentation in pediatric MR images with combined regularization from shape priors and adversarial network. Artif Intell Med 2022; 132:102364. [DOI: 10.1016/j.artmed.2022.102364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 05/13/2022] [Accepted: 07/10/2022] [Indexed: 11/02/2022]
|
46
|
Multi-Organ Segmentation Using a Low-Resource Architecture. INFORMATION 2022. [DOI: 10.3390/info13100472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Since their inception, deep-learning architectures have shown promising results for automatic segmentation. However, despite the technical advances introduced by fully convolutional networks, generative adversarial networks or recurrent neural networks, and their usage in hybrid architectures, automatic segmentation in the medical field is still not used at scale. One main reason is related to data scarcity and quality, which in turn generates a lack of annotated data that hinder the generalization of the models. The second main issue refers to challenges in training deep models. This process uses large amounts of GPU memory (that might exceed current hardware limitations) and requires high training times. In this article, we want to prove that despite these issues, good results can be obtained even when using a lower resource architecture, thus opening the way for more researchers to employ and use deep neural networks. In achieving the multi-organ segmentation, we are employing modern pre-processing techniques, a smart model design and fusion between several models trained on the same dataset. Our architecture is compared against state-of-the-art methods employed in a publicly available challenge and the notable results prove the effectiveness of our method.
Collapse
|
47
|
Ma J, Zhang Y, Gu S, An X, Wang Z, Ge C, Wang C, Zhang F, Wang Y, Xu Y, Gou S, Thaler F, Payer C, Štern D, Henderson EGA, McSweeney DM, Green A, Jackson P, McIntosh L, Nguyen QC, Qayyum A, Conze PH, Huang Z, Zhou Z, Fan DP, Xiong H, Dong G, Zhu Q, He J, Yang X. Fast and Low-GPU-memory abdomen CT organ segmentation: The FLARE challenge. Med Image Anal 2022; 82:102616. [PMID: 36179380 DOI: 10.1016/j.media.2022.102616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 06/26/2022] [Accepted: 09/02/2022] [Indexed: 11/27/2022]
Abstract
Automatic segmentation of abdominal organs in CT scans plays an important role in clinical practice. However, most existing benchmarks and datasets only focus on segmentation accuracy, while the model efficiency and its accuracy on the testing cases from different medical centers have not been evaluated. To comprehensively benchmark abdominal organ segmentation methods, we organized the first Fast and Low GPU memory Abdominal oRgan sEgmentation (FLARE) challenge, where the segmentation methods were encouraged to achieve high accuracy on the testing cases from different medical centers, fast inference speed, and low GPU memory consumption, simultaneously. The winning method surpassed the existing state-of-the-art method, achieving a 19× faster inference speed and reducing the GPU memory consumption by 60% with comparable accuracy. We provide a summary of the top methods, make their code and Docker containers publicly available, and give practical suggestions on building accurate and efficient abdominal organ segmentation models. The FLARE challenge remains open for future submissions through a live platform for benchmarking further methodology developments at https://flare.grand-challenge.org/.
Collapse
Affiliation(s)
- Jun Ma
- Department of Mathematics, Nanjing University of Science and Technology, 210094, Nanjing, China
| | - Yao Zhang
- Institute of Computing Technology, Chinese Academy of Sciences and the University of Chinese Academy of Sciences, 100019, Beijing, China
| | - Song Gu
- Department of Image Reconstruction, Nanjing Anke Medical Technology Co., Ltd., 211113, Nanjing, China
| | - Xingle An
- Infervision Technology Co. Ltd., 100020, Beijing, China
| | - Zhihe Wang
- Shenzhen Haichuang Medical Co., Ltd., 518049, Shenzhen, China
| | - Cheng Ge
- Institute of Bioinformatics and Medical Engineering, Jiangsu University of Technology, 213001, Changzhou, China
| | - Congcong Wang
- School of Computer Science and Engineering, Tianjin University of Technology, 300384, Tianjin, China; Engineering Research Center of Learning-Based Intelligent System, Ministry of Education, 300384, Tianjin, China
| | - Fan Zhang
- Radiological Algorithm, Fosun Aitrox Information Technology Co., Ltd., 200033, Shanghai, China
| | - Yu Wang
- Radiological Algorithm, Fosun Aitrox Information Technology Co., Ltd., 200033, Shanghai, China
| | - Yinan Xu
- Key Lab of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, 710071, Shaanxi, China
| | - Shuiping Gou
- Key Lab of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, 710071, Shaanxi, China
| | - Franz Thaler
- Gottfried Schatz Research Center: Biophysics, Medical University of Graz, 8010, Graz, Austria; Institute of Computer Graphics and Vision, Graz University of Technology, 8010, Graz, Austria
| | - Christian Payer
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010, Graz, Austria
| | - Darko Štern
- Gottfried Schatz Research Center: Biophysics, Medical University of Graz, 8010, Graz, Austria
| | - Edward G A Henderson
- Division of Cancer Sciences, The University of Manchester, M139PL, Manchester, UK; Radiotherapy Related Research, The Christie NHS Foundation Trust, M139PL, Manchester, UK
| | - Dónal M McSweeney
- Division of Cancer Sciences, The University of Manchester, M139PL, Manchester, UK; Radiotherapy Related Research, The Christie NHS Foundation Trust, M139PL, Manchester, UK
| | - Andrew Green
- Division of Cancer Sciences, The University of Manchester, M139PL, Manchester, UK; Radiotherapy Related Research, The Christie NHS Foundation Trust, M139PL, Manchester, UK
| | - Price Jackson
- Peter MacCallum Cancer Centre, 3000, Melbourne, Australia
| | | | - Quoc-Cuong Nguyen
- University of Information Technology, VNU-HCM, 700000, Ho Chi Minh City, Viet Nam
| | - Abdul Qayyum
- Brest National School of Engineering, UMR CNRS 6285 LabSTICC, 29280, Brest, France
| | | | - Ziyan Huang
- Institute of Medical Robotics, Shanghai Jiao Tong University, 200240, Shanghai, China
| | - Ziqi Zhou
- Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University, 518000, Shenzhen, China
| | - Deng-Ping Fan
- College of Computer Science, Nankai University, 300071, Tianjin, China; Inception Institute of Artificial Intelligence, Abu Dhabi, United Arab Emirates
| | - Huan Xiong
- Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates; Harbin Institute of Technology, 150001, Harbin, China
| | - Guoqiang Dong
- Department of Nuclear Medicine, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, 210008, Nanjing, China; Department of Interventional Radiology, The Second Affiliated Hospital of Bengbu Medical College, 233017, Bengbu, China
| | - Qiongjie Zhu
- Department of Nuclear Medicine, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, 210008, Nanjing, China; Department of Radiology, Shidong Hospital, 200438, Shanghai, China
| | - Jian He
- Department of Nuclear Medicine, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, 210008, Nanjing, China
| | - Xiaoping Yang
- Department of Mathematics, Nanjing University, 210093, Nanjing, China.
| |
Collapse
|
48
|
RE-3DLVNet: Refined estimation of the left ventricle volume via interactive 3D segmentation and reinforced quantification. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
49
|
|
50
|
Zhang Y, Lin X, Zhuang Y, Sun L, Huang Y, Ding X, Wang G, Yang L, Yu Y. Harmonizing Pathological and Normal Pixels for Pseudo-Healthy Synthesis. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:2457-2468. [PMID: 35363612 DOI: 10.1109/tmi.2022.3164095] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Synthesizing a subject-specific pathology-free image from a pathological image is valuable for algorithm development and clinical practice. In recent years, several approaches based on the Generative Adversarial Network (GAN) have achieved promising results in pseudo-healthy synthesis. However, the discriminator (i.e., a classifier) in the GAN cannot accurately identify lesions and further hampers from generating admirable pseudo-healthy images. To address this problem, we present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. Then, we apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem existing in medical image segmentation. Furthermore, a reliable metric is proposed by utilizing two attributes of label noise to measure the health of synthetic images. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods. The method achieves better performance than the existing methods with only 30% of the training data. The effectiveness of the proposed method is also demonstrated on the LiTS and the T1 modality of BraTS. The code and the pre-trained model of this study are publicly available at https://github.com/Au3C2/Generator-Versus-Segmentor.
Collapse
|