1
|
Wang X, Ren X, Jin G, Ying S, Wang J, Li J, Shi J. B-mode ultrasound-based CAD by learning using privileged information with dual-level missing modality completion. Comput Biol Med 2024; 182:109106. [PMID: 39241326 DOI: 10.1016/j.compbiomed.2024.109106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Revised: 07/23/2024] [Accepted: 09/01/2024] [Indexed: 09/09/2024]
Abstract
Learning using privileged information (LUPI) has shown its effectiveness to improve the B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) by transferring knowledge from the elasticity ultrasound (EUS). However, LUPI only performs transfer learning between the paired data with shared labels, and cannot handle the scenario of modality imbalance. In order to conduct the supervised transfer learning between the paired ultrasound data together with the additional single-modal BUS images, a novel multi-view LUPI algorithm with Dual-Level Modality Completion, named DLMC-LUPI, is proposed to improve the performance of BUS-based CAD. The DLMC-LUPI implements both image-level and feature-level (dual-level) completions of missing EUS modality, and then performs multi-view LUPI for knowledge transfer. Specifically, in the dual-level modality completion stage, a variational autoencoder (VAE) model for feature generation and a novel generative adversarial network (VAE-based GAN) model for image generation are sequentially trained. The proposed VAE-based GAN can improve the synthesis quality of EUS images by adopting the features generated by VAE from the BUS images as the model constrain to make the features generated from the synthesized EUS images more similar to them. In the multi-view LUPI stage, two feature vectors are generated from the real or pseudo images as two source domains, and then fed them to the multi-view support vector machine plus classifier for model training. The experiments on two ultrasound datasets indicate that the DLMC-LUPI outperforms all the compared algorithms, and it can effectively improve the performance of single-modal BUS-based CAD.
Collapse
Affiliation(s)
- Xiao Wang
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China
| | - Xinping Ren
- Ultrasound Department, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Ge Jin
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China; School of Communication and Information Engineering, Jiangsu Open University, Jiangsu, China
| | - Shihui Ying
- Department of Mathematics, School of Science, Shanghai University, Shanghai, China
| | - Jun Wang
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China
| | - Juncheng Li
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China
| | - Jun Shi
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China.
| |
Collapse
|
2
|
Yoon JT, Lee KM, Oh JH, Kim HG, Jeong JW. Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know. Diagnostics (Basel) 2024; 14:1756. [PMID: 39202244 PMCID: PMC11353572 DOI: 10.3390/diagnostics14161756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2024] [Accepted: 08/05/2024] [Indexed: 09/03/2024] Open
Abstract
The rapid development of deep learning in medical imaging has significantly enhanced the capabilities of artificial intelligence while simultaneously introducing challenges, including the need for vast amounts of training data and the labor-intensive tasks of labeling and segmentation. Generative adversarial networks (GANs) have emerged as a solution, offering synthetic image generation for data augmentation and streamlining medical image processing tasks through models such as cGAN, CycleGAN, and StyleGAN. These innovations not only improve the efficiency of image augmentation, reconstruction, and segmentation, but also pave the way for unsupervised anomaly detection, markedly reducing the reliance on labeled datasets. Our investigation into GANs in medical imaging addresses their varied architectures, the considerations for selecting appropriate GAN models, and the nuances of model training and performance evaluation. This paper aims to provide radiologists who are new to GAN technology with a thorough understanding, guiding them through the practical application and evaluation of GANs in brain imaging with two illustrative examples using CycleGAN and pixel2style2pixel (pSp)-combined StyleGAN. It offers a comprehensive exploration of the transformative potential of GANs in medical imaging research. Ultimately, this paper strives to equip radiologists with the knowledge to effectively utilize GANs, encouraging further research and application within the field.
Collapse
Affiliation(s)
- Jeong Taek Yoon
- Department of Radiology, Kyung Hee University Hospital, Kyung Hee University College of Medicine, 23 Kyungheedae-ro, Dongdaemun-gu, Seoul 02447, Republic of Korea; (J.T.Y.); (H.-G.K.)
| | - Kyung Mi Lee
- Department of Radiology, Kyung Hee University Hospital, Kyung Hee University College of Medicine, 23 Kyungheedae-ro, Dongdaemun-gu, Seoul 02447, Republic of Korea; (J.T.Y.); (H.-G.K.)
| | - Jang-Hoon Oh
- Department of Radiology, Kyung Hee University Hospital, Kyung Hee University College of Medicine, 23 Kyungheedae-ro, Dongdaemun-gu, Seoul 02447, Republic of Korea; (J.T.Y.); (H.-G.K.)
| | - Hyug-Gi Kim
- Department of Radiology, Kyung Hee University Hospital, Kyung Hee University College of Medicine, 23 Kyungheedae-ro, Dongdaemun-gu, Seoul 02447, Republic of Korea; (J.T.Y.); (H.-G.K.)
| | - Ji Won Jeong
- Department of Medicine, Graduate School, Kyung Hee University, 23 Kyungheedae-ro, Dongdaemun-gu, Seoul 02447, Republic of Korea;
| |
Collapse
|
3
|
Rabe M, Kurz C, Thummerer A, Landry G. Artificial intelligence for treatment delivery: image-guided radiotherapy. Strahlenther Onkol 2024:10.1007/s00066-024-02277-9. [PMID: 39138806 DOI: 10.1007/s00066-024-02277-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 07/07/2024] [Indexed: 08/15/2024]
Abstract
Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.
Collapse
Affiliation(s)
- Moritz Rabe
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Marchioninistraße 15, 81377, Munich, Bavaria, Germany
| | - Christopher Kurz
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Marchioninistraße 15, 81377, Munich, Bavaria, Germany
| | - Adrian Thummerer
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Marchioninistraße 15, 81377, Munich, Bavaria, Germany
| | - Guillaume Landry
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Marchioninistraße 15, 81377, Munich, Bavaria, Germany.
- German Cancer Consortium (DKTK), partner site Munich, a partnership between the DKFZ and the LMU University Hospital Munich, Marchioninistraße 15, 81377, Munich, Bavaria, Germany.
- Bavarian Cancer Research Center (BZKF), Marchioninistraße 15, 81377, Munich, Bavaria, Germany.
| |
Collapse
|
4
|
Zeng J, Fu Q. A review: artificial intelligence in image-guided spinal surgery. Expert Rev Med Devices 2024; 21:689-700. [PMID: 39115295 DOI: 10.1080/17434440.2024.2384541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Accepted: 07/22/2024] [Indexed: 08/28/2024]
Abstract
INTRODUCTION Due to the complex anatomy of the spine and the intricate surgical procedures involved, spinal surgery demands a high level of technical expertise from surgeons. The clinical application of image-guided spinal surgery has significantly enhanced lesion visualization, reduced operation time, and improved surgical outcomes. AREAS COVERED This article reviews the latest advancements in deep learning and artificial intelligence in image-guided spinal surgery, aiming to provide references and guidance for surgeons, engineers, and researchers involved in this field. EXPERT OPINION Our analysis indicates that image-guided spinal surgery, augmented by artificial intelligence, outperforms traditional spinal surgery techniques. Moving forward, it is imperative to collect a more expansive dataset to further ensure the procedural safety of such surgeries. These insights carry significant implications for the integration of artificial intelligence in the medical field, ultimately poised to enhance the proficiency of surgeons and improve surgical outcomes.
Collapse
Affiliation(s)
- Jiahang Zeng
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
- Department of Orthopedics, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Qiang Fu
- Department of Orthopedics, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| |
Collapse
|
5
|
Sun H, Wei J, Yuan W, Li R. Semi-supervised multi-modal medical image segmentation with unified translation. Comput Biol Med 2024; 176:108570. [PMID: 38749326 DOI: 10.1016/j.compbiomed.2024.108570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 03/08/2024] [Accepted: 05/05/2024] [Indexed: 05/31/2024]
Abstract
The two major challenges to deep-learning-based medical image segmentation are multi-modality and a lack of expert annotations. Existing semi-supervised segmentation models can mitigate the problem of insufficient annotations by utilizing a small amount of labeled data. However, most of these models are limited to single-modal data and cannot exploit the complementary information from multi-modal medical images. A few semi-supervised multi-modal models have been proposed recently, but they have rigid structures and require additional training steps for each modality. In this work, we propose a novel flexible method, semi-supervised multi-modal medical image segmentation with unified translation (SMSUT), and a unique semi-supervised procedure that can leverage multi-modal information to improve the semi-supervised segmentation performance. Our architecture capitalizes on unified translation to extract complementary information from multi-modal data which compels the network to focus on the disparities and salient features among each modality. Furthermore, we impose constraints on the model at both pixel and feature levels, to cope with the lack of annotation information and the diverse representations within semi-supervised multi-modal data. We introduce a novel training procedure tailored for semi-supervised multi-modal medical image analysis, by integrating the concept of conditional translation. Our method has a remarkable ability for seamless adaptation to varying numbers of distinct modalities in the training data. Experiments show that our model exceeds the semi-supervised segmentation counterparts in the public datasets which proves our network's high-performance capabilities and the transferability of our proposed method. The code of our method will be openly available at https://github.com/Sue1347/SMSUT-MedicalImgSegmentation.
Collapse
Affiliation(s)
- Huajun Sun
- South China University of Technology, Guangzhou, 510006, China.
| | - Jia Wei
- South China University of Technology, Guangzhou, 510006, China.
| | - Wenguang Yuan
- Huawei Cloud BU EI Innovation Laboratory, Dongguan, 523000, China.
| | - Rui Li
- Rochester Institute of Technology, Rochester, NY 14623, USA.
| |
Collapse
|
6
|
Showrav TT, Hasan MK. Hi- gMISnet: generalized medical image segmentation using DWT based multilayer fusion and dual mode attention into high resolution pGAN. Phys Med Biol 2024; 69:115019. [PMID: 38593830 DOI: 10.1088/1361-6560/ad3cb3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 04/09/2024] [Indexed: 04/11/2024]
Abstract
Objective.Automatic medical image segmentation is crucial for accurately isolating target tissue areas in the image from background tissues, facilitating precise diagnoses and procedures. While the proliferation of publicly available clinical datasets led to the development of deep learning-based medical image segmentation methods, a generalized, accurate, robust, and reliable approach across diverse imaging modalities remains elusive.Approach.This paper proposes a novel high-resolution parallel generative adversarial network (pGAN)-based generalized deep learning method for automatic segmentation of medical images from diverse imaging modalities. The proposed method showcases better performance and generalizability by incorporating novel components such as partial hybrid transfer learning, discrete wavelet transform (DWT)-based multilayer and multiresolution feature fusion in the encoder, and a dual mode attention gate in the decoder of the multi-resolution U-Net-based GAN. With multi-objective adversarial training loss functions including a unique reciprocal loss for enforcing cooperative learning inpGANs, it further enhances the robustness and accuracy of the segmentation map.Main results.Experimental evaluations conducted on nine diverse publicly available medical image segmentation datasets, including PhysioNet ICH, BUSI, CVC-ClinicDB, MoNuSeg, GLAS, ISIC-2018, DRIVE, Montgomery, and PROMISE12, demonstrate the proposed method's superior performance. The proposed method achieves mean F1 scores of 79.53%, 88.68%, 82.50%, 93.25%, 90.40%, 94.19%, 81.65%, 98.48%, and 90.79%, respectively, on the above datasets, surpass state-of-the-art segmentation methods. Furthermore, our proposed method demonstrates robust multi-domain segmentation capabilities, exhibiting consistent and reliable performance. The assessment of the model's proficiency in accurately identifying small details indicates that the high-resolution generalized medical image segmentation network (Hi-gMISnet) is more precise in segmenting even when the target area is very small.Significance.The proposed method provides robust and reliable segmentation performance on medical images, and thus it has the potential to be used in a clinical setting for the diagnosis of patients.
Collapse
Affiliation(s)
- Tushar Talukder Showrav
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, 1205, Bangladesh
| | - Md Kamrul Hasan
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, 1205, Bangladesh
| |
Collapse
|
7
|
Zhao X, Lai L, Li Y, Zhou X, Cheng X, Chen Y, Huang H, Guo J, Wang G. A lightweight bladder tumor segmentation method based on attention mechanism. Med Biol Eng Comput 2024; 62:1519-1534. [PMID: 38308022 DOI: 10.1007/s11517-024-03018-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Accepted: 01/05/2024] [Indexed: 02/04/2024]
Abstract
In the endoscopic images of bladder, accurate segmentation of different grade bladder tumor from blurred boundary regions and highly variable shapes is of great significance for doctors' diagnosis and patients' later treatment. We propose a nested attentional feature fusion segmentation network (NAFF-Net) based on the encoder-decoder structure formed by the combination of weighted pyramid pooling module (WPPM) and nested attentional feature fusion (NAFF). Among them, WPPM applies the cascade of atrous convolution to enhance the overall perceptual field while introducing adaptive weights to optimize multi-scale feature extraction, NAFF integrates deep semantic information into shallow feature maps, effectively focusing on edge and detail information in bladder tumor images. Additionally, a weighted mixed loss function is constructed to alleviate the impact of imbalance between positive and negative sample distribution on segmentation accuracy. Experiments illustrate the proposed NAFF-Net achieves better segmentation results compared to other mainstream models, with a MIoU of 84.05%, MPrecision of 91.52%, MRecall of 90.81%, and F1-score of 91.16%, and also achieves good results on the public datasets Kvasir-SEG and CVC-ClinicDB. Compared to other models, NAFF-Net has a smaller number of parameters, which is a significant advantage in model deployment.
Collapse
Affiliation(s)
- Xiushun Zhao
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China
| | - Libing Lai
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Yunjiao Li
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China
| | - Xiaochen Zhou
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Xiaofeng Cheng
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Yujun Chen
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Haohui Huang
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China
| | - Jing Guo
- School of Automation, Guangdong University of Technology, Guangzhou, 510006, China.
| | - Gongxian Wang
- Department of Urology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China.
| |
Collapse
|
8
|
Yu X, Qin Y, Zhang F, Zhang Z. A recurrent positional encoding circular attention mechanism network for biomedical image segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 246:108054. [PMID: 38350190 DOI: 10.1016/j.cmpb.2024.108054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 01/16/2024] [Accepted: 01/28/2024] [Indexed: 02/15/2024]
Abstract
Deep-learning-based medical image segmentation techniques can assist doctors in disease diagnosis and rapid treatment. However, existing medical image segmentation models do not fully consider the dependence between feature segments in the feature extraction process, and the correlated features can be further extracted. Therefore, a recurrent positional encoding circular attention mechanism network (RPECAMNet) is proposed based on relative positional encoding for medical image segmentation. Multiple residual modules are used to extract the primary features of the medical images, which are thereafter converted into one-dimensional data for relative positional encoding. The recursive former is used to further extract features from medical images, and decoding is performed using deconvolution. An adaptive loss function is designed to train the model and achieve accurate medical-image segmentation. Finally, the proposed model is used to conduct comparative experiments on the synapse and self-constructed kidney datasets to verify the accuracy of the proposed model for medical image segmentation.
Collapse
Affiliation(s)
- Xiaoxia Yu
- College of Mechanical Engineering, Chongqing University of Technology, Chongqing, 400054, China.
| | - Yong Qin
- Ph.D. degree at Children's Hospital of Chongqing Medical University, Chongqing, 400016, China
| | - Fanghong Zhang
- Ghent University in Belgium in 2012. She is now a professor at Chongqing Normal University, Chongqing, 400054, China
| | - Zhigang Zhang
- College of Mechanical Engineering, Chongqing University of Technology, Chongqing, 400045, China
| |
Collapse
|
9
|
Yang S, Kim KD, Ariji E, Kise Y. Generative adversarial networks in dental imaging: a systematic review. Oral Radiol 2024; 40:93-108. [PMID: 38001347 DOI: 10.1007/s11282-023-00719-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 10/27/2023] [Indexed: 11/26/2023]
Abstract
OBJECTIVES This systematic review on generative adversarial network (GAN) architectures for dental image analysis provides a comprehensive overview to readers regarding current GAN trends in dental imagery and potential future applications. METHODS Electronic databases (PubMed/MEDLINE, Scopus, Embase, and Cochrane Library) were searched to identify studies involving GANs for dental image analysis. Eighteen full-text articles describing the applications of GANs in dental imagery were reviewed. Risk of bias and applicability concerns were assessed using the QUADAS-2 tool. RESULTS GANs were used for various imaging modalities, including two-dimensional and three-dimensional images. In dental imaging, GANs were utilized for tasks such as artifact reduction, denoising, and super-resolution, domain transfer, image generation for augmentation, outcome prediction, and identification. The generated images were incorporated into tasks such as landmark detection, object detection and classification. Because of heterogeneity among the studies, a meta-analysis could not be conducted. Most studies (72%) had a low risk of bias in all four domains. However, only three (17%) studies had a low risk of applicability concerns. CONCLUSIONS This extensive analysis of GANs in dental imaging highlighted their broad application potential within the dental field. Future studies should address limitations related to the stability, repeatability, and overall interpretability of GAN architectures. By overcoming these challenges, the applicability of GANs in dentistry can be enhanced, ultimately benefiting the dental field in its use of GANs and artificial intelligence.
Collapse
Affiliation(s)
- Sujin Yang
- Department of Advanced General Dentistry, College of Dentistry, Yonsei University, Seoul, Korea
| | - Kee-Deog Kim
- Department of Advanced General Dentistry, College of Dentistry, Yonsei University, Seoul, Korea
| | - Eiichiro Ariji
- Department of Oral and Maxillofacial Radiology, School of Dentistry, Aichi Gakuin University, 2-11 Suemori-dori, Chikusa-ku, Nagoya, 464-8651, Japan
| | - Yoshitaka Kise
- Department of Oral and Maxillofacial Radiology, School of Dentistry, Aichi Gakuin University, 2-11 Suemori-dori, Chikusa-ku, Nagoya, 464-8651, Japan.
| |
Collapse
|
10
|
Vagni M, Tran HE, Catucci F, Chiloiro G, D’Aviero A, Re A, Romano A, Boldrini L, Kawula M, Lombardo E, Kurz C, Landry G, Belka C, Indovina L, Gambacorta MA, Cusumano D, Placidi L. Impact of bias field correction on 0.35 T pelvic MR images: evaluation on generative adversarial network-based OARs' auto-segmentation and visual grading assessment. Front Oncol 2024; 14:1294252. [PMID: 38606108 PMCID: PMC11007142 DOI: 10.3389/fonc.2024.1294252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 03/11/2024] [Indexed: 04/13/2024] Open
Abstract
Purpose Magnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance. Materials and methods 3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett's S score and Fleiss' kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively. Results In the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63 mm and 0.92/5.91 mm for the bladder and 0.84/10.61 mm and 0.83/9.71 mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs' boundaries compared with the original MRI. Conclusion The bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs' auto-segmentation outputs generated by the GAN.
Collapse
Affiliation(s)
- Marica Vagni
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | - Huong Elena Tran
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | | | - Giuditta Chiloiro
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | | | | | - Angela Romano
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | - Luca Boldrini
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | - Maria Kawula
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Elia Lombardo
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Christopher Kurz
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Guillaume Landry
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Claus Belka
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany
- German Cancer Consortium (DKTK), Partner Site Munich, A Partnership Between DKFZ and LMU University Hospital Munich, Munich, Germany
- Bavarian Cancer Research Center (BZKF), Munich, Germany
| | - Luca Indovina
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | - Maria Antonietta Gambacorta
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| | - Davide Cusumano
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
- Mater Olbia Hospital, Olbia, Italy
| | - Lorenzo Placidi
- Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario “A. Gemelli” IRCCS, Rome, Italy
| |
Collapse
|
11
|
Milosevic M, Jin Q, Singh A, Amal S. Applications of AI in multi-modal imaging for cardiovascular disease. FRONTIERS IN RADIOLOGY 2024; 3:1294068. [PMID: 38283302 PMCID: PMC10811170 DOI: 10.3389/fradi.2023.1294068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 12/22/2023] [Indexed: 01/30/2024]
Abstract
Data for healthcare is diverse and includes many different modalities. Traditional approaches to Artificial Intelligence for cardiovascular disease were typically limited to single modalities. With the proliferation of diverse datasets and new methods in AI, we are now able to integrate different modalities, such as magnetic resonance scans, computerized tomography scans, echocardiography, x-rays, and electronic health records. In this paper, we review research from the last 5 years in applications of AI to multi-modal imaging. There have been many promising results in registration, segmentation, and fusion of different magnetic resonance imaging modalities with each other and computer tomography scans, but there are still many challenges that need to be addressed. Only a few papers have addressed modalities such as x-ray, echocardiography, or non-imaging modalities. As for prediction or classification tasks, there have only been a couple of papers that use multiple modalities in the cardiovascular domain. Furthermore, no models have been implemented or tested in real world cardiovascular clinical settings.
Collapse
Affiliation(s)
- Marko Milosevic
- Roux Institute, Northeastern University, Portland, ME, United States
| | - Qingchu Jin
- Roux Institute, Northeastern University, Portland, ME, United States
| | - Akarsh Singh
- College of Engineering, Northeastern University, Boston, MA, United States
| | - Saeed Amal
- Roux Institute, Northeastern University, Portland, ME, United States
| |
Collapse
|
12
|
Ding F, Shen Z, Zhu G, Kwong S, Zhou Y, Lyu S. ExS-GAN: Synthesizing Anti-Forensics Images via Extra Supervised GAN. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:7162-7173. [PMID: 36264736 DOI: 10.1109/tcyb.2022.3210294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
So far, researchers have proposed many forensics tools to protect the authenticity and integrity of digital information. However, with the explosive development of machine learning, existing forensics tools may compromise against new attacks anytime. Hence, it is always necessary to investigate anti-forensics to expose the vulnerabilities of forensics tools. It is beneficial for forensics researchers to develop new tools as countermeasures. To date, one of the potential threats is the generative adversarial networks (GANs), which could be employed for fabricating or forging falsified data to attack forensics detectors. In this article, we investigate the anti-forensics performance of GANs by proposing a novel model, the ExS-GAN, which features an extra supervision system. After training, the proposed model could launch anti-forensics attacks on various manipulated images. Evaluated by experiments, the proposed method could achieve high anti-forensics performance while preserving satisfying image quality. We also justify the proposed extra supervision via an ablation study.
Collapse
|
13
|
P. J, B. K. SK, Jayaraman S. Automatic foot ulcer segmentation using conditional generative adversarial network (AFSegGAN): A wound management system. PLOS DIGITAL HEALTH 2023; 2:e0000344. [PMID: 37930982 PMCID: PMC10627472 DOI: 10.1371/journal.pdig.0000344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Accepted: 08/07/2023] [Indexed: 11/08/2023]
Abstract
Effective wound care is essential to prevent further complications, promote healing, and reduce the risk of infection and other health issues. Chronic wounds, particularly in older adults, patients with disabilities, and those with pressure, venous, or diabetic foot ulcers, cause significant morbidity and mortality. Due to the positive trend in the number of individuals with chronic wounds, particularly among the growing elderly and diabetes populations, it is imperative to develop novel technologies and practices for the best practice clinical management of chronic wounds to minimize the potential health and economic burdens on society. As wound care is managed in hospitals and community care, it is crucial to have quantitative metrics like wound boundary and morphological features. The traditional visual inspection technique is purely subjective and error-prone, and digitization provides an appealing alternative. Various deep-learning models have earned confidence; however, their accuracy primarily relies on the image quality, the dataset size to learn the features, and experts' annotation. This work aims to develop a wound management system that automates wound segmentation using a conditional generative adversarial network (cGAN) and estimate the wound morphological parameters. AFSegGAN was developed and validated on the MICCAI 2021-foot ulcer segmentation dataset. In addition, we use adversarial loss and patch-level comparison at the discriminator network to improve the segmentation performance and balance the GAN network training. Our model outperformed state-of-the-art methods with a Dice score of 93.11% and IoU of 99.07%. The proposed wound management system demonstrates its abilities in wound segmentation and parameter estimation, thereby reducing healthcare workers' efforts to diagnose or manage wounds and facilitating remote healthcare.
Collapse
Affiliation(s)
- Jishnu P.
- TCS Research, Digital Medicine and Medical Technology- B&T Group, TATA Consultancy Services, Bangalore, Karnataka, India
| | - Shreyamsha Kumar B. K.
- TCS Research, Digital Medicine and Medical Technology- B&T Group, TATA Consultancy Services, Bangalore, Karnataka, India
| | - Srinivasan Jayaraman
- TCS Research, Digital Medicine and Medical Technology- B&T Group, TATA Consultancy Services, Cincinnati, Ohio, United States of America
| |
Collapse
|
14
|
Hao D, Li H, Zhang Y, Zhang Q. MUE-CoT: multi-scale uncertainty entropy-aware co-training framework for left atrial segmentation. Phys Med Biol 2023; 68:215008. [PMID: 37567214 DOI: 10.1088/1361-6560/acef8e] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 08/11/2023] [Indexed: 08/13/2023]
Abstract
Objective.Accurate left atrial segmentation is the basis of the recognition and clinical analysis of atrial fibrillation. Supervised learning has achieved some competitive segmentation results, but the high annotation cost often limits its performance. Semi-supervised learning is implemented from limited labeled data and a large amount of unlabeled data and shows good potential in solving practical medical problems.Approach. In this study, we proposed a collaborative training framework for multi-scale uncertain entropy perception (MUE-CoT) and achieved efficient left atrial segmentation from a small amount of labeled data. Based on the pyramid feature network, learning is implemented from unlabeled data by minimizing the pyramid prediction difference. In addition, novel loss constraints are proposed for co-training in the study. The diversity loss is defined as a soft constraint so as to accelerate the convergence and a novel multi-scale uncertainty entropy calculation method and a consistency regularization term are proposed to measure the consistency between prediction results. The quality of pseudo-labels cannot be guaranteed in the pre-training period, so a confidence-dependent empirical Gaussian function is proposed to weight the pseudo-supervised loss.Main results.The experimental results of a publicly available dataset and an in-house clinical dataset proved that our method outperformed existing semi-supervised methods. For the two datasets with a labeled ratio of 5%, the Dice similarity coefficient scores were 84.94% ± 4.31 and 81.24% ± 2.4, the HD95values were 4.63 mm ± 2.13 and 3.94 mm ± 2.72, and the Jaccard similarity coefficient scores were 74.00% ± 6.20 and 68.49% ± 3.39, respectively.Significance.The proposed model effectively addresses the challenges of limited data samples and high costs associated with manual annotation in the medical field, leading to enhanced segmentation accuracy.
Collapse
Affiliation(s)
- Dechen Hao
- School of Software, North University of China, Taiyuan Shanxi, People's Republic of China
| | - Hualing Li
- School of Software, North University of China, Taiyuan Shanxi, People's Republic of China
| | - Yonglai Zhang
- School of Software, North University of China, Taiyuan Shanxi, People's Republic of China
| | - Qi Zhang
- Department of Cardiology, The Second Hospital of Shanxi Medical University, Taiyuan Shanxi, People's Republic of China
| |
Collapse
|
15
|
Xiao H, Li Z, Zhou Y, Gao Z. High-Density Functional Near-Infrared Spectroscopy and Machine Learning for Visual Perception Quantification. SENSORS (BASEL, SWITZERLAND) 2023; 23:8696. [PMID: 37960396 PMCID: PMC10650008 DOI: 10.3390/s23218696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 10/18/2023] [Accepted: 10/23/2023] [Indexed: 11/15/2023]
Abstract
The main application scenario for wearable sensors involves the generation of data and monitoring metrics. fNIRS (functional near-infrared spectroscopy) allows the nonintrusive monitoring of human visual perception. The quantification of visual perception by fNIRS facilitates applications in engineering-related fields. This study designed a set of experimental procedures to effectively induce visible alterations and to quantify visual perception in conjunction with the acquisition of Hbt (total hemoglobin), Hb (hemoglobin), and HbO2 (oxygenated hemoglobin) data obtained from HfNIRS (high-density functional near-infrared spectroscopy). Volunteers completed the visual task separately in response to different visible changes in the simulated scene. HfNIRS recorded the changes in Hbt, Hb, and HbO2 during the study, the time point of the visual difference, and the time point of the task change. This study consisted of one simulated scene, two visual variations, and four visual tasks. The simulation scene featured a car driving location. The visible change suggested that the brightness and saturation of the car operator interface would change. The visual task represented the completion of the layout, color, design, and information questions answered in response to the visible change. This study collected data from 29 volunteers. The volunteers completed the visual task separately in response to different visual changes in the same simulated scene. HfNIRS recorded the changes in Hbt, Hb, and HbO2 during the study, the time point of the visible difference, and the time point of the task change. The data analysis methods in this study comprised a combination of channel dimensionality reduction, feature extraction, task classification, and score correlation. Channel downscaling: This study used the data of 15 channels in HfNIRS to calculate the mutual information between different channels to set a threshold, and to retain the data of the channels that were higher than those of the mutual information. Feature extraction: The statistics derived from the visual task, including time, mean, median, variance, extreme variance, kurtosis, bias, information entropy, and approximate entropy were computed. Task classification: This study used the KNN (K-Nearest Neighbors) algorithm to classify different visual tasks and to calculate the accuracy, precision, recall, and F1 scores. Scoring correlation: This study matched the visual task scores with the fluctuations of Hbt, Hb, and HbO2 and observed the changes in Hbt, Hb, and HbO2 under different scoring levels. Mutual information was used to downscale the channels, and seven channels were retained for analysis under each visual task. The average accuracy was 96.3% ± 1.99%; the samples that correctly classified the visual task accounted for 96.3% of the total; and the classification accuracy was high. By analyzing the correlation between the scores on different visual tasks and the fluctuations of Hbt, Hb, and HbO2, it was found that the higher the score, the more obvious, significant, and higher the fluctuations of Hbt, Hb, and HbO2. Experiments found that changes in visual perception triggered changes in Hbt, Hb, and HbO2. HfNIRS combined with Hbt, Hb, and HbO2 recorded by machine learning algorithms can effectively quantify visual perception. However, the related research in this paper still needs to be further refined, and the mathematical relationship between HfNIRS and visual perception needs to be further explored to realize the quantitative study of subjective and objective visual perception supported by the mathematical relationship.
Collapse
Affiliation(s)
- Hongwei Xiao
- School of Automotive Engineering, Jilin University, Changchun 130022, China;
| | - Zhao Li
- School of Public Health, Jilin University, Changchun 130021, China;
| | - Yuting Zhou
- China Academy of Engineering Physics, Mianyang 621900, China;
| | - Zhenhai Gao
- State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130025, China
| |
Collapse
|
16
|
Yang S, Kim KD, Ariji E, Takata N, Kise Y. Evaluating the performance of generative adversarial network-synthesized periapical images in classifying C-shaped root canals. Sci Rep 2023; 13:18038. [PMID: 37865655 PMCID: PMC10590373 DOI: 10.1038/s41598-023-45290-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 10/18/2023] [Indexed: 10/23/2023] Open
Abstract
This study evaluated the performance of generative adversarial network (GAN)-synthesized periapical images for classifying C-shaped root canals, which are challenging to diagnose because of their complex morphology. GANs have emerged as a promising technique for generating realistic images, offering a potential solution for data augmentation in scenarios with limited training datasets. Periapical images were synthesized using the StyleGAN2-ADA framework, and their quality was evaluated based on the average Frechet inception distance (FID) and the visual Turing test. The average FID was found to be 35.353 (± 4.386) for synthesized C-shaped canal images and 25.471 (± 2.779) for non C-shaped canal images. The visual Turing test conducted by two radiologists on 100 randomly selected images revealed that distinguishing between real and synthetic images was difficult. These results indicate that GAN-synthesized images exhibit satisfactory visual quality. The classification performance of the neural network, when augmented with GAN data, showed improvements compared with using real data alone, and could be advantageous in addressing data conditions with class imbalance. GAN-generated images have proven to be an effective data augmentation method, addressing the limitations of limited training data and computational resources in diagnosing dental anomalies.
Collapse
Affiliation(s)
- Sujin Yang
- Department of Advanced General Dentistry, College of Dentistry, Yonsei University, Seoul, Korea
| | - Kee-Deog Kim
- Department of Advanced General Dentistry, College of Dentistry, Yonsei University, Seoul, Korea
| | - Eiichiro Ariji
- Department of Oral and Maxillofacial Radiology, Aichi Gakuin University, 2-11 Seuemori-Dori, Chikusa-Ku, Nagoya, 464-8651, Japan
| | - Natsuho Takata
- Department of Oral and Maxillofacial Radiology, Aichi Gakuin University, 2-11 Seuemori-Dori, Chikusa-Ku, Nagoya, 464-8651, Japan
| | - Yoshitaka Kise
- Department of Oral and Maxillofacial Radiology, Aichi Gakuin University, 2-11 Seuemori-Dori, Chikusa-Ku, Nagoya, 464-8651, Japan.
| |
Collapse
|
17
|
van der Schot A, Sikkel E, Niekolaas M, Spaanderman M, de Jong G. Placental Vessel Segmentation Using Pix2pix Compared to U-Net. J Imaging 2023; 9:226. [PMID: 37888333 PMCID: PMC10607321 DOI: 10.3390/jimaging9100226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 09/21/2023] [Accepted: 10/02/2023] [Indexed: 10/28/2023] Open
Abstract
Computer-assisted technologies have made significant progress in fetoscopic laser surgery, including placental vessel segmentation. However, the intra- and inter-procedure variabilities in the state-of-the-art segmentation methods remain a significant hurdle. To address this, we investigated the use of conditional generative adversarial networks (cGANs) for fetoscopic image segmentation and compared their performance with the benchmark U-Net technique for placental vessel segmentation. Two deep-learning models, U-Net and pix2pix (a popular cGAN model), were trained and evaluated using a publicly available dataset and an internal validation set. The overall results showed that the pix2pix model outperformed the U-Net model, with a Dice score of 0.80 [0.70; 0.86] versus 0.75 [0.0.60; 0.84] (p-value < 0.01) and an Intersection over Union (IoU) score of 0.70 [0.61; 0.77] compared to 0.66 [0.53; 0.75] (p-value < 0.01), respectively. The internal validation dataset further validated the superiority of the pix2pix model, achieving Dice and IoU scores of 0.68 [0.53; 0.79] and 0.59 [0.49; 0.69] (p-value < 0.01), respectively, while the U-Net model obtained scores of 0.53 [0.49; 0.64] and 0.49 [0.17; 0.56], respectively. This study successfully compared U-Net and pix2pix models for placental vessel segmentation in fetoscopic images, demonstrating improved results with the cGAN-based approach. However, the challenge of achieving generalizability still needs to be addressed.
Collapse
Affiliation(s)
- Anouk van der Schot
- Obstetrics & Gynecology, Radboud University Medical Center, 6525 GA Nijmegen, The Netherlands
| | - Esther Sikkel
- Obstetrics & Gynecology, Radboud University Medical Center, 6525 GA Nijmegen, The Netherlands
| | - Marèll Niekolaas
- Obstetrics & Gynecology, Radboud University Medical Center, 6525 GA Nijmegen, The Netherlands
| | - Marc Spaanderman
- Obstetrics & Gynecology, Radboud University Medical Center, 6525 GA Nijmegen, The Netherlands
- Obstetrics & Gynecology, Maastricht University Medical Center, 6229 ER Maastricht, The Netherlands
- Department of GROW, School for Oncology and Reproduction, Maastricht University, 6229 ER Maastricht, The Netherlands
| | - Guido de Jong
- 3D Lab, Radboud University Medical Center, 6525 GA Nijmegen, The Netherlands
| |
Collapse
|
18
|
Kosasih DI, Lee BG, Lim H. Multichannel One-Dimensional Data Augmentation with Generative Adversarial Network. SENSORS (BASEL, SWITZERLAND) 2023; 23:7693. [PMID: 37765750 PMCID: PMC10536615 DOI: 10.3390/s23187693] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 08/31/2023] [Accepted: 09/02/2023] [Indexed: 09/29/2023]
Abstract
Data augmentation is one of the most important problems in deep learning. There have been many algorithms proposed to solve this problem, such as simple noise injection, the generative adversarial network (GAN), and diffusion models. However, to the best of our knowledge, these works mainly focused on computer vision-related tasks, and there have not been many proposed works for one-dimensional data. This paper proposes a GAN-based data augmentation for generating multichannel one-dimensional data given single-channel inputs. Our architecture consists of multiple discriminators that adapt deep convolution GAN (DCGAN) and patchGAN to extract the overall pattern of the multichannel generated data while also considering the local information of each channel. We conducted an experiment with website fingerprinting data. The result for the three channels' data augmentation showed that our proposed model obtained FID scores of 0.005,0.017,0.051 for each channel, respectively, compared to 0.458,0.551,0.521 when using the vanilla GAN.
Collapse
Affiliation(s)
| | | | - Hyotaek Lim
- Department of Computer Engineering, Dongseo University, Busan 47011, Republic of Korea (B.-G.L.)
| |
Collapse
|
19
|
Xu B, Fan Y, Liu J, Zhang G, Wang Z, Li Z, Guo W, Tang X. CHSNet: Automatic lesion segmentation network guided by CT image features for acute cerebral hemorrhage. Comput Biol Med 2023; 164:107334. [PMID: 37573720 DOI: 10.1016/j.compbiomed.2023.107334] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 07/28/2023] [Accepted: 08/07/2023] [Indexed: 08/15/2023]
Abstract
Stroke is a cerebrovascular disease that can lead to severe sequelae such as hemiplegia and mental retardation with a mortality rate of up to 40%. In this paper, we proposed an automatic segmentation network (CHSNet) to segment the lesions in cranial CT images based on the characteristics of acute cerebral hemorrhage images, such as high density, multi-scale, and variable location, and realized the three-dimensional (3D) visualization and localization of the cranial lesions after the segmentation was completed. To enhance the feature representation of high-density regions, and capture multi-scale and up-down information on the target location, we constructed a convolutional neural network with encoding-decoding backbone, Res-RCL module, Atrous Spatial Pyramid Pooling, and Attention Gate. We collected images of 203 patients with acute cerebral hemorrhage, constructed a dataset containing 5998 cranial CT slices, and conducted comparative and ablation experiments on the dataset to verify the effectiveness of our model. Our model achieved the best results on both test sets with different segmentation difficulties, test1: Dice = 0.918, IoU = 0.853, ASD = 0.476, RVE = 0.113; test2: Dice = 0.716, IoU = 0.604, ASD = 5.402, RVE = 1.079. Based on the segmentation results, we achieved 3D visualization and localization of hemorrhage in CT images of stroke patients. The study has important implications for clinical adjuvant diagnosis.
Collapse
Affiliation(s)
- Bohao Xu
- School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China
| | - Yingwei Fan
- School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China
| | - Jingming Liu
- Emergency Department, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China
| | - Guobin Zhang
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China
| | - Zhiping Wang
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Zhili Li
- BECHOICE (Beijing) Science and Technology Development Ltd., Beijing, 100050, China
| | - Wei Guo
- Emergency Department, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100070, China.
| | - Xiaoying Tang
- School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China.
| |
Collapse
|
20
|
Dos Santos DFD, de Faria PR, Travençolo BAN, do Nascimento MZ. Influence of Data Augmentation Strategies on the Segmentation of Oral Histological Images Using Fully Convolutional Neural Networks. J Digit Imaging 2023; 36:1608-1623. [PMID: 37012446 PMCID: PMC10406800 DOI: 10.1007/s10278-023-00814-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 03/01/2023] [Accepted: 03/03/2023] [Indexed: 04/05/2023] Open
Abstract
Segmentation of tumor regions in H &E-stained slides is an important task for a pathologist while diagnosing different types of cancer, including oral squamous cell carcinoma (OSCC). Histological image segmentation is often constrained by the availability of labeled training data since labeling histological images is a highly skilled, complex, and time-consuming task. Thus, data augmentation strategies become essential to train convolutional neural networks models to overcome the overfitting problem when only a few training samples are available. This paper proposes a new data augmentation strategy, named Random Composition Augmentation (RCAug), to train fully convolutional networks (FCN) to segment OSCC tumor regions in H &E-stained histological images. Given the input image and their corresponding label, a pipeline with a random composition of geometric, distortion, color transfer, and generative image transformations is executed on the fly. Experimental evaluations were performed using an FCN-based method to segment OSCC regions through a set of different data augmentation transformations. By using RCAug, we improved the FCN-based segmentation method from 0.51 to 0.81 of intersection-over-union (IOU) in a whole slide image dataset and from 0.65 to 0.69 of IOU in a tissue microarray images dataset.
Collapse
Affiliation(s)
- Dalí F D Dos Santos
- Faculty of Computer Science, Federal University of Uberlândia, Brazil and Institute of Biomedical Science, Federal University of Uberlândia, Uberlândia, Brazil.
| | - Paulo R de Faria
- Faculty of Computer Science, Federal University of Uberlândia, Brazil and Institute of Biomedical Science, Federal University of Uberlândia, Uberlândia, Brazil
| | - Bruno A N Travençolo
- Faculty of Computer Science, Federal University of Uberlândia, Brazil and Institute of Biomedical Science, Federal University of Uberlândia, Uberlândia, Brazil
| | - Marcelo Z do Nascimento
- Faculty of Computer Science, Federal University of Uberlândia, Brazil and Institute of Biomedical Science, Federal University of Uberlândia, Uberlândia, Brazil
| |
Collapse
|
21
|
Huang KW, Yang YR, Huang ZH, Liu YY, Lee SH. Retinal Vascular Image Segmentation Using Improved UNet Based on Residual Module. Bioengineering (Basel) 2023; 10:722. [PMID: 37370653 DOI: 10.3390/bioengineering10060722] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 06/01/2023] [Accepted: 06/12/2023] [Indexed: 06/29/2023] Open
Abstract
In recent years, deep learning technology for clinical diagnosis has progressed considerably, and the value of medical imaging continues to increase. In the past, clinicians evaluated medical images according to their individual expertise. In contrast, the application of artificial intelligence technology for automatic analysis and diagnostic assistance to support clinicians in evaluating medical information more efficiently has become an important trend. In this study, we propose a machine learning architecture designed to segment images of retinal blood vessels based on an improved U-Net neural network model. The proposed model incorporates a residual module to extract features more effectively, and includes a full-scale skip connection to combine low level details with high-level features at different scales. The results of an experimental evaluation show that the model was able to segment images of retinal vessels accurately. The proposed method also outperformed several existing models on the benchmark datasets DRIVE and ROSE, including U-Net, ResUNet, U-Net3+, ResUNet++, and CaraNet.
Collapse
Affiliation(s)
- Ko-Wei Huang
- Department of Electrical Engineering, National Kaohsiung University of Science and Technology, Kaohsiung 80778, Taiwan
| | - Yao-Ren Yang
- Department of Electrical Engineering, National Kaohsiung University of Science and Technology, Kaohsiung 80778, Taiwan
| | - Zih-Hao Huang
- Department of Electrical Engineering, National Kaohsiung University of Science and Technology, Kaohsiung 80778, Taiwan
| | - Yi-Yang Liu
- Department of Electrical Engineering, National Kaohsiung University of Science and Technology, Kaohsiung 80778, Taiwan
- Department of Urology, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung 83301, Taiwan
| | - Shih-Hsiung Lee
- Department of Intelligent Commerce, National Kaohsiung University of Science and Technology, Kaohsiung 82444, Taiwan
| |
Collapse
|
22
|
An Improved Co-Training and Generative Adversarial Network (Diff-CoGAN) for Semi-Supervised Medical Image Segmentation. INFORMATION 2023. [DOI: 10.3390/info14030190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/19/2023] Open
Abstract
Semi-supervised learning is a technique that utilizes a limited set of labeled data and a large amount of unlabeled data to overcome the challenges of obtaining a perfect dataset in deep learning, especially in medical image segmentation. The accuracy of the predicted labels for the unlabeled data is a critical factor that affects the training performance, thus reducing the accuracy of segmentation. To address this issue, a semi-supervised learning method based on the Diff-CoGAN framework was proposed, which incorporates co-training and generative adversarial network (GAN) strategies. The proposed Diff-CoGAN framework employs two generators and one discriminator. The generators work together by providing mutual information guidance to produce predicted maps that are more accurate and closer to the ground truth. To further improve segmentation accuracy, the predicted maps are subjected to an intersection operation to identify a high-confidence region of interest, which reduces boundary segmentation errors. The predicted maps are then fed into the discriminator, and the iterative process of adversarial training enhances the generators’ ability to generate more precise maps, while also improving the discriminator’s ability to distinguish between the predicted maps and the ground truth. This study conducted experiments on the Hippocampus and Spleen images from the Medical Segmentation Decathlon (MSD) dataset using three semi-supervised methods: co-training, semi-GAN, and Diff-CoGAN. The experimental results demonstrated that the proposed Diff-CoGAN approach significantly enhanced segmentation accuracy compared to the other two methods by benefiting on the mutual guidance of the two generators and the adversarial training between the generators and discriminator. The introduction of the intersection operation prior to the discriminator also further reduced boundary segmentation errors.
Collapse
|
23
|
Keaton MR, Zaveri RJ, Doretto G. CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation. IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION. IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION 2023; 2023:455-466. [PMID: 38170053 PMCID: PMC10760785 DOI: 10.1109/wacv56688.2023.00053] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs.
Collapse
|
24
|
Wang H, Chen X, Yu R, Wei Z, Yao T, Gao C, Li Y, Wang Z, Yi D, Wu Y. E-DU: Deep neural network for multimodal medical image segmentation based on semantic gap compensation. Comput Biol Med 2022; 151:106206. [PMID: 36395592 DOI: 10.1016/j.compbiomed.2022.106206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Revised: 09/08/2022] [Accepted: 10/09/2022] [Indexed: 12/27/2022]
Abstract
BACKGROUND U-Net includes encoder, decoder and skip connection structures. It has become the benchmark network in medical image segmentation. However, the direct fusion of low-level and high-level convolution features with semantic gaps by traditional skip connections may lead to problems such as fuzzy generated feature maps and target region segmentation errors. OBJECTIVE We use spatial enhancement filtering technology to compensate for the semantic gap and propose an enhanced dense U-Net (E-DU), aiming to apply it to multimodal medical image segmentation to improve the segmentation performance and efficiency. METHODS Before combining encoder and decoder features, we replace the traditional skip connection with a multiscale denoise enhancement (MDE) module. The encoder features need to be deeply convolved by the spatial enhancement filter and then combined with the decoder features. We propose a simple and efficient deep full convolution network structure E-DU, which can not only fuse semantically various features but also denoise and enhance the feature map. RESULTS We performed experiments on medical image segmentation datasets with seven image modalities and combined MDE with various baseline networks to perform ablation studies. E-DU achieved the best segmentation results on evaluation indicators such as DSC on the U-Net family, with DSC values of 97.78, 97.64, 95.31, 94.42, 94.93, 98.85, and 98.38 (%), respectively. The addition of the MDE module to the attention mechanism network improves segmentation performance and efficiency, reflecting its generalization performance. In comparison to advanced methods, our method is also competitive. CONCLUSION Our proposed MDE module has a good segmentation effect and operating efficiency, and it can be easily extended to multiple modal medical segmentation datasets. Our idea and method can achieve clinical multimodal medical image segmentation and make full use of image information to provide clinical decision support. It has great application value and promotion prospects.
Collapse
Affiliation(s)
- Haojia Wang
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Xicheng Chen
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Rui Yu
- Tactical Health Service Department, NCO School of Army Medical University, Zhongshanxi Road 450, Qiaoxi District, Shijiazhuang, 050081, China
| | - Zeliang Wei
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Tianhua Yao
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Chengcheng Gao
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Yang Li
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Zhenyan Wang
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China
| | - Dong Yi
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China.
| | - Yazhou Wu
- Department of Health Statistics, College of Preventive Medicine, Army Medical University, NO.30 Gaotanyan Street, Shapingba District, Chongqing, 400038, China.
| |
Collapse
|
25
|
Hamamoto R, Koyama T, Kouno N, Yasuda T, Yui S, Sudo K, Hirata M, Sunami K, Kubo T, Takasawa K, Takahashi S, Machino H, Kobayashi K, Asada K, Komatsu M, Kaneko S, Yatabe Y, Yamamoto N. Introducing AI to the molecular tumor board: one direction toward the establishment of precision medicine using large-scale cancer clinical and biological information. Exp Hematol Oncol 2022; 11:82. [PMID: 36316731 PMCID: PMC9620610 DOI: 10.1186/s40164-022-00333-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 10/05/2022] [Indexed: 11/10/2022] Open
Abstract
Since U.S. President Barack Obama announced the Precision Medicine Initiative in his New Year's State of the Union address in 2015, the establishment of a precision medicine system has been emphasized worldwide, particularly in the field of oncology. With the advent of next-generation sequencers specifically, genome analysis technology has made remarkable progress, and there are active efforts to apply genome information to diagnosis and treatment. Generally, in the process of feeding back the results of next-generation sequencing analysis to patients, a molecular tumor board (MTB), consisting of experts in clinical oncology, genetic medicine, etc., is established to discuss the results. On the other hand, an MTB currently involves a large amount of work, with humans searching through vast databases and literature, selecting the best drug candidates, and manually confirming the status of available clinical trials. In addition, as personalized medicine advances, the burden on MTB members is expected to increase in the future. Under these circumstances, introducing cutting-edge artificial intelligence (AI) technology and information and communication technology to MTBs while reducing the burden on MTB members and building a platform that enables more accurate and personalized medical care would be of great benefit to patients. In this review, we introduced the latest status of elemental technologies that have potential for AI utilization in MTB, and discussed issues that may arise in the future as we progress with AI implementation.
Collapse
Affiliation(s)
- Ryuji Hamamoto
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Takafumi Koyama
- grid.272242.30000 0001 2168 5385Department of Experimental Therapeutics, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| | - Nobuji Kouno
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.258799.80000 0004 0372 2033Department of Surgery, Graduate School of Medicine, Kyoto University, Yoshida-konoe-cho, Sakyo-ku, Kyoto, 606-8303 Japan
| | - Tomohiro Yasuda
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.417547.40000 0004 1763 9564Research and Development Group, Hitachi, Ltd., 1-280 Higashi-koigakubo, Kokubunji, Tokyo, 185-8601 Japan
| | - Shuntaro Yui
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.417547.40000 0004 1763 9564Research and Development Group, Hitachi, Ltd., 1-280 Higashi-koigakubo, Kokubunji, Tokyo, 185-8601 Japan
| | - Kazuki Sudo
- grid.272242.30000 0001 2168 5385Department of Experimental Therapeutics, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.272242.30000 0001 2168 5385Department of Medical Oncology, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| | - Makoto Hirata
- grid.272242.30000 0001 2168 5385Department of Genetic Medicine and Services, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| | - Kuniko Sunami
- grid.272242.30000 0001 2168 5385Department of Laboratory Medicine, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| | - Takashi Kubo
- grid.272242.30000 0001 2168 5385Department of Laboratory Medicine, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| | - Ken Takasawa
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Satoshi Takahashi
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Hidenori Machino
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Kazuma Kobayashi
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Ken Asada
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Masaaki Komatsu
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Syuzo Kaneko
- grid.272242.30000 0001 2168 5385Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.509456.bCancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027 Japan
| | - Yasushi Yatabe
- grid.272242.30000 0001 2168 5385Department of Diagnostic Pathology, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan ,grid.272242.30000 0001 2168 5385Division of Molecular Pathology, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| | - Noboru Yamamoto
- grid.272242.30000 0001 2168 5385Department of Experimental Therapeutics, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045 Japan
| |
Collapse
|
26
|
Shoeibi A, Moridian P, Khodatars M, Ghassemi N, Jafari M, Alizadehsani R, Kong Y, Gorriz JM, Ramírez J, Khosravi A, Nahavandi S, Acharya UR. An overview of deep learning techniques for epileptic seizures detection and prediction based on neuroimaging modalities: Methods, challenges, and future works. Comput Biol Med 2022; 149:106053. [DOI: 10.1016/j.compbiomed.2022.106053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2022] [Revised: 08/17/2022] [Accepted: 08/17/2022] [Indexed: 02/01/2023]
|
27
|
De Asis-Cruz J, Krishnamurthy D, Jose C, Cook KM, Limperopoulos C. FetalGAN: Automated Segmentation of Fetal Functional Brain MRI Using Deep Generative Adversarial Learning and Multi-Scale 3D U-Net. Front Neurosci 2022; 16:887634. [PMID: 35747213 PMCID: PMC9209698 DOI: 10.3389/fnins.2022.887634] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Accepted: 05/16/2022] [Indexed: 01/02/2023] Open
Abstract
An important step in the preprocessing of resting state functional magnetic resonance images (rs-fMRI) is the separation of brain from non-brain voxels. Widely used imaging tools such as FSL's BET2 and AFNI's 3dSkullStrip accomplish this task effectively in children and adults. In fetal functional brain imaging, however, the presence of maternal tissue around the brain coupled with the non-standard position of the fetal head limit the usefulness of these tools. Accurate brain masks are thus generated manually, a time-consuming and tedious process that slows down preprocessing of fetal rs-fMRI. Recently, deep learning-based segmentation models such as convolutional neural networks (CNNs) have been increasingly used for automated segmentation of medical images, including the fetal brain. Here, we propose a computationally efficient end-to-end generative adversarial neural network (GAN) for segmenting the fetal brain. This method, which we call FetalGAN, yielded whole brain masks that closely approximated the manually labeled ground truth. FetalGAN performed better than 3D U-Net model and BET2: FetalGAN, Dice score = 0.973 ± 0.013, precision = 0.977 ± 0.015; 3D U-Net, Dice score = 0.954 ± 0.054, precision = 0.967 ± 0.037; BET2, Dice score = 0.856 ± 0.084, precision = 0.758 ± 0.113. FetalGAN was also faster than 3D U-Net and the manual method (7.35 s vs. 10.25 s vs. ∼5 min/volume). To the best of our knowledge, this is the first successful implementation of 3D CNN with GAN on fetal fMRI brain images and represents a significant advance in fully automating processing of rs-MRI images.
Collapse
Affiliation(s)
- Josepheen De Asis-Cruz
- Developing Brain Institute, Department of Diagnostic Radiology, Children’s National Hospital, Washington, DC, United States
| | - Dhineshvikram Krishnamurthy
- Developing Brain Institute, Department of Diagnostic Radiology, Children’s National Hospital, Washington, DC, United States
| | - Chris Jose
- Department of Computer Science, University of Maryland, College Park, MD, United States
| | - Kevin M. Cook
- Developing Brain Institute, Department of Diagnostic Radiology, Children’s National Hospital, Washington, DC, United States
| | - Catherine Limperopoulos
- Developing Brain Institute, Department of Diagnostic Radiology, Children’s National Hospital, Washington, DC, United States
| |
Collapse
|
28
|
Holmes K, Sharma P, Fernandes S. Facial skin disease prediction using StarGAN v2 and transfer learning. INTELLIGENT DECISION TECHNOLOGIES 2022. [DOI: 10.3233/idt-228046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/22/2023]
Abstract
Deep learning algorithms have become the most prominent methods for medical image analysis over the past years, leading to enhanced performances in various medical applications. In this paper, we focus on applying intelligent skin disease detection to face images, where the crucial challenge is the low availability of training data. To achieve high disease detection and classification success rates, we adapt the state-of-the-art StarGAN v2 network to augment images of faces and combine it with a transfer learning approach. The experimental results show that the classification accuracies of transfer learning models are in the range of 77.46–99.80% when trained on datasets that are extended with StarGAN v2 augmented data.
Collapse
Affiliation(s)
- Kristen Holmes
- Department of Computer Science, Design and Journalism, Creighton University, NE, USA
| | - Poonam Sharma
- Department of Pathology, Creighton University, NE, USA
| | - Steven Fernandes
- Department of Computer Science, Design and Journalism, Creighton University, NE, USA
| |
Collapse
|