1
|
Tang Y, Lyu T, Jin H, Du Q, Wang J, Li Y, Li M, Chen Y, Zheng J. Domain adaptive noise reduction with iterative knowledge transfer and style generalization learning. Med Image Anal 2024; 98:103327. [PMID: 39191093 DOI: 10.1016/j.media.2024.103327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 08/20/2024] [Accepted: 08/21/2024] [Indexed: 08/29/2024]
Abstract
Low-dose computed tomography (LDCT) denoising tasks face significant challenges in practical imaging scenarios. Supervised methods encounter difficulties in real-world scenarios as there are no paired data for training. Moreover, when applied to datasets with varying noise patterns, these methods may experience decreased performance owing to the domain gap. Conversely, unsupervised methods do not require paired data and can be directly trained on real-world data. However, they often exhibit inferior performance compared to supervised methods. To address this issue, it is necessary to leverage the strengths of these supervised and unsupervised methods. In this paper, we propose a novel domain adaptive noise reduction framework (DANRF), which integrates both knowledge transfer and style generalization learning to effectively tackle the domain gap problem. Specifically, an iterative knowledge transfer method with knowledge distillation is selected to train the target model using unlabeled target data and a pre-trained source model trained with paired simulation data. Meanwhile, we introduce the mean teacher mechanism to update the source model, enabling it to adapt to the target domain. Furthermore, an iterative style generalization learning process is also designed to enrich the style diversity of the training dataset. We evaluate the performance of our approach through experiments conducted on multi-source datasets. The results demonstrate the feasibility and effectiveness of our proposed DANRF model in multi-source LDCT image processing tasks. Given its hybrid nature, which combines the advantages of supervised and unsupervised learning, and its ability to bridge domain gaps, our approach is well-suited for improving practical low-dose CT imaging in clinical settings. Code for our proposed approach is publicly available at https://github.com/tyfeiii/DANRF.
Collapse
Affiliation(s)
- Yufei Tang
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Tianling Lyu
- Research Center of Augmented Intelligence, Zhejiang Lab, Hangzhou, 310000, China
| | - Haoyang Jin
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Qiang Du
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Jiping Wang
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Yunxiang Li
- Nanovision Technology Co., Ltd., Beiqing Road, Haidian District, Beijing, 100094, China
| | - Ming Li
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China.
| | - Yang Chen
- Laboratory of Image Science and Technology, the School of Computer Science and Engineering, Southeast University, Nanjing, 210096, China
| | - Jian Zheng
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Shandong Laboratory of Advanced Biomaterials and Medical Devices in Weihai, Weihai, 264200, China.
| |
Collapse
|
2
|
Santhirasekaram A, Winkler M, Rockall A, Glocker B. A geometric approach to robust medical image segmentation. Med Image Anal 2024; 97:103260. [PMID: 38970862 DOI: 10.1016/j.media.2024.103260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2023] [Revised: 06/12/2024] [Accepted: 06/26/2024] [Indexed: 07/08/2024]
Abstract
Robustness of deep learning segmentation models is crucial for their safe incorporation into clinical practice. However, these models can falter when faced with distributional changes. This challenge is evident in magnetic resonance imaging (MRI) scans due to the diverse acquisition protocols across various domains, leading to differences in image characteristics such as textural appearances. We posit that the restricted anatomical differences between subjects could be harnessed to refine the latent space into a set of shape components. The learned set then aims to encompass the relevant anatomical shape variation found within the patient population. We explore this by utilising multiple MRI sequences to learn texture invariant and shape equivariant features which are used to construct a shape dictionary using vector quantisation. We investigate shape equivariance to a number of different types of groups. We hypothesise and prove that the greater the group order, i.e., the denser the constraint, the better becomes the model robustness. We achieve shape equivariance either with a contrastive based approach or by imposing equivariant constraints on the convolutional kernels. The resulting shape equivariant dictionary is then sampled to compose the segmentation output. Our method achieves state-of-the-art performance for the task of single domain generalisation for prostate and cardiac MRI segmentation. Code is available at https://github.com/AinkaranSanthi/A_Geometric_Perspective_For_Robust_Segmentation.
Collapse
Affiliation(s)
| | - Mathias Winkler
- Department of Surgery and Cancer, Imperial College London, United Kingdom
| | - Andrea Rockall
- Department of Surgery and Cancer, Imperial College London, United Kingdom
| | - Ben Glocker
- Department of Computing, Imperial College London, United Kingdom
| |
Collapse
|
3
|
Hallaj S, Chuter BG, Lieu AC, Singh P, Kalpathy-Cramer J, Xu BY, Christopher M, Zangwill LM, Weinreb RN, Baxter SL. Federated Learning in Glaucoma: A Comprehensive Review and Future Perspectives. Ophthalmol Glaucoma 2024:S2589-4196(24)00143-1. [PMID: 39214457 DOI: 10.1016/j.ogla.2024.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Revised: 08/20/2024] [Accepted: 08/23/2024] [Indexed: 09/04/2024]
Abstract
Current approaches to developing artificial intelligence (AI) models for widespread glaucoma screening have encountered several obstacles. First, glaucoma is a complex condition with a wide range of morphological and clinical presentations. There exists no consensus definition of glaucoma or glaucomatous optic neuropathy. Further, training effective deep learning algorithms poses numerous challenges, including susceptibility to overfitting and lack of generalizability on external data. Therefore, training data should ideally be sourced from large, well-curated, multi-client cohorts to ensure diversity in patient populations, disease presentations, and imaging protocols. However, the construction of centralized repositories for multimodal data faces hurdles such as concerns regarding data sharing, re-identification, storage, regulations, patient privacy, and intellectual property. Federated learning (FL) has emerged as a proposed solution to address some of these concerns by enabling data to remain locally hosted while facilitating distributed model training. This article aims to provide a comprehensive review of the existing literature on FL in the context of its applications for AI tasks related to glaucoma.
Collapse
Affiliation(s)
- Shahin Hallaj
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92037, USA
| | - Benton G Chuter
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92037, USA
| | - Alexander C Lieu
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92037, USA
| | - Praveer Singh
- Division of Artificial Medical Intelligence, Department of Ophthalmology, University of Colorado School of Medicine, Aurora, CO, USA; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Jayashree Kalpathy-Cramer
- Division of Artificial Medical Intelligence, Department of Ophthalmology, University of Colorado School of Medicine, Aurora, CO, USA
| | - Benjamin Y Xu
- Roski Eye Institute, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
| | - Mark Christopher
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA
| | - Linda M Zangwill
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA
| | - Robert N Weinreb
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA
| | - Sally L Baxter
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037, USA; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92037, USA.
| |
Collapse
|
4
|
Shao Y, Tian N, Li X, Zhang Q, Zhao W. Mask-Shift-Inference: A novel paradigm for domain generalization. Neural Netw 2024; 179:106629. [PMID: 39153401 DOI: 10.1016/j.neunet.2024.106629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 04/12/2024] [Accepted: 08/10/2024] [Indexed: 08/19/2024]
Abstract
Domain Generalization (DG) focuses on the Out-Of-Distribution (OOD) generalization, which is able to learn a robust model that generalizes the knowledge acquired from the source domain to the unseen target domain. However, due to the existence of the domain shift, domain-invariant representation learning is challenging. Guided by fine-grained knowledge, we propose a novel paradigm Mask-Shift-Inference (MSI) for DG based on the architecture of Convolutional Neural Networks (CNN). Different from relying on a series of constraints and assumptions for model optimization, this paradigm novelly shifts the focus to feature channels in the latent space for domain-invariant representation learning. We put forward a two-branch working mode of a main module and multiple domain-specific sub-modules. The latter can only achieve good prediction performance in its own specific domain but poor predictions in other source domains, which provides the main module with the fine-grained knowledge guidance and contributes to the improvement of the cognitive ability of MSI. Firstly, during the forward propagation of the main module, the proposed MSI accurately discards unstable channels based on spurious classifications varying across domains, which have domain-specific prediction limitations and are not conducive to generalization. In this process, a progressive scheme is adopted to adaptively increase the masking ratio according to the training progress to further reduce the risk of overfitting. Subsequently, our paradigm enters the compatible shifting stage before the formal prediction. Based on maximizing semantic retention, we implement the domain style matching and shifting through the simple transformation through Fourier transform, which can explicitly and safely shift the target domain back to the source domain whose style is closest to it, requiring no additional model updates and reducing the domain gap. Eventually, the paradigm MSI enters the formal inference stage. The updated target domain is predicted in the main module trained in the previous stage with the benefit of familiar knowledge from the nearest source domain masking scheme. Our paradigm is logically progressive, which can intuitively exclude the confounding influence of domain-specific spurious information along with mitigating domain shifts and implicitly perform semantically invariant representation learning, achieving robust OOD generalization. Extensive experimental results on PACS, VLCS, Office-Home and DomainNet datasets verify the superiority and effectiveness of the proposed method.
Collapse
Affiliation(s)
- Youjia Shao
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao Shandong 266061, China
| | - Na Tian
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao Shandong 266061, China
| | - Xinyi Li
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao Shandong 266061, China
| | - Qinghao Zhang
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao Shandong 266061, China
| | - Wencang Zhao
- College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao Shandong 266061, China; Qingdao Institute of Intelligent Navigation and Control, Qingdao Shandong 266071, China.
| |
Collapse
|
5
|
Huang L, Zhang N, Yi Y, Zhou W, Zhou B, Dai J, Wang J. SAMCF: Adaptive global style alignment and multi-color spaces fusion for joint optic cup and disc segmentation. Comput Biol Med 2024; 178:108639. [PMID: 38878394 DOI: 10.1016/j.compbiomed.2024.108639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2024] [Revised: 04/21/2024] [Accepted: 05/18/2024] [Indexed: 07/24/2024]
Abstract
The optic cup (OC) and optic disc (OD) are two critical structures in retinal fundus images, and their relative positions and sizes are essential for effectively diagnosing eye diseases. With the success of deep learning in computer vision, deep learning-based segmentation models have been widely used for joint optic cup and disc segmentation. However, there are three prominent issues that impact the segmentation performance. First, significant differences among datasets collecting from various institutions, protocols, and devices lead to performance degradation of models. Second, we find that images with only RGB information struggle to counteract the interference caused by brightness variations, affecting color representation capability. Finally, existing methods typically ignored the edge perception, facing the challenges in obtaining clear and smooth edge segmentation results. To address these drawbacks, we propose a novel framework based on Style Alignment and Multi-Color Fusion (SAMCF) for joint OC and OD segmentation. Initially, we introduce a domain generalization method to generate uniformly styled images without damaged image content for mitigating domain shift issues. Next, based on multiple color spaces, we propose a feature extraction and fusion network aiming to handle brightness variation interference and improve color representation capability. Lastly, an edge aware loss is designed to generate fine edge segmentation results. Our experiments conducted on three public datasets, DGS, RIM, and REFUGE, demonstrate that our proposed SAMCF achieves superior performance to existing state-of-the-art methods. Moreover, SAMCF exhibits remarkable generalization ability across multiple retinal fundus image datasets, showcasing its outstanding generality.
Collapse
Affiliation(s)
- Longjun Huang
- School of Software, Nanchang Key Laboratory for Blindness and Visual Impairment Prevention Technology and Equipment, Jiangxi Normal University, Nanchang, 330022, China
| | - Ningyi Zhang
- School of Software, Nanchang Key Laboratory for Blindness and Visual Impairment Prevention Technology and Equipment, Jiangxi Normal University, Nanchang, 330022, China
| | - Yugen Yi
- School of Software, Nanchang Key Laboratory for Blindness and Visual Impairment Prevention Technology and Equipment, Jiangxi Normal University, Nanchang, 330022, China.
| | - Wei Zhou
- College of Computer Science, Shenyang Aerospace University, Shenyang, 110136, China
| | - Bin Zhou
- School of Software, Nanchang Key Laboratory for Blindness and Visual Impairment Prevention Technology and Equipment, Jiangxi Normal University, Nanchang, 330022, China
| | - Jiangyan Dai
- School of Computer Engineering, Weifang University, 261061, China.
| | - Jianzhong Wang
- College of Information Science and Technology, Northeast Normal University, Changchun, 130117, China
| |
Collapse
|
6
|
Ni R, Han K, Haibe-Kains B, Rink A. Generalizability of deep learning in organ-at-risk segmentation: A transfer learning study in cervical brachytherapy. Radiother Oncol 2024; 197:110332. [PMID: 38763356 DOI: 10.1016/j.radonc.2024.110332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 05/02/2024] [Accepted: 05/03/2024] [Indexed: 05/21/2024]
Abstract
PURPOSE Deep learning can automate delineation in radiation therapy, reducing time and variability. Yet, its efficacy varies across different institutions, scanners, or settings, emphasizing the need for adaptable and robust models in clinical environments. Our study demonstrates the effectiveness of the transfer learning (TL) approach in enhancing the generalizability of deep learning models for auto-segmentation of organs-at-risk (OARs) in cervical brachytherapy. METHODS A pre-trained model was developed using 120 scans with ring and tandem applicator on a 3T magnetic resonance (MR) scanner (RT3). Four OARs were segmented and evaluated. Segmentation performance was evaluated by Volumetric Dice Similarity Coefficient (vDSC), 95 % Hausdorff Distance (HD95), surface DSC, and Added Path Length (APL). The model was fine-tuned on three out-of-distribution target groups. Pre- and post-TL outcomes, and influence of number of fine-tuning scans, were compared. A model trained with one group (Single) and a model trained with all four groups (Mixed) were evaluated on both seen and unseen data distributions. RESULTS TL enhanced segmentation accuracy across target groups, matching the pre-trained model's performance. The first five fine-tuning scans led to the most noticeable improvements, with performance plateauing with more data. TL outperformed training-from-scratch given the same training data. The Mixed model performed similarly to the Single model on RT3 scans but demonstrated superior performance on unseen data. CONCLUSIONS TL can improve a model's generalizability for OAR segmentation in MR-guided cervical brachytherapy, requiring less fine-tuning data and reduced training time. These results provide a foundation for developing adaptable models to accommodate clinical settings.
Collapse
Affiliation(s)
- Ruiyan Ni
- Department of Medical Biophysics, University of Toronto, Toronto, Canada
| | - Kathy Han
- Princess Margaret Cancer Center, University Health Network, Toronto, CA, Canada; Department of Radiation Oncology, University of Toronto, Toronto, CA, Canada
| | - Benjamin Haibe-Kains
- Department of Medical Biophysics, University of Toronto, Toronto, Canada; Princess Margaret Cancer Center, University Health Network, Toronto, CA, Canada; Vector Institute, Toronto, Toronto, CA, Canada.
| | - Alexandra Rink
- Department of Medical Biophysics, University of Toronto, Toronto, Canada; Princess Margaret Cancer Center, University Health Network, Toronto, CA, Canada; Department of Radiation Oncology, University of Toronto, Toronto, CA, Canada.
| |
Collapse
|
7
|
Liu W, Zhang B, Liu T, Jiang J, Liu Y. Artificial Intelligence in Pancreatic Image Analysis: A Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:4749. [PMID: 39066145 PMCID: PMC11280964 DOI: 10.3390/s24144749] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2024] [Revised: 07/15/2024] [Accepted: 07/16/2024] [Indexed: 07/28/2024]
Abstract
Pancreatic cancer is a highly lethal disease with a poor prognosis. Its early diagnosis and accurate treatment mainly rely on medical imaging, so accurate medical image analysis is especially vital for pancreatic cancer patients. However, medical image analysis of pancreatic cancer is facing challenges due to ambiguous symptoms, high misdiagnosis rates, and significant financial costs. Artificial intelligence (AI) offers a promising solution by relieving medical personnel's workload, improving clinical decision-making, and reducing patient costs. This study focuses on AI applications such as segmentation, classification, object detection, and prognosis prediction across five types of medical imaging: CT, MRI, EUS, PET, and pathological images, as well as integrating these imaging modalities to boost diagnostic accuracy and treatment efficiency. In addition, this study discusses current hot topics and future directions aimed at overcoming the challenges in AI-enabled automated pancreatic cancer diagnosis algorithms.
Collapse
Affiliation(s)
- Weixuan Liu
- Sydney Smart Technology College, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China; (W.L.); (B.Z.)
| | - Bairui Zhang
- Sydney Smart Technology College, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China; (W.L.); (B.Z.)
| | - Tao Liu
- School of Mathematics and Statistics, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China;
| | - Juntao Jiang
- College of Control Science and Engineering, Zhejiang University, Hangzhou 310058, China
| | - Yong Liu
- College of Control Science and Engineering, Zhejiang University, Hangzhou 310058, China
| |
Collapse
|
8
|
Li X, Jia L, Lin F, Chai F, Liu T, Zhang W, Wei Z, Xiong W, Li H, Zhang M, Wang Y. Synthetic CT generation for pelvic cases based on deep learning in multi-center datasets. Radiat Oncol 2024; 19:89. [PMID: 38982452 PMCID: PMC11232325 DOI: 10.1186/s13014-024-02467-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Accepted: 06/10/2024] [Indexed: 07/11/2024] Open
Abstract
BACKGROUND AND PURPOSE To investigate the feasibility of synthesizing computed tomography (CT) images from magnetic resonance (MR) images in multi-center datasets using generative adversarial networks (GANs) for rectal cancer MR-only radiotherapy. MATERIALS AND METHODS Conventional T2-weighted MR and CT images were acquired from 90 rectal cancer patients at Peking University People's Hospital and 19 patients in public datasets. This study proposed a new model combining contrastive learning loss and consistency regularization loss to enhance the generalization of model for multi-center pelvic MRI-to-CT synthesis. The CT-to-sCT image similarity was evaluated by computing the mean absolute error (MAE), peak signal-to-noise ratio (SNRpeak), structural similarity index (SSIM) and Generalization Performance (GP). The dosimetric accuracy of synthetic CT was verified against CT-based dose distributions for the photon plan. Relative dose differences in the planning target volume and organs at risk were computed. RESULTS Our model presented excellent generalization with a GP of 0.911 on unseen datasets and outperformed the plain CycleGAN, where MAE decreased from 47.129 to 42.344, SNRpeak improved from 25.167 to 26.979, SSIM increased from 0.978 to 0.992. The dosimetric analysis demonstrated that most of the relative differences in dose and volume histogram (DVH) indicators between synthetic CT and real CT were less than 1%. CONCLUSION The proposed model can generate accurate synthetic CT in multi-center datasets from T2w-MR images. Most dosimetric differences were within clinically acceptable criteria for photon radiotherapy, demonstrating the feasibility of an MRI-only workflow for patients with rectal cancer.
Collapse
Affiliation(s)
- Xianan Li
- Peking University People's Hospital, Beijing, China
| | - Lecheng Jia
- Shenzhen United Imaging Research Institute of Innovative Medical Equipment, Shenzhen, China.
- Zhejiang Engineering Research Center for Innovation and Application of Intelligent Radiotherapy Technology, Wenzhou, China.
| | - Fengyu Lin
- Shenzhen United Imaging Research Institute of Innovative Medical Equipment, Shenzhen, China
| | - Fan Chai
- Peking University People's Hospital, Beijing, China
| | - Tao Liu
- Peking University People's Hospital, Beijing, China
| | - Wei Zhang
- Shanghai United Imaging Healthcare Co., Ltd. Shanghai, Shanghai, China
| | - Ziquan Wei
- Shenzhen United Imaging Research Institute of Innovative Medical Equipment, Shenzhen, China
| | - Weiqi Xiong
- Shanghai United Imaging Healthcare Co., Ltd. Shanghai, Shanghai, China
| | - Hua Li
- Shenzhen United Imaging Research Institute of Innovative Medical Equipment, Shenzhen, China
| | - Min Zhang
- Peking University People's Hospital, Beijing, China
| | - Yi Wang
- Peking University People's Hospital, Beijing, China.
| |
Collapse
|
9
|
Zhang L, Ning G, Liang H, Han B, Liao H. One-shot neuroanatomy segmentation through online data augmentation and confidence aware pseudo label. Med Image Anal 2024; 95:103182. [PMID: 38688039 DOI: 10.1016/j.media.2024.103182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 11/26/2023] [Accepted: 04/18/2024] [Indexed: 05/02/2024]
Abstract
Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg.
Collapse
Affiliation(s)
- Liutong Zhang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Guochen Ning
- School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Hanying Liang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Boxuan Han
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China; School of Biomedical Engineering, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
10
|
Hu D, Li H, Liu H, Oguz I. Domain generalization for retinal vessel segmentation via Hessian-based vector field. Med Image Anal 2024; 95:103164. [PMID: 38615431 DOI: 10.1016/j.media.2024.103164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 03/27/2024] [Accepted: 04/02/2024] [Indexed: 04/16/2024]
Abstract
Blessed by vast amounts of data, learning-based methods have achieved remarkable performance in countless tasks in computer vision and medical image analysis. Although these deep models can simulate highly nonlinear mapping functions, they are not robust with regard to the domain shift of input data. This is a significant concern that impedes the large-scale deployment of deep models in medical images since they have inherent variation in data distribution due to the lack of imaging standardization. Therefore, researchers have explored many domain generalization (DG) methods to alleviate this problem. In this work, we introduce a Hessian-based vector field that can effectively model the tubular shape of vessels, which is an invariant feature for data across various distributions. The vector field serves as a good embedding feature to take advantage of the self-attention mechanism in a vision transformer. We design paralleled transformer blocks that stress the local features with different scales. Furthermore, we present a novel data augmentation method that introduces perturbations in image style while the vessel structure remains unchanged. In experiments conducted on public datasets of different modalities, we show that our model achieves superior generalizability compared with the existing algorithms. Our code and trained model are publicly available at https://github.com/MedICL-VU/Vector-Field-Transformer.
Collapse
Affiliation(s)
- Dewei Hu
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Hao Li
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Han Liu
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA
| | - Ipek Oguz
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA; Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA.
| |
Collapse
|
11
|
Jain S, Dhir R, Sikka G. View adaptive unified self-supervised technique for abdominal organ segmentation. Comput Biol Med 2024; 177:108659. [PMID: 38823366 DOI: 10.1016/j.compbiomed.2024.108659] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 03/05/2024] [Accepted: 05/24/2024] [Indexed: 06/03/2024]
Abstract
Automatic abdominal organ segmentation is an essential prerequisite for accurate volumetric analysis, disease diagnosis, and tracking by medical practitioners. However, the deformable shapes, variable locations, overlapping with nearby organs, and similar contrast make the segmentation challenging. Moreover, the requirement of a large manually labeled dataset makes it harder. Hence, a semi-supervised contrastive learning approach is utilized to perform the automatic abdominal organ segmentation. Existing 3D deep learning models based on contrastive learning are not able to capture the 3D context of medical volumetric data along three planes/views: axial, sagittal, and coronal views. In this work, a semi-supervised view-adaptive unified model (VAU-model) is proposed to make the 3D deep learning model as view-adaptive to learn 3D context along each view in a unified manner. This method utilizes the novel optimization function that assists the 3D model to learn the 3D context of volumetric medical data along each view in a single model. The effectiveness of the proposed approach is validated on the three types of datasets: BTCV, NIH, and MSD quantitatively and qualitatively. The results demonstrate that the VAU model achieves an average Dice score of 81.61% which is a 3.89% improvement compared to the previous best results for pancreas segmentation in multi-organ dataset BTCV. It also achieves an average Dice score of 77.76% and 76.76% for the pancreas under the single organ non-pathological NIH dataset, and pathological MSD dataset.
Collapse
Affiliation(s)
- Suchi Jain
- Computer Science and Engineering, Dr. B.R. Ambedkar National Institute of Technology, Jalandhar, Punjab, 144008, India.
| | - Renu Dhir
- Computer Science and Engineering, Dr. B.R. Ambedkar National Institute of Technology, Jalandhar, Punjab, 144008, India
| | - Geeta Sikka
- Computer Science and Engineering, National Institute of Technology, Delhi, 110036, India
| |
Collapse
|
12
|
Lteif D, Sreerama S, Bargal SA, Plummer BA, Au R, Kolachalama VB. Disease-driven domain generalization for neuroimaging-based assessment of Alzheimer's disease. Hum Brain Mapp 2024; 45:e26707. [PMID: 38798082 PMCID: PMC11128757 DOI: 10.1002/hbm.26707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 04/22/2024] [Accepted: 04/25/2024] [Indexed: 05/29/2024] Open
Abstract
Development of deep learning models to evaluate structural brain changes caused by cognitive impairment in MRI scans holds significant translational value. The efficacy of these models often encounters challenges due to variabilities arising from different data generation protocols, imaging equipment, radiological artifacts, and shifts in demographic distributions. Domain generalization (DG) techniques show promise in addressing these challenges by enabling the model to learn from one or more source domains and apply this knowledge to new, unseen target domains. Here we present a framework that utilizes model interpretability to enhance the generalizability of classification models across various cohorts. We used MRI scans and clinical diagnoses from four independent cohorts: Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 1821), the Framingham Heart Study (FHS, n = 304), the Australian Imaging Biomarkers & Lifestyle Study of Ageing (AIBL, n = 661), and the National Alzheimer's Coordinating Center (NACC, n = 4647). With this data, we trained a deep neural network to focus on areas of the brain identified as relevant to the disease for model training. Our approach involved training a classifier to differentiate between structural neurodegeneration in individuals with normal cognition (NC), mild cognitive impairment (MCI), and dementia due to Alzheimer's disease (AD). This was achieved by aligning class-wise attention with a unified visual saliency prior, which was computed offline for each class using all the training data. Our method not only competes with state-of-the-art approaches but also shows improved correlation with postmortem histology. This alignment with the gold standard evidence is a significant step towards validating the effectiveness of DG frameworks, paving the way for their broader application in the field.
Collapse
Affiliation(s)
- Diala Lteif
- Department of Computer ScienceBoston UniversityBostonMassachusettsUSA
- Department of MedicineBoston University Chobanian & Avedisian School of MedicineBostonMassachusettsUSA
| | - Sandeep Sreerama
- Department of MedicineBoston University Chobanian & Avedisian School of MedicineBostonMassachusettsUSA
| | - Sarah A. Bargal
- Department of Computer ScienceGeorgetown UniversityWashingtonDCUSA
| | - Bryan A. Plummer
- Department of Computer ScienceBoston UniversityBostonMassachusettsUSA
| | - Rhoda Au
- Department of MedicineBoston University Chobanian & Avedisian School of MedicineBostonMassachusettsUSA
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMassachusettsUSA
- Department of NeurologyBoston University Chobanian & Avedisian School of MedicineBostonMassachusettsUSA
- Department of EpidemiologyBoston University School of Public HealthBostonMassachusettsUSA
- The Framingham Heart StudyBostonMassachusettsUSA
- Boston University Alzheimer's Disease Research CenterBostonMassachusettsUSA
| | - Vijaya B. Kolachalama
- Department of Computer ScienceBoston UniversityBostonMassachusettsUSA
- Department of MedicineBoston University Chobanian & Avedisian School of MedicineBostonMassachusettsUSA
- Boston University Alzheimer's Disease Research CenterBostonMassachusettsUSA
- Faculty of Computing & Data SciencesBoston UniversityBostonMassachusettsUSA
| |
Collapse
|
13
|
Liu X, Sanchez P, Thermos S, O'Neil AQ, Tsaftaris SA. Compositionally Equivariant Representation Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2169-2179. [PMID: 38277249 DOI: 10.1109/tmi.2024.3358955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/28/2024]
Abstract
Deep learning models often need sufficient supervision (i.e., labelled data) in order to be trained effectively. By contrast, humans can swiftly learn to identify important anatomy in medical images like MRI and CT scans, with minimal guidance. This recognition capability easily generalises to new images from different medical facilities and to new tasks in different settings. This rapid and generalisable learning ability is largely due to the compositional structure of image patterns in the human brain, which are not well represented in current medical models. In this paper, we study the utilisation of compositionality in learning more interpretable and generalisable representations for medical image segmentation. Overall, we propose that the underlying generative factors that are used to generate the medical images satisfy compositional equivariance property, where each factor is compositional (e.g., corresponds to human anatomy) and also equivariant to the task. Hence, a good representation that approximates well the ground truth factor has to be compositionally equivariant. By modelling the compositional representations with learnable von-Mises-Fisher (vMF) kernels, we explore how different design and learning biases can be used to enforce the representations to be more compositionally equivariant under un-, weakly-, and semi-supervised settings. Extensive results show that our methods achieve the best performance over several strong baselines on the task of semi-supervised domain-generalised medical image segmentation. Code will be made publicly available upon acceptance at https://github.com/vios-s.
Collapse
|
14
|
Han X, Chen Z, Lin G, Lv W, Zheng C, Lu W, Sun Y, Lu L. Semi-supervised model based on implicit neural representation and mutual learning (SIMN) for multi-center nasopharyngeal carcinoma segmentation on MRI. Comput Biol Med 2024; 175:108368. [PMID: 38663351 DOI: 10.1016/j.compbiomed.2024.108368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 03/06/2024] [Accepted: 03/24/2024] [Indexed: 05/15/2024]
Abstract
BACKGROUND The issue of using deep learning to obtain accurate gross tumor volume (GTV) and metastatic lymph nodes (MLN) segmentation for nasopharyngeal carcinoma (NPC) on heterogeneous magnetic resonance imaging (MRI) images with limited labeling remains unsolved. METHOD We collected 918 patients with MRI images from three hospitals to develop and validate models and proposed a semi-supervised framework for the fine delineation of multi-center NPC boundaries by integrating uncertainty-based implicit neural representations named SIMN. The framework utilizes the deep mutual learning approach with CNN and Transformer, incorporating dynamic thresholds. Additionally, domain adaptive algorithms are employed to enhance the performance. RESULTS SIMN predictions have a high overlap ratio with the ground truth. Under the 20 % labeled cases, for the internal test cohorts, the average DSC in GTV and MLN are 0.7981 and 0.7804, respectively; for external test cohort Wu Zhou Red Cross Hospital, the average DSC in GTV and MLN are 0.7217 and 0.7581, respectively; for external test cohorts First People Hospital of Foshan, the average DSC in GTV and MLN are 0.7004 and 0.7692, respectively. No significant differences are found in DSC, HD95, ASD, and Recall for patients with different clinical categories. Moreover, SIMN outperformed existing classical semi-supervised methods. CONCLUSIONS SIMN showed a highly accurate GTV and MLN segmentation for NPC on multi-center MRI images under Semi-Supervised Learning (SSL), which can easily transfer to other centers without fine-tuning. It suggests that it has the potential to act as a generalized delineation solution for heterogeneous MRI images with limited labels in clinical deployment.
Collapse
Affiliation(s)
- Xu Han
- School of Biomedical Engineering, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Pazhou Lab, Guangzhou, 510515, China
| | - Zihang Chen
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, 510060, China
| | - Guoyu Lin
- School of Biomedical Engineering, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, 510060, China
| | - Wenbing Lv
- School of Information Science and Engineering, Yunnan University, Kunming, 650504, China
| | - Chundan Zheng
- School of Biomedical Engineering, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Pazhou Lab, Guangzhou, 510515, China
| | - Wantong Lu
- School of Biomedical Engineering, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Pazhou Lab, Guangzhou, 510515, China
| | - Ying Sun
- Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, 510060, China.
| | - Lijun Lu
- School of Biomedical Engineering, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology, Southern Medical University, 1023 Shatai Road, Guangzhou, 510515, China; Pazhou Lab, Guangzhou, 510515, China.
| |
Collapse
|
15
|
Alrashdi I. Fog-based deep learning framework for real-time pandemic screening in smart cities from multi-site tomographies. BMC Med Imaging 2024; 24:123. [PMID: 38797827 PMCID: PMC11129417 DOI: 10.1186/s12880-024-01302-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Accepted: 05/20/2024] [Indexed: 05/29/2024] Open
Abstract
The quick proliferation of pandemic diseases has been imposing many concerns on the international health infrastructure. To combat pandemic diseases in smart cities, Artificial Intelligence of Things (AIoT) technology, based on the integration of artificial intelligence (AI) with the Internet of Things (IoT), is commonly used to promote efficient control and diagnosis during the outbreak, thereby minimizing possible losses. However, the presence of multi-source institutional data remains one of the major challenges hindering the practical usage of AIoT solutions for pandemic disease diagnosis. This paper presents a novel framework that utilizes multi-site data fusion to boost the accurateness of pandemic disease diagnosis. In particular, we focus on a case study of COVID-19 lesion segmentation, a crucial task for understanding disease progression and optimizing treatment strategies. In this study, we propose a novel multi-decoder segmentation network for efficient segmentation of infections from cross-domain CT scans in smart cities. The multi-decoder segmentation network leverages data from heterogeneous domains and utilizes strong learning representations to accurately segment infections. Performance evaluation of the multi-decoder segmentation network was conducted on three publicly accessible datasets, demonstrating robust results with an average dice score of 89.9% and an average surface dice of 86.87%. To address scalability and latency issues associated with centralized cloud systems, fog computing (FC) emerges as a viable solution. FC brings resources closer to the operator, offering low latency and energy-efficient data management and processing. In this context, we propose a unique FC technique called PANDFOG to deploy the multi-decoder segmentation network on edge nodes for practical and clinical applications of automated COVID-19 pneumonia analysis. The results of this study highlight the efficacy of the multi-decoder segmentation network in accurately segmenting infections from cross-domain CT scans. Moreover, the proposed PANDFOG system demonstrates the practical deployment of the multi-decoder segmentation network on edge nodes, providing real-time access to COVID-19 segmentation findings for improved patient monitoring and clinical decision-making.
Collapse
Affiliation(s)
- Ibrahim Alrashdi
- Department of Computer Science, College of Computer and Information Sciences, Jouf University, 72388, Sakaka, Aljouf, Saudi Arabia.
| |
Collapse
|
16
|
Cheng Z, Wang S, Gao Y, Zhu Z, Yan C. Invariant Content Representation for Generalizable Medical Image Segmentation. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01088-9. [PMID: 38758420 DOI: 10.1007/s10278-024-01088-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 01/20/2024] [Accepted: 02/09/2024] [Indexed: 05/18/2024]
Abstract
Domain generalization (DG) for medical image segmentation due to privacy preservation prefers learning from a single-source domain and expects good robustness on unseen target domains. To achieve this goal, previous methods mainly use data augmentation to expand the distribution of samples and learn invariant content from them. However, most of these methods commonly perform global augmentation, leading to limited augmented sample diversity. In addition, the style of the augmented image is more scattered than the source domain, which may cause the model to overfit the style of the source domain. To address the above issues, we propose an invariant content representation network (ICRN) to enhance the learning of invariant content and suppress the learning of variability styles. Specifically, we first design a gamma correction-based local style augmentation (LSA) to expand the distribution of samples by augmenting foreground and background styles, respectively. Then, based on the augmented samples, we introduce invariant content learning (ICL) to learn generalizable invariant content from both augmented and source-domain samples. Finally, we design domain-specific batch normalization (DSBN) based style adversarial learning (SAL) to suppress the learning of preferences for source-domain styles. Experimental results show that our proposed method improves by 8.74% and 11.33% in overall dice coefficient (Dice) and reduces 15.88 mm and 3.87 mm in overall average surface distance (ASD) on two publicly available cross-domain datasets, Fundus and Prostate, compared to the state-of-the-art DG methods. The code is available at https://github.com/ZMC-IIIM/ICRN-DG .
Collapse
Affiliation(s)
- Zhiming Cheng
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, China
| | - Shuai Wang
- School of Cyberspace, Hangzhou Dianzi University, Hangzhou, 310018, China.
- Suzhou Research Institute of Shandong University, SuZhou, 215123, China.
| | - Yuhan Gao
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, China
- Lishui Institute of Hangzhou Dianzi Universitu, Lishui, 323010, China
| | - Zunjie Zhu
- Lishui Institute of Hangzhou Dianzi Universitu, Lishui, 323010, China
- School of Communication, Engineering, Hangzhou Dianzi Universitu, Hangzhou, 310018, China
| | - Chenggang Yan
- School of Communication, Engineering, Hangzhou Dianzi Universitu, Hangzhou, 310018, China
| |
Collapse
|
17
|
Santhirasekaram A, Winkler M, Rockall A, Glocker B. Robust prostate disease classification using transformers with discrete representations. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03153-8. [PMID: 38740720 DOI: 10.1007/s11548-024-03153-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Accepted: 04/15/2024] [Indexed: 05/16/2024]
Abstract
PURPOSE Automated prostate disease classification on multi-parametric MRI has recently shown promising results with the use of convolutional neural networks (CNNs). The vision transformer (ViT) is a convolutional free architecture which only exploits the self-attention mechanism and has surpassed CNNs in some natural imaging classification tasks. However, these models are not very robust to textural shifts in the input space. In MRI, we often have to deal with textural shift arising from varying acquisition protocols. Here, we focus on the ability of models to generalise well to new magnet strengths for MRI. METHOD We propose a new framework to improve the robustness of vision transformer-based models for disease classification by constructing discrete representations of the data using vector quantisation. We sample a subset of the discrete representations to form the input into a transformer-based model. We use cross-attention in our transformer model to combine the discrete representations of T2-weighted and apparent diffusion coefficient (ADC) images. RESULTS We analyse the robustness of our model by training on a 1.5 T scanner and test on a 3 T scanner and vice versa. Our approach achieves SOTA performance for classification of lesions on prostate MRI and outperforms various other CNN and transformer-based models in terms of robustness to domain shift and perturbations in the input space. CONCLUSION We develop a method to improve the robustness of transformer-based disease classification of prostate lesions on MRI using discrete representations of the T2-weighted and ADC images.
Collapse
Affiliation(s)
| | - Mathias Winkler
- Department of Surgery and Cancer, Imperial College London, London, UK
| | - Andrea Rockall
- Department of Surgery and Cancer, Imperial College London, London, UK
| | - Ben Glocker
- Department of Surgery and Cancer, Imperial College London, London, UK
| |
Collapse
|
18
|
Hresko DJ, Drotar P. BucketAugment: Reinforced Domain Generalisation in Abdominal CT Segmentation. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2024; 5:353-361. [PMID: 38899027 PMCID: PMC11186658 DOI: 10.1109/ojemb.2024.3397623] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Revised: 02/28/2024] [Accepted: 05/03/2024] [Indexed: 06/21/2024] Open
Abstract
Goal: In recent years, deep neural networks have consistently outperformed previously proposed methods in the domain of medical segmentation. However, due to their nature, these networks often struggle to delineate desired structures in data that fall outside their training distribution. The goal of this study is to address the challenges associated with domain generalization in CT segmentation by introducing a novel method called BucketAugment for deep neural networks. Methods: BucketAugment leverages principles from the Q-learning algorithm and employs validation loss to search for an optimal policy within a search space comprised of distributed stacks of 3D volumetric augmentations, termed 'buckets.' These buckets have tunable parameters and can be seamlessly integrated into existing neural network architectures, offering flexibility for customization. Results: In our experiments, we focus on segmenting kidney and liver structures across three distinct medical datasets, each containing CT scans of the abdominal region collected from various clinical institutions and scanner vendors. Our results indicate that BucketAugment significantly enhances domain generalization across diverse medical datasets, requiring only minimal modifications to existing network architectures. Conclusions: The introduction of BucketAugment provides a promising solution to the challenges of domain generalization in CT segmentation. By leveraging Q-learning principles and distributed stacks of 3D augmentations, this method improves the performance of deep neural networks on medical segmentation tasks, demonstrating its potential to enhance the applicability of such models across different datasets and clinical scenarios.
Collapse
Affiliation(s)
| | - Peter Drotar
- Technical University of Kosice040 01KosiceSlovakia
| |
Collapse
|
19
|
Qu J, Xiao X, Wei X, Qian X. A causality-inspired generalized model for automated pancreatic cancer diagnosis. Med Image Anal 2024; 94:103154. [PMID: 38552527 DOI: 10.1016/j.media.2024.103154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Revised: 02/29/2024] [Accepted: 03/20/2024] [Indexed: 04/16/2024]
Abstract
Pancreatic cancer (PC) is a severely malignant cancer variant with high mortality. Since PC has no obvious symptoms, most PC patients are belatedly diagnosed at advanced disease stages. Recently, artificial intelligence (AI) approaches have demonstrated promising prospects for early diagnosis of pancreatic cancer. However, certain non-causal factors (such as intensity and texture appearance variations, also called confounders) tend to induce spurious correlation with PC diagnosis. This undermines the generalization performance and the clinical applicability of the AI-based PC diagnosis approaches. Therefore, we propose a causal intervention based automated method for pancreatic cancer diagnosis with contrast-enhanced computerized tomography (CT) images, where a confounding effects reduction scheme is developed for alleviating spurious correlations to achieve unbiased learning, thereby improving the generalization performance. Specifically, a continuous image generation strategy was developed to simulate wide variations of intensity differences caused by imaging heterogeneities, where Monte Carlo sampling is added to further enhance the continuity of simulated images. Then, to enhance the pancreatic texture variability, a texture diversification method was introduced in conjunction with gradient-based data augmentation. Finally, a causal intervention strategy was proposed to alleviate the adverse confounding effects by decoupling the causal and non-causal factors and combining them randomly. Extensive experiments showed remarkable diagnosis performance on a cross-validation dataset. Also, promising generalization performance with an average accuracy of 0.87 was attained on three independent test sets of a total of 782 subjects. Therefore, the proposed method shows high clinical feasibility and applicability for pancreatic cancer diagnosis.
Collapse
Affiliation(s)
- Jiaqi Qu
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, PR China
| | - Xiang Xiao
- Department of Medical Imaging, Nanfang Hospital, Southern Medical University, Guangzhou, 510515, PR China
| | - Xunbin Wei
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, PR China; Peking University Cancer Hospital & Institute, Beijing, 100142, PR China; Biomedical Engineering Department, Peking University, Beijing, 100081, PR China; Institute of Medical Technology, Peking University Health Science Center, Beijing, 100191, PR China; International Cancer Institute, Peking University, Beijing 100191, PR China.
| | - Xiaohua Qian
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, PR China.
| |
Collapse
|
20
|
Fidon L, Aertsen M, Kofler F, Bink A, David AL, Deprest T, Emam D, Guffens F, Jakab A, Kasprian G, Kienast P, Melbourne A, Menze B, Mufti N, Pogledic I, Prayer D, Stuempflen M, Van Elslander E, Ourselin S, Deprest J, Vercauteren T. A Dempster-Shafer Approach to Trustworthy AI With Application to Fetal Brain MRI Segmentation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024; 46:3784-3795. [PMID: 38198270 DOI: 10.1109/tpami.2023.3346330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of four backbone AI models for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.
Collapse
|
21
|
Wang Y, Liu Y, Bai Y, Zhou Q, Xu S, Pang X. A generalization performance study on the boosting radiotherapy dose calculation engine based on super-resolution. Z Med Phys 2024; 34:208-217. [PMID: 36631314 DOI: 10.1016/j.zemedi.2022.10.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Revised: 10/10/2022] [Accepted: 10/10/2022] [Indexed: 01/11/2023]
Abstract
PURPOSE During the radiation treatment planning process, one of the time-consuming procedures is the final high-resolution dose calculation, which obstacles the wide application of the emerging online adaptive radiotherapy techniques (OLART). There is an urgent desire for highly accurate and efficient dose calculation methods. This study aims to develop a dose super resolution-based deep learning model for fast and accurate dose prediction in clinical practice. METHOD A Multi-stage Dose Super-Resolution Network (MDSR Net) architecture with sparse masks module and multi-stage progressive dose distribution restoration method were developed to predict high-resolution dose distribution using low-resolution data. A total of 340 VMAT plans from different disease sites were used, among which 240 randomly selected nasopharyngeal, lung, and cervix cases were used for model training, and the remaining 60 cases from the same sites for model benchmark testing, and additional 40 cases from the unseen site (breast and rectum) was used for model generalizability evaluation. The clinical calculated dose with a grid size of 2 mm was used as baseline dose distribution. The input included the dose distribution with 4 mm grid size and CT images. The model performance was compared with HD U-Net and cubic interpolation methods using Dose-volume histograms (DVH) metrics and global gamma analysis with 1%/1 mm and 10% low dose threshold. The correlation between the prediction error and the dose, dose gradient, and CT values was also evaluated. RESULTS The prediction errors of MDSR were 0.06-0.84% of Dmean indices, and the gamma passing rate was 83.1-91.0% on the benchmark testing dataset, and 0.02-1.03% and 71.3-90.3% for the generalization dataset respectively. The model performance was significantly higher than the HD U-Net and interpolation methods (p < 0.05). The mean errors of the MDSR model decreased (monotonously by 0.03-0.004%) with dose and increased (by 0.01-0.73%) with the dose gradient. There was no correlation between prediction errors and the CT values. CONCLUSION The proposed MDSR model achieved good agreement with the baseline high-resolution dose distribution, with small prediction errors for DVH indices and high gamma passing rate for both seen and unseen sites, indicating a robust and generalizable dose prediction model. The model can provide fast and accurate high-resolution dose distribution for clinical dose calculation, particularly for the routine practice of OLART.
Collapse
Affiliation(s)
- Yewei Wang
- Department of Radiation Physics, The Affiliated Tumor Hospital of Harbin Medical University, Harbin, China
| | - Yaoying Liu
- School of Physics, Beihang University, Beijing, China; Manteia Technologies Co, Ltd, Xiamen, Fujian, China; Department of Radiation Oncology, PLA General Hospital, Beijing, China
| | - Yanlin Bai
- Department of Radiation Physics, The Affiliated Tumor Hospital of Harbin Medical University, Harbin, China
| | - Qichao Zhou
- Manteia Technologies Co, Ltd, Xiamen, Fujian, China
| | - Shouping Xu
- Department of Radiation Oncology, PLA General Hospital, Beijing, China
| | - Xueying Pang
- Department of Oncology, First Affiliated Hospital, Heilongjiang University of Chinese Medicine, Harbin, China.
| |
Collapse
|
22
|
Valabregue R, Girka F, Pron A, Rousseau F, Auzias G. Comprehensive analysis of synthetic learning applied to neonatal brain MRI segmentation. Hum Brain Mapp 2024; 45:e26674. [PMID: 38651625 PMCID: PMC11036377 DOI: 10.1002/hbm.26674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 03/09/2024] [Accepted: 03/18/2024] [Indexed: 04/25/2024] Open
Abstract
Brain segmentation from neonatal MRI images is a very challenging task due to large changes in the shape of cerebral structures and variations in signal intensities reflecting the gestational process. In this context, there is a clear need for segmentation techniques that are robust to variations in image contrast and to the spatial configuration of anatomical structures. In this work, we evaluate the potential of synthetic learning, a contrast-independent model trained using synthetic images generated from the ground truth labels of very few subjects. We base our experiments on the dataset released by the developmental Human Connectome Project, for which high-quality images are available for more than 700 babies aged between 26 and 45 weeks postconception. First, we confirm the impressive performance of a standard UNet trained on a few volumes, but also confirm that such models learn intensity-related features specific to the training domain. We then confirm the robustness of the synthetic learning approach to variations in image contrast. However, we observe a clear influence of the age of the baby on the predictions. We improve the performance of this model by enriching the synthetic training set with realistic motion artifacts and over-segmentation of the white matter. Based on extensive visual assessment, we argue that the better performance of the model trained on real T2w data may be due to systematic errors in the ground truth. We propose an original experiment allowing us to show that learning from real data will reproduce any systematic bias affecting the training set, while synthetic models can avoid this limitation. Overall, our experiments confirm that synthetic learning is an effective solution for segmenting neonatal brain MRI. Our adapted synthetic learning approach combines key features that will be instrumental for large multisite studies and clinical applications.
Collapse
Affiliation(s)
- R. Valabregue
- CENIR, Institut du Cerveau (ICM)—Paris Brain Institute, Inserm U 1127, CNRS UMR 7225, Sorbonne UniversitéParisFrance
| | - F. Girka
- CENIR, Institut du Cerveau (ICM)—Paris Brain Institute, Inserm U 1127, CNRS UMR 7225, Sorbonne UniversitéParisFrance
| | - A. Pron
- Aix‐Marseille Université, CNRS, Institut de Neurosciences de la Timone, UMR 7289MarseilleFrance
| | - F. Rousseau
- IMT Atlantique, LaTIM INSERM U1101BrestFrance
| | - G. Auzias
- Aix‐Marseille Université, CNRS, Institut de Neurosciences de la Timone, UMR 7289MarseilleFrance
| |
Collapse
|
23
|
Xu Z, Lu D, Luo J, Zheng Y, Tong RKY. Separated collaborative learning for semi-supervised prostate segmentation with multi-site heterogeneous unlabeled MRI data. Med Image Anal 2024; 93:103095. [PMID: 38310678 DOI: 10.1016/j.media.2024.103095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 09/11/2023] [Accepted: 01/24/2024] [Indexed: 02/06/2024]
Abstract
Segmenting prostate from magnetic resonance imaging (MRI) is a critical procedure in prostate cancer staging and treatment planning. Considering the nature of labeled data scarcity for medical images, semi-supervised learning (SSL) becomes an appealing solution since it can simultaneously exploit limited labeled data and a large amount of unlabeled data. However, SSL relies on the assumption that the unlabeled images are abundant, which may not be satisfied when the local institute has limited image collection capabilities. An intuitive solution is to seek support from other centers to enrich the unlabeled image pool. However, this further introduces data heterogeneity, which can impede SSL that works under identical data distribution with certain model assumptions. Aiming at this under-explored yet valuable scenario, in this work, we propose a separated collaborative learning (SCL) framework for semi-supervised prostate segmentation with multi-site unlabeled MRI data. Specifically, on top of the teacher-student framework, SCL exploits multi-site unlabeled data by: (i) Local learning, which advocates local distribution fitting, including the pseudo label learning that reinforces confirmation of low-entropy easy regions and the cyclic propagated real label learning that leverages class prototypes to regularize the distribution of intra-class features; (ii) External multi-site learning, which aims to robustly mine informative clues from external data, mainly including the local-support category mutual dependence learning, which takes the spirit that mutual information can effectively measure the amount of information shared by two variables even from different domains, and the stability learning under strong adversarial perturbations to enhance robustness to heterogeneity. Extensive experiments on prostate MRI data from six different clinical centers show that our method can effectively generalize SSL on multi-site unlabeled data and significantly outperform other semi-supervised segmentation methods. Besides, we validate the extensibility of our method on the multi-class cardiac MRI segmentation task with data from four different clinical centers.
Collapse
Affiliation(s)
- Zhe Xu
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, NT, Hong Kong, China.
| | - Donghuan Lu
- Tencent Jarvis Research Center, Youtu Lab, Shenzhen, China.
| | - Jie Luo
- Massachusetts General Hospital, Harvard Medical School, Boston, USA
| | - Yefeng Zheng
- Tencent Jarvis Research Center, Youtu Lab, Shenzhen, China
| | - Raymond Kai-Yu Tong
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, NT, Hong Kong, China.
| |
Collapse
|
24
|
Zhang L, Liu Z, Zhang L, Wu Z, Yu X, Holmes J, Feng H, Dai H, Li X, Li Q, Wong WW, Vora SA, Zhu D, Liu T, Liu W. Technical Note: Generalizable and Promptable Artificial Intelligence Model to Augment Clinical Delineation in Radiation Oncology. Med Phys 2024; 51:2187-2199. [PMID: 38319676 PMCID: PMC10939804 DOI: 10.1002/mp.16965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 12/29/2023] [Accepted: 01/14/2024] [Indexed: 03/13/2024] Open
Abstract
BACKGROUND Efficient and accurate delineation of organs at risk (OARs) is a critical procedure for treatment planning and dose evaluation. Deep learning-based auto-segmentation of OARs has shown promising results and is increasingly being used in radiation therapy. However, existing deep learning-based auto-segmentation approaches face two challenges in clinical practice: generalizability and human-AI interaction. A generalizable and promptable auto-segmentation model, which segments OARs of multiple disease sites simultaneously and supports on-the-fly human-AI interaction, can significantly enhance the efficiency of radiation therapy treatment planning. PURPOSE Meta's segment anything model (SAM) was proposed as a generalizable and promptable model for next-generation natural image segmentation. We further evaluated the performance of SAM in radiotherapy segmentation. METHODS Computed tomography (CT) images of clinical cases from four disease sites at our institute were collected: prostate, lung, gastrointestinal, and head & neck. For each case, we selected the OARs important in radiotherapy treatment planning. We then compared both the Dice coefficients and Jaccard indices derived from three distinct methods: manual delineation (ground truth), automatic segmentation using SAM's 'segment anything' mode, and automatic segmentation using SAM's 'box prompt' mode that implements manual interaction via live prompts during segmentation. RESULTS Our results indicate that SAM's segment anything mode can achieve clinically acceptable segmentation results in most OARs with Dice scores higher than 0.7. SAM's box prompt mode further improves Dice scores by 0.1∼0.5. Similar results were observed for Jaccard indices. The results show that SAM performs better for prostate and lung, but worse for gastrointestinal and head & neck. When considering the size of organs and the distinctiveness of their boundaries, SAM shows better performance for large organs with distinct boundaries, such as lung and liver, and worse for smaller organs with less distinct boundaries, like parotid and cochlea. CONCLUSIONS Our results demonstrate SAM's robust generalizability with consistent accuracy in automatic segmentation for radiotherapy. Furthermore, the advanced box-prompt method enables the users to augment auto-segmentation interactively and dynamically, leading to patient-specific auto-segmentation in radiation therapy. SAM's generalizability across different disease sites and different modalities makes it feasible to develop a generic auto-segmentation model in radiotherapy.
Collapse
Affiliation(s)
- Lian Zhang
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Zhengliang Liu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Lu Zhang
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
| | - Zihao Wu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Xiaowei Yu
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
| | - Jason Holmes
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Hongying Feng
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Haixing Dai
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Xiang Li
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02115, USA
| | - Quanzheng Li
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02115, USA
| | - William W. Wong
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Sujay A. Vora
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Dajiang Zhu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Tianming Liu
- School of Computing, University of Georgia, Athens, GA 30602, USA
| | - Wei Liu
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA
| |
Collapse
|
25
|
Guo Z, Wu T, Lockhart TE, Soangra R, Yoon H. Correlation enhanced distribution adaptation for prediction of fall risk. Sci Rep 2024; 14:3477. [PMID: 38347050 PMCID: PMC10861595 DOI: 10.1038/s41598-024-54053-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 02/08/2024] [Indexed: 02/15/2024] Open
Abstract
With technological advancements in diagnostic imaging, smart sensing, and wearables, a multitude of heterogeneous sources or modalities are available to proactively monitor the health of the elderly. Due to the increasing risks of falls among older adults, an early diagnosis tool is crucial to prevent future falls. However, during the early stage of diagnosis, there is often limited or no labeled data (expert-confirmed diagnostic information) available in the target domain (new cohort) to determine the proper treatment for older adults. Instead, there are multiple related but non-identical domain data with labels from the existing cohort or different institutions. Integrating different data sources with labeled and unlabeled samples to predict a patient's condition poses a significant challenge. Traditional machine learning models assume that data for new patients follow a similar distribution. If the data does not satisfy this assumption, the trained models do not achieve the expected accuracy, leading to potential misdiagnosing risks. To address this issue, we utilize domain adaptation (DA) techniques, which employ labeled data from one or more related source domains. These DA techniques promise to tackle discrepancies in multiple data sources and achieve a robust diagnosis for new patients. In our research, we have developed an unsupervised DA model to align two domains by creating a domain-invariant feature representation. Subsequently, we have built a robust fall-risk prediction model based on these new feature representations. The results from simulation studies and real-world applications demonstrate that our proposed approach outperforms existing models.
Collapse
Affiliation(s)
- Ziqi Guo
- Department of Systems Science and Industrial Engineering, The State University of New York at Binghamton, Binghamton, USA
| | - Teresa Wu
- School of Computing and Augmented Intelligence, Arizona State University, Tempe, USA
| | - Thurmon E Lockhart
- School of Biological and Health Systems Engineering, Arizona State University, Tempe, USA
| | - Rahul Soangra
- Department of Physical Therapy, Chapman University, Orange, USA
| | - Hyunsoo Yoon
- Department of Industrial Engineering, Yonsei University, Seoul, Korea.
| |
Collapse
|
26
|
Hognon C, Conze PH, Bourbonne V, Gallinato O, Colin T, Jaouen V, Visvikis D. Contrastive image adaptation for acquisition shift reduction in medical imaging. Artif Intell Med 2024; 148:102747. [PMID: 38325919 DOI: 10.1016/j.artmed.2023.102747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 10/21/2023] [Accepted: 12/10/2023] [Indexed: 02/09/2024]
Abstract
The domain shift, or acquisition shift in medical imaging, is responsible for potentially harmful differences between development and deployment conditions of medical image analysis techniques. There is a growing need in the community for advanced methods that could mitigate this issue better than conventional approaches. In this paper, we consider configurations in which we can expose a learning-based pixel level adaptor to a large variability of unlabeled images during its training, i.e. sufficient to span the acquisition shift expected during the training or testing of a downstream task model. We leverage the ability of convolutional architectures to efficiently learn domain-agnostic features and train a many-to-one unsupervised mapping between a source collection of heterogeneous images from multiple unknown domains subjected to the acquisition shift and a homogeneous subset of this source set of lower cardinality, potentially constituted of a single image. To this end, we propose a new cycle-free image-to-image architecture based on a combination of three loss functions : a contrastive PatchNCE loss, an adversarial loss and an edge preserving loss allowing for rich domain adaptation to the target image even under strong domain imbalance and low data regimes. Experiments support the interest of the proposed contrastive image adaptation approach for the regularization of downstream deep supervised segmentation and cross-modality synthesis models.
Collapse
Affiliation(s)
- Clément Hognon
- UMR U1101 Inserm LaTIM, IMT Atlantique, Université de Bretagne Occidentale, France; SOPHiA Genetics, Pessac, France
| | - Pierre-Henri Conze
- UMR U1101 Inserm LaTIM, IMT Atlantique, Université de Bretagne Occidentale, France
| | - Vincent Bourbonne
- UMR U1101 Inserm LaTIM, IMT Atlantique, Université de Bretagne Occidentale, France
| | | | | | - Vincent Jaouen
- UMR U1101 Inserm LaTIM, IMT Atlantique, Université de Bretagne Occidentale, France.
| | - Dimitris Visvikis
- UMR U1101 Inserm LaTIM, IMT Atlantique, Université de Bretagne Occidentale, France
| |
Collapse
|
27
|
Jiao R, Zhang Y, Ding L, Xue B, Zhang J, Cai R, Jin C. Learning with limited annotations: A survey on deep semi-supervised learning for medical image segmentation. Comput Biol Med 2024; 169:107840. [PMID: 38157773 DOI: 10.1016/j.compbiomed.2023.107840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 10/30/2023] [Accepted: 12/07/2023] [Indexed: 01/03/2024]
Abstract
Medical image segmentation is a fundamental and critical step in many image-guided clinical approaches. Recent success of deep learning-based segmentation methods usually relies on a large amount of labeled data, which is particularly difficult and costly to obtain, especially in the medical imaging domain where only experts can provide reliable and accurate annotations. Semi-supervised learning has emerged as an appealing strategy and been widely applied to medical image segmentation tasks to train deep models with limited annotations. In this paper, we present a comprehensive review of recently proposed semi-supervised learning methods for medical image segmentation and summarize both the technical novelties and empirical results. Furthermore, we analyze and discuss the limitations and several unsolved problems of existing approaches. We hope this review can inspire the research community to explore solutions to this challenge and further advance the field of medical image segmentation.
Collapse
Affiliation(s)
- Rushi Jiao
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; School of Engineering Medicine, Beihang University, Beijing, 100191, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
| | - Yichi Zhang
- School of Data Science, Fudan University, Shanghai, 200433, China; Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, 200433, China.
| | - Le Ding
- School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China.
| | - Bingsen Xue
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
| | - Jicong Zhang
- School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China; Hefei Innovation Research Institute, Beihang University, Hefei, 230012, China.
| | - Rong Cai
- School of Engineering Medicine, Beihang University, Beijing, 100191, China; Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, Beihang University, Beijing, 100191, China.
| | - Cheng Jin
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China; Beijing Anding Hospital, Capital Medical University, Beijing, 100088, China.
| |
Collapse
|
28
|
Borges P, Shaw R, Varsavsky T, Kläser K, Thomas D, Drobnjak I, Ourselin S, Cardoso MJ. Acquisition-invariant brain MRI segmentation with informative uncertainties. Med Image Anal 2024; 92:103058. [PMID: 38104403 DOI: 10.1016/j.media.2023.103058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Revised: 08/24/2023] [Accepted: 12/05/2023] [Indexed: 12/19/2023]
Abstract
Combining multi-site data can strengthen and uncover trends, but is a task that is marred by the influence of site-specific covariates that can bias the data and, therefore, any downstream analyses. Post-hoc multi-site correction methods exist but have strong assumptions that often do not hold in real-world scenarios. Algorithms should be designed in a way that can account for site-specific effects, such as those that arise from sequence parameter choices, and in instances where generalisation fails, should be able to identify such a failure by means of explicit uncertainty modelling. This body of work showcases such an algorithm that can become robust to the physics of acquisition in the context of segmentation tasks while simultaneously modelling uncertainty. We demonstrate that our method not only generalises to complete holdout datasets, preserving segmentation quality but does so while also accounting for site-specific sequence choices, which also allows it to perform as a harmonisation tool.
Collapse
Affiliation(s)
- Pedro Borges
- Department of Medical Physics and Biomedical Engineering, UCL, UK; School of Biomedical Engineering and Imaging Sciences, KCL, UK.
| | - Richard Shaw
- Department of Medical Physics and Biomedical Engineering, UCL, UK; School of Biomedical Engineering and Imaging Sciences, KCL, UK
| | - Thomas Varsavsky
- Department of Medical Physics and Biomedical Engineering, UCL, UK; School of Biomedical Engineering and Imaging Sciences, KCL, UK
| | - Kerstin Kläser
- School of Biomedical Engineering and Imaging Sciences, KCL, UK
| | | | - Ivana Drobnjak
- Department of Medical Physics and Biomedical Engineering, UCL, UK
| | | | - M Jorge Cardoso
- School of Biomedical Engineering and Imaging Sciences, KCL, UK
| |
Collapse
|
29
|
Zhu J, Wang C, Teng S, Lu J, Lyu P, Zhang P, Xu J, Lu L, Teng GJ. Embedding expertise knowledge into inverse treatment planning for low-dose-rate brachytherapy of hepatic malignancies. Med Phys 2024; 51:348-362. [PMID: 37475484 DOI: 10.1002/mp.16627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 06/14/2023] [Accepted: 06/23/2023] [Indexed: 07/22/2023] Open
Abstract
BACKGROUND Leveraging the precision of its radiation dose distribution and the minimization of postoperative complications, low-dose-rate (LDR) permanent seed brachytherapy is progressively adopted in addressing hepatic malignancies. PURPOSE The present study endeavors to devise a sophisticated treatment planning system (TPS) to optimize LDR brachytherapy for hepatic lesions. METHODS Our TPS encompasses four integral modules: multi-organ segmentation, seed distribution initialization, puncture pathway selection, and inverse dose planning. By amalgamating an array of deep learning models, the segmentation module proficiently labels 17 discrete abdominal targets within the images. We introduce a knowledge-based seed distribution initialization methodology that discerns the most analogous tumor shape in the reference treatment plan from the knowledge base. Subsequently, the seed distribution from the reference plan is transmuted to the current case, thus establishing seed distribution initialization. Furthermore, we parameterize the puncture needles and seeds, while concurrently constraining the puncture needle angle through the employment of a virtual puncture panel to augment planning algorithm efficiency. We also presented a user interface that includes a range of interactive features, seamlessly integrated with the treatment planning generation function. RESULTS The multi-organ segmentation module, which is trained by 50 cases of in-house CT scans and 694 cases of publicly available CT scans, achieved average Dice of 0.80 and Hausdorff distance of 5.2 mm in testing datasets. The results demonstrate that knowledge-based initialization exhibits a marked enhancement in expediting the convergence rate. Our TPS also demonstrates a dominant advantage in dose-volume-histogram criteria and execution time in comparison to commercial TPS. CONCLUSION The study proposes an innovative treatment planning system for low-dose-rate permanent seed brachytherapy for hepatic malignancies. We show that the generated treatment plans meet clinical requirement.
Collapse
Affiliation(s)
- Jianjun Zhu
- Hanglok-Tech Co., Ltd., Hengqin, China
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing, China
| | | | | | - Jian Lu
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing, China
| | | | | | - Jun Xu
- Nanjing University of Information Science & Technology, Nanjing, China
| | - Ligong Lu
- Zhuhai People's Hospital, Zhuhai Hospital Affiliated with Jinan University, Zhuhai, Guangdong, China
| | - Gao-Jun Teng
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing, China
| |
Collapse
|
30
|
Zhang Z, Wang B, Jha D, Demir U, Bagci U. Domain Generalization with Correlated Style Uncertainty. IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION. IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION 2024; 2024:1989-1998. [PMID: 38978834 PMCID: PMC11230655 DOI: 10.1109/wacv57701.2024.00200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/10/2024]
Abstract
Domain generalization (DG) approaches intend to extract domain invariant features that can lead to a more robust deep learning model. In this regard, style augmentation is a strong DG method taking advantage of instance-specific feature statistics containing informative style characteristics to synthetic novel domains. While it is one of the state-of-the-art methods, prior works on style augmentation have either disregarded the interdependence amongst distinct feature channels or have solely constrained style augmentation to linear interpolation. To address these research gaps, in this work, we introduce a novel augmentation approach, named Correlated Style Uncertainty (CSU), surpassing the limitations of linear interpolation in style statistic space and simultaneously preserving vital correlation information. Our method's efficacy is established through extensive experimentation on diverse cross-domain computer vision and medical imaging classification tasks: PACS, Office-Home, and Camelyon17 datasets, and the Duke-Market1501 instance retrieval task. The results showcase a remarkable improvement margin over existing state-of-the-art techniques. The source code is available https://github.com/freshman97/CSU.
Collapse
Affiliation(s)
- Zheyuan Zhang
- Machine & Hybrid Intelligence Lab, Northwestern University, USA
| | - Bin Wang
- Machine & Hybrid Intelligence Lab, Northwestern University, USA
| | - Debesh Jha
- Machine & Hybrid Intelligence Lab, Northwestern University, USA
| | - Ugur Demir
- Machine & Hybrid Intelligence Lab, Northwestern University, USA
| | - Ulas Bagci
- Machine & Hybrid Intelligence Lab, Northwestern University, USA
| |
Collapse
|
31
|
He W, Zhang C, Dai J, Liu L, Wang T, Liu X, Jiang Y, Li N, Xiong J, Wang L, Xie Y, Liang X. A statistical deformation model-based data augmentation method for volumetric medical image segmentation. Med Image Anal 2024; 91:102984. [PMID: 37837690 DOI: 10.1016/j.media.2023.102984] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2022] [Revised: 07/15/2023] [Accepted: 09/28/2023] [Indexed: 10/16/2023]
Abstract
The accurate delineation of organs-at-risk (OARs) is a crucial step in treatment planning during radiotherapy, as it minimizes the potential adverse effects of radiation on surrounding healthy organs. However, manual contouring of OARs in computed tomography (CT) images is labor-intensive and susceptible to errors, particularly for low-contrast soft tissue. Deep learning-based artificial intelligence algorithms surpass traditional methods but require large datasets. Obtaining annotated medical images is both time-consuming and expensive, hindering the collection of extensive training sets. To enhance the performance of medical image segmentation, augmentation strategies such as rotation and Gaussian smoothing are employed during preprocessing. However, these conventional data augmentation techniques cannot generate more realistic deformations, limiting improvements in accuracy. To address this issue, this study introduces a statistical deformation model-based data augmentation method for volumetric medical image segmentation. By applying diverse and realistic data augmentation to CT images from a limited patient cohort, our method significantly improves the fully automated segmentation of OARs across various body parts. We evaluate our framework on three datasets containing tumor OARs from the head, neck, chest, and abdomen. Test results demonstrate that the proposed method achieves state-of-the-art performance in numerous OARs segmentation challenges. This innovative approach holds considerable potential as a powerful tool for various medical imaging-related sub-fields, effectively addressing the challenge of limited data access.
Collapse
Affiliation(s)
- Wenfeng He
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China; School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China
| | - Chulong Zhang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Jingjing Dai
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Lin Liu
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Tangsheng Wang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Xuan Liu
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Yuming Jiang
- Department of Radiation Oncology, Wake Forest University School of Medicine, Winston Salem, North Carolina 27157, USA
| | - Na Li
- Department of Biomedical Engineering, Guangdong Medical University, Dongguan, 523808, China
| | - Jing Xiong
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Lei Wang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Yaoqin Xie
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Xiaokun Liang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China.
| |
Collapse
|
32
|
Lyu J, Bartlett PF, Nasrallah FA, Tang X. Toward hippocampal volume measures on ultra-high field magnetic resonance imaging: a comprehensive comparison study between deep learning and conventional approaches. Front Neurosci 2023; 17:1238646. [PMID: 38156266 PMCID: PMC10752989 DOI: 10.3389/fnins.2023.1238646] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 12/04/2023] [Indexed: 12/30/2023] Open
Abstract
The hippocampus is a complex brain structure that plays an important role in various cognitive aspects such as memory, intelligence, executive function, and path integration. The volume of this highly plastic structure is identified as one of the most important biomarkers of specific neuropsychiatric and neurodegenerative diseases. It has also been extensively investigated in numerous aging studies. However, recent studies on aging show that the performance of conventional approaches in measuring the hippocampal volume is still far from satisfactory, especially in terms of delivering longitudinal measures from ultra-high field magnetic resonance images (MRIs), which can visualize more boundary details. The advancement of deep learning provides an alternative solution to measuring the hippocampal volume. In this work, we comprehensively compared a deep learning pipeline based on nnU-Net with several conventional approaches including Freesurfer, FSL and DARTEL, for automatically delivering hippocampal volumes: (1) Firstly, we evaluated the segmentation accuracy and precision on a public dataset through cross-validation. Results showed that the deep learning pipeline had the lowest mean (L = 1.5%, R = 1.7%) and the lowest standard deviation (L = 5.2%, R = 6.2%) in terms of volume percentage error. (2) Secondly, sub-millimeter MRIs of a group of healthy adults with test-retest 3T and 7T sessions were used to extensively assess the test-retest reliability. Results showed that the deep learning pipeline achieved very high intraclass correlation coefficients (L = 0.990, R = 0.986 for 7T; L = 0.985, R = 0.983 for 3T) and very small volume percentage differences (L = 1.2%, R = 0.9% for 7T; L = 1.3%, R = 1.3% for 3T). (3) Thirdly, a Bayesian linear mixed effect model was constructed with respect to the hippocampal volumes of two healthy adult datasets with longitudinal 7T scans and one disease-related longitudinal dataset. It was found that the deep learning pipeline detected both the subtle and disease-related changes over time with high sensitivity as well as the mild differences across subjects. Comparison results from the aforementioned three aspects showed that the deep learning pipeline significantly outperformed the conventional approaches by large margins. Results also showed that the deep learning pipeline can better accommodate longitudinal analysis purposes.
Collapse
Affiliation(s)
- Junyan Lyu
- Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, Guangdong, China
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD, Australia
| | - Perry F. Bartlett
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD, Australia
| | - Fatima A. Nasrallah
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD, Australia
| | - Xiaoying Tang
- Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, Guangdong, China
| |
Collapse
|
33
|
Karunanayake N, Moodleah S, Makhanov SS. Edge-Driven Multi-Agent Reinforcement Learning: A Novel Approach to Ultrasound Breast Tumor Segmentation. Diagnostics (Basel) 2023; 13:3611. [PMID: 38132195 PMCID: PMC10742763 DOI: 10.3390/diagnostics13243611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Revised: 11/05/2023] [Accepted: 11/20/2023] [Indexed: 12/23/2023] Open
Abstract
A segmentation model of the ultrasound (US) images of breast tumors based on virtual agents trained using reinforcement learning (RL) is proposed. The agents, living in the edge map, are able to avoid false boundaries, connect broken parts, and finally, accurately delineate the contour of the tumor. The agents move similarly to robots navigating in the unknown environment with the goal of maximizing the rewards. The individual agent does not know the goal of the entire population. However, since the robots communicate, the model is able to understand the global information and fit the irregular boundaries of complicated objects. Combining the RL with a neural network makes it possible to automatically learn and select the local features. In particular, the agents handle the edge leaks and artifacts typical for the US images. The proposed model outperforms 13 state-of-the-art algorithms, including selected deep learning models and their modifications.
Collapse
Affiliation(s)
- Nalan Karunanayake
- Sirindhorn International Institute of Technology, Thammasat University, Pathum Thani 12120, Thailand;
| | - Samart Moodleah
- King Mongkut’s Institute of Technology Ladkrabang, Bangkok 10520, Thailand;
| | - Stanislav S. Makhanov
- Sirindhorn International Institute of Technology, Thammasat University, Pathum Thani 12120, Thailand;
| |
Collapse
|
34
|
Aggarwal K, Manso Jimeno M, Ravi KS, Gonzalez G, Geethanath S. Developing and deploying deep learning models in brain magnetic resonance imaging: A review. NMR IN BIOMEDICINE 2023; 36:e5014. [PMID: 37539775 DOI: 10.1002/nbm.5014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/05/2023]
Abstract
Magnetic resonance imaging (MRI) of the brain has benefited from deep learning (DL) to alleviate the burden on radiologists and MR technologists, and improve throughput. The easy accessibility of DL tools has resulted in a rapid increase of DL models and subsequent peer-reviewed publications. However, the rate of deployment in clinical settings is low. Therefore, this review attempts to bring together the ideas from data collection to deployment in the clinic, building on the guidelines and principles that accreditation agencies have espoused. We introduce the need for and the role of DL to deliver accessible MRI. This is followed by a brief review of DL examples in the context of neuropathologies. Based on these studies and others, we collate the prerequisites to develop and deploy DL models for brain MRI. We then delve into the guiding principles to develop good machine learning practices in the context of neuroimaging, with a focus on explainability. A checklist based on the United States Food and Drug Administration's good machine learning practices is provided as a summary of these guidelines. Finally, we review the current challenges and future opportunities in DL for brain MRI.
Collapse
Affiliation(s)
- Kunal Aggarwal
- Accessible MR Laboratory, Biomedical Engineering and Imaging Institute, Department of Diagnostic, Molecular and Interventional Radiology, Mount Sinai Hospital, New York, USA
- Department of Electrical and Computer Engineering, Technical University Munich, Munich, Germany
| | - Marina Manso Jimeno
- Department of Biomedical Engineering, Columbia University in the City of New York, New York, New York, USA
- Columbia Magnetic Resonance Research Center, Columbia University in the City of New York, New York, New York, USA
| | - Keerthi Sravan Ravi
- Department of Biomedical Engineering, Columbia University in the City of New York, New York, New York, USA
- Columbia Magnetic Resonance Research Center, Columbia University in the City of New York, New York, New York, USA
| | - Gilberto Gonzalez
- Division of Neuroradiology, Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Sairam Geethanath
- Accessible MR Laboratory, Biomedical Engineering and Imaging Institute, Department of Diagnostic, Molecular and Interventional Radiology, Mount Sinai Hospital, New York, USA
| |
Collapse
|
35
|
Liao W, Luo X, He Y, Dong Y, Li C, Li K, Zhang S, Zhang S, Wang G, Xiao J. Comprehensive Evaluation of a Deep Learning Model for Automatic Organs-at-Risk Segmentation on Heterogeneous Computed Tomography Images for Abdominal Radiation Therapy. Int J Radiat Oncol Biol Phys 2023; 117:994-1006. [PMID: 37244625 DOI: 10.1016/j.ijrobp.2023.05.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 03/13/2023] [Accepted: 05/18/2023] [Indexed: 05/29/2023]
Abstract
PURPOSE Our purpose was to develop a deep learning model (AbsegNet) that produces accurate contours of 16 organs at risk (OARs) for abdominal malignancies as an essential part of fully automated radiation treatment planning. METHODS AND MATERIALS Three data sets with 544 computed tomography scans were retrospectively collected. Data set 1 was split into 300 training cases and 128 test cases (cohort 1) for AbsegNet. Data set 2, including cohort 2 (n = 24) and cohort 3 (n = 20), were used to validate AbsegNet externally. Data set 3, including cohort 4 (n = 40) and cohort 5 (n = 32), were used to clinically assess the accuracy of AbsegNet-generated contours. Each cohort was from a different center. The Dice similarity coefficient and 95th-percentile Hausdorff distance were calculated to evaluate the delineation quality for each OAR. Clinical accuracy evaluation was classified into 4 levels: no revision, minor revisions (0% < volumetric revision degrees [VRD] ≤ 10%), moderate revisions (10% ≤ VRD < 20%), and major revisions (VRD ≥20%). RESULTS For all OARs, AbsegNet achieved a mean Dice similarity coefficient of 86.73%, 85.65%, and 88.04% in cohorts 1, 2, and 3, respectively, and a mean 95th-percentile Hausdorff distance of 8.92, 10.18, and 12.40 mm, respectively. The performance of AbsegNet outperformed SwinUNETR, DeepLabV3+, Attention-UNet, UNet, and 3D-UNet. When experts evaluated contours from cohorts 4 and 5, 4 OARs (liver, kidney_L, kidney_R, and spleen) of all patients were scored as having no revision, and over 87.5% of patients with contours of the stomach, esophagus, adrenals, or rectum were considered as having no or minor revisions. Only 15.0% of patients with colon and small bowel contours required major revisions. CONCLUSIONS We propose a novel deep-learning model to delineate OARs on diverse data sets. Most contours produced by AbsegNet are accurate and robust and are, therefore, clinically applicable and helpful to facilitate radiation therapy workflow.
Collapse
Affiliation(s)
- Wenjun Liao
- Department of Radiation Oncology, Radiation Oncology Key Laboratory of Sichuan Province, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Chengdu, China
| | - Xiangde Luo
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China; Shanghai AI Laboratory, Shanghai, China
| | - Yuan He
- Department of Radiation Oncology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China
| | - Ye Dong
- Department of NanFang PET Center, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Churong Li
- Department of Radiation Oncology, Radiation Oncology Key Laboratory of Sichuan Province, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Chengdu, China
| | - Kang Li
- West China Biomedical Big Data Center
| | - Shichuan Zhang
- Department of Radiation Oncology, Radiation Oncology Key Laboratory of Sichuan Province, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Chengdu, China
| | - Shaoting Zhang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China; Shanghai AI Laboratory, Shanghai, China
| | - Guotai Wang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China; Shanghai AI Laboratory, Shanghai, China
| | - Jianghong Xiao
- Radiotherapy Physics & Technology Center, Department of Radiation Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, China.
| |
Collapse
|
36
|
Yi W, Zhao J, Tang W, Yin H, Yu L, Wang Y, Tian W. Deep learning-based high-accuracy detection for lumbar and cervical degenerative disease on T2-weighted MR images. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2023; 32:3807-3814. [PMID: 36943484 DOI: 10.1007/s00586-023-07641-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 01/31/2023] [Accepted: 03/05/2023] [Indexed: 03/23/2023]
Abstract
PURPOSE To develop and validate a deep learning (DL) model for detecting lumbar degenerative disease in both sagittal and axial views of T2-weighted MRI and evaluate its generalized performance in detecting cervical degenerative disease. METHODS T2-weighted MRI scans of 804 patients with symptoms of lumbar degenerative disease were retrospectively collected from three hospitals. The training dataset (n = 456) and internal validation dataset (n = 134) were randomly selected from the center I. Two external validation datasets comprising 100 and 114 patients were from center II and center III, respectively. A DL model based on 3D ResNet18 and transformer architecture was proposed to detect lumbar degenerative disease. In addition, a cervical MR image dataset comprising 200 patients from an independent hospital was used to evaluate the generalized performance of the DL model. The diagnostic performance was assessed by the free-response receiver operating characteristic (fROC) curve and precision-recall (PR) curve. Precision, recall, and F1-score were used to measure the DL model. RESULTS A total of 2497 three-dimension retrogression annotations were labeled for training (n = 1157) and multicenter validation (n = 1340). The DL model showed excellent detection efficiency in the internal validation dataset, with F1-score achieving 0.971 and 0.903 on the sagittal and axial MR images, respectively. Good performance was also observed in the external validation dataset I (F1-score, 0.768 on sagittal MR images and 0.837 on axial MR images) and external validation dataset II (F1-score, 0.787 on sagittal MR images and 0.770 on axial MR images). Furthermore, the robustness of the DL model was demonstrated via transfer learning and generalized performance evaluation on the external cervical dataset, with the F1-score yielding 0.931 and 0.919 on the sagittal and axial MR images, respectively. CONCLUSION The proposed DL model can automatically detect lumbar and cervical degenerative disease on T2-weighted MR images with good performance, robustness, and feasibility in clinical practice.
Collapse
Affiliation(s)
- Wei Yi
- Department of Spine Surgery, Beijing Jishuitan Hospital, Beijing, 100035, China.
| | - Jingwei Zhao
- Beijing Jishuitan Hospital, Research Unit of Intelligent Orthopedics, Chinese Academy of Medical Sciences, Beijing, China.
| | - Wen Tang
- Institute of Advanced Research, Infervision Medical Technology Co., Ltd, Beijing, Beijing, China
| | - Hongkun Yin
- Institute of Advanced Research, Infervision Medical Technology Co., Ltd, Beijing, Beijing, China
| | - Lifeng Yu
- The Second Hospital of Zhangjiakou City, Zhangjiakou, Hebei Province, China
| | - Yaohui Wang
- Department of Trauma, Beijing Water Conservancy Hospital, Beijing, 10036, China
| | - Wei Tian
- Beijing Jishuitan Hospital, Research Unit of Intelligent Orthopedics, Chinese Academy of Medical Sciences, Beijing, China
| |
Collapse
|
37
|
Sang Y, McNitt-Gray M, Yang Y, Cao M, Low D, Ruan D. Target-oriented deep learning-based image registration with individualized test-time adaptation. Med Phys 2023; 50:7016-7026. [PMID: 37222565 DOI: 10.1002/mp.16477] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2022] [Revised: 02/20/2023] [Accepted: 02/26/2023] [Indexed: 05/25/2023] Open
Abstract
BACKGROUND A classic approach in medical image registration is to formulate an optimization problem based on the image pair of interest, and seek a deformation vector field (DVF) to minimize the corresponding objective, often iteratively. It has a clear focus on the targeted pair, but is typically slow. In contrast, more recent deep-learning-based registration offers a much faster alternative and can benefit from data-driven regularization. However, learning is a process to "fit" the training cohort, whose image or motion characteristics or both may differ from the pair of images to be tested, which is the ultimate goal of registration. Therefore, generalization gap poses a high risk with direct inference alone. PURPOSE In this study, we propose an individualized adaptation to improve test sample targeting, to achieve a synergy of efficiency and performance in registration. METHODS Using a previously developed network with an integrated motion representation prior module as the implementation backbone, we propose to adapt the trained registration network further for image pairs at test time to optimize the individualized performance. The adaptation method was tested against various characteristics shifts caused by cross-protocol, cross-platform, and cross-modality, with test evaluation performed on lung CBCT, cardiac MRI, and lung MRI, respectively. RESULTS Landmark-based registration errors and motion-compensated image enhancement results demonstrated significantly improved test registration performance from our method, compared to tuned classic B-spline registration and network solutions without adaptation. CONCLUSIONS We have developed a method to synergistically combine the effectiveness of pre-trained deep network and the target-centric perspective of optimization-based registration to improve performance on individual test data.
Collapse
Affiliation(s)
- Yudi Sang
- Department of Bioengineering, University of California, Los Angeles, California, USA
- Department of Radiation Oncology, University of California, Los Angeles, California, USA
| | - Michael McNitt-Gray
- Department of Radiology, University of California, Los Angeles, California, USA
| | - Yingli Yang
- Department of Radiation Oncology, University of California, Los Angeles, California, USA
| | - Minsong Cao
- Department of Radiation Oncology, University of California, Los Angeles, California, USA
| | - Daniel Low
- Department of Radiation Oncology, University of California, Los Angeles, California, USA
| | - Dan Ruan
- Department of Bioengineering, University of California, Los Angeles, California, USA
- Department of Radiation Oncology, University of California, Los Angeles, California, USA
| |
Collapse
|
38
|
Pereg D. Domain-Aware Few-Shot Learning for Optical Coherence Tomography Noise Reduction. J Imaging 2023; 9:237. [PMID: 37998084 PMCID: PMC10672362 DOI: 10.3390/jimaging9110237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 10/08/2023] [Accepted: 10/10/2023] [Indexed: 11/25/2023] Open
Abstract
Speckle noise has long been an extensively studied problem in medical imaging. In recent years, there have been significant advances in leveraging deep learning methods for noise reduction. Nevertheless, adaptation of supervised learning models to unseen domains remains a challenging problem. Specifically, deep neural networks (DNNs) trained for computational imaging tasks are vulnerable to changes in the acquisition system's physical parameters, such as: sampling space, resolution, and contrast. Even within the same acquisition system, performance degrades across datasets of different biological tissues. In this work, we propose a few-shot supervised learning framework for optical coherence tomography (OCT) noise reduction, that offers high-speed training (of the order of seconds) and requires only a single image, or part of an image, and a corresponding speckle-suppressed ground truth, for training. Furthermore, we formulate the domain shift problem for OCT diverse imaging systems and prove that the output resolution of a despeckling trained model is determined by the source domain resolution. We also provide possible remedies. We propose different practical implementations of our approach, verify and compare their applicability, robustness, and computational efficiency. Our results demonstrate the potential to improve sample complexity, generalization, and time efficiency, for coherent and non-coherent noise reduction via supervised learning models, that can also be leveraged for other real-time computer vision applications.
Collapse
Affiliation(s)
- Deborah Pereg
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA;
- School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA
| |
Collapse
|
39
|
Bianconi A, Rossi LF, Bonada M, Zeppa P, Nico E, De Marco R, Lacroce P, Cofano F, Bruno F, Morana G, Melcarne A, Ruda R, Mainardi L, Fiaschi P, Garbossa D, Morra L. Deep learning-based algorithm for postoperative glioblastoma MRI segmentation: a promising new tool for tumor burden assessment. Brain Inform 2023; 10:26. [PMID: 37801128 PMCID: PMC10558414 DOI: 10.1186/s40708-023-00207-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 09/16/2023] [Indexed: 10/07/2023] Open
Abstract
OBJECTIVE Clinical and surgical decisions for glioblastoma patients depend on a tumor imaging-based evaluation. Artificial Intelligence (AI) can be applied to magnetic resonance imaging (MRI) assessment to support clinical practice, surgery planning and prognostic predictions. In a real-world context, the current obstacles for AI are low-quality imaging and postoperative reliability. The aim of this study is to train an automatic algorithm for glioblastoma segmentation on a clinical MRI dataset and to obtain reliable results both pre- and post-operatively. METHODS The dataset used for this study comprises 237 (71 preoperative and 166 postoperative) MRIs from 71 patients affected by a histologically confirmed Grade IV Glioma. The implemented U-Net architecture was trained by transfer learning to perform the segmentation task on postoperative MRIs. The training was carried out first on BraTS2021 dataset for preoperative segmentation. Performance is evaluated using DICE score (DS) and Hausdorff 95% (H95). RESULTS In preoperative scenario, overall DS is 91.09 (± 0.60) and H95 is 8.35 (± 1.12), considering tumor core, enhancing tumor and whole tumor (ET and edema). In postoperative context, overall DS is 72.31 (± 2.88) and H95 is 23.43 (± 7.24), considering resection cavity (RC), gross tumor volume (GTV) and whole tumor (WT). Remarkably, the RC segmentation obtained a mean DS of 63.52 (± 8.90) in postoperative MRIs. CONCLUSIONS The performances achieved by the algorithm are consistent with previous literature for both pre-operative and post-operative glioblastoma's MRI evaluation. Through the proposed algorithm, it is possible to reduce the impact of low-quality images and missing sequences.
Collapse
Affiliation(s)
- Andrea Bianconi
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy.
| | | | - Marta Bonada
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Pietro Zeppa
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Elsa Nico
- Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, USA
| | - Raffaele De Marco
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | | | - Fabio Cofano
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Francesco Bruno
- Neurooncology, Department of Neuroscience, University of Turin, Turin, Italy
| | - Giovanni Morana
- Neuroradiology, Department of Neuroscience, University of Turin, Turin, Italy
| | - Antonio Melcarne
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Roberta Ruda
- Neurooncology, Department of Neuroscience, University of Turin, Turin, Italy
| | - Luca Mainardi
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
| | - Pietro Fiaschi
- IRCCS Ospedale Policlinico S. Martino, Genoa, Italy
- Dipartimento di Neuroscienze, Riabilitazione, Oftalmologia, Genetica e Scienze Materno-Infantili, Univeristy of Genoa, Genoa, Italy
| | - Diego Garbossa
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Lia Morra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, Turin, Italy
| |
Collapse
|
40
|
Patil SS, Ramteke M, Verma M, Seth S, Bhargava R, Mittal S, Rathore AS. A Domain-Shift Invariant CNN Framework for Cardiac MRI Segmentation Across Unseen Domains. J Digit Imaging 2023; 36:2148-2163. [PMID: 37430062 PMCID: PMC10501982 DOI: 10.1007/s10278-023-00873-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 06/09/2023] [Accepted: 06/13/2023] [Indexed: 07/12/2023] Open
Abstract
The emergence of various deep learning approaches in diagnostic medical image segmentation has made machines capable of accomplishing human-level accuracy. However, the generalizability of these architectures across patients from different countries, Magnetic Resonance Imaging (MRI) scans from distinct vendors, and varying imaging conditions remains questionable. In this work, we propose a translatable deep learning framework for diagnostic segmentation of cine MRI scans. This study aims to render the available SOTA (state-of-the-art) architectures domain-shift invariant by utilizing the heterogeneity of multi-sequence cardiac MRI. To develop and test our approach, we curated a diverse group of public datasets and a dataset obtained from private source. We evaluated 3 SOTA CNN (Convolution neural network) architectures i.e., U-Net, Attention-U-Net, and Attention-Res-U-Net. These architectures were first trained on a combination of three different cardiac MRI sequences. Next, we examined the M&M (multi-center & mutli-vendor) challenge dataset to investigate the effect of different training sets on translatability. The U-Net architecture, trained on the multi-sequence dataset, proved to be the most generalizable across multiple datasets during validation on unseen domains. This model attained mean dice scores of 0.81, 0.85, and 0.83 for myocardial wall segmentation after testing on unseen MyoPS (Myocardial Pathology Segmentation) 2020 dataset, AIIMS (All India Institute of Medical Sciences) dataset and M&M dataset, respectively. Our framework achieved Pearson's correlation values of 0.98, 0.99, and 0.95 between the observed and predicted parameters of end diastole volume, end systole volume, and ejection fraction, respectively, on the unseen Indian population dataset.
Collapse
Affiliation(s)
- Sanjeet S Patil
- Department of Chemical Engineering, Indian Institute of Technology, Delhi, New Delhi, India
| | - Manojkumar Ramteke
- Department of Chemical Engineering, Indian Institute of Technology, Delhi, New Delhi, India
- Yardi School of Artificial Intelligence, Indian Institute of Technology, Delhi, New Delhi, India
| | - Mansi Verma
- Department of Cardiology, All India Institute of Medical Science, Rishikesh, Uttarakhand, India
| | - Sandeep Seth
- Department of Cardiology, All India Institute Medical Science, New Delhi, India
| | - Rohit Bhargava
- Departments of Bioengineering, Electrical & Computer Engineering, Mechanical Science & Engineering, Chemical and Biomolecular Engineering and Chemistry, Beckman Institute for Advanced Science and Technology, Cancer Center at Illinois, University of Illinois at Urbana-Champaign, Urbana, IL, 61801, USA
| | - Shachi Mittal
- Department of Laboratory Medicine and Pathology, School of Medicine, University of Washington, Seattle, WA, USA.
| | - Anurag S Rathore
- Department of Chemical Engineering, Indian Institute of Technology, Delhi, New Delhi, India.
- Yardi School of Artificial Intelligence, Indian Institute of Technology, Delhi, New Delhi, India.
| |
Collapse
|
41
|
Kang Y, Zhao X, Zhang Y, Li H, Wang G, Cui L, Xing Y, Feng J, Yang L. Improving domain generalization performance for medical image segmentation via random feature augmentation. Methods 2023; 218:149-157. [PMID: 37572767 DOI: 10.1016/j.ymeth.2023.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 07/19/2023] [Accepted: 08/07/2023] [Indexed: 08/14/2023] Open
Abstract
Deep convolutional neural networks (DCNNs) have shown remarkable performance in medical image segmentation tasks. However, medical images frequently exhibit distribution discrepancies due to variations in scanner vendors, operators, and image quality, which pose significant challenges to the robustness of trained models when applied to unseen clinical data. To address this issue, domain generalization methods have been developed to enhance the generalization ability of DCNNs. Feature space-based data augmentation methods have been proven effective in improving domain generalization, but they often rely on prior knowledge or assumptions, which can limit the diversity of source domain data. In this study, we propose a novel random feature augmentation (RFA) method to diversify source domain data at the feature level without prior knowledge. Specifically, our RFA method perturbs domain-specific information while preserving domain-invariant information, thereby adequately diversifying the source domain data. Furthermore, we propose a dual-branches invariant synergistic learning strategy to capture domain-invariant information from the augmented features of RFA, enabling DCNNs to learn a more generalized representation. We evaluate our proposed method on two challenging medical image segmentation tasks, optic cup/disc segmentation on fundus images and prostate segmentation on MRI images. Extensive experimental results demonstrate the superior performance of our method over state-of-the-art domain generalization methods.
Collapse
Affiliation(s)
- Yuxin Kang
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China
| | - Xuan Zhao
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China
| | - Yu Zhang
- Department of Medical Oncology, Harbin Medical University Cancer Hospital, Harbin, 150081, China
| | - Hansheng Li
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China
| | - Guan Wang
- Department of Neurosurgery, Xi'an People's Hospital, Xi'an, 710004, China
| | - Lei Cui
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China.
| | - Yaqiong Xing
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China.
| | - Jun Feng
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China
| | - Lin Yang
- School of Information Science and Technology, Northwest University, Xi'an, 710127, China
| |
Collapse
|
42
|
Gao S, Zhou H, Gao Y, Zhuang X. BayeSeg: Bayesian modeling for medical image segmentation with interpretable generalizability. Med Image Anal 2023; 89:102889. [PMID: 37467643 DOI: 10.1016/j.media.2023.102889] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 06/27/2023] [Accepted: 06/29/2023] [Indexed: 07/21/2023]
Abstract
Due to the cross-domain distribution shift aroused from diverse medical imaging systems, many deep learning segmentation methods fail to perform well on unseen data, which limits their real-world applicability. Recent works have shown the benefits of extracting domain-invariant representations on domain generalization. However, the interpretability of domain-invariant features remains a great challenge. To address this problem, we propose an interpretable Bayesian framework (BayeSeg) through Bayesian modeling of image and label statistics to enhance model generalizability for medical image segmentation. Specifically, we first decompose an image into a spatial-correlated variable and a spatial-variant variable, assigning hierarchical Bayesian priors to explicitly force them to model the domain-stable shape and domain-specific appearance information respectively. Then, we model the segmentation as a locally smooth variable only related to the shape. Finally, we develop a variational Bayesian framework to infer the posterior distributions of these explainable variables. The framework is implemented with neural networks, and thus is referred to as deep Bayesian segmentation. Quantitative and qualitative experimental results on prostate segmentation and cardiac segmentation tasks have shown the effectiveness of our proposed method. Moreover, we investigated the interpretability of BayeSeg by explaining the posteriors and analyzed certain factors that affect the generalization ability through further ablation studies. Our code is released via https://zmiclab.github.io/projects.html.
Collapse
Affiliation(s)
- Shangqi Gao
- School of Data Science, Fudan University, Shanghai, 200433, China
| | - Hangqi Zhou
- School of Data Science, Fudan University, Shanghai, 200433, China
| | - Yibo Gao
- School of Data Science, Fudan University, Shanghai, 200433, China
| | - Xiahai Zhuang
- School of Data Science, Fudan University, Shanghai, 200433, China. https://www.sdspeople.fudan.edu.cn/zhuangxiahai/
| |
Collapse
|
43
|
Sahiner B, Chen W, Samala RK, Petrick N. Data drift in medical machine learning: implications and potential remedies. Br J Radiol 2023; 96:20220878. [PMID: 36971405 PMCID: PMC10546450 DOI: 10.1259/bjr.20220878] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Revised: 02/16/2023] [Accepted: 02/20/2023] [Indexed: 03/29/2023] Open
Abstract
Data drift refers to differences between the data used in training a machine learning (ML) model and that applied to the model in real-world operation. Medical ML systems can be exposed to various forms of data drift, including differences between the data sampled for training and used in clinical operation, differences between medical practices or context of use between training and clinical use, and time-related changes in patient populations, disease patterns, and data acquisition, to name a few. In this article, we first review the terminology used in ML literature related to data drift, define distinct types of drift, and discuss in detail potential causes within the context of medical applications with an emphasis on medical imaging. We then review the recent literature regarding the effects of data drift on medical ML systems, which overwhelmingly show that data drift can be a major cause for performance deterioration. We then discuss methods for monitoring data drift and mitigating its effects with an emphasis on pre- and post-deployment techniques. Some of the potential methods for drift detection and issues around model retraining when drift is detected are included. Based on our review, we find that data drift is a major concern in medical ML deployment and that more research is needed so that ML models can identify drift early, incorporate effective mitigation strategies and resist performance decay.
Collapse
Affiliation(s)
- Berkman Sahiner
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| | - Weijie Chen
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| | - Ravi K. Samala
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| | - Nicholas Petrick
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| |
Collapse
|
44
|
Gu R, Wang G, Lu J, Zhang J, Lei W, Chen Y, Liao W, Zhang S, Li K, Metaxas DN, Zhang S. CDDSA: Contrastive domain disentanglement and style augmentation for generalizable medical image segmentation. Med Image Anal 2023; 89:102904. [PMID: 37506556 DOI: 10.1016/j.media.2023.102904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 06/06/2023] [Accepted: 07/12/2023] [Indexed: 07/30/2023]
Abstract
Generalization to previously unseen images with potential domain shifts is essential for clinically applicable medical image segmentation. Disentangling domain-specific and domain-invariant features is key for Domain Generalization (DG). However, existing DG methods struggle to achieve effective disentanglement. To address this problem, we propose an efficient framework called Contrastive Domain Disentanglement and Style Augmentation (CDDSA) for generalizable medical image segmentation. First, a disentangle network decomposes the image into domain-invariant anatomical representation and domain-specific style code, where the former is sent for further segmentation that is not affected by domain shift, and the disentanglement is regularized by a decoder that combines the anatomical representation and style code to reconstruct the original image. Second, to achieve better disentanglement, a contrastive loss is proposed to encourage the style codes from the same domain and different domains to be compact and divergent, respectively. Finally, to further improve generalizability, we propose a style augmentation strategy to synthesize images with various unseen styles in real time while maintaining anatomical information. Comprehensive experiments on a public multi-site fundus image dataset and an in-house multi-site Nasopharyngeal Carcinoma Magnetic Resonance Image (NPC-MRI) dataset show that the proposed CDDSA achieved remarkable generalizability across different domains, and it outperformed several state-of-the-art methods in generalizable segmentation. Code is available at https://github.com/HiLab-git/DAG4MIA.
Collapse
Affiliation(s)
- Ran Gu
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Guotai Wang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China; Shanghai AI Lab, Shanghai, China.
| | - Jiangshan Lu
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Jingyang Zhang
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China; School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - Wenhui Lei
- School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China; Shanghai AI Lab, Shanghai, China
| | - Yinan Chen
- SenseTime Research, Shanghai, China; West China Hospital-SenseTime Joint Lab, West China Biomedical Big Data Center, Sichuan University, Chengdu, China
| | - Wenjun Liao
- Department of Radiation Oncology, Sichuan Cancer Hospital and Institute, University of Electronic Science and Technology of China, Chengdu, China
| | - Shichuan Zhang
- Department of Radiation Oncology, Sichuan Cancer Hospital and Institute, University of Electronic Science and Technology of China, Chengdu, China
| | - Kang Li
- West China Hospital-SenseTime Joint Lab, West China Biomedical Big Data Center, Sichuan University, Chengdu, China
| | - Dimitris N Metaxas
- Department of Computer Science, Rutgers University, Piscataway NJ 08854, USA
| | - Shaoting Zhang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China; SenseTime Research, Shanghai, China; Shanghai AI Lab, Shanghai, China.
| |
Collapse
|
45
|
Liu X, Wang K, Liu F, Zhao W, Liu J. 3D Convolution neural network with multiscale spatial and temporal cues for motor imagery EEG classification. Cogn Neurodyn 2023; 17:1357-1380. [PMID: 37786651 PMCID: PMC10542086 DOI: 10.1007/s11571-022-09906-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 08/01/2022] [Accepted: 09/06/2022] [Indexed: 10/04/2023] Open
Abstract
Recently, deep learning-based methods have achieved meaningful results in the Motor imagery electroencephalogram (MI EEG) classification. However, because of the low signal-to-noise ratio and the various characteristics of brain activities among subjects, these methods lack a subject adaptive feature extraction mechanism. Another issue is that they neglect important spatial topological information and the global temporal variation trend of MI EEG signals. These issues limit the classification accuracy. Here, we propose an end-to-end 3D CNN to extract multiscale spatial and temporal dependent features for improving the accuracy performance of 4-class MI EEG classification. The proposed method adaptively assigns higher weights to motor-related spatial channels and temporal sampling cues than the motor-unrelated ones across all brain regions, which can prevent influences caused by biological and environmental artifacts. Experimental evaluation reveals that the proposed method achieved an average classification accuracy of 93.06% and 97.05% on two commonly used datasets, demonstrating excellent performance and robustness for different subjects compared to other state-of-the-art methods.In order to verify the real-time performance in actual applications, the proposed method is applied to control the robot based on MI EEG signals. The proposed approach effectively addresses the issues of existing methods, improves the classification accuracy and the performance of BCI system, and has great application prospects.
Collapse
Affiliation(s)
- Xiuling Liu
- College of Electronic and Information Engineering, Hebei University, Baoding, 071002 China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, 071002 China
| | - Kaidong Wang
- College of Electronic and Information Engineering, Hebei University, Baoding, 071002 China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, 071002 China
| | - Fengshuang Liu
- College of Electronic and Information Engineering, Hebei University, Baoding, 071002 China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, 071002 China
| | - Wei Zhao
- College of Computer and Cyber Security, Hebei Normal University, Street, Shijiazhuang, 050024 China
| | - Jing Liu
- College of Computer and Cyber Security, Hebei Normal University, Street, Shijiazhuang, 050024 China
| |
Collapse
|
46
|
Lteif D, Sreerama S, Bargal SA, Plummer BA, Au R, Kolachalama VB. Disease-driven domain generalization for neuroimaging-based assessment of Alzheimer's disease. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.09.22.23295984. [PMID: 37808872 PMCID: PMC10557812 DOI: 10.1101/2023.09.22.23295984] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/10/2023]
Abstract
Development of deep learning models to assess the degree of cognitive impairment on magnetic resonance imaging (MRI) scans has high translational significance. Performance of such models is often affected by potential variabilities stemming from independent protocols for data generation, imaging equipment, radiology artifacts, and demographic distributional shifts. Domain generalization (DG) frameworks have the potential to overcome these issues by learning signal from one or more source domains that can be transferable to unseen target domains. We developed an approach that leverages model interpretability as a means to improve generalizability of classification models across multiple cohorts. Using MRI scans and clinical diagnosis obtained from four independent cohorts (Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 1,821), the Framingham Heart Study (FHS, n = 304), the Australian Imaging Biomarkers and Lifestyle Study of Ageing (AIBL, n = 661), and the National Alzheimer's Coordinating Center (NACC, n = 4,647)), we trained a deep neural network that used model-identified regions of disease relevance to inform model training. We trained a classifier to distinguish persons with normal cognition (NC) from those with mild cognitive impairment (MCI) and Alzheimer's disease (AD) by aligning class-wise attention with a unified visual saliency prior computed offline per class over all training data. Our proposed method competes with state-of-the-art methods with improved correlation with postmortem histology, thus grounding our findings with gold standard evidence and paving a way towards validating DG frameworks.
Collapse
Affiliation(s)
- Diala Lteif
- Department of Computer Science, Boston University, Boston, MA, USA
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
| | - Sandeep Sreerama
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
| | - Sarah A. Bargal
- Department of Computer Science, Georgetown University, Washington, DC, USA
| | - Bryan A. Plummer
- Department of Computer Science, Boston University, Boston, MA, USA
| | - Rhoda Au
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
- Departments of Anatomy & Neurobiology and Neurology, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA; Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA; the Framingham Heart Study, Boston, MA, USA
- Boston University Alzheimer’s Disease Research Center, Boston, MA, USA
| | - Vijaya B. Kolachalama
- Department of Computer Science, Boston University, Boston, MA, USA
- Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
- Boston University Alzheimer’s Disease Research Center, Boston, MA, USA
- Faculty of Computing & Data Sciences, Boston University, Boston, MA, USA
| |
Collapse
|
47
|
Wei X, Li H, Zhu T, Li W, Li Y, Sui R. Deep Learning with Automatic Data Augmentation for Segmenting Schisis Cavities in the Optical Coherence Tomography Images of X-Linked Juvenile Retinoschisis Patients. Diagnostics (Basel) 2023; 13:3035. [PMID: 37835778 PMCID: PMC10572414 DOI: 10.3390/diagnostics13193035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 09/09/2023] [Accepted: 09/15/2023] [Indexed: 10/15/2023] Open
Abstract
X-linked juvenile retinoschisis (XLRS) is an inherited disorder characterized by retinal schisis cavities, which can be observed in optical coherence tomography (OCT) images. Monitoring disease progression necessitates the accurate segmentation and quantification of these cavities; yet, current manual methods are time consuming and result in subjective interpretations, highlighting the need for automated and precise solutions. We employed five state-of-the-art deep learning models-U-Net, U-Net++, Attention U-Net, Residual U-Net, and TransUNet-for the task, leveraging a dataset of 1500 OCT images from 30 patients. To enhance the models' performance, we utilized data augmentation strategies that were optimized via deep reinforcement learning. The deep learning models achieved a human-equivalent accuracy level in the segmentation of schisis cavities, with U-Net++ surpassing others by attaining an accuracy of 0.9927 and a Dice coefficient of 0.8568. By utilizing reinforcement-learning-based automatic data augmentation, deep learning segmentation models demonstrate a robust and precise method for the automated segmentation of schisis cavities in OCT images. These findings are a promising step toward enhancing clinical evaluation and treatment planning for XLRS.
Collapse
Affiliation(s)
| | | | | | | | | | - Ruifang Sui
- Department of Ophthalmology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, No. 1, Shuai Fu Yuan, Beijing 100730, China; (X.W.); (H.L.); (T.Z.); (W.L.); (Y.L.)
| |
Collapse
|
48
|
Karim MR, Islam T, Shajalal M, Beyan O, Lange C, Cochez M, Rebholz-Schuhmann D, Decker S. Explainable AI for Bioinformatics: Methods, Tools and Applications. Brief Bioinform 2023; 24:bbad236. [PMID: 37478371 DOI: 10.1093/bib/bbad236] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 05/10/2023] [Accepted: 05/26/2023] [Indexed: 07/23/2023] Open
Abstract
Artificial intelligence (AI) systems utilizing deep neural networks and machine learning (ML) algorithms are widely used for solving critical problems in bioinformatics, biomedical informatics and precision medicine. However, complex ML models that are often perceived as opaque and black-box methods make it difficult to understand the reasoning behind their decisions. This lack of transparency can be a challenge for both end-users and decision-makers, as well as AI developers. In sensitive areas such as healthcare, explainability and accountability are not only desirable properties but also legally required for AI systems that can have a significant impact on human lives. Fairness is another growing concern, as algorithmic decisions should not show bias or discrimination towards certain groups or individuals based on sensitive attributes. Explainable AI (XAI) aims to overcome the opaqueness of black-box models and to provide transparency in how AI systems make decisions. Interpretable ML models can explain how they make predictions and identify factors that influence their outcomes. However, the majority of the state-of-the-art interpretable ML methods are domain-agnostic and have evolved from fields such as computer vision, automated reasoning or statistics, making direct application to bioinformatics problems challenging without customization and domain adaptation. In this paper, we discuss the importance of explainability and algorithmic transparency in the context of bioinformatics. We provide an overview of model-specific and model-agnostic interpretable ML methods and tools and outline their potential limitations. We discuss how existing interpretable ML methods can be customized and fit to bioinformatics research problems. Further, through case studies in bioimaging, cancer genomics and text mining, we demonstrate how XAI methods can improve transparency and decision fairness. Our review aims at providing valuable insights and serving as a starting point for researchers wanting to enhance explainability and decision transparency while solving bioinformatics problems. GitHub: https://github.com/rezacsedu/XAI-for-bioinformatics.
Collapse
Affiliation(s)
- Md Rezaul Karim
- Computer Science 5 - Information Systems and Databases, RWTH Aachen University, Germany
- Department of Data Science and Artificial Intelligence, Fraunhofer FIT, Germany
| | - Tanhim Islam
- Computer Science 9 - Process and Data Science, RWTH Aachen University, Germany
| | | | - Oya Beyan
- Computer Science 5 - Information Systems and Databases, RWTH Aachen University, Germany
- University of Cologne, Faculty of Medicine and University Hospital Cologne, Institute for Medical Informatics, Germany
| | - Christoph Lange
- Computer Science 5 - Information Systems and Databases, RWTH Aachen University, Germany
- Department of Data Science and Artificial Intelligence, Fraunhofer FIT, Germany
| | - Michael Cochez
- Department of Computer Science, Vrije Universiteit Amsterdam, the Netherlands
- Elsevier Discovery Lab, Amsterdam, the Netherlands
| | - Dietrich Rebholz-Schuhmann
- ZBMED - Information Center for Life Sciences, Cologne, Germany
- Faculty of Medicine, University of Cologne, Germany
| | - Stefan Decker
- Computer Science 5 - Information Systems and Databases, RWTH Aachen University, Germany
- Department of Data Science and Artificial Intelligence, Fraunhofer FIT, Germany
| |
Collapse
|
49
|
Kazemimoghadam M, Yang Z, Chen M, Ma L, Lu W, Gu X. Leveraging global binary masks for structure segmentation in medical images. Phys Med Biol 2023; 68:10.1088/1361-6560/acf2e2. [PMID: 37607564 PMCID: PMC10511220 DOI: 10.1088/1361-6560/acf2e2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Accepted: 08/22/2023] [Indexed: 08/24/2023]
Abstract
Deep learning (DL) models for medical image segmentation are highly influenced by intensity variations of input images and lack generalization due to primarily utilizing pixels' intensity information for inference. Acquiring sufficient training data is another challenge limiting models' applications. Here, we proposed to leverage the consistency of organs' anatomical position and shape information in medical images. We introduced a framework leveraging recurring anatomical patterns through global binary masks for organ segmentation. Two scenarios were studied: (1) global binary masks were the only input for the U-Net based model, forcing exclusively encoding organs' position and shape information for rough segmentation or localization. (2) Global binary masks were incorporated as an additional channel providing position/shape clues to mitigate training data scarcity. Two datasets of the brain and heart computed tomography (CT) images with their ground-truth were split into (26:10:10) and (12:3:5) for training, validation, and test respectively. The two scenarios were evaluated using full training split as well as reduced subsets of training data. In scenario (1), training exclusively on global binary masks led to Dice scores of 0.77 ± 0.06 and 0.85 ± 0.04 for the brain and heart structures respectively. Average Euclidian distance of 3.12 ± 1.43 mm and 2.5 ± 0.93 mm were obtained relative to the center of mass of the ground truth for the brain and heart structures respectively. The outcomes indicated encoding a surprising degree of position and shape information through global binary masks. In scenario (2), incorporating global binary masks led to significantly higher accuracy relative to the model trained on only CT images in small subsets of training data; the performance improved by 4.3%-125.3% and 1.3%-48.1% for 1-8 training cases of the brain and heart datasets respectively. The findings imply the advantages of utilizing global binary masks for building models that are robust to image intensity variations as well as an effective approach to boost performance when access to labeled training data is highly limited.
Collapse
Affiliation(s)
- Mahdieh Kazemimoghadam
- Department of Radiation Oncology, the University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Zi Yang
- Department of Radiation Oncology, the University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Mingli Chen
- Department of Radiation Oncology, the University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Lin Ma
- Department of Radiation Oncology, the University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Weiguo Lu
- Department of Radiation Oncology, the University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
| | - Xuejun Gu
- Department of Radiation Oncology, the University of Texas Southwestern Medical Center, Dallas TX, 75390 USA
- Department of Radiation Oncology, Stanford University, Stanford, CA 94305
| |
Collapse
|
50
|
Yang Z, Yao S, Heng Y, Shen P, Lv T, Feng S, Tao L, Zhang W, Qiu W, Lu H, Cai W. Automated diagnosis and management of follicular thyroid nodules based on the devised small-dataset interpretable foreground optimization network deep learning: a multicenter diagnostic study. Int J Surg 2023; 109:2732-2741. [PMID: 37204464 PMCID: PMC10498847 DOI: 10.1097/js9.0000000000000506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Accepted: 05/10/2023] [Indexed: 05/20/2023]
Abstract
BACKGROUND Currently, follicular thyroid carcinoma (FTC) has a relatively low incidence with a lack of effective preoperative diagnostic means. To reduce the need for invasive diagnostic procedures and to address information deficiencies inherent in a small dataset, we utilized interpretable foreground optimization network deep learning to develop a reliable preoperative FTC detection system. METHODS In this study, a deep learning model (FThyNet) was established using preoperative ultrasound images. Data on patients in the training and internal validation cohort ( n =432) were obtained from Ruijin Hospital, China. Data on patients in the external validation cohort ( n =71) were obtained from four other clinical centers. We evaluated the predictive performance of FThyNet and its ability to generalize across multiple external centers and compared the results yielded with assessments from physicians directly predicting FTC outcomes. In addition, the influence of texture information around the nodule edge on the prediction results was evaluated. RESULTS FThyNet had a consistently high accuracy in predicting FTC with an area under the receiver operating characteristic curve (AUC) of 89.0% [95% CI 87.0-90.9]. Particularly, the AUC for grossly invasive FTC reached 90.3%, which was significantly higher than that of the radiologists (56.1% [95% CI 51.8-60.3]). The parametric visualization study found that those nodules with blurred edges and relatively distorted surrounding textures were more likely to have FTC. Furthermore, edge texture information played an important role in FTC prediction with an AUC of 68.3% [95% CI 61.5-75.5], and highly invasive malignancies had the highest texture complexity. CONCLUSION FThyNet could effectively predict FTC, provide explanations consistent with pathological knowledge, and improve clinical understanding of the disease.
Collapse
Affiliation(s)
- Zheyu Yang
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine
| | - Siqiong Yao
- School of Life Sciences and Biotechnology, Shanghai Jiao Tong University
| | - Yu Heng
- Department of Otolaryngology, Eye, Ear, Nose and Throat Hospital, Fudan University
| | - Pengcheng Shen
- School of Life Sciences and Biotechnology, Shanghai Jiao Tong University
| | - Tian Lv
- Department of Head, Neck and Thyroid Surgery, Zhejiang Provincial People’s Hospital, People’s Hospital of Hangzhou Medical College, Hangzhou, People’s Republic of China
| | - Siqi Feng
- Department of General Surgery, Liaoning Cancer Hospital & Institute, Shenyang
| | - Lei Tao
- Department of Otolaryngology, Eye, Ear, Nose and Throat Hospital, Fudan University
| | - Weituo Zhang
- Shanghai Tong Ren Hospital and Clinical Research Institute
- Hong Qiao International Institute of Medicine, Shanghai
| | - Weihua Qiu
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine
- Department of General Surgery, Ruijin Hospital Gubei Campus, Shanghai Jiao Tong University School of Medicine
| | - Hui Lu
- School of Life Sciences and Biotechnology, Shanghai Jiao Tong University
| | - Wei Cai
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine
| |
Collapse
|