1
|
Qin C, Wang Y, Zhang J. URCA: Uncertainty-based region clipping algorithm for semi-supervised medical image segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108278. [PMID: 38878360 DOI: 10.1016/j.cmpb.2024.108278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 05/24/2024] [Accepted: 06/06/2024] [Indexed: 07/28/2024]
Abstract
BACKGROUND AND OBJECTIVE Training convolutional neural networks based on large amount of labeled data has made great progress in the field of image segmentation. However, in medical image segmentation tasks, annotating the data is expensive and time-consuming because pixel-level annotation requires experts in the relevant field. Currently, the combination of consistent regularization and pseudo labeling-based semi-supervised methods has shown good performance in image segmentation. However, in the training process, a portion of low-confidence pseudo labels are generated by the model. And the semi-supervised segmentation method still has the problem of distribution bias between labeled and unlabeled data. The objective of this study is to address the challenges of semi-supervised learning and improve the segmentation accuracy of semi-supervised models on medical images. METHODS To address these issues, we propose an Uncertainty-based Region Clipping Algorithm for semi-supervised medical image segmentation, which consists of two main modules. A module is introduced to compute the uncertainty of two sub-networks predictions with diversity using Monte Carlo Dropout, allowing the model to gradually learn from more reliable targets. To retain model diversity, we use different loss functions for different branches and use Non-Maximum Suppression in one of the branches. The other module is proposed to generate new samples by masking the low-confidence pixels in the original image based on uncertainty information. New samples are fed into the model to facilitate the model to generate pseudo labels with high confidence and enlarge the training data distribution. RESULTS Comprehensive experiments on the combination of two benchmarks ACDC and BraTS2019 show that our proposed model outperforms state-of-the-art methods in terms of Dice, HD95 and ASD. The results reach an average Dice score of 87.86 % and a HD95 score of 4.214 mm on ACDC dataset. For the brain tumor segmentation, the results reach an average Dice score of 84.79 % and a HD score of 10.13 mm. CONCLUSIONS Our proposed method improves the accuracy of semi-supervised medical image segmentation. Extensive experiments on two public medical image datasets including 2D and 3D modalities demonstrate the superiority of our model. The code is available at: https://github.com/QuintinDong/URCA.
Collapse
Affiliation(s)
- Chendong Qin
- University of Shanghai for Science and Technology, School of Opto-Electronic Information and Computer Engineering, Department of Control Science and Engineering, 516 War Industrial Road, Shanghai 200093, China
| | - Yongxiong Wang
- University of Shanghai for Science and Technology, School of Opto-Electronic Information and Computer Engineering, Department of Control Science and Engineering, 516 War Industrial Road, Shanghai 200093, China.
| | - Jiapeng Zhang
- University of Shanghai for Science and Technology, School of Opto-Electronic Information and Computer Engineering, Department of Control Science and Engineering, 516 War Industrial Road, Shanghai 200093, China
| |
Collapse
|
2
|
Lüscher TF, Wenzl FA, D'Ascenzo F, Friedman PA, Antoniades C. Artificial intelligence in cardiovascular medicine: clinical applications. Eur Heart J 2024:ehae465. [PMID: 39158472 DOI: 10.1093/eurheartj/ehae465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Revised: 06/07/2024] [Accepted: 07/03/2024] [Indexed: 08/20/2024] Open
Abstract
Clinical medicine requires the integration of various forms of patient data including demographics, symptom characteristics, electrocardiogram findings, laboratory values, biomarker levels, and imaging studies. Decision-making on the optimal management should be based on a high probability that the envisaged treatment is appropriate, provides benefit, and bears no or little potential harm. To that end, personalized risk-benefit considerations should guide the management of individual patients to achieve optimal results. These basic clinical tasks have become more and more challenging with the massively growing data now available; artificial intelligence and machine learning (AI/ML) can provide assistance for clinicians by obtaining and comprehensively preparing the history of patients, analysing face and voice and other clinical features, by integrating laboratory results, biomarkers, and imaging. Furthermore, AI/ML can provide a comprehensive risk assessment as a basis of optimal acute and chronic care. The clinical usefulness of AI/ML algorithms should be carefully assessed, validated with confirmation datasets before clinical use, and repeatedly re-evaluated as patient phenotypes change. This review provides an overview of the current data revolution that has changed and will continue to change the face of clinical medicine radically, if properly used, to the benefit of physicians and patients alike.
Collapse
Affiliation(s)
- Thomas F Lüscher
- Royal Brompton and Harefield Hospitals, London, UK
- National Heart and Lung Institute, Imperial College London, UK
- Cardiovascular Academic Group, King's College, London, UK
- Center for Molecular Cardiology, University of Zurich, Wagistrasse 12, 8952 Schlieren - Zurich, Switzerland
| | - Florian A Wenzl
- Center for Molecular Cardiology, University of Zurich, Wagistrasse 12, 8952 Schlieren - Zurich, Switzerland
- National Disease Registration and Analysis Service, NHS, London, UK
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
- Department of Clinical Sciences, Karolinska Institutet, Stockholm, Sweden
| | - Fabrizio D'Ascenzo
- Division of Cardiology, Cardiovascular and Thoracic Department, Città della Salute e della Scienza Hospital, Turin, Italy
| | - Paul A Friedman
- Department of Cardiovascular Medicine, Mayo Clinic and Mayo Foundation, Rochester, MN, USA
| | - Charalambos Antoniades
- Acute Multidisciplinary Imaging and Interventional Centre, RDM Division of Cardiovascular Medicine, University of Oxford, Headley Way, Headington, Oxford OX39DU, UK
| |
Collapse
|
3
|
Crawley R, Amirrajab S, Lustermans D, Holtackers RJ, Plein S, Veta M, Breeuwer M, Chiribiri A, Scannell CM. Automated cardiovascular MR myocardial scar quantification with unsupervised domain adaptation. Eur Radiol Exp 2024; 8:93. [PMID: 39143405 PMCID: PMC11324636 DOI: 10.1186/s41747-024-00497-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Accepted: 07/15/2024] [Indexed: 08/16/2024] Open
Abstract
Quantification of myocardial scar from late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) images can be facilitated by automated artificial intelligence (AI)-based analysis. However, AI models are susceptible to domain shifts in which the model performance is degraded when applied to data with different characteristics than the original training data. In this study, CycleGAN models were trained to translate local hospital data to the appearance of a public LGE CMR dataset. After domain adaptation, an AI scar quantification pipeline including myocardium segmentation, scar segmentation, and computation of scar burden, previously developed on the public dataset, was evaluated on an external test set including 44 patients clinically assessed for ischemic scar. The mean ± standard deviation Dice similarity coefficients between the manual and AI-predicted segmentations in all patients were similar to those previously reported: 0.76 ± 0.05 for myocardium and 0.75 ± 0.32 for scar, 0.41 ± 0.12 for scar in scans with pathological findings. Bland-Altman analysis showed a mean bias in scar burden percentage of -0.62% with limits of agreement from -8.4% to 7.17%. These results show the feasibility of deploying AI models, trained with public data, for LGE CMR quantification on local clinical data using unsupervised CycleGAN-based domain adaptation. RELEVANCE STATEMENT: Our study demonstrated the possibility of using AI models trained from public databases to be applied to patient data acquired at a specific institution with different acquisition settings, without additional manual labor to obtain further training labels.
Collapse
Affiliation(s)
- Richard Crawley
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Sina Amirrajab
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Didier Lustermans
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Robert J Holtackers
- Cardiovascular Research Institute Maastricht (CARIM), Maastricht University, Maastricht, the Netherlands
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Sven Plein
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
- Leeds Institute of Cardiovascular and Metabolic Medicine, University of Leeds, Leeds, UK
| | - Mitko Veta
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Marcel Breeuwer
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Amedeo Chiribiri
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Cian M Scannell
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK.
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands.
| |
Collapse
|
4
|
De Santi LA, Meloni A, Santarelli MF, Pistoia L, Spasiano A, Casini T, Putti MC, Cuccia L, Cademartiri F, Positano V. Left Ventricle Detection from Cardiac Magnetic Resonance Relaxometry Images Using Visual Transformer. SENSORS (BASEL, SWITZERLAND) 2023; 23:3321. [PMID: 36992032 PMCID: PMC10052975 DOI: 10.3390/s23063321] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 03/16/2023] [Accepted: 03/20/2023] [Indexed: 06/19/2023]
Abstract
Left Ventricle (LV) detection from Cardiac Magnetic Resonance (CMR) imaging is a fundamental step, preliminary to myocardium segmentation and characterization. This paper focuses on the application of a Visual Transformer (ViT), a novel neural network architecture, to automatically detect LV from CMR relaxometry sequences. We implemented an object detector based on the ViT model to identify LV from CMR multi-echo T2* sequences. We evaluated performances differentiated by slice location according to the American Heart Association model using 5-fold cross-validation and on an independent dataset of CMR T2*, T2, and T1 acquisitions. To the best of our knowledge, this is the first attempt to localize LV from relaxometry sequences and the first application of ViT for LV detection. We collected an Intersection over Union (IoU) index of 0.68 and a Correct Identification Rate (CIR) of blood pool centroid of 0.99, comparable with other state-of-the-art methods. IoU and CIR values were significantly lower in apical slices. No significant differences in performances were assessed on independent T2* dataset (IoU = 0.68, p = 0.405; CIR = 0.94, p = 0.066). Performances were significantly worse on the T2 and T1 independent datasets (T2: IoU = 0.62, CIR = 0.95; T1: IoU = 0.67, CIR = 0.98), but still encouraging considering the different types of acquisition. This study confirms the feasibility of the application of ViT architectures in LV detection and defines a benchmark for relaxometry imaging.
Collapse
Affiliation(s)
- Lisa Anita De Santi
- Department of Information Engineering, University of Pisa, 56122 Pisa, Italy;
- U.O.C. Bioingegneria, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy;
| | - Antonella Meloni
- U.O.C. Bioingegneria, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy;
- Department of Radiology, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy; (L.P.)
| | | | - Laura Pistoia
- Department of Radiology, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy; (L.P.)
| | - Anna Spasiano
- Unità Operativa Semplice Dipartimentale Malattie Rare del Globulo Rosso, Azienda Ospedaliera di Rilievo Nazionale “A. Cardarelli”, 80131 Napoli, Italy
| | - Tommaso Casini
- Centro Talassemie ed Emoglobinopatie, Ospedale “Meyer”, 50139 Firenze, Italy
| | - Maria Caterina Putti
- Clinica di Emato-Oncologia Pediatrica, Dipartimento di Salute della Donna e del Bambino, Azienda Ospedale Università, 35128 Padova, Italy
| | - Liana Cuccia
- Unità Operativa Complessa Ematologia con Talassemia, ARNAS Civico “Benfratelli-Di Cristina”, 90127 Palermo, Italy
| | - Filippo Cademartiri
- Department of Radiology, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy; (L.P.)
| | - Vincenzo Positano
- U.O.C. Bioingegneria, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy;
- Department of Radiology, Fondazione G. Monasterio CNR-Regione Toscana, 56124 Pisa, Italy; (L.P.)
| |
Collapse
|
5
|
Zhang HT, Sun ZY, Zhou J, Gao S, Dong JH, Liu Y, Bai X, Ma JL, Li M, Li G, Cai JM, Sheng FG. Computed tomography-based COVID-19 triage through a deep neural network using mask-weighted global average pooling. Front Cell Infect Microbiol 2023; 13:1116285. [PMID: 36936770 PMCID: PMC10020619 DOI: 10.3389/fcimb.2023.1116285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 02/13/2023] [Indexed: 03/06/2023] Open
Abstract
Background There is an urgent need to find an effective and accurate method for triaging coronavirus disease 2019 (COVID-19) patients from millions or billions of people. Therefore, this study aimed to develop a novel deep-learning approach for COVID-19 triage based on chest computed tomography (CT) images, including normal, pneumonia, and COVID-19 cases. Methods A total of 2,809 chest CT scans (1,105 COVID-19, 854 normal, and 850 non-3COVID-19 pneumonia cases) were acquired for this study and classified into the training set (n = 2,329) and test set (n = 480). A U-net-based convolutional neural network was used for lung segmentation, and a mask-weighted global average pooling (GAP) method was proposed for the deep neural network to improve the performance of COVID-19 classification between COVID-19 and normal or common pneumonia cases. Results The results for lung segmentation reached a dice value of 96.5% on 30 independent CT scans. The performance of the mask-weighted GAP method achieved the COVID-19 triage with a sensitivity of 96.5% and specificity of 87.8% using the testing dataset. The mask-weighted GAP method demonstrated 0.9% and 2% improvements in sensitivity and specificity, respectively, compared with the normal GAP. In addition, fusion images between the CT images and the highlighted area from the deep learning model using the Grad-CAM method, indicating the lesion region detected using the deep learning method, were drawn and could also be confirmed by radiologists. Conclusions This study proposed a mask-weighted GAP-based deep learning method and obtained promising results for COVID-19 triage based on chest CT images. Furthermore, it can be considered a convenient tool to assist doctors in diagnosing COVID-19.
Collapse
Affiliation(s)
- Hong-Tao Zhang
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Ze-Yu Sun
- Algorithm Center, Keya Medical Technology Co., Ltd, Shenzhen, China
| | - Juan Zhou
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Shen Gao
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Jing-Hui Dong
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Yuan Liu
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Xu Bai
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Jin-Lin Ma
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Ming Li
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
| | - Guang Li
- Algorithm Center, Keya Medical Technology Co., Ltd, Shenzhen, China
| | - Jian-Ming Cai
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
- *Correspondence: Jian-Ming Cai, ; Fu-Geng Sheng,
| | - Fu-Geng Sheng
- Department of Radiology, the Fifth Medical Center of Chinese PLA General Hospital, Beijing, China
- *Correspondence: Jian-Ming Cai, ; Fu-Geng Sheng,
| |
Collapse
|