1
|
Crawley R, Amirrajab S, Lustermans D, Holtackers RJ, Plein S, Veta M, Breeuwer M, Chiribiri A, Scannell CM. Automated cardiovascular MR myocardial scar quantification with unsupervised domain adaptation. Eur Radiol Exp 2024; 8:93. [PMID: 39143405 PMCID: PMC11324636 DOI: 10.1186/s41747-024-00497-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Accepted: 07/15/2024] [Indexed: 08/16/2024] Open
Abstract
Quantification of myocardial scar from late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) images can be facilitated by automated artificial intelligence (AI)-based analysis. However, AI models are susceptible to domain shifts in which the model performance is degraded when applied to data with different characteristics than the original training data. In this study, CycleGAN models were trained to translate local hospital data to the appearance of a public LGE CMR dataset. After domain adaptation, an AI scar quantification pipeline including myocardium segmentation, scar segmentation, and computation of scar burden, previously developed on the public dataset, was evaluated on an external test set including 44 patients clinically assessed for ischemic scar. The mean ± standard deviation Dice similarity coefficients between the manual and AI-predicted segmentations in all patients were similar to those previously reported: 0.76 ± 0.05 for myocardium and 0.75 ± 0.32 for scar, 0.41 ± 0.12 for scar in scans with pathological findings. Bland-Altman analysis showed a mean bias in scar burden percentage of -0.62% with limits of agreement from -8.4% to 7.17%. These results show the feasibility of deploying AI models, trained with public data, for LGE CMR quantification on local clinical data using unsupervised CycleGAN-based domain adaptation. RELEVANCE STATEMENT: Our study demonstrated the possibility of using AI models trained from public databases to be applied to patient data acquired at a specific institution with different acquisition settings, without additional manual labor to obtain further training labels.
Collapse
Affiliation(s)
- Richard Crawley
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Sina Amirrajab
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Didier Lustermans
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Robert J Holtackers
- Cardiovascular Research Institute Maastricht (CARIM), Maastricht University, Maastricht, the Netherlands
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Sven Plein
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
- Leeds Institute of Cardiovascular and Metabolic Medicine, University of Leeds, Leeds, UK
| | - Mitko Veta
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Marcel Breeuwer
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Amedeo Chiribiri
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Cian M Scannell
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK.
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands.
| |
Collapse
|
2
|
Jaspers TJM, Martens B, Crawley R, Jada L, Amirrajab S, Breeuwer M, Holtackers RJ, Chiribiri A, Scannell CM. Deep Learning Synthesis of White-Blood From Dark-Blood Late Gadolinium Enhancement Cardiac Magnetic Resonance. Invest Radiol 2024:00004424-990000000-00213. [PMID: 38687025 DOI: 10.1097/rli.0000000000001086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
OBJECTIVES Dark-blood late gadolinium enhancement (DB-LGE) cardiac magnetic resonance has been proposed as an alternative to standard white-blood LGE (WB-LGE) imaging protocols to enhance scar-to-blood contrast without compromising scar-to-myocardium contrast. In practice, both DB and WB contrasts may have clinical utility, but acquiring both has the drawback of additional acquisition time. The aim of this study was to develop and evaluate a deep learning method to generate synthetic WB-LGE images from DB-LGE, allowing the assessment of both contrasts without additional scan time. MATERIALS AND METHODS DB-LGE and WB-LGE data from 215 patients were used to train 2 types of unpaired image-to-image translation deep learning models, cycle-consistent generative adversarial network (CycleGAN) and contrastive unpaired translation, with 5 different loss function hyperparameter settings each. Initially, the best hyperparameter setting was determined for each model type based on the Fréchet inception distance and the visual assessment of expert readers. Then, the CycleGAN and contrastive unpaired translation models with the optimal hyperparameters were directly compared. Finally, with the best model chosen, the quantification of scar based on the synthetic WB-LGE images was compared with the truly acquired WB-LGE. RESULTS The CycleGAN architecture for unpaired image-to-image translation was found to provide the most realistic synthetic WB-LGE images from DB-LGE images. The results showed that it was difficult for visual readers to distinguish if an image was true or synthetic (55% correctly classified). In addition, scar burden quantification with the synthetic data was highly correlated with the analysis of the truly acquired images. Bland-Altman analysis found a mean bias in percentage scar burden between the quantification of the real WB and synthetic white-blood images of 0.44% with limits of agreement from -10.85% to 11.74%. The mean image quality of the real WB images (3.53/5) was scored higher than the synthetic white-blood images (3.03), P = 0.009. CONCLUSIONS This study proposed a CycleGAN model to generate synthetic WB-LGE from DB-LGE images to allow assessment of both image contrasts without additional scan time. This work represents a clinically focused assessment of synthetic medical images generated by artificial intelligence, a topic with significant potential for a multitude of applications. However, further evaluation is warranted before clinical adoption.
Collapse
Affiliation(s)
- Tim J M Jaspers
- From the Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands (T.J.M.J., S.A., M.B., C.M.S.); School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom (T.J.M.J., R.C., L.J., R.J.H., A.C., C.M.S.); Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands (T.J.M.J.); Cardiovascular Research Institute Maastricht, Maastricht University, Maastricht, the Netherlands (B.M., R.J.H.); and Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre, Maastricht, the Netherlands (B.M., R.J.H.)
| | | | | | | | | | | | | | | | | |
Collapse
|
3
|
Hoh T, Margolis I, Weine J, Joyce T, Manka R, Weisskopf M, Cesarovic N, Fuetterer M, Kozerke S. Impact of late gadolinium enhancement image acquisition resolution on neural network based automatic scar segmentation. J Cardiovasc Magn Reson 2024; 26:101031. [PMID: 38431078 PMCID: PMC10981112 DOI: 10.1016/j.jocmr.2024.101031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Accepted: 02/23/2024] [Indexed: 03/05/2024] Open
Abstract
BACKGROUND Automatic myocardial scar segmentation from late gadolinium enhancement (LGE) images using neural networks promises an alternative to time-consuming and observer-dependent semi-automatic approaches. However, alterations in data acquisition, reconstruction as well as post-processing may compromise network performance. The objective of the present work was to systematically assess network performance degradation due to a mismatch of point-spread function between training and testing data. METHODS Thirty-six high-resolution (0.7×0.7×2.0 mm3) LGE k-space datasets were acquired post-mortem in porcine models of myocardial infarction. The in-plane point-spread function and hence in-plane resolution Δx was retrospectively degraded using k-space lowpass filtering, while field-of-view and matrix size were kept constant. Manual segmentation of the left ventricle (LV) and healthy remote myocardium was performed to quantify location and area (% of myocardium) of scar by thresholding (≥ SD5 above remote). Three standard U-Nets were trained on training resolutions Δxtrain = 0.7, 1.2 and 1.7 mm to predict endo- and epicardial borders of LV myocardium and scar. The scar prediction of the three networks for varying test resolutions (Δxtest = 0.7 to 1.7 mm) was compared against the reference SD5 thresholding at 0.7 mm. Finally, a fourth network trained on a combination of resolutions (Δxtrain = 0.7 to 1.7 mm) was tested. RESULTS The prediction of relative scar areas showed the highest precision when the resolution of the test data was identical to or close to the resolution used during training. The median fractional scar errors and precisions (IQR) from networks trained and tested on the same resolution were 0.0 percentage points (p.p.) (1.24 - 1.45), and - 0.5 - 0.0 p.p. (2.00 - 3.25) for networks trained and tested on the most differing resolutions, respectively. Deploying the network trained on multiple resolutions resulted in reduced resolution dependency with median scar errors and IQRs of 0.0 p.p. (1.24 - 1.69) for all investigated test resolutions. CONCLUSION A mismatch of the imaging point-spread function between training and test data can lead to degradation of scar segmentation when using current U-Net architectures as demonstrated on LGE porcine myocardial infarction data. Training networks on multi-resolution data can alleviate the resolution dependency.
Collapse
Affiliation(s)
- Tobias Hoh
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland.
| | - Isabel Margolis
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland.
| | - Jonathan Weine
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland
| | - Thomas Joyce
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland.
| | - Robert Manka
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland; Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland; Department of Cardiology, University Heart Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland.
| | - Miriam Weisskopf
- Center of Surgical Research, University Hospital Zurich, University of Zurich, Zurich, Switzerland.
| | - Nikola Cesarovic
- Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland; Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.
| | - Maximilian Fuetterer
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland.
| | - Sebastian Kozerke
- Institute for Biomedical Engineering, University and ETH Zurich, Zurich, Switzerland.
| |
Collapse
|
4
|
Scannell CM, Crawley R, Alskaf E, Breeuwer M, Plein S, Chiribiri A. High-resolution quantification of stress perfusion defects by cardiac magnetic resonance. EUROPEAN HEART JOURNAL. IMAGING METHODS AND PRACTICE 2024; 2:qyae001. [PMID: 38283662 PMCID: PMC10810243 DOI: 10.1093/ehjimp/qyae001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 01/04/2024] [Indexed: 01/30/2024]
Abstract
Aims Quantitative stress perfusion cardiac magnetic resonance (CMR) is becoming more widely available, but it is still unclear how to integrate this information into clinical decision-making. Typically, pixel-wise perfusion maps are generated, but diagnostic and prognostic studies have summarized perfusion as just one value per patient or in 16 myocardial segments. In this study, the reporting of quantitative perfusion maps is extended from the standard 16 segments to a high-resolution bullseye. Cut-off thresholds are established for the high-resolution bullseye, and the identified perfusion defects are compared with visual assessment. Methods and results Thirty-four patients with known or suspected coronary artery disease were retrospectively analysed. Visual perfusion defects were contoured on the CMR images and pixel-wise quantitative perfusion maps were generated. Cut-off values were established on the high-resolution bullseye consisting of 1800 points and compared with the per-segment, per-coronary, and per-patient resolution thresholds. Quantitative stress perfusion was significantly lower in visually abnormal pixels, 1.11 (0.75-1.57) vs. 2.35 (1.82-2.9) mL/min/g (Mann-Whitney U test P < 0.001), with an optimal cut-off of 1.72 mL/min/g. This was lower than the segment-wise optimal threshold of 1.92 mL/min/g. The Bland-Altman analysis showed that visual assessment underestimated large perfusion defects compared with the quantification with good agreement for smaller defect burdens. A Dice overlap of 0.68 (0.57-0.78) was found. Conclusion This study introduces a high-resolution bullseye consisting of 1800 points, rather than 16, per patient for reporting quantitative stress perfusion, which may improve sensitivity. Using this representation, the threshold required to identify areas of reduced perfusion is lower than for segmental analysis.
Collapse
Affiliation(s)
- Cian M Scannell
- Department of Biomedical Engineering, Eindhoven University of Technology, Groene Loper 5, 5612 AEEindhoven, The Netherlands
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, UK
| | - Richard Crawley
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, UK
| | - Ebraham Alskaf
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, UK
| | - Marcel Breeuwer
- Department of Biomedical Engineering, Eindhoven University of Technology, Groene Loper 5, 5612 AEEindhoven, The Netherlands
| | - Sven Plein
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, UK
- Leeds Institute of Cardiovascular and Metabolic Medicine, University of Leeds, Leeds LS2 9JT, UK
| | - Amedeo Chiribiri
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, UK
| |
Collapse
|
5
|
Oliveira GC, Ngo QC, Passos LA, Papa JP, Jodas DS, Kumar D. Tabular data augmentation for video-based detection of hypomimia in Parkinson's disease. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107713. [PMID: 37531692 DOI: 10.1016/j.cmpb.2023.107713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 04/19/2023] [Accepted: 07/07/2023] [Indexed: 08/04/2023]
Abstract
BACKGROUND AND OBJECTIVE This paper presents a method for the computerized detection of hypomimia in people with Parkinson's disease (PD). It overcomes the difficulty of the small and unbalanced size of available datasets. METHODS A public dataset consisting of features of the video recordings of people with PD with four facial expressions was used. Synthetic data was generated using a Conditional Generative Adversarial Network (CGAN) for training augmentation. After training the model, Test-Time Augmentation was performed. The classification was conducted using the original test set to prevent bias in the results. RESULTS The employment of CGAN followed by Test-Time Augmentation led to an accuracy of classification of the videos of 83%, specificity of 82%, and sensitivity of 85% in the test set that the prevalence of PD was around 7% and where real data was used for testing. This is a significant improvement compared with other similar studies. The results show that while the technique was able to detect people with PD, there were a number of false positives. Hence this is suitable for applications such as population screening or assisting clinicians, but at this stage is not suitable for diagnosis. CONCLUSIONS This work has the potential for assisting neurologists to perform online diagnose and monitoring their patients. However, it is essential to test this for different ethnicity and to test its repeatability.
Collapse
Affiliation(s)
- Guilherme C Oliveira
- School of Sciences, São Paulo State University, São Paulo, Brazil; School of Engineering, Royal Melbourne Institute of Technology, Victoria, Australia.
| | - Quoc C Ngo
- School of Engineering, Royal Melbourne Institute of Technology, Victoria, Australia.
| | - Leandro A Passos
- CMI Lab, School of Engineering and Informatics, University of Wolverhampton, Wolverhampton, UK.
| | - João P Papa
- School of Sciences, São Paulo State University, São Paulo, Brazil.
| | - Danilo S Jodas
- School of Sciences, São Paulo State University, São Paulo, Brazil.
| | - Dinesh Kumar
- School of Engineering, Royal Melbourne Institute of Technology, Victoria, Australia.
| |
Collapse
|
6
|
Mamalakis M, Garg P, Nelson T, Lee J, Swift AJ, Wild JM, Clayton RH. Artificial Intelligence framework with traditional computer vision and deep learning approaches for optimal automatic segmentation of left ventricle with scar. Artif Intell Med 2023; 143:102610. [PMID: 37673578 DOI: 10.1016/j.artmed.2023.102610] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 05/17/2023] [Accepted: 06/06/2023] [Indexed: 09/08/2023]
Abstract
Automatic segmentation of the cardiac left ventricle with scars remains a challenging and clinically significant task, as it is essential for patient diagnosis and treatment pathways. This study aimed to develop a novel framework and cost function to achieve optimal automatic segmentation of the left ventricle with scars using LGE-MRI images. To ensure the generalization of the framework, an unbiased validation protocol was established using out-of-distribution (OOD) internal and external validation cohorts, and intra-observation and inter-observer variability ground truths. The framework employs a combination of traditional computer vision techniques and deep learning, to achieve optimal segmentation results. The traditional approach uses multi-atlas techniques, active contours, and k-means methods, while the deep learning approach utilizes various deep learning techniques and networks. The study found that the traditional computer vision technique delivered more accurate results than deep learning, except in cases where there was breath misalignment error. The optimal solution of the framework achieved robust and generalized results with Dice scores of 82.8 ± 6.4% and 72.1 ± 4.6% in the internal and external OOD cohorts, respectively. The developed framework offers a high-performance solution for automatic segmentation of the left ventricle with scars using LGE-MRI. Unlike existing state-of-the-art approaches, it achieves unbiased results across different hospitals and vendors without the need for training or tuning in hospital cohorts. This framework offers a valuable tool for experts to accomplish the task of fully automatic segmentation of the left ventricle with scars based on a single-modality cardiac scan.
Collapse
Affiliation(s)
- Michail Mamalakis
- Insigneo Institute for in-silico, Medicine, University of Sheffield, Sheffield, S1 4DP, UK; Department of Computer Science, University of Sheffield, Regent Court, Sheffield, S1 4DP, UK.
| | - Pankaj Garg
- Department of Cardiology, Sheffield Teaching Hospitals Sheffield S5 7AU, UK
| | - Tom Nelson
- Department of Cardiology, Sheffield Teaching Hospitals Sheffield S5 7AU, UK
| | - Justin Lee
- Department of Cardiology, Sheffield Teaching Hospitals Sheffield S5 7AU, UK
| | - Andrew J Swift
- Department of Computer Science, University of Sheffield, Regent Court, Sheffield, S1 4DP, UK; Department of Infection, Immunity & Cardiovascular Disease, University of Sheffield, Sheffield, UK
| | - James M Wild
- Insigneo Institute for in-silico, Medicine, University of Sheffield, Sheffield, S1 4DP, UK; Polaris, Imaging Sciences, Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Sheffield, UK
| | - Richard H Clayton
- Insigneo Institute for in-silico, Medicine, University of Sheffield, Sheffield, S1 4DP, UK; Department of Computer Science, University of Sheffield, Regent Court, Sheffield, S1 4DP, UK.
| |
Collapse
|
7
|
Rather IH, Kumar S. Generative adversarial network based synthetic data training model for lightweight convolutional neural networks. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-23. [PMID: 37362646 PMCID: PMC10199442 DOI: 10.1007/s11042-023-15747-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 04/17/2023] [Accepted: 04/25/2023] [Indexed: 06/28/2023]
Abstract
Inadequate training data is a significant challenge for deep learning techniques, particularly in applications where data is difficult to get, and publicly available datasets are uncommon owing to ethical and privacy concerns. Various approaches, such as data augmentation and transfer learning, are employed to address this problem, which help to some extent in removing this limitation. However, after a certain amount of data augmentation, the quality of the generated data stalls, and transfer learning suffers from the issue of negative transfer. This paper proposes a novel generative adversarial network-based synthetic data training (GAN-ST) model to generate synthetic data for training a lightweight convolutional neural network (CNN). An enhanced generator is proposed to quickly saturate and cover the colour space of the training distribution. The GAN-ST model is based on Deep Convolutional Generative Adversarial Network(s) (DCGAN) and Conditional Generative Adversarial Network(s) (CGAN) models, which consist of an enhanced generator. The study evaluates the accuracy of a CNN model on the MNIST and CIFAR 10 datasets using both original and synthetic data. The results revealed an impressive classifier accuracy on the MNIST dataset, achieving an accuracy of 99.38% on GAN-ST-generated synthetic training data, which is only 0.05% lower than the performance on original data-based training. The classifier performance on the CIFAR dataset is also remarkable, achieving an accuracy of 90.23%. The performance of CNN trained using GAN-ST-based synthetic data is notable, with the most considerable improvement of 0.66% and 7.06%, over a single GAN-based synthetic data training for the MNIST and CIFAR datasets, respectively. By training two GANs independently, the GAN-ST model covers different parts of the original data distribution, resulting in a more diverse and realistic training data set for the classifier. This diverse set of synthetic data, when used to train a CNN, shows better generalization to new data, leading to improved classification accuracy.
Collapse
Affiliation(s)
- Ishfaq Hussain Rather
- School of Computer & Systems Sciences, Jawaharlal Nehru University, New Delhi, India
| | - Sushil Kumar
- School of Computer & Systems Sciences, Jawaharlal Nehru University, New Delhi, India
| |
Collapse
|