1
|
Jain SS, Elias P, Poterucha T, Randazzo M, Lopez Jimenez F, Khera R, Perez M, Ouyang D, Pirruccello J, Salerno M, Einstein AJ, Avram R, Tison GH, Nadkarni G, Natarajan V, Pierson E, Beecy A, Kumaraiah D, Haggerty C, Avari Silva JN, Maddox TM. Artificial Intelligence in Cardiovascular Care-Part 2: Applications: JACC Review Topic of the Week. J Am Coll Cardiol 2024; 83:2487-2496. [PMID: 38593945 DOI: 10.1016/j.jacc.2024.03.401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Accepted: 03/14/2024] [Indexed: 04/11/2024]
Abstract
Recent artificial intelligence (AI) advancements in cardiovascular care offer potential enhancements in effective diagnosis, treatment, and outcomes. More than 600 U.S. Food and Drug Administration-approved clinical AI algorithms now exist, with 10% focusing on cardiovascular applications, highlighting the growing opportunities for AI to augment care. This review discusses the latest advancements in the field of AI, with a particular focus on the utilization of multimodal inputs and the field of generative AI. Further discussions in this review involve an approach to understanding the larger context in which AI-augmented care may exist, and include a discussion of the need for rigorous evaluation, appropriate infrastructure for deployment, ethics and equity assessments, regulatory oversight, and viable business cases for deployment. Embracing this rapidly evolving technology while setting an appropriately high evaluation benchmark with careful and patient-centered implementation will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.
Collapse
Affiliation(s)
- Sneha S Jain
- Division of Cardiology, Stanford University School of Medicine, Palo Alto, California, USA
| | - Pierre Elias
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA; Department of Biomedical Informatics Columbia University Irving Medical Center, New York, New York, USA
| | - Timothy Poterucha
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA
| | - Michael Randazzo
- Division of Cardiology, University of Chicago Medical Center, Chicago, Illinois, USA
| | | | - Rohan Khera
- Division of Cardiology, Yale School of Medicine, New Haven, Connecticut, USA
| | - Marco Perez
- Division of Cardiology, Stanford University School of Medicine, Palo Alto, California, USA
| | - David Ouyang
- Division of Cardiology, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | - James Pirruccello
- Division of Cardiology, University of California San Francisco, San Francisco, California, USA
| | - Michael Salerno
- Division of Cardiology, Stanford University School of Medicine, Palo Alto, California, USA
| | - Andrew J Einstein
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA
| | - Robert Avram
- Division of Cardiology, Montreal Heart Institute, Montreal, Quebec, Canada
| | - Geoffrey H Tison
- Division of Cardiology, University of California San Francisco, San Francisco, California, USA
| | - Girish Nadkarni
- Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | | | - Emma Pierson
- Department of Computer Science, Cornell Tech, New York, New York, USA
| | - Ashley Beecy
- NewYork-Presbyterian Health System, New York, New York, USA; Division of Cardiology, Weill Cornell Medical College, New York, New York, USA
| | - Deepa Kumaraiah
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA; NewYork-Presbyterian Health System, New York, New York, USA
| | - Chris Haggerty
- Department of Biomedical Informatics Columbia University Irving Medical Center, New York, New York, USA; NewYork-Presbyterian Health System, New York, New York, USA
| | - Jennifer N Avari Silva
- Division of Cardiology, Washington University School of Medicine, St Louis, Missouri, USA
| | - Thomas M Maddox
- Division of Cardiology, Washington University School of Medicine, St Louis, Missouri, USA.
| |
Collapse
|
2
|
Plappert F, Engström G, Platonov PG, Wallman M, Sandberg F. ECG-based estimation of respiration-induced autonomic modulation of AV nodal conduction during atrial fibrillation. Front Physiol 2024; 15:1281343. [PMID: 38779321 PMCID: PMC11110927 DOI: 10.3389/fphys.2024.1281343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 04/08/2024] [Indexed: 05/25/2024] Open
Abstract
Introduction: Information about autonomic nervous system (ANS) activity may offer insights about atrial fibrillation (AF) progression and support personalized AF treatment but is not easily accessible from the ECG. In this study, we propose a new approach for ECG-based assessment of respiratory modulation in atrioventricular (AV) nodal refractory period and conduction delay. Methods: A 1-dimensional convolutional neural network (1D-CNN) was trained to estimate respiratory modulation of AV nodal conduction properties from 1-minute segments of RR series, respiration signals, and atrial fibrillatory rates (AFR) using synthetic data that replicates clinical ECG-derived data. The synthetic data were generated using a network model of the AV node and 4 million unique model parameter sets. The 1D-CNN was then used to analyze respiratory modulation in clinical deep breathing test data of 28 patients in AF, where an ECG-derived respiration signal was extracted using a novel approach based on periodic component analysis. Results: We demonstrated using synthetic data that the 1D-CNN can estimate the respiratory modulation from RR series alone with a Pearson sample correlation of r = 0.805 and that the addition of either respiration signal (r = 0.830), AFR (r = 0.837), or both (r = 0.855) improves the estimation. Discussion: Initial results from analysis of ECG data suggest that our proposed estimate of respiration-induced autonomic modulation, a resp, is reproducible and sufficiently sensitive to monitor changes and detect individual differences. However, further studies are needed to verify the reproducibility, sensitivity, and clinical significance of a resp.
Collapse
Affiliation(s)
- Felix Plappert
- Department of Biomedical Engineering, Lund University, Lund, Sweden
| | - Gunnar Engström
- Department of Clinical Sciences, Cardiovascular Research–Epidemiology, Malmö, Sweden
| | - Pyotr G. Platonov
- Department of Cardiology, Clinical Sciences, Lund University, Lund, Sweden
| | - Mikael Wallman
- Fraunhofer-Chalmers Centre, Department of Systems and Data Analysis, Gothenburg, Sweden
| | - Frida Sandberg
- Department of Biomedical Engineering, Lund University, Lund, Sweden
| |
Collapse
|
3
|
Meng Q, Bai W, O’Regan DP, Rueckert D. DeepMesh: Mesh-Based Cardiac Motion Tracking Using Deep Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:1489-1500. [PMID: 38064325 PMCID: PMC7615801 DOI: 10.1109/tmi.2023.3340118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/22/2024]
Abstract
3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and the diagnosis of cardiovascular diseases. Current state-of-the art methods focus on estimating dense pixel-/voxel-wise motion fields in image space, which ignores the fact that motion estimation is only relevant and useful within the anatomical objects of interest, e.g., the heart. In this work, we model the heart as a 3D mesh consisting of epi- and endocardial surfaces. We propose a novel learning framework, DeepMesh, which propagates a template heart mesh to a subject space and estimates the 3D motion of the heart mesh from CMR images for individual subjects. In DeepMesh, the heart mesh of the end-diastolic frame of an individual subject is first reconstructed from the template mesh. Mesh-based 3D motion fields with respect to the end-diastolic frame are then estimated from 2D short- and long-axis CMR images. By developing a differentiable mesh-to-image rasterizer, DeepMesh is able to leverage 2D shape information from multiple anatomical views for 3D mesh reconstruction and mesh motion estimation. The proposed method estimates vertex-wise displacement and thus maintains vertex correspondences between time frames, which is important for the quantitative assessment of cardiac function across different subjects and populations. We evaluate DeepMesh on CMR images acquired from the UK Biobank. We focus on 3D motion estimation of the left ventricle in this work. Experimental results show that the proposed method quantitatively and qualitatively outperforms other image-based and mesh-based cardiac motion tracking methods.
Collapse
Affiliation(s)
- Qingjie Meng
- The Biomedical Image Analysis Group, Department of Computing, Imperial College London, SW7 2AZ, UK
| | - Wenjia Bai
- The Biomedical Image Analysis Group, Department of Computing, Imperial College London, SW7 2AZ, UK; Department of Brain Sciences, Imperial College London
| | - Declan P O’Regan
- The MRC London Institute of Medical Sciences, Imperial College London, W12 0HS, UK
| | - Daniel Rueckert
- The Biomedical Image Analysis Group, Department of Computing, Imperial College London, SW7 2AZ, UK; Klinikum rechts der Isar, Technical University Munich, Germany
| |
Collapse
|
4
|
de-la-Torre R, Oña ED, Victores JG, Jardón A. SpasticSim: a synthetic data generation method for upper limb spasticity modelling in neurorehabilitation. Sci Rep 2024; 14:1646. [PMID: 38238475 PMCID: PMC10796340 DOI: 10.1038/s41598-024-51993-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 01/11/2024] [Indexed: 01/22/2024] Open
Abstract
In neurorehabilitation, assessment of functional problems is essential to define optimal rehabilitation treatments. Usually, this assessment process requires distinguishing between impaired and non-impaired behavior of limbs. One of the common muscle motor disorders affecting limbs is spasticity, which is complicated to quantify objectively due to the complex nature of motor control. Thus, the lack of heterogeneous samples of patients constituting an acceptable amount of data is an obstacle which is relevant to understanding the behavior of spasticity and, consequently, quantifying it. In this article, we use the 3D creation suite Blender combined with the MBLab add-on to generate synthetic samples of human body models, aiming to be as sufficiently representative as possible to real human samples. Exporting these samples to OpenSim and performing four specific upper limb movements, we analyze the muscle behavior by simulating the six degrees of spasticity contemplated by the Modified Ashworth Scale (MAS). The complete dataset of patients and movements is open-source and available for future research. This approach advocates the potential to generate synthetic data for testing and validating musculoskeletal models.
Collapse
Affiliation(s)
- Rubén de-la-Torre
- Department of Systems Engineering and Automation, Universidad Carlos III de Madrid, Avda. de la Universidad 30, Leganés, 28911, Madrid, Spain
| | - Edwin Daniel Oña
- Department of Systems Engineering and Automation, Universidad Carlos III de Madrid, Avda. de la Universidad 30, Leganés, 28911, Madrid, Spain.
| | - Juan G Victores
- Department of Systems Engineering and Automation, Universidad Carlos III de Madrid, Avda. de la Universidad 30, Leganés, 28911, Madrid, Spain
| | - Alberto Jardón
- Department of Systems Engineering and Automation, Universidad Carlos III de Madrid, Avda. de la Universidad 30, Leganés, 28911, Madrid, Spain
| |
Collapse
|
5
|
Chen Z, Stapleton MC, Xie Y, Li D, Wu YL, Christodoulou AG. Physics-informed deep learning for T2-deblurred superresolution turbo spin echo MRI. Magn Reson Med 2023; 90:2362-2374. [PMID: 37578085 DOI: 10.1002/mrm.29814] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 07/03/2023] [Accepted: 07/12/2023] [Indexed: 08/15/2023]
Abstract
PURPOSE Deep learning superresolution (SR) is a promising approach to reduce MRI scan time without requiring custom sequences or iterative reconstruction. Previous deep learning SR approaches have generated low-resolution training images by simple k-space truncation, but this does not properly model in-plane turbo spin echo (TSE) MRI resolution degradation, which has variable T2 relaxation effects in different k-space regions. To fill this gap, we developed a T2 -deblurred deep learning SR method for the SR of 3D-TSE images. METHODS A SR generative adversarial network was trained using physically realistic resolution degradation (asymmetric T2 weighting of raw high-resolution k-space data). For comparison, we trained the same network structure on previous degradation models without TSE physics modeling. We tested all models for both retrospective and prospective SR with 3 × 3 acceleration factor (in the two phase-encoding directions) of genetically engineered mouse embryo model TSE-MR images. RESULTS The proposed method can produce high-quality 3 × 3 SR images for a typical 500-slice volume with 6-7 mouse embryos. Because 3 × 3 SR was performed, the image acquisition time can be reduced from 15 h to 1.7 h. Compared to previous SR methods without TSE modeling, the proposed method achieved the best quantitative imaging metrics for both retrospective and prospective evaluations and achieved the best imaging-quality expert scores for prospective evaluation. CONCLUSION The proposed T2 -deblurring method improved accuracy and image quality of deep learning-based SR of TSE MRI. This method has the potential to accelerate TSE image acquisition by a factor of up to 9.
Collapse
Affiliation(s)
- Zihao Chen
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, California, USA
- Department of Bioengineering, University of California, Los Angeles, California, USA
| | - Margaret Caroline Stapleton
- Department of Developmental Biology, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Yibin Xie
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | - Debiao Li
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, California, USA
- Department of Bioengineering, University of California, Los Angeles, California, USA
| | - Yijen L Wu
- Department of Developmental Biology, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
- Rangos Research Center Animal Imaging Core, Children's Hospital of Pittsburgh of UPMC, Pittsburgh, Pennsylvania, USA
| | - Anthony G Christodoulou
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, California, USA
- Department of Bioengineering, University of California, Los Angeles, California, USA
| |
Collapse
|
6
|
Morales MA, Yoon S, Fahmy A, Ghanbari F, Nakamori S, Rodriguez J, Yue J, Street JA, Herzka DA, Manning WJ, Nezafat R. Highly accelerated free-breathing real-time myocardial tagging for exercise cardiovascular magnetic resonance. J Cardiovasc Magn Reson 2023; 25:56. [PMID: 37784153 PMCID: PMC10544487 DOI: 10.1186/s12968-023-00961-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Accepted: 09/11/2023] [Indexed: 10/04/2023] Open
Abstract
BACKGROUND Exercise cardiovascular magnetic resonance (Ex-CMR) myocardial tagging would enable quantification of myocardial deformation after exercise. However, current electrocardiogram (ECG)-segmented sequences are limited for Ex-CMR. METHODS We developed a highly accelerated balanced steady-state free-precession real-time tagging technique for 3 T. A 12-fold acceleration was achieved using incoherent sixfold random Cartesian sampling, twofold truncated outer phase encoding, and a deep learning resolution enhancement model. The technique was tested in two prospective studies. In a rest study of 27 patients referred for clinical CMR and 19 healthy subjects, a set of ECG-segmented for comparison and two sets of real-time tagging images for repeatability assessment were collected in 2-chamber and short-axis views with spatiotemporal resolution 2.0 × 2.0 mm2 and 29 ms. In an Ex-CMR study of 26 patients with known or suspected cardiac disease and 23 healthy subjects, real-time images were collected before and after exercise. Deformation was quantified using measures of short-axis global circumferential strain (GCS). Two experienced CMR readers evaluated the image quality of all real-time data pooled from both studies using a 4-point Likert scale for tagline quality (1-excellent; 2-good; 3-moderate; 4-poor) and artifact level (1-none; 2-minimal; 3-moderate; 4-significant). Statistical evaluation included Pearson correlation coefficient (r), intraclass correlation coefficient (ICC), and coefficient of variation (CoV). RESULTS In the rest study, deformation was successfully quantified in 90% of cases. There was a good correlation (r = 0.71) between ECG-segmented and real-time measures of GCS, and repeatability was good to excellent (ICC = 0.86 [0.71, 0.94]) with a CoV of 4.7%. In the Ex-CMR study, deformation was successfully quantified in 96% of subjects pre-exercise and 84% of subjects post-exercise. Short-axis and 2-chamber tagline quality were 1.6 ± 0.7 and 1.9 ± 0.8 at rest and 1.9 ± 0.7 and 2.5 ± 0.8 after exercise, respectively. Short-axis and 2-chamber artifact level was 1.2 ± 0.5 and 1.4 ± 0.7 at rest and 1.3 ± 0.6 and 1.5 ± 0.8 post-exercise, respectively. CONCLUSION We developed a highly accelerated real-time tagging technique and demonstrated its potential for Ex-CMR quantification of myocardial deformation. Further studies are needed to assess the clinical utility of our technique.
Collapse
Affiliation(s)
- Manuel A Morales
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Siyeop Yoon
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Ahmed Fahmy
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Fahime Ghanbari
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Shiro Nakamori
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Jennifer Rodriguez
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Jennifer Yue
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | - Jordan A Street
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
| | | | - Warren J Manning
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA
- Department of Radiology, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, USA
| | - Reza Nezafat
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Ave., Boston, MA, 02215, USA.
| |
Collapse
|
7
|
Buoso S, Joyce T, Schulthess N, Kozerke S. MRXCAT2.0: Synthesis of realistic numerical phantoms by combining left-ventricular shape learning, biophysical simulations and tissue texture generation. J Cardiovasc Magn Reson 2023; 25:25. [PMID: 37076840 PMCID: PMC10116689 DOI: 10.1186/s12968-023-00934-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 03/15/2023] [Indexed: 04/21/2023] Open
Abstract
BACKGROUND Standardised performance assessment of image acquisition, reconstruction and processing methods is limited by the absence of images paired with ground truth reference values. To this end, we propose MRXCAT2.0 to generate synthetic data, covering healthy and pathological function, using a biophysical model. We exemplify the approach by generating cardiovascular magnetic resonance (CMR) images of healthy, infarcted, dilated and hypertrophic left-ventricular (LV) function. METHOD In MRXCAT2.0, the XCAT torso phantom is coupled with a statistical shape model, describing population (patho)physiological variability, and a biophysical model, providing known and detailed functional ground truth of LV morphology and function. CMR balanced steady-state free precession images are generated using MRXCAT2.0 while realistic image appearance is ensured by assigning texturized tissue properties to the phantom labels. FINDING Paired CMR image and ground truth data of LV function were generated with a range of LV masses (85-140 g), ejection fractions (34-51%) and peak radial and circumferential strains (0.45 to 0.95 and - 0.18 to - 0.13, respectively). These ranges cover healthy and pathological cases, including infarction, dilated and hypertrophic cardiomyopathy. The generation of the anatomy takes a few seconds and it improves on current state-of-the-art models where the pathological representation is not explicitly addressed. For the full simulation framework, the biophysical models require approximately two hours, while image generation requires a few minutes per slice. CONCLUSION MRXCAT2.0 offers synthesis of realistic images embedding population-based anatomical and functional variability and associated ground truth parameters to facilitate a standardized assessment of CMR acquisition, reconstruction and processing methods.
Collapse
Affiliation(s)
- Stefano Buoso
- Institute for Biomedical Engineering, ETH Zurich and University Zurich, Zurich, Switzerland.
| | - Thomas Joyce
- Institute for Biomedical Engineering, ETH Zurich and University Zurich, Zurich, Switzerland
| | - Nico Schulthess
- Institute for Biomedical Engineering, ETH Zurich and University Zurich, Zurich, Switzerland
| | - Sebastian Kozerke
- Institute for Biomedical Engineering, ETH Zurich and University Zurich, Zurich, Switzerland
| |
Collapse
|
8
|
Zhang X, Cui C, Zhao S, Xie L, Tian Y. Cardiac magnetic resonance radiomics for disease classification. Eur Radiol 2023; 33:2312-2323. [PMID: 36378251 DOI: 10.1007/s00330-022-09236-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2022] [Revised: 10/07/2022] [Accepted: 10/12/2022] [Indexed: 11/17/2022]
Abstract
OBJECTIVES This study investigated the discriminability of quantitative radiomics features extracted from cardiac magnetic resonance (CMR) images for hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), and healthy (NOR) patients. METHODS The data of two hundred and eighty-three patients with HCM (n = 48) or DCM (n = 52) and NOR (n = 123) were extracted from two publicly available datasets. Ten feature selection methods were first performed on twenty-one different sets of radiomics features extracted from the left ventricle, right ventricle, and myocardium segmented from CMR images in the end-diastolic frame, end-systolic frame, and a combination of both; then, nine classical machine learning methods were trained with the selected radiomics features to distinguish HCM, DCM, and NOR. Ninety classification models were constructed based on combinations of the ten feature selection methods and nine classifiers. The classification models were evaluated, and the optimal model was selected. The diagnostic performance of the selected model was also compared to that of state-of-the-art methods. RESULTS The random forest minimum redundancy maximum relevance model with features based on LeastAxisLength, Maximum2DDiameterSlice, Median, MinorAxisLength, Sphericity, VoxelVolume, Kurtosis, Flatness, and Skewness was the highest performing model, achieving 91.2% classification accuracy. The cross-validated areas under the curve on the test dataset were 0.938, 0.966, and 0.936 for NOR, DCM, and HCM, respectively. Furthermore, compared with those of the state-of-the-art methods, the sensitivity and accuracy of this model were greatly improved. CONCLUSIONS A predictive model was proposed based on CMR radiomics features for classifying HCM, DCM, and NOR patients. The model had good discriminability. KEY POINTS • The first-order features and the features extracted from the LOG-filtered images have potential in distinguishing HCM patients from DCM patients. • The features extracted from the RV play little role in distinguishing DCM from HCM. • The VoxelVolume of the myocardium in the ED frame is important in the recognition of DCM.
Collapse
Affiliation(s)
- Xiaoxuan Zhang
- School of Artificial Intelligence, Beijing Normal University, Beijing, 100875, China
| | - Caixia Cui
- Department of Medical Imaging, The Second Hospital of Hebei Medical University, Shijiazhuang, 050000, China
| | - Shifeng Zhao
- School of Artificial Intelligence, Beijing Normal University, Beijing, 100875, China.
| | - Lizhi Xie
- MR Research China, GE Healthcare, Beijing, 100176, China
| | - Yun Tian
- School of Artificial Intelligence, Beijing Normal University, Beijing, 100875, China.
| |
Collapse
|
9
|
Deveshwar N, Rajagopal A, Sahin S, Shimron E, Larson PEZ. Synthesizing Complex-Valued Multicoil MRI Data from Magnitude-Only Images. Bioengineering (Basel) 2023; 10:bioengineering10030358. [PMID: 36978749 PMCID: PMC10045391 DOI: 10.3390/bioengineering10030358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 03/08/2023] [Accepted: 03/12/2023] [Indexed: 03/18/2023] Open
Abstract
Despite the proliferation of deep learning techniques for accelerated MRI acquisition and enhanced image reconstruction, the construction of large and diverse MRI datasets continues to pose a barrier to effective clinical translation of these technologies. One major challenge is in collecting the MRI raw data (required for image reconstruction) from clinical scanning, as only magnitude images are typically saved and used for clinical assessment and diagnosis. The image phase and multi-channel RF coil information are not retained when magnitude-only images are saved in clinical imaging archives. Additionally, preprocessing used for data in clinical imaging can lead to biased results. While several groups have begun concerted efforts to collect large amounts of MRI raw data, current databases are limited in the diversity of anatomy, pathology, annotations, and acquisition types they contain. To address this, we present a method for synthesizing realistic MR data from magnitude-only data, allowing for the use of diverse data from clinical imaging archives in advanced MRI reconstruction development. Our method uses a conditional GAN-based framework to generate synthetic phase images from input magnitude images. We then applied ESPIRiT to derive RF coil sensitivity maps from fully sampled real data to generate multi-coil data. The synthetic data generation method was evaluated by comparing image reconstruction results from training Variational Networks either with real data or synthetic data. We demonstrate that the Variational Network trained on synthetic MRI data from our method, consisting of GAN-derived synthetic phase and multi-coil information, outperformed Variational Networks trained on data with synthetic phase generated using current state-of-the-art methods. Additionally, we demonstrate that the Variational Networks trained with synthetic k-space data from our method perform comparably to image reconstruction networks trained on undersampled real k-space data.
Collapse
Affiliation(s)
- Nikhil Deveshwar
- UC Berkeley-UCSF Graduate Program in Bioengineering, Berkeley, CA 94701, USA
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94701, USA
| | - Abhejit Rajagopal
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
| | - Sule Sahin
- UC Berkeley-UCSF Graduate Program in Bioengineering, Berkeley, CA 94701, USA
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
| | - Efrat Shimron
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94701, USA
| | - Peder E. Z. Larson
- UC Berkeley-UCSF Graduate Program in Bioengineering, Berkeley, CA 94701, USA
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
- Correspondence:
| |
Collapse
|
10
|
Dhaene AP, Loecher M, Wilson AJ, Ennis DB. Myocardial Segmentation of Tagged Magnetic Resonance Images with Transfer Learning Using Generative Cine-To-Tagged Dataset Transformation. Bioengineering (Basel) 2023; 10:bioengineering10020166. [PMID: 36829660 PMCID: PMC9952238 DOI: 10.3390/bioengineering10020166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 01/19/2023] [Accepted: 01/23/2023] [Indexed: 01/31/2023] Open
Abstract
The use of deep learning (DL) segmentation in cardiac MRI has the potential to streamline the radiology workflow, particularly for the measurement of myocardial strain. Recent efforts in DL motion tracking models have drastically reduced the time needed to measure the heart's displacement field and the subsequent myocardial strain estimation. However, the selection of initial myocardial reference points is not automated and still requires manual input from domain experts. Segmentation of the myocardium is a key step for initializing reference points. While high-performing myocardial segmentation models exist for cine images, this is not the case for tagged images. In this work, we developed and compared two novel DL models (nnU-net and Segmentation ResNet VAE) for the segmentation of myocardium from tagged CMR images. We implemented two methods to transform cardiac cine images into tagged images, allowing us to leverage large public annotated cine datasets. The cine-to-tagged methods included (i) a novel physics-driven transformation model, and (ii) a generative adversarial network (GAN) style transfer model. We show that pretrained models perform better (+2.8 Dice coefficient percentage points) and converge faster (6×) than models trained from scratch. The best-performing method relies on a pretraining with an unpaired, unlabeled, and structure-preserving generative model trained to transform cine images into their tagged-appearing equivalents. Our state-of-the-art myocardium segmentation network reached a Dice coefficient of 0.828 and 95th percentile Hausdorff distance of 4.745 mm on a held-out test set. This performance is comparable to existing state-of-the-art segmentation networks for cine images.
Collapse
Affiliation(s)
- Arnaud P. Dhaene
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
- Signal Processing Laboratory (LTS4), École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
| | - Michael Loecher
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
- Stanford Cardiovascular Institute, Stanford University, Stanford, CA 94305, USA
- Correspondence: (M.L.); (D.B.E.)
| | - Alexander J. Wilson
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
- Stanford Cardiovascular Institute, Stanford University, Stanford, CA 94305, USA
| | - Daniel B. Ennis
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
- Stanford Cardiovascular Institute, Stanford University, Stanford, CA 94305, USA
- Correspondence: (M.L.); (D.B.E.)
| |
Collapse
|
11
|
Yang Q, Ma L, Zhou Z, Bao J, Yang Q, Huang H, Cai S, He H, Chen Z, Zhong J, Cai C. Rapid high-fidelity T 2 * mapping using single-shot overlapping-echo acquisition and deep learning reconstruction. Magn Reson Med 2023; 89:2157-2170. [PMID: 36656132 DOI: 10.1002/mrm.29585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Revised: 12/07/2022] [Accepted: 12/29/2022] [Indexed: 01/20/2023]
Abstract
PURPOSE To develop and evaluate a single-shot quantitative MRI technique called GRE-MOLED (gradient-echo multiple overlapping-echo detachment) for rapid T 2 * $$ {T}_2^{\ast } $$ mapping. METHODS In GRE-MOLED, multiple echoes with different TEs are generated and captured in a single shot of the k-space through MOLED encoding and EPI readout. A deep neural network, trained by synthetic data, was employed for end-to-end parametric mapping from overlapping-echo signals. GRE-MOLED uses pure GRE acquisition with a single echo train to deliver T 2 * $$ {T}_2^{\ast } $$ maps less than 90 ms per slice. The self-registered B0 information modulated in image phase was utilized for distortion-corrected parametric mapping. The proposed method was evaluated in phantoms, healthy volunteers, and task-based FMRI experiments. RESULTS The quantitative results of GRE-MOLED T 2 * $$ {T}_2^{\ast } $$ mapping demonstrated good agreement with those obtained from the multi-echo GRE method (Pearson's correlation coefficient = 0.991 and 0.973 for phantom and in vivo brains, respectively). High intrasubject repeatability (coefficient of variation <1.0%) were also achieved in scan-rescan test. Enabled by deep learning reconstruction, GRE-MOLED showed excellent robustness to geometric distortion, noise, and random subject motion. Compared to the conventional FMRI approach, GRE-MOLED also achieved a higher temporal SNR and BOLD sensitivity in task-based FMRI. CONCLUSION GRE-MOLED is a new real-time technique for T 2 * $$ {T}_2^{\ast } $$ quantification with high efficiency and quality, and it has the potential to be a better quantitative BOLD detection method.
Collapse
Affiliation(s)
- Qinqin Yang
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| | - Lingceng Ma
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| | - Zihan Zhou
- The Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrumental Science, Zhejiang University, Hangzhou, Zhejiang, China
| | - Jianfeng Bao
- Department of Magnetic Resonance Imaging, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, Zhengzhou, Henan, China
| | - Qizhi Yang
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| | - Haitao Huang
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| | - Shuhui Cai
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| | - Hongjian He
- The Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrumental Science, Zhejiang University, Hangzhou, Zhejiang, China
| | - Zhong Chen
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| | - Jianhui Zhong
- The Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrumental Science, Zhejiang University, Hangzhou, Zhejiang, China.,Department of Imaging Sciences, University of Rochester, Rochester, New York, USA
| | - Congbo Cai
- Department of Electronic Science, Xiamen University, Xiamen, Fujian, China
| |
Collapse
|
12
|
Morales MA, Cirillo J, Nakata K, Kucukseymen S, Ngo LH, Izquierdo-Garcia D, Catana C, Nezafat R. Comparison of DeepStrain and Feature Tracking for Cardiac MRI Strain Analysis. J Magn Reson Imaging 2022; 57:1507-1515. [PMID: 35900119 DOI: 10.1002/jmri.28374] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Revised: 07/12/2022] [Accepted: 07/14/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND Myocardial feature tracking (FT) provides a comprehensive analysis of myocardial deformation from cine balanced steady-state free-precession images (bSSFP). However, FT remains time-consuming, precluding its clinical adoption. PURPOSE To compare left-ventricular global radial strain (GRS) and global circumferential strain (GCS) values measured using automated DeepStrain analysis of short-axis cine images to those calculated using manual commercially available FT analysis. STUDY TYPE Retrospective, single-center. POPULATION A total of 30 healthy subjects and 120 patients with cardiac disease for DeepStrain development. For evaluation, 47 healthy subjects (36 male, 53 ± 5 years) and 533 patients who had undergone a clinical cardiac MRI (373 male, 59 ± 14 years). FIELD STRENGTH/SEQUENCE: bSSFP sequence at 1.5 T (Phillips) and 3 T (Siemens). ASSESSMENT Automated DeepStrain measurements of GRS and GCS were compared to commercially available FT (Circle, cvi42) measures obtained by readers with 1 year and 3 years of experience. Comparisons were performed overall and stratified by scanner manufacturer. STATISTICAL TESTS Paired t-test, linear regression slope, Pearson correlation coefficient (r). RESULTS Overall, FT and DeepStrain measurements of GCS were not significantly different (P = 0.207), but measures of GRS were significantly different. Measurements of GRS from Philips (slope = 1.06 [1.03 1.08], r = 0.85) and Siemens (slope = 1.04 [0.99 1.09], r = 0.83) data showed a very strong correlation and agreement between techniques. Measurements of GCS from Philips (slope = 0.98 [0.98 1.01], r = 0.91) and Siemens (slope = 1.0 [0.96 1.03], r = 0.88) data similarly showed a very strong correlation. The average analysis time per subject was 4.1 ± 1.2 minutes for FT and 34.7 ± 3.3 seconds for DeepStrain, representing a 7-fold reduction in analysis time. DATA CONCLUSION This study demonstrated high correlation of myocardial GCS and GRS measurements between freely available fully automated DeepStrain and commercially available manual FT software, with substantial time-saving in the analysis. EVIDENCE LEVEL 3 TECHNICAL EFFICACY: Stage 3.
Collapse
Affiliation(s)
- Manuel A Morales
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| | - Julia Cirillo
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| | - Kei Nakata
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| | - Selcuk Kucukseymen
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| | - Long H Ngo
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| | - David Izquierdo-Garcia
- Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Boston, Massachusetts, USA
- Harvard-MIT Division of Health Sciences and Technology, Cambridge, Massachusetts, USA
| | - Ciprian Catana
- Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Boston, Massachusetts, USA
| | - Reza Nezafat
- Department of Medicine (Cardiovascular Division), Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
13
|
Chen Y, Yang XH, Wei Z, Heidari AA, Zheng N, Li Z, Chen H, Hu H, Zhou Q, Guan Q. Generative Adversarial Networks in Medical Image augmentation: A review. Comput Biol Med 2022; 144:105382. [PMID: 35276550 DOI: 10.1016/j.compbiomed.2022.105382] [Citation(s) in RCA: 49] [Impact Index Per Article: 24.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Revised: 02/25/2022] [Accepted: 03/02/2022] [Indexed: 12/31/2022]
Abstract
OBJECT With the development of deep learning, the number of training samples for medical image-based diagnosis and treatment models is increasing. Generative Adversarial Networks (GANs) have attracted attention in medical image processing due to their excellent image generation capabilities and have been widely used in data augmentation. In this paper, a comprehensive and systematic review and analysis of medical image augmentation work are carried out, and its research status and development prospects are reviewed. METHOD This paper reviews 105 medical image augmentation related papers, which mainly collected by ELSEVIER, IEEE Xplore, and Springer from 2018 to 2021. We counted these papers according to the parts of the organs corresponding to the images, and sorted out the medical image datasets that appeared in them, the loss function in model training, and the quantitative evaluation metrics of image augmentation. At the same time, we briefly introduce the literature collected in three journals and three conferences that have received attention in medical image processing. RESULT First, we summarize the advantages of various augmentation models, loss functions, and evaluation metrics. Researchers can use this information as a reference when designing augmentation tasks. Second, we explore the relationship between augmented models and the amount of the training set, and tease out the role that augmented models may play when the quality of the training set is limited. Third, the statistical number of papers shows that the development momentum of this research field remains strong. Furthermore, we discuss the existing limitations of this type of model and suggest possible research directions. CONCLUSION We discuss GAN-based medical image augmentation work in detail. This method effectively alleviates the challenge of limited training samples for medical image diagnosis and treatment models. It is hoped that this review will benefit researchers interested in this field.
Collapse
Affiliation(s)
- Yizhou Chen
- College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China.
| | - Xu-Hua Yang
- College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China.
| | - Zihan Wei
- College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China.
| | - Ali Asghar Heidari
- School of Surveying and Geospatial Engineering, College of Engineering, University of Tehran, Tehran, Iran; Department of Computer Science, School of Computing, National University of Singapore, Singapore, Singapore.
| | - Nenggan Zheng
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, Zhejiang, China.
| | - Zhicheng Li
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
| | - Huiling Chen
- College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, Zhejiang, 325035, China.
| | - Haigen Hu
- College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China.
| | - Qianwei Zhou
- College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China.
| | - Qiu Guan
- College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China.
| |
Collapse
|
14
|
Yassine IA, Ghanem AM, Metwalli NS, Hamimi A, Ouwerkerk R, Matta JR, Solomon MA, Elinoff JM, Gharib AM, Abd-Elmoniem KZ. Native-resolution myocardial principal Eulerian strain mapping using convolutional neural networks and Tagged Magnetic Resonance Imaging. Comput Biol Med 2022; 141:105041. [PMID: 34836627 PMCID: PMC8900530 DOI: 10.1016/j.compbiomed.2021.105041] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 11/12/2021] [Accepted: 11/13/2021] [Indexed: 02/03/2023]
Abstract
BACKGROUND Assessment of regional myocardial function at native pixel-level resolution can play a crucial role in recognizing the early signs of the decline in regional myocardial function. Extensive data processing in existing techniques limits the effective resolution and accuracy of the generated strain maps. The purpose of this study is to compute myocardial principal strain maps εp1 and εp2 from tagged MRI (tMRI) at the native image resolution using deep-learning local patch convolutional neural network (CNN) models (DeepStrain). METHODS For network training, validation, and testing, realistic tMRI datasets were generated and consisted of 53,606 cine images simulating the heart, the liver, blood pool, and backgrounds, including ranges of shapes, positions, motion patterns, noise, and strain. In addition, 102 in-vivo image datasets from three healthy subjects, and three Pulmonary Arterial Hypertension patients, were acquired and used to assess the network's in-vivo performance. Four convolutional neural networks were trained for mapping input tagging patterns to corresponding ground-truth principal strains using different cost functions. Strain maps using harmonic phase analysis (HARP) were obtained with various spectral filtering settings for comparison. CNN and HARP strain maps were compared at the pixel level versus the ground-truth and versus the least-loss in-vivo maps using Pearson correlation coefficients (R) and the median error and Inter-Quartile Range (IQR) histograms. RESULTS CNN-based local patch DeepStrain maps at a phantom resolution of 1.1mm × 1.1 mm and in-vivo resolution of 2.1mm × 1.6 mm were artifact-free with multiple fold improvement with εp1 ground-truth median error of 0.009(0.007) vs. 0.32(0.385) using HARP and εp2 ground-truth error of 0.016(0.021) vs. 0.181(0.08) using HARP. CNN-based strain maps showed substantially higher agreement with the ground-truth maps with correlation coefficients R > 0.91 for εp1 and εp2 compared to R < 0.21 and R < 0.82 for HARP-generated maps, respectively. CONCLUSION CNN-generated Eulerian strain mapping permits artifact-free visualization of myocardial function at the native image resolution.
Collapse
Affiliation(s)
- Inas A. Yassine
- Systems and Biomedical Engineering Department, Faculty of Engineering, Cairo University, Egypt
| | - Ahmed M. Ghanem
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA
| | - Nader S. Metwalli
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA
| | - Ahmed Hamimi
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA
| | - Ronald Ouwerkerk
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA
| | - Jatin R. Matta
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA
| | - Michael A. Solomon
- Cardiovascular Branch of the National Heart, Lung, and Blood Institute (NHLBI), NIH, Bethesda, MD, USA.,Critical Care Medicine Department, NIH Clinical Center, Bethesda, MD, USA
| | - Jason M. Elinoff
- Critical Care Medicine Department, NIH Clinical Center, Bethesda, MD, USA
| | - Ahmed M. Gharib
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA
| | - Khaled Z. Abd-Elmoniem
- Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, Bethesda, MD, USA,Corresponding author: Khaled Z Abd-Elmoniem, PhD, MHS, Biomedical and Metabolic Imaging Branch, National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, 10 Center Drive, Bldg. 10, CRC, Rm. 3-5340, Bethesda, MD 20892, Tel: 301-451-8982/Fax: 301-480-3166,
| |
Collapse
|