Using synthetic data generation to train a cardiac motion tag tracking neural network

Artificial Intelligence in Cardiovascular Care-Part 2: Applications: JACC Review Topic of the Week

Recent artificial intelligence (AI) advancements in cardiovascular care offer potential enhancements in effective diagnosis, treatment, and outcomes. More than 600 U.S. Food and Drug Administration-approved clinical AI algorithms now exist, with 10% focusing on cardiovascular applications, highlighting the growing opportunities for AI to augment care. This review discusses the latest advancements in the field of AI, with a particular focus on the utilization of multimodal inputs and the field of generative AI. Further discussions in this review involve an approach to understanding the larger context in which AI-augmented care may exist, and include a discussion of the need for rigorous evaluation, appropriate infrastructure for deployment, ethics and equity assessments, regulatory oversight, and viable business cases for deployment. Embracing this rapidly evolving technology while setting an appropriately high evaluation benchmark with careful and patient-centered implementation will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.

ECG-based estimation of respiration-induced autonomic modulation of AV nodal conduction during atrial fibrillation

Introduction: Information about autonomic nervous system (ANS) activity may offer insights about atrial fibrillation (AF) progression and support personalized AF treatment but is not easily accessible from the ECG. In this study, we propose a new approach for ECG-based assessment of respiratory modulation in atrioventricular (AV) nodal refractory period and conduction delay. Methods: A 1-dimensional convolutional neural network (1D-CNN) was trained to estimate respiratory modulation of AV nodal conduction properties from 1-minute segments of RR series, respiration signals, and atrial fibrillatory rates (AFR) using synthetic data that replicates clinical ECG-derived data. The synthetic data were generated using a network model of the AV node and 4 million unique model parameter sets. The 1D-CNN was then used to analyze respiratory modulation in clinical deep breathing test data of 28 patients in AF, where an ECG-derived respiration signal was extracted using a novel approach based on periodic component analysis. Results: We demonstrated using synthetic data that the 1D-CNN can estimate the respiratory modulation from RR series alone with a Pearson sample correlation of r = 0.805 and that the addition of either respiration signal (r = 0.830), AFR (r = 0.837), or both (r = 0.855) improves the estimation. Discussion: Initial results from analysis of ECG data suggest that our proposed estimate of respiration-induced autonomic modulation, a resp, is reproducible and sufficiently sensitive to monitor changes and detect individual differences. However, further studies are needed to verify the reproducibility, sensitivity, and clinical significance of a resp.

DeepMesh: Mesh-Based Cardiac Motion Tracking Using Deep Learning

3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and the diagnosis of cardiovascular diseases. Current state-of-the art methods focus on estimating dense pixel-/voxel-wise motion fields in image space, which ignores the fact that motion estimation is only relevant and useful within the anatomical objects of interest, e.g., the heart. In this work, we model the heart as a 3D mesh consisting of epi- and endocardial surfaces. We propose a novel learning framework, DeepMesh, which propagates a template heart mesh to a subject space and estimates the 3D motion of the heart mesh from CMR images for individual subjects. In DeepMesh, the heart mesh of the end-diastolic frame of an individual subject is first reconstructed from the template mesh. Mesh-based 3D motion fields with respect to the end-diastolic frame are then estimated from 2D short- and long-axis CMR images. By developing a differentiable mesh-to-image rasterizer, DeepMesh is able to leverage 2D shape information from multiple anatomical views for 3D mesh reconstruction and mesh motion estimation. The proposed method estimates vertex-wise displacement and thus maintains vertex correspondences between time frames, which is important for the quantitative assessment of cardiac function across different subjects and populations. We evaluate DeepMesh on CMR images acquired from the UK Biobank. We focus on 3D motion estimation of the left ventricle in this work. Experimental results show that the proposed method quantitatively and qualitatively outperforms other image-based and mesh-based cardiac motion tracking methods.

SpasticSim: a synthetic data generation method for upper limb spasticity modelling in neurorehabilitation

In neurorehabilitation, assessment of functional problems is essential to define optimal rehabilitation treatments. Usually, this assessment process requires distinguishing between impaired and non-impaired behavior of limbs. One of the common muscle motor disorders affecting limbs is spasticity, which is complicated to quantify objectively due to the complex nature of motor control. Thus, the lack of heterogeneous samples of patients constituting an acceptable amount of data is an obstacle which is relevant to understanding the behavior of spasticity and, consequently, quantifying it. In this article, we use the 3D creation suite Blender combined with the MBLab add-on to generate synthetic samples of human body models, aiming to be as sufficiently representative as possible to real human samples. Exporting these samples to OpenSim and performing four specific upper limb movements, we analyze the muscle behavior by simulating the six degrees of spasticity contemplated by the Modified Ashworth Scale (MAS). The complete dataset of patients and movements is open-source and available for future research. This approach advocates the potential to generate synthetic data for testing and validating musculoskeletal models.

Physics-informed deep learning for T2-deblurred superresolution turbo spin echo MRI

PURPOSE

Deep learning superresolution (SR) is a promising approach to reduce MRI scan time without requiring custom sequences or iterative reconstruction. Previous deep learning SR approaches have generated low-resolution training images by simple k-space truncation, but this does not properly model in-plane turbo spin echo (TSE) MRI resolution degradation, which has variable T2 relaxation effects in different k-space regions. To fill this gap, we developed a T2 -deblurred deep learning SR method for the SR of 3D-TSE images.

METHODS

A SR generative adversarial network was trained using physically realistic resolution degradation (asymmetric T2 weighting of raw high-resolution k-space data). For comparison, we trained the same network structure on previous degradation models without TSE physics modeling. We tested all models for both retrospective and prospective SR with 3 × 3 acceleration factor (in the two phase-encoding directions) of genetically engineered mouse embryo model TSE-MR images.

RESULTS

The proposed method can produce high-quality 3 × 3 SR images for a typical 500-slice volume with 6-7 mouse embryos. Because 3 × 3 SR was performed, the image acquisition time can be reduced from 15 h to 1.7 h. Compared to previous SR methods without TSE modeling, the proposed method achieved the best quantitative imaging metrics for both retrospective and prospective evaluations and achieved the best imaging-quality expert scores for prospective evaluation.

CONCLUSION

The proposed T2 -deblurring method improved accuracy and image quality of deep learning-based SR of TSE MRI. This method has the potential to accelerate TSE image acquisition by a factor of up to 9.

Highly accelerated free-breathing real-time myocardial tagging for exercise cardiovascular magnetic resonance

BACKGROUND

Exercise cardiovascular magnetic resonance (Ex-CMR) myocardial tagging would enable quantification of myocardial deformation after exercise. However, current electrocardiogram (ECG)-segmented sequences are limited for Ex-CMR.

METHODS

We developed a highly accelerated balanced steady-state free-precession real-time tagging technique for 3 T. A 12-fold acceleration was achieved using incoherent sixfold random Cartesian sampling, twofold truncated outer phase encoding, and a deep learning resolution enhancement model. The technique was tested in two prospective studies. In a rest study of 27 patients referred for clinical CMR and 19 healthy subjects, a set of ECG-segmented for comparison and two sets of real-time tagging images for repeatability assessment were collected in 2-chamber and short-axis views with spatiotemporal resolution 2.0 × 2.0 mm2 and 29 ms. In an Ex-CMR study of 26 patients with known or suspected cardiac disease and 23 healthy subjects, real-time images were collected before and after exercise. Deformation was quantified using measures of short-axis global circumferential strain (GCS). Two experienced CMR readers evaluated the image quality of all real-time data pooled from both studies using a 4-point Likert scale for tagline quality (1-excellent; 2-good; 3-moderate; 4-poor) and artifact level (1-none; 2-minimal; 3-moderate; 4-significant). Statistical evaluation included Pearson correlation coefficient (r), intraclass correlation coefficient (ICC), and coefficient of variation (CoV).

RESULTS

In the rest study, deformation was successfully quantified in 90% of cases. There was a good correlation (r = 0.71) between ECG-segmented and real-time measures of GCS, and repeatability was good to excellent (ICC = 0.86 [0.71, 0.94]) with a CoV of 4.7%. In the Ex-CMR study, deformation was successfully quantified in 96% of subjects pre-exercise and 84% of subjects post-exercise. Short-axis and 2-chamber tagline quality were 1.6 ± 0.7 and 1.9 ± 0.8 at rest and 1.9 ± 0.7 and 2.5 ± 0.8 after exercise, respectively. Short-axis and 2-chamber artifact level was 1.2 ± 0.5 and 1.4 ± 0.7 at rest and 1.3 ± 0.6 and 1.5 ± 0.8 post-exercise, respectively.

CONCLUSION

We developed a highly accelerated real-time tagging technique and demonstrated its potential for Ex-CMR quantification of myocardial deformation. Further studies are needed to assess the clinical utility of our technique.

MRXCAT2.0: Synthesis of realistic numerical phantoms by combining left-ventricular shape learning, biophysical simulations and tissue texture generation

BACKGROUND

Standardised performance assessment of image acquisition, reconstruction and processing methods is limited by the absence of images paired with ground truth reference values. To this end, we propose MRXCAT2.0 to generate synthetic data, covering healthy and pathological function, using a biophysical model. We exemplify the approach by generating cardiovascular magnetic resonance (CMR) images of healthy, infarcted, dilated and hypertrophic left-ventricular (LV) function.

METHOD

In MRXCAT2.0, the XCAT torso phantom is coupled with a statistical shape model, describing population (patho)physiological variability, and a biophysical model, providing known and detailed functional ground truth of LV morphology and function. CMR balanced steady-state free precession images are generated using MRXCAT2.0 while realistic image appearance is ensured by assigning texturized tissue properties to the phantom labels.

FINDING

Paired CMR image and ground truth data of LV function were generated with a range of LV masses (85-140 g), ejection fractions (34-51%) and peak radial and circumferential strains (0.45 to 0.95 and - 0.18 to - 0.13, respectively). These ranges cover healthy and pathological cases, including infarction, dilated and hypertrophic cardiomyopathy. The generation of the anatomy takes a few seconds and it improves on current state-of-the-art models where the pathological representation is not explicitly addressed. For the full simulation framework, the biophysical models require approximately two hours, while image generation requires a few minutes per slice.

CONCLUSION

MRXCAT2.0 offers synthesis of realistic images embedding population-based anatomical and functional variability and associated ground truth parameters to facilitate a standardized assessment of CMR acquisition, reconstruction and processing methods.

Cardiac magnetic resonance radiomics for disease classification

OBJECTIVES

This study investigated the discriminability of quantitative radiomics features extracted from cardiac magnetic resonance (CMR) images for hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), and healthy (NOR) patients.

METHODS

The data of two hundred and eighty-three patients with HCM (n = 48) or DCM (n = 52) and NOR (n = 123) were extracted from two publicly available datasets. Ten feature selection methods were first performed on twenty-one different sets of radiomics features extracted from the left ventricle, right ventricle, and myocardium segmented from CMR images in the end-diastolic frame, end-systolic frame, and a combination of both; then, nine classical machine learning methods were trained with the selected radiomics features to distinguish HCM, DCM, and NOR. Ninety classification models were constructed based on combinations of the ten feature selection methods and nine classifiers. The classification models were evaluated, and the optimal model was selected. The diagnostic performance of the selected model was also compared to that of state-of-the-art methods.

RESULTS

The random forest minimum redundancy maximum relevance model with features based on LeastAxisLength, Maximum2DDiameterSlice, Median, MinorAxisLength, Sphericity, VoxelVolume, Kurtosis, Flatness, and Skewness was the highest performing model, achieving 91.2% classification accuracy. The cross-validated areas under the curve on the test dataset were 0.938, 0.966, and 0.936 for NOR, DCM, and HCM, respectively. Furthermore, compared with those of the state-of-the-art methods, the sensitivity and accuracy of this model were greatly improved.

CONCLUSIONS

A predictive model was proposed based on CMR radiomics features for classifying HCM, DCM, and NOR patients. The model had good discriminability.

KEY POINTS

• The first-order features and the features extracted from the LOG-filtered images have potential in distinguishing HCM patients from DCM patients. • The features extracted from the RV play little role in distinguishing DCM from HCM. • The VoxelVolume of the myocardium in the ED frame is important in the recognition of DCM.

Synthesizing Complex-Valued Multicoil MRI Data from Magnitude-Only Images

Despite the proliferation of deep learning techniques for accelerated MRI acquisition and enhanced image reconstruction, the construction of large and diverse MRI datasets continues to pose a barrier to effective clinical translation of these technologies. One major challenge is in collecting the MRI raw data (required for image reconstruction) from clinical scanning, as only magnitude images are typically saved and used for clinical assessment and diagnosis. The image phase and multi-channel RF coil information are not retained when magnitude-only images are saved in clinical imaging archives. Additionally, preprocessing used for data in clinical imaging can lead to biased results. While several groups have begun concerted efforts to collect large amounts of MRI raw data, current databases are limited in the diversity of anatomy, pathology, annotations, and acquisition types they contain. To address this, we present a method for synthesizing realistic MR data from magnitude-only data, allowing for the use of diverse data from clinical imaging archives in advanced MRI reconstruction development. Our method uses a conditional GAN-based framework to generate synthetic phase images from input magnitude images. We then applied ESPIRiT to derive RF coil sensitivity maps from fully sampled real data to generate multi-coil data. The synthetic data generation method was evaluated by comparing image reconstruction results from training Variational Networks either with real data or synthetic data. We demonstrate that the Variational Network trained on synthetic MRI data from our method, consisting of GAN-derived synthetic phase and multi-coil information, outperformed Variational Networks trained on data with synthetic phase generated using current state-of-the-art methods. Additionally, we demonstrate that the Variational Networks trained with synthetic k-space data from our method perform comparably to image reconstruction networks trained on undersampled real k-space data.

Myocardial Segmentation of Tagged Magnetic Resonance Images with Transfer Learning Using Generative Cine-To-Tagged Dataset Transformation

The use of deep learning (DL) segmentation in cardiac MRI has the potential to streamline the radiology workflow, particularly for the measurement of myocardial strain. Recent efforts in DL motion tracking models have drastically reduced the time needed to measure the heart's displacement field and the subsequent myocardial strain estimation. However, the selection of initial myocardial reference points is not automated and still requires manual input from domain experts. Segmentation of the myocardium is a key step for initializing reference points. While high-performing myocardial segmentation models exist for cine images, this is not the case for tagged images. In this work, we developed and compared two novel DL models (nnU-net and Segmentation ResNet VAE) for the segmentation of myocardium from tagged CMR images. We implemented two methods to transform cardiac cine images into tagged images, allowing us to leverage large public annotated cine datasets. The cine-to-tagged methods included (i) a novel physics-driven transformation model, and (ii) a generative adversarial network (GAN) style transfer model. We show that pretrained models perform better (+2.8 Dice coefficient percentage points) and converge faster (6×) than models trained from scratch. The best-performing method relies on a pretraining with an unpaired, unlabeled, and structure-preserving generative model trained to transform cine images into their tagged-appearing equivalents. Our state-of-the-art myocardium segmentation network reached a Dice coefficient of 0.828 and 95th percentile Hausdorff distance of 4.745 mm on a held-out test set. This performance is comparable to existing state-of-the-art segmentation networks for cine images.

Rapid high-fidelity T 2 * mapping using single-shot overlapping-echo acquisition and deep learning reconstruction

PURPOSE

To develop and evaluate a single-shot quantitative MRI technique called GRE-MOLED (gradient-echo multiple overlapping-echo detachment) for rapid T 2 * $$ {T}_2^{\ast } $$ mapping.

METHODS

In GRE-MOLED, multiple echoes with different TEs are generated and captured in a single shot of the k-space through MOLED encoding and EPI readout. A deep neural network, trained by synthetic data, was employed for end-to-end parametric mapping from overlapping-echo signals. GRE-MOLED uses pure GRE acquisition with a single echo train to deliver T 2 * $$ {T}_2^{\ast } $$ maps less than 90 ms per slice. The self-registered B₀ information modulated in image phase was utilized for distortion-corrected parametric mapping. The proposed method was evaluated in phantoms, healthy volunteers, and task-based FMRI experiments.

RESULTS

The quantitative results of GRE-MOLED T 2 * $$ {T}_2^{\ast } $$ mapping demonstrated good agreement with those obtained from the multi-echo GRE method (Pearson's correlation coefficient = 0.991 and 0.973 for phantom and in vivo brains, respectively). High intrasubject repeatability (coefficient of variation <1.0%) were also achieved in scan-rescan test. Enabled by deep learning reconstruction, GRE-MOLED showed excellent robustness to geometric distortion, noise, and random subject motion. Compared to the conventional FMRI approach, GRE-MOLED also achieved a higher temporal SNR and BOLD sensitivity in task-based FMRI.

CONCLUSION

GRE-MOLED is a new real-time technique for T 2 * $$ {T}_2^{\ast } $$ quantification with high efficiency and quality, and it has the potential to be a better quantitative BOLD detection method.

Comparison of DeepStrain and Feature Tracking for Cardiac MRI Strain Analysis

BACKGROUND

Myocardial feature tracking (FT) provides a comprehensive analysis of myocardial deformation from cine balanced steady-state free-precession images (bSSFP). However, FT remains time-consuming, precluding its clinical adoption.

PURPOSE

To compare left-ventricular global radial strain (GRS) and global circumferential strain (GCS) values measured using automated DeepStrain analysis of short-axis cine images to those calculated using manual commercially available FT analysis.

STUDY TYPE

Retrospective, single-center.

POPULATION

A total of 30 healthy subjects and 120 patients with cardiac disease for DeepStrain development. For evaluation, 47 healthy subjects (36 male, 53 ± 5 years) and 533 patients who had undergone a clinical cardiac MRI (373 male, 59 ± 14 years). FIELD STRENGTH/SEQUENCE: bSSFP sequence at 1.5 T (Phillips) and 3 T (Siemens).

ASSESSMENT

Automated DeepStrain measurements of GRS and GCS were compared to commercially available FT (Circle, cvi42) measures obtained by readers with 1 year and 3 years of experience. Comparisons were performed overall and stratified by scanner manufacturer.

STATISTICAL TESTS

Paired t-test, linear regression slope, Pearson correlation coefficient (r).

RESULTS

Overall, FT and DeepStrain measurements of GCS were not significantly different (P = 0.207), but measures of GRS were significantly different. Measurements of GRS from Philips (slope = 1.06 [1.03 1.08], r = 0.85) and Siemens (slope = 1.04 [0.99 1.09], r = 0.83) data showed a very strong correlation and agreement between techniques. Measurements of GCS from Philips (slope = 0.98 [0.98 1.01], r = 0.91) and Siemens (slope = 1.0 [0.96 1.03], r = 0.88) data similarly showed a very strong correlation. The average analysis time per subject was 4.1 ± 1.2 minutes for FT and 34.7 ± 3.3 seconds for DeepStrain, representing a 7-fold reduction in analysis time.

DATA CONCLUSION

This study demonstrated high correlation of myocardial GCS and GRS measurements between freely available fully automated DeepStrain and commercially available manual FT software, with substantial time-saving in the analysis.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 3.

Generative Adversarial Networks in Medical Image augmentation: A review

OBJECT

With the development of deep learning, the number of training samples for medical image-based diagnosis and treatment models is increasing. Generative Adversarial Networks (GANs) have attracted attention in medical image processing due to their excellent image generation capabilities and have been widely used in data augmentation. In this paper, a comprehensive and systematic review and analysis of medical image augmentation work are carried out, and its research status and development prospects are reviewed.

METHOD

This paper reviews 105 medical image augmentation related papers, which mainly collected by ELSEVIER, IEEE Xplore, and Springer from 2018 to 2021. We counted these papers according to the parts of the organs corresponding to the images, and sorted out the medical image datasets that appeared in them, the loss function in model training, and the quantitative evaluation metrics of image augmentation. At the same time, we briefly introduce the literature collected in three journals and three conferences that have received attention in medical image processing.

RESULT

First, we summarize the advantages of various augmentation models, loss functions, and evaluation metrics. Researchers can use this information as a reference when designing augmentation tasks. Second, we explore the relationship between augmented models and the amount of the training set, and tease out the role that augmented models may play when the quality of the training set is limited. Third, the statistical number of papers shows that the development momentum of this research field remains strong. Furthermore, we discuss the existing limitations of this type of model and suggest possible research directions.

CONCLUSION

We discuss GAN-based medical image augmentation work in detail. This method effectively alleviates the challenge of limited training samples for medical image diagnosis and treatment models. It is hoped that this review will benefit researchers interested in this field.

Native-resolution myocardial principal Eulerian strain mapping using convolutional neural networks and Tagged Magnetic Resonance Imaging

BACKGROUND

Assessment of regional myocardial function at native pixel-level resolution can play a crucial role in recognizing the early signs of the decline in regional myocardial function. Extensive data processing in existing techniques limits the effective resolution and accuracy of the generated strain maps. The purpose of this study is to compute myocardial principal strain maps ε_p1 and ε_p2 from tagged MRI (tMRI) at the native image resolution using deep-learning local patch convolutional neural network (CNN) models (DeepStrain).

METHODS

For network training, validation, and testing, realistic tMRI datasets were generated and consisted of 53,606 cine images simulating the heart, the liver, blood pool, and backgrounds, including ranges of shapes, positions, motion patterns, noise, and strain. In addition, 102 in-vivo image datasets from three healthy subjects, and three Pulmonary Arterial Hypertension patients, were acquired and used to assess the network's in-vivo performance. Four convolutional neural networks were trained for mapping input tagging patterns to corresponding ground-truth principal strains using different cost functions. Strain maps using harmonic phase analysis (HARP) were obtained with various spectral filtering settings for comparison. CNN and HARP strain maps were compared at the pixel level versus the ground-truth and versus the least-loss in-vivo maps using Pearson correlation coefficients (R) and the median error and Inter-Quartile Range (IQR) histograms.

RESULTS

CNN-based local patch DeepStrain maps at a phantom resolution of 1.1mm × 1.1 mm and in-vivo resolution of 2.1mm × 1.6 mm were artifact-free with multiple fold improvement with ε_p1 ground-truth median error of 0.009(0.007) vs. 0.32(0.385) using HARP and ε_p2 ground-truth error of 0.016(0.021) vs. 0.181(0.08) using HARP. CNN-based strain maps showed substantially higher agreement with the ground-truth maps with correlation coefficients R > 0.91 for ε_p1 and ε_p2 compared to R < 0.21 and R < 0.82 for HARP-generated maps, respectively.

CONCLUSION

CNN-generated Eulerian strain mapping permits artifact-free visualization of myocardial function at the native image resolution.