Deep neural network ensemble for on-the-fly quality control-driven segmentation of cardiac MRI T1 mapping

Development and performance evaluation of fully automated deep learning-based models for myocardial segmentation on T1 mapping MRI data

To develop a deep learning-based model capable of segmenting the left ventricular (LV) myocardium on native T1 maps from cardiac MRI in both long-axis and short-axis orientations. Models were trained on native myocardial T1 maps from 50 healthy volunteers and 75 patients using manual segmentation as the reference standard. Based on a U-Net architecture, we systematically optimized the model design using two different training metrics (Sørensen-Dice coefficient = DSC and Intersection-over-Union = IOU), two different activation functions (ReLU and LeakyReLU) and various numbers of training epochs. Training with DSC metric and a ReLU activation function over 35 epochs achieved the highest overall performance (mean error in T1 10.6 ± 17.9 ms, mean DSC 0.88 ± 0.07). Limits of agreement between model results and ground truth were from -35.5 to + 36.1 ms. This was superior to the agreement between two human raters (-34.7 to + 59.1 ms). Segmentation was as accurate for long-axis views (mean error T1: 6.77 ± 8.3 ms, mean DSC: 0.89 ± 0.03) as for short-axis images (mean error ΔT1: 11.6 ± 19.7 ms, mean DSC: 0.88 ± 0.08). Fully automated segmentation and quantitative analysis of native myocardial T1 maps is possible in both long-axis and short-axis orientations with very high accuracy.

Improved Robustness for Deep Learning-based Segmentation of Multi-Center Myocardial Perfusion MRI Datasets Using Data Adaptive Uncertainty-guided Space-time Analysis

BACKGROUND

Fully automatic analysis of myocardial perfusion MRI datasets enables rapid and objective reporting of stress/rest studies in patients with suspected ischemic heart disease. Developing deep learning techniques that can analyze multi-center datasets despite limited training data and variations in software (pulse sequence) and hardware (scanner vendor) is an ongoing challenge.

METHODS

Datasets from 3 medical centers acquired at 3T (n = 150 subjects; 21,150 first-pass images) were included: an internal dataset (inD; n = 95) and two external datasets (exDs; n = 55) used for evaluating the robustness of the trained deep neural network (DNN) models against differences in pulse sequence (exD-1) and scanner vendor (exD-2). A subset of inD (n = 85) was used for training/validation of a pool of DNNs for segmentation, all using the same spatiotemporal U-Net architecture and hyperparameters but with different parameter initializations. We employed a space-time sliding-patch analysis approach that automatically yields a pixel-wise "uncertainty map" as a byproduct of the segmentation process. In our approach, dubbed Data Adaptive Uncertainty-Guided Space-time (DAUGS) analysis, a given test case is segmented by all members of the DNN pool and the resulting uncertainty maps are leveraged to automatically select the "best" one among the pool of solutions. For comparison, we also trained a DNN using the established approach with the same settings (hyperparameters, data augmentation, etc.).

RESULTS

The proposed DAUGS analysis approach performed similarly to the established approach on the internal dataset (Dice score for the testing subset of inD: 0.896 ± 0.050 vs. 0.890 ± 0.049; p = n.s.) whereas it significantly outperformed on the external datasets (Dice for exD-1: 0.885 ± 0.040 vs. 0.849 ± 0.065, p < 0.005; Dice for exD-2: 0.811 ± 0.070 vs. 0.728 ± 0.149, p < 0.005). Moreover, the number of image series with "failed" segmentation (defined as having myocardial contours that include bloodpool or are noncontiguous in ≥1 segment) was significantly lower for the proposed vs. the established approach (4.3% vs. 17.1%, p < 0.0005).

CONCLUSIONS

The proposed DAUGS analysis approach has the potential to improve the robustness of deep learning methods for segmentation of multi-center stress perfusion datasets with variations in the choice of pulse sequence, site location or scanner vendor.

Improved Robustness for Deep Learning-based Segmentation of Multi-Center Myocardial Perfusion MRI Datasets Using Data Adaptive Uncertainty-guided Space-time Analysis

Background

Fully automatic analysis of myocardial perfusion MRI datasets enables rapid and objective reporting of stress/rest studies in patients with suspected ischemic heart disease. Developing deep learning techniques that can analyze multi-center datasets despite limited training data and variations in software (pulse sequence) and hardware (scanner vendor) is an ongoing challenge.

Methods

Datasets from 3 medical centers acquired at 3T (n = 150 subjects; 21,150 first-pass images) were included: an internal dataset (inD; n = 95) and two external datasets (exDs; n = 55) used for evaluating the robustness of the trained deep neural network (DNN) models against differences in pulse sequence (exD-1) and scanner vendor (exD-2). A subset of inD (n = 85) was used for training/validation of a pool of DNNs for segmentation, all using the same spatiotemporal U-Net architecture and hyperparameters but with different parameter initializations. We employed a space-time sliding-patch analysis approach that automatically yields a pixel-wise "uncertainty map" as a byproduct of the segmentation process. In our approach, dubbed Data Adaptive Uncertainty-Guided Space-time (DAUGS) analysis, a given test case is segmented by all members of the DNN pool and the resulting uncertainty maps are leveraged to automatically select the "best" one among the pool of solutions. For comparison, we also trained a DNN using the established approach with the same settings (hyperparameters, data augmentation, etc.).

Results

The proposed DAUGS analysis approach performed similarly to the established approach on the internal dataset (Dice score for the testing subset of inD: 0.896 ± 0.050 vs. 0.890 ± 0.049; p = n.s.) whereas it significantly outperformed on the external datasets (Dice for exD-1: 0.885 ± 0.040 vs. 0.849 ± 0.065, p < 0.005; Dice for exD-2: 0.811 ± 0.070 vs. 0.728 ± 0.149, p < 0.005). Moreover, the number of image series with "failed" segmentation (defined as having myocardial contours that include bloodpool or are noncontiguous in ≥1 segment) was significantly lower for the proposed vs. the established approach (4.3% vs. 17.1%, p < 0.0005).

Conclusions

The proposed DAUGS analysis approach has the potential to improve the robustness of deep learning methods for segmentation of multi-center stress perfusion datasets with variations in the choice of pulse sequence, site location or scanner vendor.

Ventricular volume asymmetry as a novel imaging biomarker for disease discrimination and outcome prediction

Aims

Disruption of the predictable symmetry of the healthy heart may be an indicator of cardiovascular risk. This study defines the population distribution of ventricular asymmetry and its relationships across a range of prevalent and incident cardiorespiratory diseases.

Methods and results

The analysis includes 44 796 UK Biobank participants (average age 64.1 ± 7.7 years; 51.9% women). Cardiovascular magnetic resonance (CMR) metrics were derived using previously validated automated pipelines. Ventricular asymmetry was expressed as the ratio of left and right ventricular (LV and RV) end-diastolic volumes. Clinical outcomes were defined through linked health records. Incident events were those occurring for the first time after imaging, longitudinally tracked over an average follow-up time of 4.75 ± 1.52 years. The normal range for ventricular symmetry was defined in a healthy subset. Participants with values outside the 5th-95th percentiles of the healthy distribution were classed as either LV dominant (LV/RV > 112%) or RV dominant (LV/RV < 80%) asymmetry. Associations of LV and RV dominant asymmetry with vascular risk factors, CMR features, and prevalent and incident cardiovascular diseases (CVDs) were examined using regression models, adjusting for vascular risk factors, prevalent diseases, and conventional CMR measures. Left ventricular dominance was linked to an array of pre-existing vascular risk factors and CVDs, and a two-fold increased risk of incident heart failure, non-ischaemic cardiomyopathies, and left-sided valvular disorders. Right ventricular dominance was associated with an elevated risk of all-cause mortality.

Conclusion

Ventricular asymmetry has clinical utility for cardiovascular risk assessment, providing information that is incremental to traditional risk factors and conventional CMR metrics.

[Artificial intelligence in cardiovascular radiology : Image acquisition, image reconstruction and workflow optimization]

BACKGROUND

Artificial intelligence (AI) has the potential to fundamentally change radiology workflow.

OBJECTIVES

This review article provides an overview of AI applications in cardiovascular radiology with a focus on image acquisition, image reconstruction, and workflow optimization.

MATERIALS AND METHODS

First, established applications of AI are presented for cardiovascular computed tomography (CT) and magnetic resonance imaging (MRI). Building on this, we describe the range of applications that are currently being developed and evaluated. The practical benefits, opportunities, and potential risks of artificial intelligence in cardiovascular imaging are critically discussed. The presentation is based on the relevant specialist literature and our own clinical and scientific experience.

RESULTS

AI-based techniques for image reconstruction are already commercially available and enable dose reduction in cardiovascular CT and accelerated image acquisition in cardiac MRI. Postprocessing of cardiovascular CT and MRI examinations can already be considerably simplified using established AI-based segmentation algorithms. In contrast, the practical benefits of many AI applications aimed at the diagnosis of cardiovascular diseases are less evident. Potential risks such as automation bias and considerations regarding cost efficiency should also be taken into account.

CONCLUSIONS

In a market characterized by great expectations and rapid technical development, it is important to realistically assess the practical benefits of AI applications for your own hospital or practice.

Improving the efficiency and accuracy of cardiovascular magnetic resonance with artificial intelligence-review of evidence and proposition of a roadmap to clinical translation

BACKGROUND

Cardiovascular magnetic resonance (CMR) is an important imaging modality for the assessment of heart disease; however, limitations of CMR include long exam times and high complexity compared to other cardiac imaging modalities. Recently advancements in artificial intelligence (AI) technology have shown great potential to address many CMR limitations. While the developments are remarkable, translation of AI-based methods into real-world CMR clinical practice remains at a nascent stage and much work lies ahead to realize the full potential of AI for CMR.

METHODS

Herein we review recent cutting-edge and representative examples demonstrating how AI can advance CMR in areas such as exam planning, accelerated image reconstruction, post-processing, quality control, classification and diagnosis.

RESULTS

These advances can be applied to speed up and simplify essentially every application including cine, strain, late gadolinium enhancement, parametric mapping, 3D whole heart, flow, perfusion and others. AI is a unique technology based on training models using data. Beyond reviewing the literature, this paper discusses important AI-specific issues in the context of CMR, including (1) properties and characteristics of datasets for training and validation, (2) previously published guidelines for reporting CMR AI research, (3) considerations around clinical deployment, (4) responsibilities of clinicians and the need for multi-disciplinary teams in the development and deployment of AI in CMR, (5) industry considerations, and (6) regulatory perspectives.

CONCLUSIONS

Understanding and consideration of all these factors will contribute to the effective and ethical deployment of AI to improve clinical CMR.

Trustworthy clinical AI solutions: A unified review of uncertainty quantification in Deep Learning models for medical image analysis

The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. End users are particularly reluctant to rely on the opaque predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential solution, to reduce the black-box effect of DL models and increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associated with DL predictions. We focus on applications to medical image analysis, which present specific challenges due to the high dimensionality of images and their variable quality, as well as constraints associated with real-world clinical routine. Moreover, we discuss the concept of structural uncertainty, a corpus of methods to facilitate the alignment of segmentation uncertainty estimates with clinical attention. We then discuss the evaluation protocols to validate the relevance of uncertainty estimates. Finally, we highlight the open challenges for uncertainty quantification in the medical field.

A 3D Sparse Autoencoder for Fully Automated Quality Control of Affine Registrations in Big Data Brain MRI Studies

This paper presents a fully automated pipeline using a sparse convolutional autoencoder for quality control (QC) of affine registrations in large-scale T1-weighted (T1w) and T2-weighted (T2w) magnetic resonance imaging (MRI) studies. Here, a customized 3D convolutional encoder-decoder (autoencoder) framework is proposed and the network is trained in a fully unsupervised manner. For cross-validating the proposed model, we used 1000 correctly aligned MRI images of the human connectome project young adult (HCP-YA) dataset. We proposed that the quality of the registration is proportional to the reconstruction error of the autoencoder. Further, to make this method applicable to unseen datasets, we have proposed dataset-specific optimal threshold calculation (using the reconstruction error) from ROC analysis that requires a subset of the correctly aligned and artificially generated misalignments specific to that dataset. The calculated optimum threshold is used for testing the quality of remaining affine registrations from the corresponding datasets. The proposed framework was tested on four unseen datasets from autism brain imaging data exchange (ABIDE I, 215 subjects), information eXtraction from images (IXI, 577 subjects), Open Access Series of Imaging Studies (OASIS4, 646 subjects), and "Food and Brain" study (77 subjects). The framework has achieved excellent performance for T1w and T2w affine registrations with an accuracy of 100% for HCP-YA. Further, we evaluated the generality of the model on four unseen datasets and obtained accuracies of 81.81% for ABIDE I (only T1w), 93.45% (T1w) and 81.75% (T2w) for OASIS4, and 92.59% for "Food and Brain" study (only T1w) and in the range 88-97% for IXI (for both T1w and T2w and stratified concerning scanner vendor and magnetic field strengths). Moreover, the real failures from "Food and Brain" and OASIS4 datasets were detected with sensitivities of 100% and 80% for T1w and T2w, respectively. In addition, AUCs of > 0.88 in all scenarios were obtained during threshold calculation on the four test sets.

An efficient semi-supervised quality control system trained using physics-based MRI-artefact generators and adversarial training

Large medical imaging data sets are becoming increasingly available. A common challenge in these data sets is to ensure that each sample meets minimum quality requirements devoid of significant artefacts. Despite a wide range of existing automatic methods having been developed to identify imperfections and artefacts in medical imaging, they mostly rely on data-hungry methods. In particular, the scarcity of artefact-containing scans available for training has been a major obstacle in the development and implementation of machine learning in clinical research. To tackle this problem, we propose a novel framework having four main components: (1) a set of artefact generators inspired by magnetic resonance physics to corrupt brain MRI scans and augment a training dataset, (2) a set of abstract and engineered features to represent images compactly, (3) a feature selection process that depends on the class of artefact to improve classification performance, and (4) a set of Support Vector Machine (SVM) classifiers trained to identify artefacts. Our novel contributions are threefold: first, we use the novel physics-based artefact generators to generate synthetic brain MRI scans with controlled artefacts as a data augmentation technique. This will avoid the labour-intensive collection and labelling process of scans with rare artefacts. Second, we propose a large pool of abstract and engineered image features developed to identify 9 different artefacts for structural MRI. Finally, we use an artefact-based feature selection block that, for each class of artefacts, finds the set of features that provide the best classification performance. We performed validation experiments on a large data set of scans with artificially-generated artefacts, and in a multiple sclerosis clinical trial where real artefacts were identified by experts, showing that the proposed pipeline outperforms traditional methods. In particular, our data augmentation increases performance by up to 12.5 percentage points on the accuracy, F1, F2, precision and recall. At the same time, the computation cost of our pipeline remains low - less than a second to process a single scan - with the potential for real-time deployment. Our artefact simulators obtained using adversarial learning enable the training of a quality control system for brain MRI that otherwise would have required a much larger number of scans in both supervised and unsupervised settings. We believe that systems for quality control will enable a wide range of high-throughput clinical applications based on the use of automatic image-processing pipelines.

Present and Future Innovations in AI and Cardiac MRI

Cardiac MRI is used to diagnose and treat patients with a multitude of cardiovascular diseases. Despite the growth of clinical cardiac MRI, complicated image prescriptions and long acquisition protocols limit the specialty and restrain its impact on the practice of medicine. Artificial intelligence (AI)-the ability to mimic human intelligence in learning and performing tasks-will impact nearly all aspects of MRI. Deep learning (DL) primarily uses an artificial neural network to learn a specific task from example data sets. Self-driving scanners are increasingly available, where AI automatically controls cardiac image prescriptions. These scanners offer faster image collection with higher spatial and temporal resolution, eliminating the need for cardiac triggering or breath holding. In the future, fully automated inline image analysis will most likely provide all contour drawings and initial measurements to the reader. Advanced analysis using radiomic or DL features may provide new insights and information not typically extracted in the current analysis workflow. AI may further help integrate these features with clinical, genetic, wearable-device, and "omics" data to improve patient outcomes. This article presents an overview of AI and its application in cardiac MRI, including in image acquisition, reconstruction, and processing, and opportunities for more personalized cardiovascular care through extraction of novel imaging markers.

Selective ensemble methods for deep learning segmentation of major vessels in invasive coronary angiography

BACKGROUND

Invasive coronary angiography (ICA) is a primary imaging modality that visualizes the lumen area of coronary arteries for diagnosis and interventional guidance. In the current practice of quantitative coronary analysis (QCA), semi-automatic segmentation tools require labor-intensive and time-consuming manual correction, limiting their application in the catheterization room.

PURPOSE

This study aims to propose rank-based selective ensemble methods that improve the segmentation performance and reduce morphological errors that limit fully automated quantification of coronary artery using deep-learning segmentation of ICA.

METHODS

Two selective ensemble methods proposed in this work integrated the weighted ensemble approach with per-image quality estimation. The segmentation outcomes from five base models with different loss functions were ranked either by mask morphology or estimated dice similarity coefficient (DSC). The final output was determined by imposing different weights according to the ranks. The ranking criteria based on mask morphology were formulated from empirical insight to avoid frequent types of segmentation errors (MSEN), while the estimation of DSCs was performed by comparing the pseudo-ground truth generated from a meta-learner (ESEN). Five-fold cross-validation was performed with the internal dataset of 7426 coronary angiograms from 2924 patients, and prediction model was externally validated with 556 images of 226 patients.

RESULTS

The selective ensemble methods improved the segmentation performance with DSCs up to 93.07% and provided a better delineation of coronary lesion with local DSCs of up to 93.93%, outperforming all individual models. Proposed methods also minimized the chances of mask disconnection in the most narrowed regions to 2.10%. The robustness of the proposed methods was also evident in the external validation. Inference time for major vessel segmentation was approximately one-sixth of a second.

CONCLUSION

Proposed methods successfully reduced morphological errors in the predicted masks and were able to enhance the robustness of the automatic segmentation. The results suggest better applicability of real-time QCA-based diagnostic methods in routine clinical settings.

Temporal Uncertainty Localization to Enable Human-in-the-loop Analysis of Dynamic Contrast-enhanced Cardiac MRI Datasets

Dynamic contrast-enhanced (DCE) cardiac magnetic resonance imaging (CMRI) is a widely used modality for diagnosing myocardial blood flow (perfusion) abnormalities. During a typical free-breathing DCE-CMRI scan, close to 300 time-resolved images of myocardial perfusion are acquired at various contrast "wash in/out" phases. Manual segmentation of myocardial contours in each time-frame of a DCE image series can be tedious and time-consuming, particularly when non-rigid motion correction has failed or is unavailable. While deep neural networks (DNNs) have shown promise for analyzing DCE-CMRI datasets, a "dynamic quality control" (dQC) technique for reliably detecting failed segmentations is lacking. Here we propose a new space-time uncertainty metric as a dQC tool for DNN-based segmentation of free-breathing DCE-CMRI datasets by validating the proposed metric on an external dataset and establishing a human-in-the-loop framework to improve the segmentation results. In the proposed approach, we referred the top 10% most uncertain segmentations as detected by our dQC tool to the human expert for refinement. This approach resulted in a significant increase in the Dice score ( p < 0.001 ) and a notable decrease in the number of images with failed segmentation (16.2% to 11.3%) whereas the alternative approach of randomly selecting the same number of segmentations for human referral did not achieve any significant improvement. Our results suggest that the proposed dQC framework has the potential to accurately identify poor-quality segmentations and may enable efficient DNN-based analysis of DCE-CMRI in a human-in-the-loop pipeline for clinical interpretation and reporting of dynamic CMRI datasets.

Neuroticism personality traits are linked to adverse cardiovascular phenotypes in the UK Biobank

AIMS

To evaluate the relationship between neuroticism personality traits and cardiovascular magnetic resonance (CMR) measures of cardiac morphology and function, considering potential differential associations in men and women.

METHODS AND RESULTS

The analysis includes 36 309 UK Biobank participants (average age = 63.9 ± 7.7 years; 47.8% men) with CMR available and neuroticism score assessed by the 12-item Eysenck Personality Questionnaire-Revised Short Form. CMR scans were performed on 1.5 Tesla scanners (MAGNETOM Aera, Siemens Healthcare, Erlangen, Germany) according to pre-defined protocols and analysed using automated pipelines. We considered measures of left ventricular (LV) and right ventricular (RV) structure and function, and indicators of arterial compliance. Multivariable linear regression was used to estimate association of neuroticism score with individual CMR metrics, with adjustment for age, sex, obesity, deprivation, smoking, diabetes, hypertension, hypercholesterolaemia, alcohol use, exercise, and education. Higher neuroticism scores were associated with smaller LV and RV end-diastolic volumes, lower LV mass, greater concentricity (higher LV mass to volume ratio), and higher native T1. Greater neuroticism was also linked to poorer LV and RV function (lower stroke volumes) and greater arterial stiffness. In sex-stratified analyses, the relationships between neuroticism and LV stroke volume, concentricity, and arterial stiffness were attenuated in women. In men, association (with exception of native T1) remained robust.

CONCLUSION

Greater tendency towards neuroticism personality traits is linked to smaller, poorer functioning ventricles with lower LV mass, higher myocardial fibrosis, and higher arterial stiffness. These relationships are independent of traditional vascular risk factors and are more robust in men than women.

Temporal Uncertainty Localization to Enable Human-in-the-Loop Analysis of Dynamic Contrast-Enhanced Cardiac MRI Datasets

Dynamic contrast-enhanced (DCE) cardiac magnetic resonance imaging (CMRI) is a widely used modality for diagnosing myocardial blood flow (perfusion) abnormalities. During a typical free-breathing DCE-CMRI scan, close to 300 time-resolved images of myocardial perfusion are acquired at various contrast "wash in/out" phases. Manual segmentation of myocardial contours in each time-frame of a DCE image series can be tedious and time-consuming, particularly when non-rigid motion correction has failed or is unavailable. While deep neural networks (DNNs) have shown promise for analyzing DCE-CMRI datasets, a "dynamic quality control" (dQC) technique for reliably detecting failed segmentations is lacking. Here we propose a new space-time uncertainty metric as a dQC tool for DNN-based segmentation of free-breathing DCE-CMRI datasets by validating the proposed metric on an external dataset and establishing a human-in-the-loop framework to improve the segmentation results. In the proposed approach, we referred the top 10% most uncertain segmentations as detected by our dQC tool to the human expert for refinement. This approach resulted in a significant increase in the Dice score (p < 0.001) and a notable decrease in the number of images with failed segmentation (16.2% to 11.3%) whereas the alternative approach of randomly selecting the same number of segmentations for human referral did not achieve any significant improvement. Our results suggest that the proposed dQC framework has the potential to accurately identify poor-quality segmentations and may enable efficient DNN-based analysis of DCE-CMRI in a human-in-the-loop pipeline for clinical interpretation and reporting of dynamic CMRI datasets.

Validation of a deep learning-based software for automated analysis of T2 mapping in cardiac magnetic resonance imaging

Background

The reliability and diagnostic performance of deep learning (DL)-based automated T2 measurements on T2 map of 3.0-T cardiac magnetic resonance imaging (MRI) using multi-institutional datasets have not been investigated. We aimed to evaluate the performance of a DL-based software for measuring automated T2 values from 3.0-T cardiac MRI obtained at two centers.

Methods

Eighty-three subjects were retrospectively enrolled from two centers (42 healthy subjects and 41 patients with myocarditis) to validate a commercial DL-based software that was trained to segment the left ventricular myocardium and measure T2 values on T2 mapping sequences. Manual reference T2 values by two experienced radiologists and those calculated by the DL-based software were obtained. The segmentation performance of the DL-based software and the non-inferiority of automated T2 values were assessed compared with the manual reference standard per segment level. The software's performance in detecting elevated T2 values was assessed by calculating the sensitivity, specificity, and accuracy per segment.

Results

The average Dice similarity coefficient for segmentation of myocardium on T2 maps was 0.844. The automated T2 values were non-inferior to the manual reference T2 values on a per-segment analysis (45.35 vs. 44.32 ms). The DL-based software exhibited good performance (sensitivity: 83.6-92.8%; specificity: 82.5-92.0%; accuracy: 82.7-92.2%) in detecting elevated T2 values.

Conclusions

The DL-based software for automated T2 map analysis yields non-inferior measurements at the per-segment level and good performance for detecting myocardial segments with elevated T2 values compared with manual analysis.

Deep Ensembles Are Robust to Occasional Catastrophic Failures of Individual DNNs for Organs Segmentations in CT Images

Deep neural networks (DNNs) have recently showed remarkable performance in various computer vision tasks, including classification and segmentation of medical images. Deep ensembles (an aggregated prediction of multiple DNNs) were shown to improve a DNN's performance in various classification tasks. Here we explore how deep ensembles perform in the image segmentation task, in particular, organ segmentations in CT (Computed Tomography) images. Ensembles of V-Nets were trained to segment multiple organs using several in-house and publicly available clinical studies. The ensembles segmentations were tested on images from a different set of studies, and the effects of ensemble size as well as other ensemble parameters were explored for various organs. Compared to single models, Deep Ensembles significantly improved the average segmentation accuracy, especially for those organs where the accuracy was lower. More importantly, Deep Ensembles strongly reduced occasional "catastrophic" segmentation failures characteristic of single models and variability of the segmentation accuracy from image to image. To quantify this we defined the "high risk images": images for which at least one model produced an outlier metric (performed in the lower 5% percentile). These images comprised about 12% of the test images across all organs. Ensembles performed without outliers for 68%-100% of the "high risk images" depending on the performance metric used.

Characterizing the hypertensive cardiovascular phenotype in the UK Biobank

AIMS

To describe hypertension-related cardiovascular magnetic resonance (CMR) phenotypes in the UK Biobank considering variations across patient populations.

METHODS AND RESULTS

We studied 39 095 (51.5% women, mean age: 63.9 ± 7.7 years, 38.6% hypertensive) participants with CMR data available. Hypertension status was ascertained through health record linkage. Associations between hypertension and CMR metrics were estimated using multivariable linear regression adjusting for major vascular risk factors. Stratified analyses were performed by sex, ethnicity, time since hypertension diagnosis, and blood pressure (BP) control. Results are standardized beta coefficients, 95% confidence intervals, and P-values corrected for multiple testing. Hypertension was associated with concentric left ventricular (LV) hypertrophy (increased LV mass, wall thickness, concentricity index), poorer LV function (lower global function index, worse global longitudinal strain), larger left atrial (LA) volumes, lower LA ejection fraction, and lower aortic distensibility. Hypertension was linked to significantly lower myocardial native T1 and increased LV ejection fraction. Women had greater hypertension-related reduction in aortic compliance than men. The degree of hypertension-related LV hypertrophy was greatest in Black ethnicities. Increasing time since diagnosis of hypertension was linked to adverse remodelling. Hypertension-related remodelling was substantially attenuated in hypertensives with good BP control.

CONCLUSION

Hypertension was associated with concentric LV hypertrophy, reduced LV function, dilated poorer functioning LA, and reduced aortic compliance. Whilst the overall pattern of remodelling was consistent across populations, women had greater hypertension-related reduction in aortic compliance and Black ethnicities showed the greatest LV mass increase. Importantly, adverse cardiovascular remodelling was markedly attenuated in hypertensives with good BP control.

Quality control-driven deep ensemble for accountable automated segmentation of cardiac magnetic resonance LGE and VNE images

Background

Late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging is the gold standard for non-invasive myocardial tissue characterisation. However, accurate segmentation of the left ventricular (LV) myocardium remains a challenge due to limited training data and lack of quality control. This study addresses these issues by leveraging generative adversarial networks (GAN)-generated virtual native enhancement (VNE) images to expand the training set and incorporating an automated quality control-driven (QCD) framework to improve segmentation reliability.

Methods

A dataset comprising 4,716 LGE images (from 1,363 patients with hypertrophic cardiomyopathy and myocardial infarction) was used for development. To generate additional clinically validated data, LGE data were augmented with a GAN-based generator to produce VNE images. LV was contoured on these images manually by clinical observers. To create diverse candidate segmentations, the QCD framework involved multiple U-Nets, which were combined using statistical rank filters. The framework predicted the Dice Similarity Coefficient (DSC) for each candidate segmentation, with the highest predicted DSC indicating the most accurate and reliable result. The performance of the QCD ensemble framework was evaluated on both LGE and VNE test datasets (309 LGE/VNE images from 103 patients), assessing segmentation accuracy (DSC) and quality prediction (mean absolute error (MAE) and binary classification accuracy).

Results

The QCD framework effectively and rapidly segmented the LV myocardium (<1 s per image) on both LGE and VNE images, demonstrating robust performance on both test datasets with similar mean DSC (LGE: 0.845 ± 0.075 ; VNE: 0.845 ± 0.071 ; p = n s ). Incorporating GAN-generated VNE data into the training process consistently led to enhanced performance for both individual models and the overall framework. The quality control mechanism yielded a high performance (MAE = 0.043 , accuracy = 0.951 ) emphasising the accuracy of the quality control-driven strategy in predicting segmentation quality in clinical settings. Overall, no statistical difference (p = n s ) was found when comparing the LGE and VNE test sets across all experiments.

Conclusions

The QCD ensemble framework, leveraging GAN-generated VNE data and an automated quality control mechanism, significantly improved the accuracy and reliability of LGE segmentation, paving the way for enhanced and accountable diagnostic imaging in routine clinical use.

Automated T1 and T2 mapping segmentation on cardiovascular magnetic resonance imaging using deep learning

Introduction

Structural and functional heart abnormalities can be examined non-invasively with cardiac magnetic resonance imaging (CMR). Thanks to the development of MR devices, diagnostic scans can capture more and more relevant information about possible heart diseases. T1 and T2 mapping are such novel technology, providing tissue specific information even without the administration of contrast material. Artificial intelligence solutions based on deep learning have demonstrated state-of-the-art results in many application areas, including medical imaging. More specifically, automated tools applied at cine sequences have revolutionized volumetric CMR reporting in the past five years. Applying deep learning models to T1 and T2 mapping images can similarly improve the efficiency of post-processing pipelines and consequently facilitate diagnostic processes.

Methods

In this paper, we introduce a deep learning model for myocardium segmentation trained on over 7,000 raw CMR images from 262 subjects of heterogeneous disease etiology. The data were labeled by three experts. As part of the evaluation, Dice score and Hausdorff distance among experts is calculated, and the expert consensus is compared with the model's predictions.

Results

Our deep learning method achieves 86% mean Dice score, while contours provided by three experts on the same data show 90% mean Dice score. The method's accuracy is consistent across epicardial and endocardial contours, and on basal, midventricular slices, with only 5% lower results on apical slices, which are often challenging even for experts.

Conclusions

We trained and evaluated a deep learning based segmentation model on 262 heterogeneous CMR cases. Applying deep neural networks to T1 and T2 mapping could similarly improve diagnostic practices. Using the fine details of T1 and T2 mapping images and high-quality labels, the objective of this research is to approach human segmentation accuracy with deep learning.

The role of obesity-related cardiovascular remodelling in mediating incident cardiovascular outcomes: a population-based observational study

AIMS

We examined associations of obesity with incident cardiovascular outcomes and cardiovascular magnetic resonance (CMR) phenotypes, integrating information from body mass index (BMI) and waist-to-hip ratio (WHR). Then, we used multiple mediation to define the role of obesity-related cardiac remodelling in driving obesity-outcome associations, independent of cardiometabolic diseases.

METHODS AND RESULTS

In 491 606 UK Biobank participants, using Cox proportional hazard models, greater obesity (higher WHR, higher BMI) was linked to significantly greater risk of incident ischaemic heart disease, atrial fibrillation (AF), heart failure (HF), all-cause mortality, and cardiovascular disease (CVD) mortality. In combined stratification by BMI and WHR thresholds, elevated WHR was associated with greater risk of adverse outcomes at any BMI level. Individuals with overweight BMI but normal WHR had weaker disease associations. In the subset of participants with CMR (n = 31 107), using linear regression, greater obesity was associated with higher left ventricular (LV) mass, greater LV concentricity, poorer LV systolic function, lower myocardial native T1, larger left atrial (LA) volumes, poorer LA function, and lower aortic distensibility. Of note, higher BMI was linked to higher, whilst greater WHR was linked to lower LV end-diastolic volume (LVEDV). In Cox models, greater LVEDV and LV mass (LVM) were linked to increased risk of CVD, most importantly HF and an increased LA maximal volume was the key predictive measure of new-onset AF. In multiple mediation analyses, hypertension and adverse LV remodelling (higher LVM, greater concentricity) were major independent mediators of the obesity-outcome associations. Atrial remodelling and native T1 were additional mediators in the associations of obesity with AF and HF, respectively.

CONCLUSIONS

We demonstrate associations of obesity with adverse cardiovascular phenotypes and their significant independent role in mediating obesity-outcome relationships. In addition, our findings support the integrated use of BMI and WHR to evaluate obesity-related cardiovascular risk.

A benchmark study of convolutional neural networks in fully automatic segmentation of aortic root

Recent clinical studies have suggested that introducing 3D patient-specific aortic root models into the pre-operative assessment procedure of transcatheter aortic valve replacement (TAVR) would reduce the incident rate of peri-operative complications. Tradition manual segmentation is labor-intensive and low-efficient, which cannot meet the clinical demands of processing large data volumes. Recent developments in machine learning provided a viable way for accurate and efficient medical image segmentation for 3D patient-specific models automatically. This study quantitively evaluated the auto segmentation quality and efficiency of the four popular segmentation-dedicated three-dimensional (3D) convolutional neural network (CNN) architectures, including 3D UNet, VNet, 3D Res-UNet and SegResNet. All the CNNs were implemented in PyTorch platform, and low-dose CTA image sets of 98 anonymized patients were retrospectively selected from the database for training and testing of the CNNs. The results showed that despite all four 3D CNNs having similar recall, Dice similarity coefficient (DSC), and Jaccard index on the segmentation of the aortic root, the Hausdorff distance (HD) of the segmentation results from 3D Res-UNet is 8.56 ± 2.28, which is only 9.8% higher than that of VNet, but 25.5% and 86.4% lower than that of 3D UNet and SegResNet, respectively. In addition, 3D Res-UNet and VNet also performed better in the 3D deviation location of interest analysis focusing on the aortic valve and the bottom of the aortic root. Although 3D Res-UNet and VNet are evenly matched in the aspect of classical segmentation quality evaluation metrics and 3D deviation location of interest analysis, 3D Res-UNet is the most efficient CNN architecture with an average segmentation time of 0.10 ± 0.04 s, which is 91.2%, 95.3% and 64.3% faster than 3D UNet, VNet and SegResNet, respectively. The results from this study suggested that 3D Res-UNet is a suitable candidate for accurate and fast automatic aortic root segmentation for pre-operative assessment of TAVR.

Incident cardiovascular events and imaging phenotypes in UK Biobank participants with past cancer

OBJECTIVES

To evaluate incident cardiovascular outcomes and imaging phenotypes in UK Biobank participants with previous cancer.

METHODS

Cancer and cardiovascular disease (CVD) diagnoses were ascertained using health record linkage. Participants with cancer history (breast, lung, prostate, colorectal, uterus, haematological) were propensity matched on vascular risk factors to non-cancer controls. Competing risk regression was used to calculate subdistribution HRs (SHRs) for associations of cancer history with incident CVD (ischaemic heart disease (IHD), non-ischaemic cardiomyopathy (NICM), heart failure (HF), atrial fibrillation/flutter, stroke, pericarditis, venous thromboembolism (VTE)) and mortality outcomes (any CVD, IHD, HF/NICM, stroke, hypertensive disease) over 11.8±1.7 years of prospective follow-up. Linear regression was used to assess associations of cancer history with left ventricular (LV) and left atrial metrics.

RESULTS

We studied 18 714 participants (67% women, age: 62 (IQR: 57-66) years, 97% white ethnicities) with cancer history, including 1354 individuals with cardiovascular magnetic resonance. Participants with cancer had high burden of vascular risk factors and prevalent CVDs. Haematological cancer was associated with increased risk of all incident CVDs considered (SHRs: 1.92-3.56), larger chamber volumes, lower ejection fractions, and poorer LV strain. Breast cancer was associated with increased risk of selected CVDs (NICM, HF, pericarditis and VTE; SHRs: 1.34-2.03), HF/NICM death, hypertensive disease death, lower LV ejection fraction, and lower LV global function index. Lung cancer was associated with increased risk of pericarditis, HF, and CVD death. Prostate cancer was linked to increased VTE risk.

CONCLUSIONS

Cancer history is linked to increased risk of incident CVDs and adverse cardiac remodelling independent of shared vascular risk factors.

Incident Clinical and Mortality Associations of Myocardial Native T1 in the UK Biobank

BACKGROUND

Cardiac magnetic resonance native T1-mapping provides noninvasive, quantitative, and contrast-free myocardial characterization. However, its predictive value in population cohorts has not been studied.

OBJECTIVES

The associations of native T1 with incident events were evaluated in 42,308 UK Biobank participants over 3.17 ± 1.53 years of prospective follow-up.

METHODS

Native T1-mapping was performed in 1 midventricular short-axis slice using the Shortened Modified Look-Locker Inversion recovery technique (WIP780B) in 1.5-T scanners (Siemens Healthcare). Global myocardial T1 was calculated using an automated tool. Associations of T1 with: 1) prevalent risk factors (eg, diabetes, hypertension, and high cholesterol); 2) prevalent and incident diseases (eg, any cardiovascular disease [CVD], any brain disease, valvular heart disease, heart failure, nonischemic cardiomyopathies, cardiac arrhythmias, atrial fibrillation [AF], myocardial infarction, ischemic heart disease [IHD], and stroke); and 3) mortality (eg, all-cause, CVD, and IHD) were examined. Results are reported as odds ratios (ORs) or HRs per SD increment of T1 value with 95% CIs and corrected P values, from logistic and Cox proportional hazards regression models.

RESULTS

Higher myocardial T1 was associated with greater odds of a range of prevalent conditions (eg, any CVD, brain disease, heart failure, nonischemic cardiomyopathies, AF, stroke, and diabetes). The strongest relationships were with heart failure (OR: 1.41 [95% CI: 1.26-1.57]; P = 1.60 × 10-9) and nonischemic cardiomyopathies (OR: 1.40 [95% CI: 1.16-1.66]; P = 2.42 × 10-4). Native T1 was positively associated with incident AF (HR: 1.25 [95% CI: 1.10-1.43]; P = 9.19 × 10-4), incident heart failure (HR: 1.47 [95% CI: 1.31-1.65]; P = 4.79 × 10-11), all-cause mortality (HR: 1.24 [95% CI: 1.12-1.36]; P = 1.51 × 10-5), CVD mortality (HR: 1.40 [95% CI: 1.14-1.73]; P = 0.0014), and IHD mortality (HR: 1.36 [95% CI: 1.03-1.80]; P = 0.0310).

CONCLUSIONS

This large population study demonstrates the utility of myocardial native T1-mapping for disease discrimination and outcome prediction.

Cardiovascular magnetic resonance for evaluation of cardiac involvement in COVID-19: recommendations by the Society for Cardiovascular Magnetic Resonance

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic that has affected nearly 600 million people to date across the world. While COVID-19 is primarily a respiratory illness, cardiac injury is also known to occur. Cardiovascular magnetic resonance (CMR) imaging is uniquely capable of characterizing myocardial tissue properties in-vivo, enabling insights into the pattern and degree of cardiac injury. The reported prevalence of myocardial involvement identified by CMR in the context of COVID-19 infection among previously hospitalized patients ranges from 26 to 60%. Variations in the reported prevalence of myocardial involvement may result from differing patient populations (e.g. differences in severity of illness) and the varying intervals between acute infection and CMR evaluation. Standardized methodologies in image acquisition, analysis, interpretation, and reporting of CMR abnormalities across would likely improve concordance between studies. This consensus document by the Society for Cardiovascular Magnetic Resonance (SCMR) provides recommendations on CMR imaging and reporting metrics towards the goal of improved standardization and uniform data acquisition and analytic approaches when performing CMR in patients with COVID-19 infection.

Epoch and accuracy based empirical study for cardiac MRI segmentation using deep learning technique

Cardiac magnetic resonance imaging (CMRI) is a non-invasive imaging technique to analyse the structure and function of the heart. It was enhanced considerably over several years to deliver functional information for diagnosing and managing cardiovascular disease. CMRI image delivers non-invasive, clear access to the heart and great vessels. The segmentation of CMRI provides quantification parameters such as myocardial viability, ejection fraction, cardiac chamber volume, and morphological details. In general, experts interpret the CMR images by delineating the images manually. The manual segmentation process is time-consuming, and it has been observed that the final observation varied with the opinion of the different experts. Convolution neural network is a new-age technology that provides impressive results compared to manual ones. In this study convolution neural network model is used for the segmentation task. The neural network parameters have been optimized to perform on the novel data set for accurate predictions. With other parameters, epochs play an essential role in training the network, as the network should not be under-fitted or over-fitted. The relationship between the hyperparameter epoch and accuracy is established in the model. The model delivers the accuracy of 0.88 in terms of the IoU coefficient.

Automatic uncertainty-based quality controlled T1 mapping and ECV analysis from native and post-contrast cardiac T1 mapping images using Bayesian vision transformer

Deep learning-based methods for cardiac MR segmentation have achieved state-of-the-art results. However, these methods can generate incorrect segmentation results which can lead to wrong clinical decisions in the downstream tasks. Automatic and accurate analysis of downstream tasks, such as myocardial tissue characterization, is highly dependent on the quality of the segmentation results. Therefore, it is of paramount importance to use quality control methods to detect the failed segmentations before further analysis. In this work, we propose a fully automatic uncertainty-based quality control framework for T1 mapping and extracellular volume (ECV) analysis. The framework consists of three parts. The first one focuses on segmentation of cardiac structures from a native and post-contrast T1 mapping dataset (n=295) using a Bayesian Swin transformer-based U-Net. In the second part, we propose a novel uncertainty-based quality control (QC) to detect inaccurate segmentation results. The QC method utilizes image-level uncertainty features as input to a random forest-based classifier/regressor to determine the quality of the segmentation outputs. The experimental results from four different types of segmentation results show that the proposed QC method achieves a mean area under the ROC curve (AUC) of 0.927 on binary classification and a mean absolute error (MAE) of 0.021 on Dice score regression, significantly outperforming other state-of-the-art uncertainty based QC methods. The performance gap is notably higher in predicting the segmentation quality from poor-performing models which shows the robustness of our method in detecting failed segmentations. After the inaccurate segmentation results are detected and rejected by the QC method, in the third part, T1 mapping and ECV values are computed automatically to characterize the myocardial tissues of healthy and cardiac pathological cases. The native myocardial T1 and ECV values computed from automatic and manual segmentations show an excellent agreement yielding Pearson coefficients of 0.990 and 0.975 (on the combined validation and test sets), respectively. From the results, we observe that the automatically computed myocardial T1 and ECV values have the ability to characterize myocardial tissues of healthy and cardiac diseases like myocardial infarction, amyloidosis, Tako-Tsubo syndrome, dilated cardiomyopathy, and hypertrophic cardiomyopathy.

Introduction of a cascaded segmentation pipeline for parametric T1 mapping in cardiovascular magnetic resonance to improve segmentation performance

The manual and often time-consuming segmentation of the myocardium in cardiovascular magnetic resonance is increasingly automated using convolutional neural networks (CNNs). This study proposes a cascaded segmentation (CASEG) approach to improve automatic image segmentation quality. First, an object detection algorithm predicts a bounding box (BB) for the left ventricular myocardium whose 1.5 times enlargement defines the region of interest (ROI). Then, the ROI image section is fed into a U-Net based segmentation. Two CASEG variants were evaluated: one using the ROI cropped image solely (cropU) and the other using a 2-channel-image additionally containing the original BB image section (crinU). Both were compared to a classical U-Net segmentation (refU). All networks share the same hyperparameters and were tested on basal and midventricular slices of native and contrast enhanced (CE) MOLLI T1 maps. Dice Similarity Coefficient improved significantly (p < 0.05) in cropU and crinU compared to refU (81.06%, 81.22%, 72.79% for native and 80.70%, 79.18%, 71.41% for CE data), while no significant improvement (p < 0.05) was achieved in the mean absolute error of the T1 time (11.94 ms, 12.45 ms, 14.22 ms for native and 5.32 ms, 6.07 ms, 5.89 ms for CE data). In conclusion, CASEG provides an improved geometric concordance but needs further improvement in the quantitative outcome.

TMS-Net: A segmentation network coupled with a run-time quality control method for robust cardiac image segmentation

Recently, deep networks have shown impressive performance for the segmentation of cardiac Magnetic Resonance Imaging (MRI) images. However, their achievement is proving slow to transition to widespread use in medical clinics because of robustness issues leading to low trust of clinicians to their results. Predicting run-time quality of segmentation masks can be useful to warn clinicians against poor results. Despite its importance, there are few studies on this problem. To address this gap, we propose a quality control method based on the agreement across decoders of a multi-view network, TMS-Net, measured by the cosine similarity. The network takes three view inputs resliced from the same 3D image along different axes. Different from previous multi-view networks, TMS-Net has a single encoder and three decoders, leading to better noise robustness, segmentation performance and run-time quality estimation in our experiments on the segmentation of the left atrium on STACOM 2013 and STACOM 2018 challenge datasets. We also present a way to generate poor segmentation masks by using noisy images generated with engineered noise and Rician noise to simulate undertraining, high anisotropy and poor imaging settings problems. Our run-time quality estimation method show a good classification of poor and good quality segmentation masks with an AUC reaching to 0.97 on STACOM 2018. We believe that TMS-Net and our run-time quality estimation method has a high potential to increase the thrust of clinicians to automatic image analysis tools.

Automated Measurement of Native T1 and Extracellular Volume Fraction in Cardiac Magnetic Resonance Imaging Using a Commercially Available Deep Learning Algorithm

OBJECTIVE

T1 mapping provides valuable information regarding cardiomyopathies. Manual drawing is time consuming and prone to subjective errors. Therefore, this study aimed to test a DL algorithm for the automated measurement of native T1 and extracellular volume (ECV) fractions in cardiac magnetic resonance (CMR) imaging with a temporally separated dataset.

MATERIALS AND METHODS

CMR images obtained for 95 participants (mean age ± standard deviation, 54.5 ± 15.2 years), including 36 left ventricular hypertrophy (12 hypertrophic cardiomyopathy, 12 Fabry disease, and 12 amyloidosis), 32 dilated cardiomyopathy, and 27 healthy volunteers, were included. A commercial deep learning (DL) algorithm based on 2D U-net (Myomics-T1 software, version 1.0.0) was used for the automated analysis of T1 maps. Four radiologists, as study readers, performed manual analysis. The reference standard was the consensus result of the manual analysis by two additional expert readers. The segmentation performance of the DL algorithm and the correlation and agreement between the automated measurement and the reference standard were assessed. Interobserver agreement among the four radiologists was analyzed.

RESULTS

DL successfully segmented the myocardium in 99.3% of slices in the native T1 map and 89.8% of slices in the post-T1 map with Dice similarity coefficients of 0.86 ± 0.05 and 0.74 ± 0.17, respectively. Native T1 and ECV showed strong correlation and agreement between DL and the reference: for T1, r = 0.967 (95% confidence interval [CI], 0.951-0.978) and bias of 9.5 msec (95% limits of agreement [LOA], -23.6-42.6 msec); for ECV, r = 0.987 (95% CI, 0.980-0.991) and bias of 0.7% (95% LOA, -2.8%-4.2%) on per-subject basis. Agreements between DL and each of the four radiologists were excellent (intraclass correlation coefficient [ICC] of 0.98-0.99 for both native T1 and ECV), comparable to the pairwise agreement between the radiologists (ICC of 0.97-1.00 and 0.99-1.00 for native T1 and ECV, respectively).

CONCLUSION

The DL algorithm allowed automated T1 and ECV measurements comparable to those of radiologists.

Artificial Intelligence for Contrast-Free MRI: Scar Assessment in Myocardial Infarction Using Deep Learning-Based Virtual Native Enhancement

BACKGROUND

Myocardial scars are assessed noninvasively using cardiovascular magnetic resonance late gadolinium enhancement (LGE) as an imaging gold standard. A contrast-free approach would provide many advantages, including a faster and cheaper scan without contrast-associated problems.

METHODS

Virtual native enhancement (VNE) is a novel technology that can produce virtual LGE-like images without the need for contrast. VNE combines cine imaging and native T1 maps to produce LGE-like images using artificial intelligence. VNE was developed for patients with previous myocardial infarction from 4271 data sets (912 patients); each data set comprises slice position-matched cine, T1 maps, and LGE images. After quality control, 3002 data sets (775 patients) were used for development and 291 data sets (68 patients) for testing. The VNE generator was trained using generative adversarial networks, using 2 adversarial discriminators to improve the image quality. The left ventricle was contoured semiautomatically. Myocardial scar volume was quantified using the full width at half maximum method. Scar transmurality was measured using the centerline chord method and visualized on bull's-eye plots. Lesion quantification by VNE and LGE was compared using linear regression, Pearson correlation (R), and intraclass correlation coefficients. Proof-of-principle histopathologic comparison of VNE in a porcine model of myocardial infarction also was performed.

RESULTS

VNE provided significantly better image quality than LGE on blinded analysis by 5 independent operators on 291 data sets (all P<0.001). VNE correlated strongly with LGE in quantifying scar size (R, 0.89; intraclass correlation coefficient, 0.94) and transmurality (R, 0.84; intraclass correlation coefficient, 0.90) in 66 patients (277 test data sets). Two cardiovascular magnetic resonance experts reviewed all test image slices and reported an overall accuracy of 84% for VNE in detecting scars when compared with LGE, with specificity of 100% and sensitivity of 77%. VNE also showed excellent visuospatial agreement with histopathology in 2 cases of a porcine model of myocardial infarction.

CONCLUSIONS

VNE demonstrated high agreement with LGE cardiovascular magnetic resonance for myocardial scar assessment in patients with previous myocardial infarction in visuospatial distribution and lesion quantification with superior image quality. VNE is a potentially transformative artificial intelligence-based technology with promise in reducing scan times and costs, increasing clinical throughput, and improving the accessibility of cardiovascular magnetic resonance in the near future.

Clinician's guide to trustworthy and responsible artificial intelligence in cardiovascular imaging

A growing number of artificial intelligence (AI)-based systems are being proposed and developed in cardiology, driven by the increasing need to deal with the vast amount of clinical and imaging data with the ultimate aim of advancing patient care, diagnosis and prognostication. However, there is a critical gap between the development and clinical deployment of AI tools. A key consideration for implementing AI tools into real-life clinical practice is their "trustworthiness" by end-users. Namely, we must ensure that AI systems can be trusted and adopted by all parties involved, including clinicians and patients. Here we provide a summary of the concepts involved in developing a "trustworthy AI system." We describe the main risks of AI applications and potential mitigation techniques for the wider application of these promising techniques in the context of cardiovascular imaging. Finally, we show why trustworthy AI concepts are important governing forces of AI development.

Synthetic Post-Contrast Imaging through Artificial Intelligence: Clinical Applications of Virtual and Augmented Contrast Media

Contrast media are widely diffused in biomedical imaging, due to their relevance in the diagnosis of numerous disorders. However, the risk of adverse reactions, the concern of potential damage to sensitive organs, and the recently described brain deposition of gadolinium salts, limit the use of contrast media in clinical practice. In recent years, the application of artificial intelligence (AI) techniques to biomedical imaging has led to the development of 'virtual' and 'augmented' contrasts. The idea behind these applications is to generate synthetic post-contrast images through AI computational modeling starting from the information available on other images acquired during the same scan. In these AI models, non-contrast images (virtual contrast) or low-dose post-contrast images (augmented contrast) are used as input data to generate synthetic post-contrast images, which are often undistinguishable from the native ones. In this review, we discuss the most recent advances of AI applications to biomedical imaging relative to synthetic contrast media.

A Deep Learning Segmentation Pipeline for Cardiac T1 Mapping Using MRI Relaxation-based Synthetic Contrast Augmentation

PURPOSE

To design and evaluate an automated deep learning method for segmentation and analysis of cardiac MRI T1 maps with use of synthetic T1-weighted images for MRI relaxation-based contrast augmentation.

MATERIALS AND METHODS

This retrospective study included MRI scans acquired between 2016 and 2019 from 100 patients (mean age ± SD, 55 years ± 13; 72 men) across various clinical abnormalities with use of a modified Look-Locker inversion recovery, or MOLLI, sequence to quantify native T1 (T1_native), postcontrast T1 (T1_post), and extracellular volume (ECV). Data were divided into training (n = 60) and internal (n = 40) test subsets. "Synthetic" T1-weighted images were generated from the T1 exponential inversion-recovery signal model at a range of optimal inversion times, yielding high blood-myocardium contrast, and were used for contrast-based image augmentation during training and testing of a convolutional neural network for myocardial segmentation. Automated segmentation, T1, and ECV were compared with experts with use of Dice similarity coefficients (DSCs), correlation coefficients, and Bland-Altman analysis. An external test dataset (n = 147) was used to assess model generalization.

RESULTS

Internal testing showed high myocardial DSC relative to experts (0.81 ± 0.08), which was similar to interobserver DSC (0.81 ± 0.08). Automated segmental measurements strongly correlated with experts (T1_native, R = 0.87; T1_post, R = 0.91; ECV, R = 0.92), which were similar to interobserver correlation (T1_native, R = 0.86; T1_post, R = 0.94; ECV, R = 0.95). External testing showed strong DSC (0.80 ± 0.09) and T1_native correlation (R = 0.88) between automatic and expert analysis.

CONCLUSION

This deep learning method leveraging synthetic contrast augmentation may provide accurate automated T1 and ECV analysis for cardiac MRI data acquired across different abnormalities, centers, scanners, and T1 sequences.Keywords: MRI, Cardiac, Tissue Characterization, Segmentation, Convolutional Neural Network, Deep Learning Algorithms, Machine Learning Algorithms, Supervised Learning Supplemental material is available for this article. © RSNA, 2022.

Impact of deep learning architectures on accelerated cardiac T1 mapping using MyoMapNet

The objective of the current study was to investigate the performance of various deep learning (DL) architectures for MyoMapNet, a DL model for T1 estimation using accelerated cardiac T1 mapping from four T1 -weighted images collected after a single inversion pulse (Look-Locker 4 [LL4]). We implemented and tested three DL architectures for MyoMapNet: (a) a fully connected neural network (FC), (b) convolutional neural networks (VGG19, ResNet50), and (c) encoder-decoder networks with skip connections (ResUNet, U-Net). Modified Look-Locker inversion recovery (MOLLI) images from 749 patients at 3 T were used for training, validation, and testing. The first four T1 -weighted images from MOLLI5(3)3 and/or MOLLI4(1)3(1)2 protocols were extracted to create accelerated cardiac T1 mapping data. We also prospectively collected data from 28 subjects using MOLLI and LL4 to further evaluate model performance. Despite rigorous training, conventional VGG19 and ResNet50 models failed to produce anatomically correct T1 maps, and T1 values had significant errors. While ResUNet yielded good quality maps, it significantly underestimated T1 . Both FC and U-Net, however, yielded excellent image quality with good T1 accuracy for both native (FC/U-Net/MOLLI = 1217 ± 64/1208 ± 61/1199 ± 61 ms, all p < 0.05) and postcontrast myocardial T1 (FC/U-Net/MOLLI = 578 ± 57/567 ± 54/574 ± 55 ms, all p < 0.05). In terms of precision, the U-Net model yielded better T1 precision compared with the FC architecture (standard deviation of 61 vs. 67 ms for the myocardium for native [p < 0.05], and 31 vs. 38 ms [p < 0.05], for postcontrast). Similar findings were observed in prospectively collected LL4 data. It was concluded that U-Net and FC DL models in MyoMapNet enable fast myocardial T1 mapping using only four T1 -weighted images collected from a single LL sequence with comparable accuracy. U-Net also provides a slight improvement in precision.

Myocardial infarct border demarcation by dual-wavelength photoacoustic spectral analysis

Myocardial infarction (MI) is a major cause of morbidity and mortality worldwide. Modern therapeutic strategies targeting the infarct border area have been shown to benefit overall cardiac function after MI. However, there is no non-invasive diagnostic technique to precisely demarcate the MI boundary till to now. In this study, the feasibility of demarcating the MI border using dual-wavelength photoacoustic spectral analysis (DWPASA) was investigated. To quantify specific molecular characteristics before and after MI, "the ratio of the areas of the power spectral densities (R APSD)" was computed from the DWPASA results. Compared to the normal tissue, MI tissue was shown to contain more collagen, resulting in higher R APSD values (p < 0.001). Cross-sectional MI lengths and the MI area border demarcated in two dimensions by DWPASA were in substantial agreement with Masson staining (ICC = 0.76, p < 0.001, IoU = 0.72). R APSD has been proved that can be used as an indicator of disease evolution to distinguish normal and pathological tissues. These findings indicate that the DWPASA method may offer a new diagnostic solution for determining MI borders.

Deep Neural Network for Cardiac Magnetic Resonance Image Segmentation

The analysis and interpretation of cardiac magnetic resonance (CMR) images are often time-consuming. The automated segmentation of cardiac structures can reduce the time required for image analysis. Spatial similarities between different CMR image types were leveraged to jointly segment multiple sequences using a segmentation model termed a multi-image type UNet (MI-UNet). This model was developed from 72 exams (46% female, mean age 63 ± 11 years) performed on patients with hypertrophic cardiomyopathy. The MI-UNet for steady-state free precession (SSFP) images achieved a superior Dice similarity coefficient (DSC) of 0.92 ± 0.06 compared to 0.87 ± 0.08 for a single-image type UNet (p < 0.001). The MI-UNet for late gadolinium enhancement (LGE) images also had a superior DSC of 0.86 ± 0.11 compared to 0.78 ± 0.11 for a single-image type UNet (p = 0.001). The difference across image types was most evident for the left ventricular myocardium in SSFP images and for both the left ventricular cavity and the left ventricular myocardium in LGE images. For the right ventricle, there were no differences in DCS when comparing the MI-UNet with single-image type UNets. The joint segmentation of multiple image types increases segmentation accuracy for CMR images of the left ventricle compared to single-image models. In clinical practice, the MI-UNet model may expedite the analysis and interpretation of CMR images of multiple types.

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

Since the rise of deep learning (DL) in the mid-2010s, cardiac magnetic resonance (CMR) image segmentation has achieved state-of-the-art performance. Despite achieving inter-observer variability in terms of different accuracy performance measures, visual inspections reveal errors in most segmentation results, indicating a lack of reliability and robustness of DL segmentation models, which can be critical if a model was to be deployed into clinical practice. In this work, we aim to bring attention to reliability and robustness, two unmet needs of cardiac image segmentation methods, which are hampering their translation into practice. To this end, we first study the performance accuracy evolution of CMR segmentation, illustrate the improvements brought by DL algorithms and highlight the symptoms of performance stagnation. Afterwards, we provide formal definitions of reliability and robustness. Based on the two definitions, we identify the factors that limit the reliability and robustness of state-of-the-art deep learning CMR segmentation techniques. Finally, we give an overview of the current set of works that focus on improving the reliability and robustness of CMR segmentation, and we categorize them into two families of methods: quality control methods and model improvement techniques. The first category corresponds to simpler strategies that only aim to flag situations where a model may be incurring poor reliability or robustness. The second one, instead, directly tackles the problem by bringing improvements into different aspects of the CMR segmentation model development process. We aim to bring the attention of more researchers towards these emerging trends regarding the development of reliable and robust CMR segmentation frameworks, which can guarantee the safe use of DL in clinical routines and studies.

Generalizable Framework for Atrial Volume Estimation for Cardiac CT Images Using Deep Learning With Quality Control Assessment

Objectives

Cardiac computed tomography (CCT) is a common pre-operative imaging modality to evaluate pulmonary vein anatomy and left atrial appendage thrombus in patients undergoing catheter ablation (CA) for atrial fibrillation (AF). These images also allow for full volumetric left atrium (LA) measurement for recurrence risk stratification, as larger LA volume (LAV) is associated with higher recurrence rates. Our objective is to apply deep learning (DL) techniques to fully automate the computation of LAV and assess the quality of the computed LAV values.

Methods

Using a dataset of 85,477 CCT images from 337 patients, we proposed a framework that consists of several processes that perform a combination of tasks including the selection of images with LA from all other images using a ResNet50 classification model, the segmentation of images with LA using a UNet image segmentation model, the assessment of the quality of the image segmentation task, the estimation of LAV, and quality control (QC) assessment.

Results

Overall, the proposed LAV estimation framework achieved accuracies of 98% (precision, recall, and F1 score metrics) in the image classification task, 88.5% (mean dice score) in the image segmentation task, 82% (mean dice score) in the segmentation quality prediction task, and R² (the coefficient of determination) value of 0.968 in the volume estimation task. It correctly identified 9 out of 10 poor LAV estimations from a total of 337 patients as poor-quality estimates.

Conclusions

We proposed a generalizable framework that consists of DL models and computational methods for LAV estimation. The framework provides an efficient and robust strategy for QC assessment of the accuracy for DL-based image segmentation and volume estimation tasks, allowing high-throughput extraction of reproducible LAV measurements to be possible.

Multiparametric Cardiovascular Magnetic Resonance Approach in Diagnosing, Monitoring, and Prognostication of Myocarditis

Myocarditis represents the entity of an inflamed myocardium and is a diagnostic challenge caused by its heterogeneous presentation. Contemporary noninvasive evaluation of patients with clinically suspected myocarditis using cardiac magnetic resonance (CMR) includes dimensions and function of the heart chambers, conventional T2-weighted imaging, late gadolinium enhancement, novel T1 and T2 mapping, and extracellular volume fraction calculation. CMR feature-tracking, texture analysis, and artificial intelligence emerge as potential modern techniques to further improve diagnosis and prognostication in this clinical setting. This review will describe the evidence surrounding different CMR methods and image postprocessing methods and highlight their values for clinical decision making, monitoring, and risk stratification across stages of this condition.

MOCOnet: Robust Motion Correction of Cardiovascular Magnetic Resonance T1 Mapping Using Convolutional Neural Networks

Background: Quantitative cardiovascular magnetic resonance (CMR) T1 mapping has shown promise for advanced tissue characterisation in routine clinical practise. However, T1 mapping is prone to motion artefacts, which affects its robustness and clinical interpretation. Current methods for motion correction on T1 mapping are model-driven with no guarantee on generalisability, limiting its widespread use. In contrast, emerging data-driven deep learning approaches have shown good performance in general image registration tasks. We propose MOCOnet, a convolutional neural network solution, for generalisable motion artefact correction in T1 maps.

Methods: The network architecture employs U-Net for producing distance vector fields and utilises warping layers to apply deformation to the feature maps in a coarse-to-fine manner. Using the UK Biobank imaging dataset scanned at 1.5T, MOCOnet was trained on 1,536 mid-ventricular T1 maps (acquired using the ShMOLLI method) with motion artefacts, generated by a customised deformation procedure, and tested on a different set of 200 samples with a diverse range of motion. MOCOnet was compared to a well-validated baseline multi-modal image registration method. Motion reduction was visually assessed by 3 human experts, with motion scores ranging from 0% (strictly no motion) to 100% (very severe motion).

Results: MOCOnet achieved fast image registration (<1 second per T1 map) and successfully suppressed a wide range of motion artefacts. MOCOnet significantly reduced motion scores from 37.1±21.5 to 13.3±10.5 (p < 0.001), whereas the baseline method reduced it to 15.8±15.6 (p < 0.001). MOCOnet was significantly better than the baseline method in suppressing motion artefacts and more consistently (p = 0.007).

Conclusion: MOCOnet demonstrated significantly better motion correction performance compared to a traditional image registration approach. Salvaging data affected by motion with robustness and in a time-efficient manner may enable better image quality and reliable images for immediate clinical interpretation.

MVnet: automated time-resolved tracking of the mitral valve plane in CMR long-axis cine images with residual neural networks: a multi-center, multi-vendor study

BACKGROUND

Mitral annular plane systolic excursion (MAPSE) and left ventricular (LV) early diastolic velocity (e') are key metrics of systolic and diastolic function, but not often measured by cardiovascular magnetic resonance (CMR). Its derivation is possible with manual, precise annotation of the mitral valve (MV) insertion points along the cardiac cycle in both two and four-chamber long-axis cines, but this process is highly time-consuming, laborious, and prone to errors. A fully automated, consistent, fast, and accurate method for MV plane tracking is lacking. In this study, we propose MVnet, a deep learning approach for MV point localization and tracking capable of deriving such clinical metrics comparable to human expert-level performance, and validated it in a multi-vendor, multi-center clinical population.

METHODS

The proposed pipeline first performs a coarse MV point annotation in a given cine accurately enough to apply an automated linear transformation task, which standardizes the size, cropping, resolution, and heart orientation, and second, tracks the MV points with high accuracy. The model was trained and evaluated on 38,854 cine images from 703 patients with diverse cardiovascular conditions, scanned on equipment from 3 main vendors, 16 centers, and 7 countries, and manually annotated by 10 observers. Agreement was assessed by the intra-class correlation coefficient (ICC) for both clinical metrics and by the distance error in the MV plane displacement. For inter-observer variability analysis, an additional pair of observers performed manual annotations in a randomly chosen set of 50 patients.

RESULTS

MVnet achieved a fast segmentation (<1 s/cine) with excellent ICCs of 0.94 (MAPSE) and 0.93 (LV e') and a MV plane tracking error of -0.10 ± 0.97 mm. In a similar manner, the inter-observer variability analysis yielded ICCs of 0.95 and 0.89 and a tracking error of -0.15 ± 1.18 mm, respectively.

CONCLUSION

A dual-stage deep learning approach for automated annotation of MV points for systolic and diastolic evaluation in CMR long-axis cine images was developed. The method is able to carefully track these points with high accuracy and in a timely manner. This will improve the feasibility of CMR methods which rely on valve tracking and increase their utility in a clinical setting.

Toward Replacing Late Gadolinium Enhancement With Artificial Intelligence Virtual Native Enhancement for Gadolinium-Free Cardiovascular Magnetic Resonance Tissue Characterization in Hypertrophic Cardiomyopathy

BACKGROUND

Late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging is the gold standard for noninvasive myocardial tissue characterization but requires intravenous contrast agent administration. It is highly desired to develop a contrast agent-free technology to replace LGE for faster and cheaper CMR scans.

METHODS

A CMR virtual native enhancement (VNE) imaging technology was developed using artificial intelligence. The deep learning model for generating VNE uses multiple streams of convolutional neural networks to exploit and enhance the existing signals in native T1 maps (pixel-wise maps of tissue T1 relaxation times) and cine imaging of cardiac structure and function, presenting them as LGE-equivalent images. The VNE generator was trained using generative adversarial networks. This technology was first developed on CMR datasets from the multicenter Hypertrophic Cardiomyopathy Registry, using hypertrophic cardiomyopathy as an exemplar. The datasets were randomized into 2 independent groups for deep learning training and testing. The test data of VNE and LGE were scored and contoured by experienced human operators to assess image quality, visuospatial agreement, and myocardial lesion burden quantification. Image quality was compared using a nonparametric Wilcoxon test. Intra- and interobserver agreement was analyzed using intraclass correlation coefficients (ICC). Lesion quantification by VNE and LGE were compared using linear regression and ICC.

RESULTS

A total of 1348 hypertrophic cardiomyopathy patients provided 4093 triplets of matched T1 maps, cines, and LGE datasets. After randomization and data quality control, 2695 datasets were used for VNE method development and 345 were used for independent testing. VNE had significantly better image quality than LGE, as assessed by 4 operators (n=345 datasets; P<0.001 [Wilcoxon test]). VNE revealed lesions characteristic of hypertrophic cardiomyopathy in high visuospatial agreement with LGE. In 121 patients (n=326 datasets), VNE correlated with LGE in detecting and quantifying both hyperintensity myocardial lesions (r=0.77-0.79; ICC=0.77-0.87; P<0.001) and intermediate-intensity lesions (r=0.70-0.76; ICC=0.82-0.85; P<0.001). The native CMR images (cine plus T1 map) required for VNE can be acquired within 15 minutes and producing a VNE image takes less than 1 second.

CONCLUSIONS

VNE is a new CMR technology that resembles conventional LGE but without the need for contrast administration. VNE achieved high agreement with LGE in the distribution and quantification of lesions, with significantly better image quality.

Emerging Techniques in Cardiac Magnetic Resonance Imaging

Cardiovascular disease is the leading cause of death and a significant contributor of health care costs. Noninvasive imaging plays an essential role in the management of patients with cardiovascular disease. Cardiac magnetic resonance (MR) can noninvasively assess heart and vascular abnormalities, including biventricular structure/function, blood hemodynamics, myocardial tissue composition, microstructure, perfusion, metabolism, coronary microvascular function, and aortic distensibility/stiffness. Its ability to characterize myocardial tissue composition is unique among alternative imaging modalities in cardiovascular disease. Significant growth in cardiac MR utilization, particularly in Europe in the last decade, has laid the necessary clinical groundwork to position cardiac MR as an important imaging modality in the workup of patients with cardiovascular disease. Although lack of availability, limited training, physician hesitation, and reimbursement issues have hampered widespread clinical adoption of cardiac MR in the United States, growing clinical evidence will ultimately overcome these challenges. Advances in cardiac MR techniques, particularly faster image acquisition, quantitative myocardial tissue characterization, and image analysis have been critical to its growth. In this review article, we discuss recent advances in established and emerging cardiac MR techniques that are expected to strengthen its capability in managing patients with cardiovascular disease. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 1.

Cardiac stress T1-mapping response and extracellular volume stability of MOLLI-based T1-mapping methods

Stress and rest T1-mapping may assess for myocardial ischemia and extracellular volume (ECV). However, the stress T1 response is method-dependent, and underestimation may lead to misdiagnosis. Further, ECV quantification may be affected by time, as well as the number and dosage of gadolinium (Gd) contrast administered. We compared two commonly available T1-mapping approaches in their stress T1 response and ECV measurement stability. Healthy subjects (n = 10, 50% female, 35 ± 8 years) underwent regadenoson stress CMR (1.5 T) on two separate days. Prototype ShMOLLI 5(1)1(1)1 sequence was used to acquire consecutive mid-ventricular T1-maps at rest, stress and post-Gd contrast to track the T1 time evolution. For comparison, standard MOLLI sequences were used: MOLLI 5(3)3 Low (256 matrix) & High (192 matrix) Heart Rate (HR) to acquire rest and stress T1-maps, and MOLLI 4(1)3(1)2 Low & High HR for post-contrast T1-maps. Stress and rest myocardial blood flow (MBF) maps were acquired after IV Gd contrast (0.05 mmol/kg each). Stress T1 reactivity (delta T1) was defined as the relative percentage increase in native T1 between rest and stress. Myocardial T1 values for delta T1 (dT1) and ECV were calculated. Residuals from the identified time dependencies were used to assess intra-method variability. ShMOLLI achieved a greater stress T1 response compared to MOLLI Low and High HR (peak dT1 = 6.4 ± 1.7% vs. 4.8 ± 1.3% vs. 3.8 ± 1.0%, respectively; both p < 0.0001). ShMOLLI dT1 correlated strongly with stress MBF (r = 0.77, p < 0.001), compared to MOLLI Low HR (r = 0.65, p < 0.01) and MOLLI High HR (r = 0.43, p = 0.07). ShMOLLI ECV was more stable to gadolinium dose with less time drift (0.006-0.04% per minute) than MOLLI variants. Overall, ShMOLLI demonstrated less intra-individual variability than MOLLI variants for stress T1 and ECV quantification. Power calculations indicate up to a fourfold (stress T1) and 7.5-fold (ECV) advantage in sample-size reduction using ShMOLLI. Our results indicate that ShMOLLI correlates strongly with increased MBF during regadenoson stress and achieves a significantly higher stress T1 response, greater effect size, and greater ECV measurement stability compared with the MOLLI variants tested.

Deep learning approach for guiding three-dimensional computed tomography reconstruction of lower limbs for robotically-assisted total knee arthroplasty

BACKGROUND

Robotic-assisted total knee arthroplasty (TKA) was performed to promote the accuracy of bone resection and mechanical alignment. Among these TKA system procedures, 3D reconstruction of CT data of lower limbs consumes significant manpower. Artificial intelligence (AI) algorithms applying deep learning has been proved efficient in automated identification and visual processing.

METHODS

CT data of a total of 200 lower limbs scanning were used for AI-based 3D model construction and CT data of 20 lower limbs scanning were utilised for verification.

RESULTS

We showed that the performance of an AI-guided 3D reconstruction of CT data of lower limbs for robotic-assisted TKA was similar to that of the operator-based approach. The time of 3D lower limb model construction using AI was 4.7 min. AI-based 3D models can be used for surgical planning.

CONCLUSION

AI was used for the first time to guide the 3D reconstruction of CT data of lower limbs for facilitating robotic-assisted TKA. Incorporation of AI in 3D model reconstruction before TKA might reduce the workload of radiologists.