Deep learning cardiac motion analysis for human survival prediction

Improving the efficiency and accuracy of CMR with AI - review of evidence and proposition of a roadmap to clinical translation

Cardiovascular magnetic resonance (CMR) is an important imaging modality for the assessment of heart disease; however, limitations of CMR include long exam times and high complexity compared to other cardiac imaging modalities. Recently advancements in artificial intelligence (AI) technology have shown great potential to address many CMR limitations. While the developments are remarkable, translation of AI-based methods into real-world CMR clinical practice remains at a nascent stage and much work lies ahead to realize the full potential of AI for CMR. Herein we review recent cutting-edge and representative examples demonstrating how AI can advance CMR in areas such as exam planning, accelerated image reconstruction, post-processing, quality control, classification and diagnosis. These advances can be applied to speed up and simplify essentially every application including cine, strain, late gadolinium enhancement, parametric mapping, 3D whole heart, flow, perfusion and others. AI is a unique technology based on training models using data. Beyond reviewing the literature, this paper discusses important AI-specific issues in the context of CMR, including (1) properties and characteristics of datasets for training and validation, (2) previously published guidelines for reporting CMR AI research, (3) considerations around clinical deployment, (4) responsibilities of clinicians and the need for multi-disciplinary teams in the development and deployment of AI in CMR, (5) industry considerations, and (6) regulatory perspectives. Understanding and consideration of all these factors will contribute to the effective and ethical deployment of AI to improve clinical CMR.

DeepMesh: Mesh-Based Cardiac Motion Tracking Using Deep Learning

3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and the diagnosis of cardiovascular diseases. Current state-of-the art methods focus on estimating dense pixel-/voxel-wise motion fields in image space, which ignores the fact that motion estimation is only relevant and useful within the anatomical objects of interest, e.g., the heart. In this work, we model the heart as a 3D mesh consisting of epi- and endocardial surfaces. We propose a novel learning framework, DeepMesh, which propagates a template heart mesh to a subject space and estimates the 3D motion of the heart mesh from CMR images for individual subjects. In DeepMesh, the heart mesh of the end-diastolic frame of an individual subject is first reconstructed from the template mesh. Mesh-based 3D motion fields with respect to the end-diastolic frame are then estimated from 2D short- and long-axis CMR images. By developing a differentiable mesh-to-image rasterizer, DeepMesh is able to leverage 2D shape information from multiple anatomical views for 3D mesh reconstruction and mesh motion estimation. The proposed method estimates vertex-wise displacement and thus maintains vertex correspondences between time frames, which is important for the quantitative assessment of cardiac function across different subjects and populations. We evaluate DeepMesh on CMR images acquired from the UK Biobank. We focus on 3D motion estimation of the left ventricle in this work. Experimental results show that the proposed method quantitatively and qualitatively outperforms other image-based and mesh-based cardiac motion tracking methods.

Automated segmentation of the left-ventricle from MRI with a fully convolutional network to investigate CTRCD in breast cancer patients

Purpose: The goal of this study was to develop a fully convolutional network (FCN) tool to automatedly segment the left-ventricular (LV) myocardium in displacement encoding with stimulated echoes MRI. The segmentation results are used for LV chamber quantification and strain analyses in breast cancer patients susceptible to cancer therapy-related cardiac dysfunction (CTRCD). Approach: A DeepLabV3+ FCN with a ResNet-101 backbone was custom-designed to conduct chamber quantification on 45 female breast cancer datasets (23 training, 11 validation, and 11 test sets). LV structural parameters and LV ejection fraction (LVEF) were measured, and myocardial strains estimated with the radial point interpolation method. Myocardial classification validation was against quantization-based ground-truth with computations of accuracy, Dice score, average perpendicular distance (APD), Hausdorff-distance, and others. Additional validations were conducted with equivalence tests and Cronbach's alpha (C - α ) intraclass correlation coefficients between the FCN and a vendor tool on chamber quantification and myocardial strain computations. Results: Myocardial classification results against ground-truth were Dice = 0.89 , APD = 2.4    mm , and accuracy = 97 % for the validation set and Dice = 0.90 , APD = 2.5    mm , and accuracy = 97 % for the test set. The confidence intervals (CI) and two one-sided t-test results of equivalence tests between the FCN and vendor-tool were CI = - 1.36 % to 2.42%, p-value < 0.001 for LVEF (58 ± 5 % versus 57 ± 6 % ), and CI = - 0.71 % to 0.63%, p-value < 0.001 for longitudinal strain (- 15 ± 2 % versus - 15 ± 3 % ). Conclusions: The validation results were found equivalent to the vendor tool-based parameter estimates, which show that accurate LV chamber quantification followed by strain analysis for CTRCD investigation can be achieved with our proposed FCN methodology.

Automated Real-Time Detection of Lung Sliding Using Artificial Intelligence: A Prospective Diagnostic Accuracy Study

BACKGROUND

Rapid evaluation for pneumothorax is a common clinical priority. Although lung ultrasound (LUS) often is used to assess for pneumothorax, its diagnostic accuracy varies based on patient and provider factors. To enhance the performance of LUS for pulmonary pathologic features, artificial intelligence (AI)-assisted imaging has been adopted; however, the diagnostic accuracy of AI-assisted LUS (AI-LUS) deployed in real time to diagnose pneumothorax remains unknown.

RESEARCH QUESTION

In patients with suspected pneumothorax, what is the real-time diagnostic accuracy of AI-LUS to recognize the absence of lung sliding?

STUDY DESIGN AND METHODS

We performed a prospective AI-assisted diagnostic accuracy study of AI-LUS to recognize the absence of lung sliding in a convenience sample of patients with suspected pneumothorax. After calibrating the model parameters and imaging settings for bedside deployment, we prospectively evaluated its diagnostic accuracy for lung sliding compared with a reference standard of expert consensus.

RESULTS

Two hundred forty-one lung sliding evaluations were derived from 62 patients. AI-LUS showed a sensitivity of 0.921 (95% CI, 0.792-0.973), specificity of 0.802 (95% CI, 0.735-0.856), area under the receiver operating characteristic curve of 0.885 (95% CI, 0.828-0.956), and accuracy of 0.824 (95% CI, 0.766-0.870) for the diagnosis of absent lung sliding.

INTERPRETATION

Real-time AI-LUS shows high sensitivity and moderate specificity to identify the absence of lung sliding. Further research to improve model performance and optimize the integration of AI-LUS into existing diagnostic pathways is warranted.

Automated inversion time selection for late gadolinium-enhanced cardiac magnetic resonance imaging

OBJECTIVES

To develop and share a deep learning method that can accurately identify optimal inversion time (TI) from multi-vendor, multi-institutional and multi-field strength inversion scout (TI scout) sequences for late gadolinium enhancement cardiac MRI.

MATERIALS AND METHODS

Retrospective multicentre study conducted on 1136 1.5-T and 3-T cardiac MRI examinations from four centres and three scanner vendors. Deep learning models, comprising a convolutional neural network (CNN) that provides input to a long short-term memory (LSTM) network, were trained on TI scout pixel data from centres 1 to 3 to identify optimal TI, using ground truth annotations by two readers. Accuracy within 50 ms, mean absolute error (MAE), Lin's concordance coefficient (LCCC) and reduced major axis regression (RMAR) were used to select the best model from validation results, and applied to holdout test data. Robustness of the best-performing model was also tested on imaging data from centre 4.

RESULTS

The best model (SE-ResNet18-LSTM) produced accuracy of 96.1%, MAE 22.9 ms and LCCC 0.47 compared to ground truth on the holdout test set and accuracy of 97.3%, MAE 15.2 ms and LCCC 0.64 when tested on unseen external (centre 4) data. Differences in vendor performance were observed, with greatest accuracy for the most commonly represented vendor in the training data.

CONCLUSION

A deep learning model was developed that can identify optimal inversion time from TI scout images on multi-vendor data with high accuracy, including on previously unseen external data. We make this model available to the scientific community for further assessment or development.

CLINICAL RELEVANCE STATEMENT

A robust automated inversion time selection tool for late gadolinium-enhanced imaging allows for reproducible and efficient cross-vendor inversion time selection.

KEY POINTS

• A model comprising convolutional and recurrent neural networks was developed to extract optimal TI from TI scout images. • Model accuracy within 50 ms of ground truth on multi-vendor holdout and external data of 96.1% and 97.3% respectively was achieved. • This model could improve workflow efficiency and standardise optimal TI selection for consistent LGE imaging.

Deep Learning Based Prediction of Pulmonary Hypertension in Newborns Using Echocardiograms

Pulmonary hypertension (PH) in newborns and infants is a complex condition associated with several pulmonary, cardiac, and systemic diseases contributing to morbidity and mortality. Thus, accurate and early detection of PH and the classification of its severity is crucial for appropriate and successful management. Using echocardiography, the primary diagnostic tool in pediatrics, human assessment is both time-consuming and expertise-demanding, raising the need for an automated approach. Little effort has been directed towards automatic assessment of PH using echocardiography, and the few proposed methods only focus on binary PH classification on the adult population. In this work, we present an explainable multi-view video-based deep learning approach to predict and classify the severity of PH for a cohort of 270 newborns using echocardiograms. We use spatio-temporal convolutional architectures for the prediction of PH from each view, and aggregate the predictions of the different views using majority voting. Our results show a mean F1-score of 0.84 for severity prediction and 0.92 for binary detection using 10-fold cross-validation and 0.63 for severity prediction and 0.78 for binary detection on the held-out test set. We complement our predictions with saliency maps and show that the learned model focuses on clinically relevant cardiac structures, motivating its usage in clinical practice. To the best of our knowledge, this is the first work for an automated assessment of PH in newborns using echocardiograms.

Advancements in artificial intelligence-driven techniques for interventional cardiology

This paper aims to thoroughly discuss the impact of artificial intelligence (AI) on clinical practice in interventional cardiology (IC) with special recognition of its most recent advancements. Thus, recent years have been exceptionally abundant in advancements in computational tools, including the development of AI. The application of AI development is currently in its early stages, nevertheless new technologies have proven to be a promising concept, particularly considering IC showing great impact on patient safety, risk stratification and outcomes during the whole therapeutic process. The primary goal is to achieve the integration of multiple cardiac imaging modalities, establish online decision support systems and platforms based on augmented and/or virtual realities, and finally to create automatic medical systems, providing electronic health data on patients. In a simplified way, two main areas of AI utilization in IC may be distinguished, namely, virtual and physical. Consequently, numerous studies have provided data regarding AI utilization in terms of automated interpretation and analysis from various cardiac modalities, including electrocardiogram, echocardiography, angiography, cardiac magnetic resonance imaging, and computed tomography as well as data collected during robotic-assisted percutaneous coronary intervention procedures. Thus, this paper aims to thoroughly discuss the impact of AI on clinical practice in IC with special recognition of its most recent advancements.

Microfluidics-based patient-derived disease detection tool for deep learning-assisted precision medicine

Cancer spatial and temporal heterogeneity fuels resistance to therapies. To realize the routine assessment of cancer prognosis and treatment, we demonstrate the development of an Intelligent Disease Detection Tool (IDDT), a microfluidic-based tumor model integrated with deep learning-assisted algorithmic analysis. IDDT was clinically validated with liquid blood biopsy samples (n = 71) from patients with various types of cancers (e.g., breast, gastric, and lung cancer) and healthy donors, requiring low sample volume (∼200 μl) and a high-throughput 3D tumor culturing system (∼300 tumor clusters). To support automated algorithmic analysis, intelligent decision-making, and precise segmentation, we designed and developed an integrative deep neural network, which includes Mask Region-Based Convolutional Neural Network (Mask R-CNN), vision transformer, and Segment Anything Model (SAM). Our approach significantly reduces the manual labeling time by up to 90% with a high mean Intersection Over Union (mIoU) of 0.902 and immediate results (<2 s per image) for clinical cohort classification. The IDDT can accurately stratify healthy donors (n = 12) and cancer patients (n = 55) within their respective treatment cycle and cancer stage, resulting in high precision (∼99.3%) and high sensitivity (∼98%). We envision that our patient-centric IDDT provides an intelligent, label-free, and cost-effective approach to help clinicians make precise medical decisions and tailor treatment strategies for each patient.

Dynamic prediction of malignant ventricular arrhythmias using neural networks in patients with an implantable cardioverter-defibrillator

BACKGROUND

Risk stratification for ventricular arrhythmias currently relies on static measurements that fail to adequately capture dynamic interactions between arrhythmic substrate and triggers over time. We trained and internally validated a dynamic machine learning (ML) model and neural network that extracted features from longitudinally collected electrocardiograms (ECG), and used these to predict the risk of malignant ventricular arrhythmias.

METHODS

A multicentre study in patients implanted with an implantable cardioverter-defibrillator (ICD) between 2007 and 2021 in two academic hospitals was performed. Variational autoencoders (VAEs), which combine neural networks with variational inference principles, and can learn patterns and structure in data without explicit labelling, were trained to encode the mean ECG waveforms from the limb leads into 16 variables. Supervised dynamic ML models using these latent ECG representations and clinical baseline information were trained to predict malignant ventricular arrhythmias treated by the ICD. Model performance was evaluated on a hold-out set, using time-dependent receiver operating characteristic (ROC) and calibration curves.

FINDINGS

2942 patients (61.7 ± 13.9 years, 25.5% female) were included, with a total of 32,129 ECG recordings during a mean follow-up of 43.9 ± 35.9 months. The mean time-varying area under the ROC curve for the dynamic model was 0.738 ± 0.07, compared to 0.639 ± 0.03 for a static (i.e. baseline-only model). Feature analyses indicated dynamic changes in latent ECG representations, particularly those affecting the T-wave morphology, were of highest importance for model predictions.

INTERPRETATION

Dynamic ML models and neural networks effectively leverage routinely collected longitudinal ECG recordings for personalised and updated predictions of malignant ventricular arrhythmias, outperforming static models.

FUNDING

This publication is part of the project DEEP RISK ICD (with project number 452019308) of the research programme Rubicon which is (partly) financed by the Dutch Research Council (NWO). This research is partly funded by the Amsterdam Cardiovascular Sciences (personal grant F.V.Y.T).

Present and Future Innovations in AI and Cardiac MRI

Cardiac MRI is used to diagnose and treat patients with a multitude of cardiovascular diseases. Despite the growth of clinical cardiac MRI, complicated image prescriptions and long acquisition protocols limit the specialty and restrain its impact on the practice of medicine. Artificial intelligence (AI)-the ability to mimic human intelligence in learning and performing tasks-will impact nearly all aspects of MRI. Deep learning (DL) primarily uses an artificial neural network to learn a specific task from example data sets. Self-driving scanners are increasingly available, where AI automatically controls cardiac image prescriptions. These scanners offer faster image collection with higher spatial and temporal resolution, eliminating the need for cardiac triggering or breath holding. In the future, fully automated inline image analysis will most likely provide all contour drawings and initial measurements to the reader. Advanced analysis using radiomic or DL features may provide new insights and information not typically extracted in the current analysis workflow. AI may further help integrate these features with clinical, genetic, wearable-device, and "omics" data to improve patient outcomes. This article presents an overview of AI and its application in cardiac MRI, including in image acquisition, reconstruction, and processing, and opportunities for more personalized cardiovascular care through extraction of novel imaging markers.

Biopsy-free AI-aided precision MRI assessment in prediction of prostate cancer biochemical recurrence

BACKGROUND

To investigate the predictive ability of high-throughput MRI with deep survival networks for biochemical recurrence (BCR) of prostate cancer (PCa) after prostatectomy.

METHODS

Clinical-MRI and histopathologic data of 579 (train/test, 463/116) PCa patients were retrospectively collected. The deep survival network (iBCR-Net) is based on stepwise processing operations, which first built an MRI radiomics signature (RadS) for BCR, and predicted the T3 stage and lymph node metastasis (LN+) of tumour using two predefined AI models. Subsequently, clinical, imaging and histopathological variables were integrated into iBCR-Net for BCR prediction.

RESULTS

RadS, derived from 2554 MRI features, was identified as an independent predictor of BCR. Two predefined AI models achieved an accuracy of 82.6% and 78.4% in staging T3 and LN+. The iBCR-Net, when expressed as a presurgical model by integrating RadS, AI-diagnosed T3 stage and PSA, can match a state-of-the-art histopathological model (C-index, 0.81 to 0.83 vs 0.79 to 0.81, p > 0.05); and has maximally 5.16-fold, 12.8-fold, and 2.09-fold (p < 0.05) benefit to conventional D'Amico score, the Cancer of the Prostate Risk Assessment (CAPRA) score and the CAPRA Postsurgical score.

CONCLUSIONS

AI-aided iBCR-Net using high-throughput MRI can predict PCa BCR accurately and thus may provide an alternative to the conventional method for PCa risk stratification.

Deep learning-based prognostic model using non-enhanced cardiac cine MRI for outcome prediction in patients with heart failure

OBJECTIVES

To evaluate the performance of a deep learning-based multi-source model for survival prediction and risk stratification in patients with heart failure.

METHODS

Patients with heart failure with reduced ejection fraction (HFrEF) who underwent cardiac magnetic resonance between January 2015 and April 2020 were retrospectively included in this study. Baseline electronic health record data, including clinical demographic information, laboratory data, and electrocardiographic information, were collected. Short-axis non-contrast cine images of the whole heart were acquired to estimate the cardiac function parameters and the motion features of the left ventricle. Model accuracy was evaluated using the Harrell's concordance index. All patients were followed up for major adverse cardiac events (MACEs), and survival prediction was assessed using Kaplan-Meier curves.

RESULTS

A total of 329 patients were evaluated (age 54 ± 14 years; men, 254) in this study. During a median follow-up period of 1041 days, 62 patients experienced MACEs and their median survival time was 495 days. When compared with conventional Cox hazard prediction models, deep learning models showed better survival prediction performance. Multi-data denoising autoencoder (DAE) model reached the concordance index of 0.8546 (95% CI: 0.7902-0.8883). Furthermore, when divided into phenogroups, the multi-data DAE model could significantly discriminate between the survival outcomes of the high-risk and low-risk groups compared with other models (p < 0.001).

CONCLUSIONS

The proposed deep learning (DL) model based on non-contrast cardiac cine magnetic resonance imaging could independently predict the outcome of patients with HFrEF and showed better prediction efficiency than conventional methods.

CLINICAL RELEVANCE STATEMENT

The proposed multi-source deep learning model based on cardiac magnetic resonance enables survival prediction in patients with heart failure.

KEY POINTS

• A multi-source deep learning model based on non-contrast cardiovascular magnetic resonance (CMR) cine images was built to make robust survival prediction in patients with heart failure. • The ground truth definition contains electronic health record data as well as DL-based motion data, and cardiac motion information is extracted by optical flow method from non-contrast CMR cine images. • The DL-based model exhibits better prognostic value and stratification performance when compared with conventional prediction models and could aid in the risk stratification in patients with HF.

WarpPINN: Cine-MR image registration with physics-informed neural networks

The diagnosis of heart failure usually includes a global functional assessment, such as ejection fraction measured by magnetic resonance imaging. However, these metrics have low discriminate power to distinguish different cardiomyopathies, which may not affect the global function of the heart. Quantifying local deformations in the form of cardiac strain can provide helpful information, but it remains a challenge. In this work, we introduce WarpPINN, a physics-informed neural network to perform image registration to obtain local metrics of heart deformation. We apply this method to cine magnetic resonance images to estimate the motion during the cardiac cycle. We inform our neural network of the near-incompressibility of cardiac tissue by penalizing the Jacobian of the deformation field. The loss function has two components: an intensity-based similarity term between the reference and the warped template images, and a regularizer that represents the hyperelastic behavior of the tissue. The architecture of the neural network allows us to easily compute the strain via automatic differentiation to assess cardiac activity. We use Fourier feature mappings to overcome the spectral bias of neural networks, allowing us to capture discontinuities in the strain field. The algorithm is tested on synthetic examples and on a cine SSFP MRI benchmark of 15 healthy volunteers, where it is trained to learn the deformation mapping of each case. We outperform current methodologies in landmark tracking and provide physiological strain estimations in the radial and circumferential directions. WarpPINN provides precise measurements of local cardiac deformations that can be used for a better diagnosis of heart failure and can be used for general image registration tasks. Source code is available at https://github.com/fsahli/WarpPINN.

Development and validation of a fully automatic tissue delineation model for brain metastasis using a deep neural network

Background

Stereotactic radiosurgery (SRS) treatment planning requires accurate delineation of brain metastases, a task that can be tedious and time-consuming. Although studies have explored the use of convolutional neural networks (CNNs) in magnetic resonance imaging (MRI) for automatic brain metastases delineation, none of these studies have performed clinical evaluation, raising concerns about clinical applicability. This study aimed to develop an artificial intelligence (AI) tool for the automatic delineation of single brain metastasis that could be integrated into clinical practice.

Methods

Data from 426 patients with postcontrast T1-weighted MRIs who underwent SRS between March 2007 and August 2019 were retrospectively collected and divided into training, validation, and testing cohorts of 299, 42, and 85 patients, respectively. Two Gamma Knife (GK) surgeons contoured the brain metastases as the ground truth. A novel 2.5D CNN network was developed for single brain metastasis delineation. The mean Dice similarity coefficient (DSC) and average surface distance (ASD) were used to assess the performance of this method.

Results

The mean DSC and ASD values were 88.34%±5.00% and 0.35±0.21 mm, respectively, for the contours generated with the AI tool based on the testing set. The DSC measure of the AI tool's performance was dependent on metastatic shape, reinforcement shape, and the existence of peritumoral edema (all P values <0.05). The clinical experts' subjective assessments showed that 415 out of 572 slices (72.6%) in the testing cohort were acceptable for clinical usage without revision. The average time spent editing an AI-generated contour compared with time spent with manual contouring was 74 vs. 196 seconds, respectively (P<0.01).

Conclusions

The contours delineated with the AI tool for single brain metastasis were in close agreement with the ground truth. The developed AI tool can effectively reduce contouring time and aid in GK treatment planning of single brain metastasis in clinical practice.

Neural network on interval-censored data with application to the prediction of Alzheimer's disease

Alzheimer's disease (AD) is a progressive and polygenic disorder that affects millions of individuals each year. Given that there have been few effective treatments yet for AD, it is highly desirable to develop an accurate model to predict the full disease progression profile based on an individual's genetic characteristics for early prevention and clinical management. This work uses data composed of all four phases of the Alzheimer's Disease Neuroimaging Initiative (ADNI) study, including 1740 individuals with 8 million genetic variants. We tackle several challenges in this data, characterized by large-scale genetic data, interval-censored outcome due to intermittent assessments, and left truncation in one study phase (ADNIGO). Specifically, we first develop a semiparametric transformation model on interval-censored and left-truncated data and estimate parameters through a sieve approach. Then we propose a computationally efficient generalized score test to identify variants associated with AD progression. Next, we implement a novel neural network on interval-censored data (NN-IC) to construct a prediction model using top variants identified from the genome-wide test. Comprehensive simulation studies show that the NN-IC outperforms several existing methods in terms of prediction accuracy. Finally, we apply the NN-IC to the full ADNI data and successfully identify subgroups with differential progression risk profiles. Data used in the preparation of this article were obtained from the ADNI database.

The Puzzle of Evaluating Moral Cognition in Artificial Agents

In developing artificial intelligence (AI), researchers often benchmark against human performance as a measure of progress. Is this kind of comparison possible for moral cognition? Given that human moral judgment often hinges on intangible properties like "intention" which may have no natural analog in artificial agents, it may prove difficult to design a "like-for-like" comparison between the moral behavior of artificial and human agents. What would a measure of moral behavior for both humans and AI look like? We unravel the complexity of this question by discussing examples within reinforcement learning and generative AI, and we examine how the puzzle of evaluating artificial agents' moral cognition remains open for further investigation within cognitive science.

Diagnosis of Parkinson's Disease via the Metabolic Fingerprint in Saliva by Deep Learning

Parkinson's disease (PD) is the second cause of the neurodegenerative disorder, affecting over 6 million people worldwide. The World Health Organization estimated that population aging will cause global PD prevalence to double in the coming 30 years. Optimal management of PD shall start at diagnosis and requires both a timely and accurate method. Conventional PD diagnosis needs observations and clinical signs assessment, which are time-consuming and low-throughput. A lack of body fluid diagnostic biomarkers for PD has been a significant challenge, although substantial progress has been made in genetic and imaging marker development. Herein, a platform that noninvasively collects saliva metabolic fingerprinting (SMF) by nanoparticle-enhanced laser desorption-ionization mass spectrometry with high-reproducibility and high-throughput, using ultra-small sample volume (down to 10 nL), is developed. Further, excellent diagnostic performance is achieved with an area-under-the-curve of 0.8496 (95% CI: 0.7393-0.8625) by constructing deep learning model from 312 participants. In conclusion, an alternative solution is provided for the molecular diagnostics of PD with SMF and metabolic biomarker screening for therapeutic intervention.

Artificial intelligence in cardiovascular imaging: enhancing image analysis and risk stratification

In this review, we summarize state-of-the-art artificial intelligence applications for non-invasive cardiovascular imaging modalities including CT, MRI, echocardiography, and nuclear myocardial perfusion imaging.

Survival prediction of heart failure patients using motion-based analysis method

BACKGROUND AND OBJECTIVE

Survival prediction of heart failure patients is critical to improve the prognostic management of the cardiovascular disease. The existing survival prediction methods focus on the clinical information while lacking the cardiac motion information. we propose a motion-based analysis method to predict the survival risk of heart failure patients for aiding clinical diagnosis and treatment.

METHODS

We propose a motion-based analysis method for survival prediction of heart failure patients. First, our method proposes the hierarchical spatial-temporal structure to capture the myocardial border. It promotes the model discrimination on border features. Second, our method explores the dense optical flow structure to capture motion fields. It improves the tracking capability on cardiac images. The cardiac motion information is obtained by fusing boundary information and motion fields of cardiac images. Finally, our method proposes the multi-modality deep-cox structure to predict the survival risk of heart failure patients. It improves the survival probability of heart failure patients.

RESULTS

The motion-based analysis method is confirmed to be able to improve the survival prediction of heart failure patients. The precision, recall, F1-score, and C-index are 0.8519, 0.8333, 0.8425, and 0.8478, respectively, which is superior to other state-of-the-art methods.

CONCLUSIONS

The experimental results show that the proposed model can effectively predict survival risk of heart failure patients. It facilitates the application of robust clinical treatment strategies.

Automated segmentation of long and short axis DENSE cardiovascular magnetic resonance for myocardial strain analysis using spatio-temporal convolutional neural networks

BACKGROUND

Cine Displacement Encoding with Stimulated Echoes (DENSE) facilitates the quantification of myocardial deformation, by encoding tissue displacements in the cardiovascular magnetic resonance (CMR) image phase, from which myocardial strain can be estimated with high accuracy and reproducibility. Current methods for analyzing DENSE images still heavily rely on user input, making this process time-consuming and subject to inter-observer variability. The present study sought to develop a spatio-temporal deep learning model for segmentation of the left-ventricular (LV) myocardium, as spatial networks often fail due to contrast-related properties of DENSE images.

METHODS

2D + time nnU-Net-based models have been trained to segment the LV myocardium from DENSE magnitude data in short- and long-axis images. A dataset of 360 short-axis and 124 long-axis slices was used to train the networks, from a combination of healthy subjects and patients with various conditions (hypertrophic and dilated cardiomyopathy, myocardial infarction, myocarditis). Segmentation performance was evaluated using ground-truth manual labels, and a strain analysis using conventional methods was performed to assess strain agreement with manual segmentation. Additional validation was performed using an externally acquired dataset to compare the inter- and intra-scanner reproducibility with respect to conventional methods.

RESULTS

Spatio-temporal models gave consistent segmentation performance throughout the cine sequence, while 2D architectures often failed to segment end-diastolic frames due to the limited blood-to-myocardium contrast. Our models achieved a DICE score of 0.83 ± 0.05 and a Hausdorff distance of 4.0 ± 1.1 mm for short-axis segmentation, and 0.82 ± 0.03 and 7.9 ± 3.9 mm respectively for long-axis segmentations. Strain measurements obtained from automatically estimated myocardial contours showed good to excellent agreement with manual pipelines, and remained within the limits of inter-user variability estimated in previous studies.

CONCLUSION

Spatio-temporal deep learning shows increased robustness for the segmentation of cine DENSE images. It provides excellent agreement with manual segmentation for strain extraction. Deep learning will facilitate the analysis of DENSE data, bringing it one step closer to clinical routine.

Artificial intelligence to enhance clinical value across the spectrum of cardiovascular healthcare

Artificial intelligence (AI) is increasingly being utilized in healthcare. This article provides clinicians and researchers with a step-wise foundation for high-value AI that can be applied to a variety of different data modalities. The aim is to improve the transparency and application of AI methods, with the potential to benefit patients in routine cardiovascular care. Following a clear research hypothesis, an AI-based workflow begins with data selection and pre-processing prior to analysis, with the type of data (structured, semi-structured, or unstructured) determining what type of pre-processing steps and machine-learning algorithms are required. Algorithmic and data validation should be performed to ensure the robustness of the chosen methodology, followed by an objective evaluation of performance. Seven case studies are provided to highlight the wide variety of data modalities and clinical questions that can benefit from modern AI techniques, with a focus on applying them to cardiovascular disease management. Despite the growing use of AI, further education for healthcare workers, researchers, and the public are needed to aid understanding of how AI works and to close the existing gap in knowledge. In addition, issues regarding data access, sharing, and security must be addressed to ensure full engagement by patients and the public. The application of AI within healthcare provides an opportunity for clinicians to deliver a more personalized approach to medical care by accounting for confounders, interactions, and the rising prevalence of multi-morbidity.

Artificial Intelligence as a Diagnostic Tool in Non-Invasive Imaging in the Assessment of Coronary Artery Disease

Coronary artery disease (CAD) remains a leading cause of mortality and morbidity worldwide, and it is associated with considerable economic burden. In an ageing, multimorbid population, it has become increasingly important to develop reliable, consistent, low-risk, non-invasive means of diagnosing CAD. The evolution of multiple cardiac modalities in this field has addressed this dilemma to a large extent, not only in providing information regarding anatomical disease, as is the case with coronary computed tomography angiography (CCTA), but also in contributing critical details about functional assessment, for instance, using stress cardiac magnetic resonance (S-CMR). The field of artificial intelligence (AI) is developing at an astounding pace, especially in healthcare. In healthcare, key milestones have been achieved using AI and machine learning (ML) in various clinical settings, from smartwatches detecting arrhythmias to retinal image analysis and skin cancer prediction. In recent times, we have seen an emerging interest in developing AI-based technology in the field of cardiovascular imaging, as it is felt that ML methods have potential to overcome some limitations of current risk models by applying computer algorithms to large databases with multidimensional variables, thus enabling the inclusion of complex relationships to predict outcomes. In this paper, we review the current literature on the various applications of AI in the assessment of CAD, with a focus on multimodality imaging, followed by a discussion on future perspectives and critical challenges that this field is likely to encounter as it continues to evolve in cardiology.

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

Guidelines and evaluation of clinical explainable AI in medical image analysis

Explainable artificial intelligence (XAI) is essential for enabling clinical users to get informed decision support from AI and comply with evidence-based medical practice. Applying XAI in clinical settings requires proper evaluation criteria to ensure the explanation technique is both technically sound and clinically useful, but specific support is lacking to achieve this goal. To bridge the research gap, we propose the Clinical XAI Guidelines that consist of five criteria a clinical XAI needs to be optimized for. The guidelines recommend choosing an explanation form based on Guideline 1 (G1) Understandability and G2 Clinical relevance. For the chosen explanation form, its specific XAI technique should be optimized for G3 Truthfulness, G4 Informative plausibility, and G5 Computational efficiency. Following the guidelines, we conducted a systematic evaluation on a novel problem of multi-modal medical image explanation with two clinical tasks, and proposed new evaluation metrics accordingly. Sixteen commonly-used heatmap XAI techniques were evaluated and found to be insufficient for clinical use due to their failure in G3 and G4. Our evaluation demonstrated the use of Clinical XAI Guidelines to support the design and evaluation of clinically viable XAI.

Generative myocardial motion tracking via latent space exploration with biomechanics-informed prior

Myocardial motion and deformation are rich descriptors that characterize cardiac function. Image registration, as the most commonly used technique for myocardial motion tracking, is an ill-posed inverse problem which often requires prior assumptions on the solution space. In contrast to most existing approaches which impose explicit generic regularization such as smoothness, in this work we propose a novel method that can implicitly learn an application-specific biomechanics-informed prior and embed it into a neural network-parameterized transformation model. Particularly, the proposed method leverages a variational autoencoder-based generative model to learn a manifold for biomechanically plausible deformations. The motion tracking then can be performed via traversing the learnt manifold to search for the optimal transformations while considering the sequence information. The proposed method is validated on three public cardiac cine MRI datasets with comprehensive evaluations. The results demonstrate that the proposed method can outperform other approaches, yielding higher motion tracking accuracy with reasonable volume preservation and better generalizability to varying data distributions. It also enables better estimates of myocardial strains, which indicates the potential of the method in characterizing spatiotemporal signatures for understanding cardiovascular diseases.

Interpretable cardiac anatomy modeling using variational mesh autoencoders

Cardiac anatomy and function vary considerably across the human population with important implications for clinical diagnosis and treatment planning. Consequently, many computer-based approaches have been developed to capture this variability for a wide range of applications, including explainable cardiac disease detection and prediction, dimensionality reduction, cardiac shape analysis, and the generation of virtual heart populations. In this work, we propose a variational mesh autoencoder (mesh VAE) as a novel geometric deep learning approach to model such population-wide variations in cardiac shapes. It embeds multi-scale graph convolutions and mesh pooling layers in a hierarchical VAE framework to enable direct processing of surface mesh representations of the cardiac anatomy in an efficient manner. The proposed mesh VAE achieves low reconstruction errors on a dataset of 3D cardiac meshes from over 1,000 patients with acute myocardial infarction, with mean surface distances between input and reconstructed meshes below the underlying image resolution. We also find that it outperforms a voxelgrid-based deep learning benchmark in terms of both mean surface distance and Hausdorff distance while requiring considerably less memory. Furthermore, we explore the quality and interpretability of the mesh VAE's latent space and showcase its ability to improve the prediction of major adverse cardiac events over a clinical benchmark. Finally, we investigate the method's ability to generate realistic virtual populations of cardiac anatomies and find good alignment between the synthesized and gold standard mesh populations in terms of multiple clinical metrics.

A spatial attention guided deep learning system for prediction of pathological complete response using breast cancer histopathology images

MOTIVATION

Predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in triple-negative breast cancer (TNBC) patients accurately is direly needed for clinical decision making. pCR is also regarded as a strong predictor of overall survival. In this work, we propose a deep learning system to predict pCR to NAC based on serial pathology images stained with hematoxylin and eosin and two immunohistochemical biomarkers (Ki67 and PHH3). To support human prior domain knowledge-based guidance and enhance interpretability of the deep learning system, we introduce a human knowledge-derived spatial attention mechanism to inform deep learning models of informative tissue areas of interest. For each patient, three serial breast tumor tissue sections from biopsy blocks were sectioned, stained in three different stains and integrated. The resulting comprehensive attention information from the image triplets is used to guide our prediction system for prognostic tissue regions.

RESULTS

The experimental dataset consists of 26 419 pathology image patches of 1000×1000 pixels from 73 TNBC patients treated with NAC. Image patches from randomly selected 43 patients are used as a training dataset and images patches from the rest 30 are used as a testing dataset. By the maximum voting from patch-level results, our proposed model achieves a 93% patient-level accuracy, outperforming baselines and other state-of-the-art systems, suggesting its high potential for clinical decision making.

AVAILABILITY AND IMPLEMENTATION

The codes, the documentation and example data are available on an open source at: https://github.com/jkonglab/PCR_Prediction_Serial_WSIs_biomarkers.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Artificial intelligence in cardiac magnetic resonance fingerprinting

Magnetic resonance fingerprinting (MRF) is a fast MRI-based technique that allows for multiparametric quantitative characterization of the tissues of interest in a single acquisition. In particular, it has gained attention in the field of cardiac imaging due to its ability to provide simultaneous and co-registered myocardial T₁ and T₂ mapping in a single breath-held cardiac MRF scan, in addition to other parameters. Initial results in small healthy subject groups and clinical studies have demonstrated the feasibility and potential of MRF imaging. Ongoing research is being conducted to improve the accuracy, efficiency, and robustness of cardiac MRF. However, these improvements usually increase the complexity of image reconstruction and dictionary generation and introduce the need for sequence optimization. Each of these steps increase the computational demand and processing time of MRF. The latest advances in artificial intelligence (AI), including progress in deep learning and the development of neural networks for MRI, now present an opportunity to efficiently address these issues. Artificial intelligence can be used to optimize candidate sequences and reduce the memory demand and computational time required for reconstruction and post-processing. Recently, proposed machine learning-based approaches have been shown to reduce dictionary generation and reconstruction times by several orders of magnitude. Such applications of AI should help to remove these bottlenecks and speed up cardiac MRF, improving its practical utility and allowing for its potential inclusion in clinical routine. This review aims to summarize the latest developments in artificial intelligence applied to cardiac MRF. Particularly, we focus on the application of machine learning at different steps of the MRF process, such as sequence optimization, dictionary generation and image reconstruction.

Understanding and Improving Risk Assessment After Myocardial Infarction Using Automated Left Ventricular Shape Analysis

BACKGROUND

Left ventricular ejection fraction (LVEF) and end-systolic volume (ESV) remain the main imaging biomarkers for post-acute myocardial infarction (AMI) risk stratification. However, they are limited to global systolic function and fail to capture functional and anatomical regional abnormalities, hindering their performance in risk stratification.

OBJECTIVES

This study aimed to identify novel 3-dimensional (3D) imaging end-systolic (ES) shape and contraction descriptors toward risk-related features and superior prognosis in AMI.

METHODS

A multicenter cohort of AMI survivors (n = 1,021; median age 63 years; 74.5% male) who underwent cardiac magnetic resonance (CMR) at a median of 3 days after infarction were considered for this study. The clinical endpoint was the 12-month rate of major adverse cardiac events (MACE; n = 73), consisting of all-cause death, reinfarction, and new congestive heart failure. A fully automated pipeline was developed to segment CMR images, build 3D statistical models of shape and contraction in AMI, and find the 3D patterns related to MACE occurrence.

RESULTS

The novel ES shape markers proved to be superior to ESV (median cross-validated area under the receiver-operating characteristic curve 0.681 [IQR: 0.679-0.684] vs 0.600 [IQR: 0.598-0.602]; P < 0.001); and 3D contraction to LVEF (0.716 [IQR: 0.714-0.718] vs 0.681 [IQR: 0.679-0.684]; P < 0.001) in MACE occurrence prediction. They also contributed to a significant improvement in a multivariable setting including CMR markers, cardiovascular risk factors, and basic patient characteristics (0.747 [IQR: 0.745-0.749]; P < 0.001). Based on these novel 3D descriptors, 3 impairments caused by AMI were identified: global, anterior, and basal, the latter being the most complementary signature to already known predictors.

CONCLUSIONS

The quantification of 3D differences in ES shape and contraction, enabled by a fully automated pipeline, improves post-AMI risk prediction and identifies shape and contraction patterns related to MACE occurrence.

Artificial intelligence and machine learning in precision medicine: A paradigm shift in big data analysis

The integration of artificial intelligence in precision medicine has revolutionized healthcare delivery. Precision medicine identifies the phenotype of particular patients with less-common responses to treatment. Recent studies have demonstrated that translational research exploring the convergence between artificial intelligence and precision medicine will help solve the most difficult challenges facing precision medicine. Here, we discuss different aspects of artificial intelligence in precision medicine that improve healthcare delivery. First, we discuss how artificial intelligence changes the landscape of precision medicine and the evolution of artificial intelligence in precision medicine. Second, we highlight the synergies between artificial intelligence and precision medicine and promises of artificial intelligence and precision medicine in healthcare delivery. Third, we briefly explain the promise of big data analytics and the integration of nanomaterials in precision medicine. Last, we highlight the challenges and opportunities of artificial intelligence in precision medicine.

Predicting adverse cardiac events in sarcoidosis: deep learning from automated characterization of regional myocardial remodeling

Recognizing early cardiac sarcoidosis (CS) imaging phenotypes can help identify opportunities for effective treatment before irreversible myocardial pathology occurs. We aimed to characterize regional CS myocardial remodeling features correlating with future adverse cardiac events by coupling automated image processing and data analysis on cardiac magnetic resonance (CMR) imaging datasets. A deep convolutional neural network (DCNN) was used to process a CMR database of a 10-year cohort of 117 consecutive biopsy-proven sarcoidosis patients. The maximum relevance - minimum redundancy method was used to select the best subset of all the features-24 (from manual processing) and 232 (from automated processing) left ventricular (LV) structural/functional features. Three machine learning (ML) algorithms, logistic regression (LogR), support vector machine (SVM) and multi-layer neural networks (MLP), were used to build classifiers to categorize endpoints. Over a median follow-up of 41.8 (inter-quartile range 20.4-60.5) months, 35 sarcoidosis patients experienced a total of 43 cardiac events. After manual processing, LV ejection fraction (LVEF), late gadolinium enhancement, abnormal segmental wall motion, LV mass (LVM), LVMI index (LVMI), septal wall thickness, lateral wall thickness, relative wall thickness, and wall thickness of 9 (out of 17) individual LV segments were significantly different between patients with and without endpoints. After automated processing, LVEF, end-diastolic volume, end-systolic volume, LV mass and wall thickness of 92 (out of 216) individual LV segments were significantly different between patients with and without endpoints. To achieve the best predictive performance, ML algorithms selected lateral wall thickness, abnormal segmental wall motion, septal wall thickness, and increased wall thickness of 3 individual segments after manual image processing, and selected end-diastolic volume and 7 individual segments after automated image processing. LogR, SVM and MLP based on automated image processing consistently showed better predictive accuracies than those based on manual image processing. Automated image processing with a DCNN improves data resolution and regional CS myocardial remodeling pattern recognition, suggesting that a framework coupling automated image processing with data analysis can help clinical risk stratification.

MulViMotion: Shape-Aware 3D Myocardial Motion Tracking From Multi-View Cardiac MRI

Recovering the 3D motion of the heart from cine cardiac magnetic resonance (CMR) imaging enables the assessment of regional myocardial function and is important for understanding and analyzing cardiovascular disease. However, 3D cardiac motion estimation is challenging because the acquired cine CMR images are usually 2D slices which limit the accurate estimation of through-plane motion. To address this problem, we propose a novel multi-view motion estimation network (MulViMotion), which integrates 2D cine CMR images acquired in short-axis and long-axis planes to learn a consistent 3D motion field of the heart. In the proposed method, a hybrid 2D/3D network is built to generate dense 3D motion fields by learning fused representations from multi-view images. To ensure that the motion estimation is consistent in 3D, a shape regularization module is introduced during training, where shape information from multi-view images is exploited to provide weak supervision to 3D motion estimation. We extensively evaluate the proposed method on 2D cine CMR images from 580 subjects of the UK Biobank study for 3D motion tracking of the left ventricular myocardium. Experimental results show that the proposed method quantitatively and qualitatively outperforms competing methods.

Application of deep learning methods: From molecular modelling to patient classification

We are now well into the information driven age with complex, heterogeneous, datasets in the biological sciences continuing to grow at a rapid pace. Moreover, distilling of such datasets, to find new governing principles, are underway. Leading the surge are new and exciting algorithmic developments in computer simulation and machine learning, most notably for the latter, those centred on deep learning. However, practical applications of cell centric computations within the biological sciences, even when carefully benchmarked against existing experimental datasets, remain challenging. Here we discuss the application of deep learning methodologies to support our understanding of cell functionality and as an aid to patient classification. Whilst comprehensive end-to-end deep learning approaches that utilise knowledge of the cell and its molecular components to aid human disease classification are yet to be implemented, important for opening the door to more effective molecular and cell-based therapies, we illustrate that many deep learning applications have been developed to tackle components of such an ambitious pipeline. We end our discussion on what the future may hold, especially how an integrated framework of computer simulations and deep learning, in conjunction with wet-bench experimentation, could enable to reveal the governing principles underlying cell functionalities within the tissue environments cells operate.

Artificial intelligence and machine learning in precision and genomic medicine

The advancement of precision medicine in medical care has led behind the conventional symptom-driven treatment process by allowing early risk prediction of disease through improved diagnostics and customization of more effective treatments. It is necessary to scrutinize overall patient data alongside broad factors to observe and differentiate between ill and relatively healthy people to take the most appropriate path toward precision medicine, resulting in an improved vision of biological indicators that can signal health changes. Precision and genomic medicine combined with artificial intelligence have the potential to improve patient healthcare. Patients with less common therapeutic responses or unique healthcare demands are using genomic medicine technologies. AI provides insights through advanced computation and inference, enabling the system to reason and learn while enhancing physician decision making. Many cell characteristics, including gene up-regulation, proteins binding to nucleic acids, and splicing, can be measured at high throughput and used as training objectives for predictive models. Researchers can create a new era of effective genomic medicine with the improved availability of a broad range of datasets and modern computer techniques such as machine learning. This review article has elucidated the contributions of ML algorithms in precision and genome medicine.

Artificial intelligence and machine learning in precision and genomic medicine

The advancement of precision medicine in medical care has led behind the conventional symptom-driven treatment process by allowing early risk prediction of disease through improved diagnostics and customization of more effective treatments. It is necessary to scrutinize overall patient data alongside broad factors to observe and differentiate between ill and relatively healthy people to take the most appropriate path toward precision medicine, resulting in an improved vision of biological indicators that can signal health changes. Precision and genomic medicine combined with artificial intelligence have the potential to improve patient healthcare. Patients with less common therapeutic responses or unique healthcare demands are using genomic medicine technologies. AI provides insights through advanced computation and inference, enabling the system to reason and learn while enhancing physician decision making. Many cell characteristics, including gene up-regulation, proteins binding to nucleic acids, and splicing, can be measured at high throughput and used as training objectives for predictive models. Researchers can create a new era of effective genomic medicine with the improved availability of a broad range of datasets and modern computer techniques such as machine learning. This review article has elucidated the contributions of ML algorithms in precision and genome medicine.

Multi-Domain Variational Autoencoders for Combined Modeling of MRI-Based Biventricular Anatomy and ECG-Based Cardiac Electrophysiology

Human cardiac function is characterized by a complex interplay of mechanical deformation and electrophysiological conduction. Similar to the underlying cardiac anatomy, these interconnected physiological patterns vary considerably across the human population with important implications for the effectiveness of clinical decision-making and the accuracy of computerized heart models. While many previous works have investigated this variability separately for either cardiac anatomy or physiology, this work aims to combine both aspects in a single data-driven approach and capture their intricate interdependencies in a multi-domain setting. To this end, we propose a novel multi-domain Variational Autoencoder (VAE) network to capture combined Electrocardiogram (ECG) and Magnetic Resonance Imaging (MRI)-based 3D anatomy information in a single model. Each VAE branch is specifically designed to address the particular challenges of the respective input domain, enabling efficient encoding, reconstruction, and synthesis of multi-domain cardiac signals. Our method achieves high reconstruction accuracy on a United Kingdom Biobank dataset, with Chamfer Distances between reconstructed and input anatomies below the underlying image resolution and ECG reconstructions outperforming multiple single-domain benchmarks by a considerable margin. The proposed VAE is capable of generating realistic virtual populations of arbitrary size with good alignment in clinical metrics between the synthesized and gold standard anatomies and Maximum Mean Discrepancy (MMD) scores of generated ECGs below those of comparable single-domain approaches. Furthermore, we observe the latent space of our VAE to be highly interpretable with separate components encoding different aspects of anatomical and ECG variability. Finally, we demonstrate that the combined anatomy and ECG representation improves the performance in a cardiac disease classification task by 3.9% in terms of Area Under the Receiver Operating Characteristic (AUROC) curve over the best corresponding single-domain modeling approach.

Machine learning cardiac-MRI features predict mortality in newly diagnosed pulmonary arterial hypertension

Aims

Pulmonary arterial hypertension (PAH) is a rare but serious disease associated with high mortality if left untreated. This study aims to assess the prognostic cardiac magnetic resonance (CMR) features in PAH using machine learning.

Methods and results

Seven hundred and twenty-three consecutive treatment-naive PAH patients were identified from the ASPIRE registry; 516 were included in the training, and 207 in the validation cohort. A multilinear principal component analysis (MPCA)-based machine learning approach was used to extract mortality and survival features throughout the cardiac cycle. The features were overlaid on the original imaging using thresholding and clustering of high- and low-risk of mortality prediction values. The 1-year mortality rate in the validation cohort was 10%. Univariable Cox regression analysis of the combined short-axis and four-chamber MPCA-based predictions was statistically significant (hazard ratios: 2.1, 95% CI: 1.3, 3.4, c-index = 0.70, P = 0.002). The MPCA features improved the 1-year mortality prediction of REVEAL from c-index = 0.71 to 0.76 (P ≤ 0.001). Abnormalities in the end-systolic interventricular septum and end-diastolic left ventricle indicated the highest risk of mortality.

Conclusion

The MPCA-based machine learning is an explainable time-resolved approach that allows visualization of prognostic cardiac features throughout the cardiac cycle at the population level, making this approach transparent and clinically interpretable. In addition, the added prognostic value over the REVEAL risk score and CMR volumetric measurements allows for a more accurate prediction of 1-year mortality risk in PAH.

Using the Plasma Proteome for Risk Stratifying Patients with Pulmonary Arterial Hypertension

Rationale: NT-proBNP (N-terminal pro-brain natriuretic peptide), a biomarker of cardiac origin, is used to risk stratify patients with pulmonary arterial hypertension (PAH). Its limitations include poor sensitivity to early vascular pathology. Other biomarkers of vascular or systemic origin may also be useful in the management of PAH. Objectives: Identify prognostic proteins in PAH that complement NT-proBNP and clinical risk scores. Methods: An aptamer-based assay (SomaScan version 4) targeting 4,152 proteins was used to measure plasma proteins in patients with idiopathic, heritable, or drug-induced PAH from the UK National Cohort of PAH (n = 357) and the French EFORT (Evaluation of Prognostic Factors and Therapeutic Targets in PAH) study (n = 79). Prognostic proteins were identified in discovery-replication analyses of UK samples. Proteins independent of 6-minute-walk distance and NT-proBNP entered least absolute shrinkage and selection operator modeling, and the best combination in a single score was evaluated against clinical targets in EFORT. Measurements and Main Results: Thirty-one proteins robustly informed prognosis independent of NT-proBNP and 6-minute-walk distance in the UK cohort. A weighted combination score of six proteins was validated at baseline (5-yr mortality; area under the curve [AUC], 0.73; 95% confidence interval [CI], 0.63-0.85) and follow-up in EFORT (AUC, 0.84; 95% CI, 0.75-0.94; P = 9.96 × 10^-6). The protein score risk stratified patients independent of established clinical targets and risk equations. The addition of the six-protein model score to NT-proBNP improved prediction of 5-year outcomes from AUC 0.762 (0.702-0.821) to 0.818 (0.767-0.869) by receiver operating characteristic analysis (P = 0.00426 for difference in AUC) in the UK replication and French samples combined. Conclusions: The plasma proteome informs prognosis beyond established factors in PAH and may provide a more sensitive measure of therapeutic response.

Characterization of motion patterns by a spatio-temporal saliency descriptor in cardiac cine MRI

BACKGROUND AND OBJECTIVE

Abnormalities of the heart motion reveal the presence of a disease. However, a quantitative interpretation of the motion is still a challenge due to the complex dynamics of the heart. This work proposes a quantitative characterization of regional cardiac motion patterns in cine magnetic resonance imaging (MRI) by a novel spatio-temporal saliency descriptor.

METHOD

The strategy starts by dividing the cardiac sequence into a progression of scales which are in due turn mapped to a feature space of regional orientation changes, mimicking the multi-resolution decomposition of oriented primitive changes of visual systems. These changes are estimated as the difference between a particular time and the rest of the sequence. This decomposition is then temporarily and regionally integrated for a particular orientation and then for the set of different orientations. A final spatio-temporal 4D saliency map is obtained as the summation of the previously integrated information for the available scales. The saliency dispersion of this map was computed in standard cardiac locations as a measure of the regional motion pattern and was applied to discriminate control and hypertrophic cardiomyopathy (HCM) subjects during the diastolic phase.

RESULTS

Salient motion patterns were estimated from an experimental set, which consisted of 3D sequences acquired by MRI from 108 subjects (33 control, 35 HCM, 20 dilated cardiomyopathy (DCM), and 20 myocardial infarction (MINF) from heterogeneous datasets). HCM and control subjects were classified by an SVM that learned the salient motion patterns estimated from the presented strategy, by achieving a 94% AUC. In addition, statistical differences (test t-student, p<0.05) were found among groups of disease in the septal and anterior ventricular segments at both the ED and ES, with salient motion characteristics aligned with existing knowledge on the diseases.

CONCLUSIONS

Regional wall motion abnormality in the apical, anterior, basal, and inferior segments was associated with the saliency dispersion in HCM, DCM, and MINF compared to healthy controls during the systolic and diastolic phases. This saliency analysis may be used to detect subtle changes in heart function.

Introduction of Lazy Luna an automatic software-driven multilevel comparison of ventricular function quantification in cardiovascular magnetic resonance imaging

Cardiovascular magnetic resonance imaging is the gold standard for cardiac function assessment. Quantification of clinical results (CR) requires precise segmentation. Clinicians statistically compare CRs to ensure reproducibility. Convolutional Neural Network developers compare their results via metrics. Aim: Introducing software capable of automatic multilevel comparison. A multilevel analysis covering segmentations and CRs builds on a generic software backend. Metrics and CRs are calculated with geometric accuracy. Segmentations and CRs are connected to track errors and their effects. An interactive GUI makes the software accessible to different users. The software’s multilevel comparison was tested on a use case based on cardiac function assessment. The software shows good reader agreement in CRs and segmentation metrics (Dice > 90%). Decomposing differences by cardiac position revealed excellent agreement in midventricular slices: > 90% but poorer segmentations in apical (> 71%) and basal slices (> 74%). Further decomposition by contour type locates the largest millilitre differences in the basal right cavity (> 3 ml). Visual inspection shows these differences being caused by different basal slice choices. The software illuminated reader differences on several levels. Producing spreadsheets and figures concerning metric values and CR differences was automated. A multilevel reader comparison is feasible and extendable to other cardiac structures in the future.

Current State and Future Perspectives of Artificial Intelligence for Automated Coronary Angiography Imaging Analysis in Patients with Ischemic Heart Disease

Purpose of Review

Artificial intelligence (AI) applications in (interventional) cardiology continue to emerge. This review summarizes the current state and future perspectives of AI for automated imaging analysis in invasive coronary angiography (ICA).

Recent Findings

Recently, 12 studies on AI for automated imaging analysis In ICA have been published. In these studies, machine learning (ML) models have been developed for frame selection, segmentation, lesion assessment, and functional assessment of coronary flow. These ML models have been developed on monocenter datasets (in range 31–14,509 patients) and showed moderate to good performance. However, only three ML models were externally validated.

Summary

Given the current pace of AI developments for the analysis of ICA, less-invasive, objective, and automated diagnosis of CAD can be expected in the near future. Further research on this technology in the catheterization laboratory may assist and improve treatment allocation, risk stratification, and cath lab logistics by integrating ICA analysis with other clinical characteristics.

Strategies for Sudden Cardiac Death Prevention

Sudden cardiac death (SCD) represents a major challenge in modern medicine. The prevention of SCD orbits on two levels, the general population level and individual level. Much research has been done with the aim to improve risk stratification of SCD, although no radical changes in evidence and in therapeutic strategy have been achieved. Artificial intelligence (AI), and in particular machine learning (ML) models, represent novel technologic tools that promise to improve predictive ability of fatal arrhythmic events. In this review, firstly, we analyzed the electrophysiological basis and the major clues of SCD prevention at population and individual level; secondly, we reviewed the main research where ML models were used for risk stratification in other field of cardiology, suggesting its potentiality in the field of SCD prevention.

Deep Learning-based Automated Aortic Area and Distensibility Assessment: the Multi-Ethnic Study of Atherosclerosis (MESA)

This study details application of deep learning for automatic segmentation of the ascending and descending aorta from 2D phase-contrast cine magnetic resonance imaging for automatic aortic analysis on the large MESA cohort with assessment on an external cohort of thoracic aortic aneurysm (TAA) patients. This study includes images and corresponding analysis of the ascending and descending aorta at the pulmonary artery bifurcation from the MESA study. Train, validation, and internal test sets consisted of 1123 studies (24,282 images), 374 studies (8067 images), and 375 studies (8069 images), respectively. The external test set of TAAs consisted of 37 studies (3224 images). CNN performance was evaluated utilizing a dice coefficient and concordance correlation coefficients (CCC) of geometric parameters. Dice coefficients were as high as 97.55% (CI: 97.47-97.62%) and 93.56% (CI: 84.63-96.68%) on the internal and external test of TAAs, respectively. CCC for maximum and minimum and ascending aortic area were 0.969 and 0.950, respectively, on the internal test set and 0.997 and 0.995, respectively, for the external test. The absolute differences between manual and deep learning segmentations for ascending and descending aortic distensibility were 0.0194 × 10^-4 ± 9.67 × 10^-4 and 0.002 ± 0.001 mmHg^-1, respectively, on the internal test set and 0.44 × 10^-4 ± 20.4 × 10^-4 and 0.002 ± 0.001 mmHg^-1, respectively, on the external test set. We successfully developed a U-Net-based aortic segmentation and analysis algorithm in both MESA and in external cases of TAA.

Artificial Intelligence in Cardiac MRI: Is Clinical Adoption Forthcoming?

Artificial intelligence (AI) refers to the area of knowledge that develops computerised models to perform tasks that typically require human intelligence. These algorithms are programmed to learn and identify patterns from "training data," that can be subsequently applied to new datasets, without being explicitly programmed to do so. AI is revolutionising the field of medical imaging and in particular of Cardiovascular Magnetic Resonance (CMR) by providing deep learning solutions for image acquisition, reconstruction and analysis, ultimately supporting the clinical decision making. Numerous methods have been developed over recent years to enhance and expedite CMR data acquisition, image reconstruction, post-processing and analysis; along with the development of promising AI-based biomarkers for a wide spectrum of cardiac conditions. The exponential rise in the availability and complexity of CMR data has fostered the development of different AI models. Integration in clinical routine in a meaningful way remains a challenge. Currently, innovations in this field are still mostly presented in proof-of-concept studies with emphasis on the engineering solutions; often recruiting small patient cohorts or relying on standardised databases such as Multi-ethnic Study on atherosclerosis (MESA), UK Biobank and others. The wider incorporation of clinically valid endpoints such as symptoms, survival, need and response to treatment remains to be seen. This review briefly summarises the current principles of AI employed in CMR and explores the relevant prospective observational studies in cardiology patient cohorts. It provides an overview of clinical studies employing undersampled reconstruction techniques to speed up the scan encompassing cine imaging, whole-heart imaging, multi-parametric mapping and magnetic resonance fingerprinting along with the clinical utility of AI applications in image post-processing, and analysis. Specific focus is given to studies that have incorporated CMR-derived prediction models for prognostication in cardiac disease. It also discusses current limitations and proposes potential developments to enable multi-disciplinary collaboration for improved evidence-based medicine. AI is an extremely promising field and the timely integration of clinician's input in the ingenious technical investigator's paradigm holds promise for a bright future in the medical field.

The Emergence of Artificial Intelligence in Cardiology: Current and Future Applications

Artificial intelligence technology is emerging as a promising entity in cardiovascular medicine, potentially improving diagnosis and patient care. In this article, we review the literature on artificial intelligence and its utility in cardiology. We provide a detailed description of concepts of artificial intelligence tools like machine learning, deep learning, and cognitive computing. This review discusses the current evidence, application, prospects, and limitations of artificial intelligence in cardiology.

Deep Learning in Prediction of Late Major Bleeding After Transcatheter Aortic Valve Replacement

Purpose

Patients and Methods

Results

Conclusion