Prediction of Coronary Artery Calcium Using Deep Learning of Echocardiograms

Artificial Intelligence in Coronary Artery Calcium Scoring

Cardiovascular disease (CVD), particularly coronary heart disease (CHD), is the leading cause of death in the US, with a high economic impact. Coronary artery calcium (CAC) is a known marker for CHD and a useful tool for estimating the risk of atherosclerotic cardiovascular disease (ASCVD). Although CACS is recommended for informing the decision to initiate statin therapy, the current standard requires a dedicated CT protocol, which is time-intensive and contributes to radiation exposure. Non-dedicated CT protocols can be taken advantage of to visualize calcium and reduce overall cost and radiation exposure; however, they mainly provide visual estimates of coronary calcium and have disadvantages such as motion artifacts. Artificial intelligence is a growing field involving software that independently performs human-level tasks, and is well suited for improving CACS efficiency and repurposing non-dedicated CT for calcium scoring. We present a review of the current studies on automated CACS across various CT protocols and discuss consideration points in clinical application and some barriers to implementation.

Vision-language foundation model for echocardiogram interpretation

The development of robust artificial intelligence models for echocardiography has been limited by the availability of annotated clinical data. Here, to address this challenge and improve the performance of cardiac imaging models, we developed EchoCLIP, a vision-language foundation model for echocardiography, that learns the relationship between cardiac ultrasound images and the interpretations of expert cardiologists across a wide range of patients and indications for imaging. After training on 1,032,975 cardiac ultrasound videos and corresponding expert text, EchoCLIP performs well on a diverse range of benchmarks for cardiac image interpretation, despite not having been explicitly trained for individual interpretation tasks. EchoCLIP can assess cardiac function (mean absolute error of 7.1% when predicting left ventricular ejection fraction in an external validation dataset) and identify implanted intracardiac devices (area under the curve (AUC) of 0.84, 0.92 and 0.97 for pacemakers, percutaneous mitral valve repair and artificial aortic valves, respectively). We also developed a long-context variant (EchoCLIP-R) using a custom tokenizer based on common echocardiography concepts. EchoCLIP-R accurately identified unique patients across multiple videos (AUC of 0.86), identified clinical transitions such as heart transplants (AUC of 0.79) and cardiac surgery (AUC 0.77) and enabled robust image-to-text search (mean cross-modal retrieval rank in the top 1% of candidate text reports). These capabilities represent a substantial step toward understanding and applying foundation models in cardiovascular imaging for preliminary interpretation of echocardiographic findings.

Deep learning evaluation of echocardiograms to identify occult atrial fibrillation

Atrial fibrillation (AF) often escapes detection, given its frequent paroxysmal and asymptomatic presentation. Deep learning of transthoracic echocardiograms (TTEs), which have structural information, could help identify occult AF. We created a two-stage deep learning algorithm using a video-based convolutional neural network model that (1) distinguished whether TTEs were in sinus rhythm or AF and then (2) predicted which of the TTEs in sinus rhythm were in patients who had experienced AF within 90 days. Our model, trained on 111,319 TTE videos, distinguished TTEs in AF from those in sinus rhythm with high accuracy in a held-out test cohort (AUC 0.96 (0.95-0.96), AUPRC 0.91 (0.90-0.92)). Among TTEs in sinus rhythm, the model predicted the presence of concurrent paroxysmal AF (AUC 0.74 (0.71-0.77), AUPRC 0.19 (0.16-0.23)). Model discrimination remained similar in an external cohort of 10,203 TTEs (AUC of 0.69 (0.67-0.70), AUPRC 0.34 (0.31-0.36)). Performance held across patients who were women (AUC 0.76 (0.72-0.81)), older than 65 years (0.73 (0.69-0.76)), or had a CHA2DS2VASc ≥2 (0.73 (0.79-0.77)). The model performed better than using clinical risk factors (AUC 0.64 (0.62-0.67)), TTE measurements (0.64 (0.62-0.67)), left atrial size (0.63 (0.62-0.64)), or CHA2DS2VASc (0.61 (0.60-0.62)). An ensemble model in a cohort subset combining the TTE model with an electrocardiogram (ECGs) deep learning model performed better than using the ECG model alone (AUC 0.81 vs. 0.79, p = 0.01). Deep learning using TTEs can predict patients with active or occult AF and could be used for opportunistic AF screening that could lead to earlier treatment.

Deep learning from latent spatiotemporal information of the heart: Identifying advanced bioimaging markers from echocardiograms

A key strength of echocardiography lies in its integration of comprehensive spatiotemporal cardiac imaging data in real-time, to aid frontline or bedside patient risk stratification and management. Nonetheless, its acquisition, processing, and interpretation are known to all be subject to heterogeneity from its reliance on manual and subjective human tracings, which challenges workflow and protocol standardization and final interpretation accuracy. In the era of advanced computational power, utilization of machine learning algorithms for big data analytics in echocardiography promises reduction in cost, cognitive errors, and intra- and inter-observer variability. Novel spatiotemporal deep learning (DL) models allow the integration of temporal arm information based on unlabeled pixel echocardiographic data for convolution of an adaptive semantic spatiotemporal calibration to construct personalized 4D heart meshes, assess global and regional cardiac function, detect early valve pathology, and differentiate uncommon cardiovascular disorders. Meanwhile, data visualization on spatiotemporal DL prediction models helps extract latent temporal imaging features to develop advanced imaging biomarkers in early disease stages and advance our understanding of pathophysiology to support the development of personalized prevention or treatment strategies. Since portable echocardiograms have been increasingly used as point-of-care imaging tools to aid rural care delivery, the application of these new spatiotemporal DL techniques show the potentials in streamlining echocardiographic acquisition, processing, and data analysis to improve workflow standardization and efficiencies, and provide risk stratification and decision supporting tools in real-time, to prompt the building of new imaging diagnostic networks to enhance rural healthcare engagement.

Machine learning-based coronary artery calcium score predicted from clinical variables as a prognostic indicator in patients referred for invasive coronary angiography

OBJECTIVES

Utilising readily available clinical variables, we aimed to develop and validate a novel machine learning (ML) model to predict severe coronary calcification, and further assessed its prognostic significance.

METHODS

This retrospective study enrolled patients who underwent coronary CT angiography and subsequent invasive coronary angiography. Multiple ML algorithms were used to train the models for predicting severe coronary calcification (cardiac CT-measured coronary artery calcium [CT-CAC] score ≥ 400). The ML-based CAC (ML-CAC) score derived from the ML predictive probability was stratified into quartiles for prognostic analysis. The primary endpoint was a composite of all-cause death, nonfatal myocardial infarction, or nonfatal stroke.

RESULTS

Overall, 5785 patients were divided into training (80%) and test sets (20%). For clinical practicability, we selected the nine-feature support vector machine model with good and satisfactory performance regarding both discrimination and calibration based on five repetitions of the 10-fold cross-validation in the training set (mean AUC = 0.715, Brier score = 0.202), and based on the test in the test set (AUC = 0.753, Brier score = 0.191). In the test set cohort (n = 1137), the primary endpoint was observed in 50 (4.4%) patients during a median 2.8 years' follow-up. The ML-CAC system was significantly associated with an increased risk of the primary endpoint (adjusted hazard ratio for trend 2.26, 95% CI 1.35-3.79, p = 0.002). There was no significant difference in the prognostic value between the ML-CAC and CT-CAC systems (C-index, 0.67 vs. 0.69; p = 0.618).

CONCLUSION

ML-CAC score predicted from clinical variables can serve as a novel prognostic indicator in patients referred for invasive coronary angiography.

CLINICAL RELEVANCE STATEMENT

In patients referred for invasive coronary angiography who have not undergone preoperative CT-measured coronary artery calcium scoring, machine learning-based coronary artery calcium score assessment can serve as an alternative for predicting the prognosis.

KEY POINTS

• The coronary artery calcium (CAC) score, a solid prognostic indicator, can be predicted using non-CT methods. • We developed a machine learning (ML)-CAC model utilising nine clinical variables to predict severe coronary calcification. • The ML-CAC system offers significant prognostic value in patients referred for invasive coronary angiography.

Machine Learning and Bias in Medical Imaging: Opportunities and Challenges

Bias in health care has been well documented and results in disparate and worsened outcomes for at-risk groups. Medical imaging plays a critical role in facilitating patient diagnoses but involves multiple sources of bias including factors related to access to imaging modalities, acquisition of images, and assessment (ie, interpretation) of imaging data. Machine learning (ML) applied to diagnostic imaging has demonstrated the potential to improve the quality of imaging-based diagnosis and the precision of measuring imaging-based traits. Algorithms can leverage subtle information not visible to the human eye to detect underdiagnosed conditions or derive new disease phenotypes by linking imaging features with clinical outcomes, all while mitigating cognitive bias in interpretation. Importantly, however, the application of ML to diagnostic imaging has the potential to either reduce or propagate bias. Understanding the potential gain as well as the potential risks requires an understanding of how and what ML models learn. Common risks of propagating bias can arise from unbalanced training, suboptimal architecture design or selection, and uneven application of models. Notwithstanding these risks, ML may yet be applied to improve gain from imaging across all 3A's (access, acquisition, and assessment) for all patients. In this review, we present a framework for understanding the balance of opportunities and challenges for minimizing bias in medical imaging, how ML may improve current approaches to imaging, and what specific design considerations should be made as part of efforts to maximize the quality of health care for all.

Development of artificial intelligence-based slow-motion echocardiography and clinical usefulness for evaluating regional wall motion abnormalities

The diagnostic accuracy of exercise stress echocardiography (ESE) for myocardial ischemia requires improvement, given that it currently depends on the physicians' experience and image quality. To address this issue, we aimed to develop artificial intelligence (AI)-based slow-motion echocardiography using inter-image interpolation. The clinical usefulness of this method was evaluated for detecting regional wall-motion abnormalities (RWMAs). In this study, an AI-based echocardiographic image-interpolation pipeline was developed using optical flow calculation and prediction for in-between images. The accuracy for detecting RWMAs and image readability among 25 patients with RWMA and 25 healthy volunteers was compared between four cardiologists using slow-motion and conventional ESE. Slow-motion echocardiography was successfully developed for arbitrary time-steps (e.g., 0.125×, and 0.5×) using 1,334 videos. The RWMA detection accuracy showed a numerical improvement, but it was not statistically significant (87.5% in slow-motion echocardiography vs. 81.0% in conventional ESE; odds ratio: 1.43 [95% CI: 0.78-2.62], p = 0.25). Interreader agreement analysis (Fleiss's Kappa) for detecting RWMAs among the four cardiologists were 0.66 (95%CI: 0.55-0.77) for slow-motion ESE and 0.53 (95%CI: 0.42-0.65) for conventional ESE. Additionally, subjective evaluations of image readability using a four-point scale showed a significant improvement for slow-motion echocardiography (2.11 ± 0.73 vs. 1.70 ± 0.78, p < 0.001).In conclusion, we successfully developed slow-motion echocardiography using in-between echocardiographic image interpolation. Although the accuracy for detecting RWMAs did not show a significant improvement with this method, we observed enhanced image readability and interreader agreement. This AI-based approach holds promise in supporting physicians' evaluations.

Opportunistic screening for coronary artery calcium deposition using chest radiographs - a multi-objective models with multi-modal data fusion

Background

To create an opportunistic screening strategy by multitask deep learning methods to stratify prediction for coronary artery calcium (CAC) and associated cardiovascular risk with frontal chest x-rays (CXR) and minimal data from electronic health records (EHR).

Methods

In this retrospective study, 2,121 patients with available computed tomography (CT) scans and corresponding CXR images were collected internally (Mayo Enterprise) with calculated CAC scores binned into 3 categories (0, 1-99, and 100+) as ground truths for model training. Results from the internal training were tested on multiple external datasets (domestic (EUH) and foreign (VGHTPE)) with significant racial and ethnic differences and classification performance was compared.

Findings

Classification performance between 0, 1-99, and 100+ CAC scores performed moderately on both the internal test and external datasets, reaching average f1-score of 0.66 for Mayo, 0.62 for EUH and 0.61 for VGHTPE. For the clinically relevant binary task of 0 vs 400+ CAC classification, the performance of our model on the internal test and external datasets reached an average AUCROC of 0.84.

Interpretation

The fusion model trained on CXR performed better (0.84 average AUROC on internal and external dataset) than existing state-of-the-art models on predicting CAC scores only on internal (0.73 AUROC), with robust performance on external datasets. Thus, our proposed model may be used as a robust, first-pass opportunistic screening method for cardiovascular risk from regular chest radiographs. For community use, trained model and the inference code can be downloaded with an academic open-source license from https://github.com/jeong-jasonji/MTL_CAC_classification .

Funding

The study was partially supported by National Institute of Health 1R01HL155410-01A1 award.

DLFFNet: A new dynamical local feature fusion network for automatic aortic valve calcification recognition using echocardiography

BACKGROUND AND OBJECTIVE

Aortic valve calcification (AVC) is a strong predictor of adverse cardiovascular events and is correlated with the degree of coronary artery stenosis. Generally, AVC is identified by echocardiography using visual "eyeballing", which results in huge differences between observers and requires a long time to learn. Therefore, accurately identifying AVC from echocardiographic images is a challenging task due to the interference of various factors.

METHOD

In this paper, we built a dynamical local feature fusion net capable of processing echocardiography to recognize AVC automatically. We proposed high-echo area which were segmented by a U-Net. Meanwhile, we fine-tuned the segmentation results by adding brightness in the mask tuning module in order to dynamically adjust the selection of local features. To better fuse multi-level and multi-scale information, we designed a pyramid-based two-branch feature fusion module in classification, which enables the network to better integrate global and local semantic representations. In addition, for the echocardiographic data collected by different devices and doctors, inconsistent aortic valve position with a small occupied area, a unified preprocessing algorithm was designed.

RESULTS

To highlight the effectiveness of the proposed approach, we compared several state-of-the-art methods on the same ultrasound dataset. The 231 patients with short-axis views of the aortic valve images were collected and labeled (masked) by experienced ultrasound doctors from The First Hospital of China Medical University. The accuracy, precision, sensitivity, specificity, and F1 score, micro-AUC, and macro-AUC of the model for the test dataset were, 82.40%, 82.50%, 82.50%, 91.23%, 82.47%, 92.39%, and 92.25%, respectively.

CONCLUSIONS

The results showed the possibility of using echocardiography to examine AVC automatically and verified by visualization methods that the Region of Interest of the model is consistent with the observed region of the experts.