Automatic stenosis recognition from coronary angiography using convolutional neural networks

A Convolutional-Transformer Model for FFR and iFR Assessment From Coronary Angiography

The quantification of stenosis severity from X-ray catheter angiography is a challenging task. Indeed, this requires to fully understand the lesion's geometry by analyzing dynamics of the contrast material, only relying on visual observation by clinicians. To support decision making for cardiac intervention, we propose a hybrid CNN-Transformer model for the assessment of angiography-based non-invasive fractional flow-reserve (FFR) and instantaneous wave-free ratio (iFR) of intermediate coronary stenosis. Our approach predicts whether a coronary artery stenosis is hemodynamically significant and provides direct FFR and iFR estimates. This is achieved through a combination of regression and classification branches that forces the model to focus on the cut-off region of FFR (around 0.8 FFR value), which is highly critical for decision-making. We also propose a spatio-temporal factorization mechanisms that redesigns the transformer's self-attention mechanism to capture both local spatial and temporal interactions between vessel geometry, blood flow dynamics, and lesion morphology. The proposed method achieves state-of-the-art performance on a dataset of 778 exams from 389 patients. Unlike existing methods, our approach employs a single angiography view and does not require knowledge of the key frame; supervision at training time is provided by a classification loss (based on a threshold of the FFR/iFR values) and a regression loss for direct estimation. Finally, the analysis of model interpretability and calibration shows that, in spite of the complexity of angiographic imaging data, our method can robustly identify the location of the stenosis and correlate prediction uncertainty to the provided output scores.

Revolutionising Acute Cardiac Care With Artificial Intelligence: Opportunities and Challenges

This article reviews the application of artificial intelligence (AI) in acute cardiac care, highlighting its potential to transform patient outcomes in the face of the global burden of cardiovascular diseases. It explores how AI algorithms can rapidly and accurately process data for the prediction and diagnosis of acute cardiac conditions. The review examines AI's impact on patient health across various diagnostic tools such as echocardiography, electrocardiography, coronary angiography, cardiac computed tomography, and magnetic resonance imaging, discusses the regulatory landscape for AI in health care, and categorises AI algorithms by their risk levels. Furthermore, it addresses the challenges of data quality, generalisability, bias, transparency, and regulatory considerations, underscoring the necessity for inclusive data and robust validation processes. The review concludes with future perspectives on integrating AI into clinical workflows and the ongoing need for research, regulation, and innovation to harness AI's full potential in improving acute cardiac care.

Artificial Intelligence for Cardiovascular Care-Part 1: Advances: JACC Review Topic of the Week

Recent artificial intelligence (AI) advancements in cardiovascular care offer potential enhancements in diagnosis, treatment, and outcomes. Innovations to date focus on automating measurements, enhancing image quality, and detecting diseases using novel methods. Applications span wearables, electrocardiograms, echocardiography, angiography, genetics, and more. AI models detect diseases from electrocardiograms at accuracy not previously achieved by technology or human experts, including reduced ejection fraction, valvular heart disease, and other cardiomyopathies. However, AI's unique characteristics necessitate rigorous validation by addressing training methods, real-world efficacy, equity concerns, and long-term reliability. Despite an exponentially growing number of studies in cardiovascular AI, trials showing improvement in outcomes remain lacking. A number are currently underway. Embracing this rapidly evolving technology while setting a high evaluation benchmark will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.

Evaluation of stenoses using AI video models applied to coronary angiography

The coronary angiogram is the gold standard for evaluating the severity of coronary artery disease stenoses. Presently, the assessment is conducted visually by cardiologists, a method that lacks standardization. This study introduces DeepCoro, a ground-breaking AI-driven pipeline that integrates advanced vessel tracking and a video-based Swin3D model that was trained and validated on a dataset comprised of 182,418 coronary angiography videos spanning 5 years. DeepCoro achieved a notable precision of 71.89% in identifying coronary artery segments and demonstrated a mean absolute error of 20.15% (95% CI: 19.88-20.40) and a classification AUROC of 0.8294 (95% CI: 0.8215-0.8373) in stenosis percentage prediction compared to traditional cardiologist assessments. When compared to two expert interventional cardiologists, DeepCoro achieved lower variability than the clinical reports (19.09%; 95% CI: 18.55-19.58 vs 21.00%; 95% CI: 20.20-21.76, respectively). In addition, DeepCoro can be fine-tuned to a different modality type. When fine-tuned on quantitative coronary angiography assessments, DeepCoro attained an even lower mean absolute error of 7.75% (95% CI: 7.37-8.07), underscoring the reduced variability inherent to this method. This study establishes DeepCoro as an innovative video-based, adaptable tool in coronary artery disease analysis, significantly enhancing the precision and reliability of stenosis assessment.

Optimizing ensemble U-Net architectures for robust coronary vessel segmentation in angiographic images

Automated coronary angiography assessment requires precise vessel segmentation, a task complicated by uneven contrast filling and background noise. Our research introduces an ensemble U-Net model, SE-RegUNet, designed to accurately segment coronary vessels using 100 labeled angiographies from angiographic images. SE-RegUNet incorporates RegNet encoders and squeeze-and-excitation blocks to enhance feature extraction. A dual-phase image preprocessing strategy further improves the model's performance, employing unsharp masking and contrast-limited adaptive histogram equalization. Following fivefold cross-validation and Ranger21 optimization, the SE-RegUNet 4GF model emerged as the most effective, evidenced by performance metrics such as a Dice score of 0.72 and an accuracy of 0.97. Its potential for real-world application is highlighted by its ability to process images at 41.6 frames per second. External validation on the DCA1 dataset demonstrated the model's consistent robustness, achieving a Dice score of 0.76 and an accuracy of 0.97. The SE-RegUNet 4GF model's precision in segmenting blood vessels in coronary angiographies showcases its remarkable efficiency and accuracy. However, further development and clinical testing are necessary before it can be routinely implemented in medical practice.

A novel approach for diabetic foot diagnosis: Deep learning-based detection of lower extremity arterial stenosis

PURPOSE OF THE STUDY

Assessing the lower extremity arterial stenosis scores (LEASS) in patients with diabetic foot ulcer (DFU) is a challenging task that requires considerable time and efforts from physicians, and it may yield varying results. The presence of vascular wall calcification and other irrelevant tissue information surrounding the vessel can further compound the difficulties of this evaluation. Automatic detection of lower extremity arterial stenosis (LEAS) is expected to help doctors develop treatment plans for patients faster.

METHODS

In this paper, we first reconstructed the 3D model of blood vessels by medical digital image processing and then utilized it as the training data for deep learning (DL) in conjunction with the non-calcified part of blood vessels in the original data. We proposed an improved model of vascular stenosis small target detection based on YOLOv5. We added Convolutional Block Attention Module (CBAM) in backbone, replaced Path Aggregation Network (PANET) with Bidirectional Feature Pyramid Network (BiFPN) and replaced C3 with GhostC3 in neck to improve the recognition of three types of stenosis targets (I: <50 %, II: 51 % - 99 %, III: completely occluded). Additionally, we utilized K-Means++ instead of K-Means for better algorithm convergence performance, and enhanced the Complete-IoU (CIoU) loss function to Alpha-Scylla-IoU (ASIoU) loss for faster reasoning and convergence. Lastly, we conducted comparisons between our approach and five other prominent models.

RESULT

Our method had the best average ability to detect three types of stenosis with 85.40% mean Average Precision (mAP) and 74.60 Frames Per Second (FPS) and explored the possibility of applying DL to the detection of LEAS in diabetic foot. The code is available at github.com/wuchongxin/yolov5_LEAS.git.

CASMatching strategy for automated detection and quantification of carotid artery stenosis based on digital subtraction angiography

BACKGROUND AND OBJECTIVE

Automated detection and quantification of carotid artery stenosis is a crucial task in establishing a computer-aided diagnostic system for brain diseases. Digital subtraction angiography (DSA) is known as the "gold standard" for carotid stenosis diagnosis. It is commonly used to identify carotid artery stenosis and measure morphological indices of the stenosis. However, using deep learning to detect stenosis based on DSA images and further quantitatively predicting the morphological indices remain a challenge due the absence of prior work. In this paper, we propose a quantitative method for predicting morphological indices of carotid stenosis.

METHODS

Our method adopts a two-stage pipeline, first locating regions suitable for predicting morphological indices by object detection model, and then using a regression model to predict indices. A novel Carotid Artery Stenosis Matching (CASMatching) strategy is introduced into the object detection to model the matching relationship between a stenosis and multiple normal vessel segments. The proposed Match-ness branch predicts a Match-ness score for each normal vessel segment to indicate the degree of matching to the stenosis. A novel Direction Distance-IoU (2DIoU) loss based on the Distance-IoU loss is proposed to make the model focused more on the bounding box regression in the direction of vessel extension. After detection, the normal vessel segment with the highest Match-ness score and the stenosis are intercepted from the original image, then fed into a regression model to predict morphological indices and calculate the degree of stenosis.

RESULTS

Our method is trained and evaluated on a dataset collected from three different manufacturers' monoplane X-ray systems. The results show that the proposed components in the object detector substantially improve the detection performance of normal vascular segments. For the prediction of morphological indices, our model achieves Mean Absolute Error of 0.378, 0.221, 4.9 on reference vessel diameter (RVD), minimum lumen diameter (MLD) and stenosis degree.

CONCLUSIONS

Our method can precisely localize the carotid stenosis and the normal vessel segment suitable for predicting RVD of the stenosis, and further achieve accurate quantification, providing a novel solution for the quantification of carotid artery stenosis.

Selective ensemble methods for deep learning segmentation of major vessels in invasive coronary angiography

BACKGROUND

Invasive coronary angiography (ICA) is a primary imaging modality that visualizes the lumen area of coronary arteries for diagnosis and interventional guidance. In the current practice of quantitative coronary analysis (QCA), semi-automatic segmentation tools require labor-intensive and time-consuming manual correction, limiting their application in the catheterization room.

PURPOSE

This study aims to propose rank-based selective ensemble methods that improve the segmentation performance and reduce morphological errors that limit fully automated quantification of coronary artery using deep-learning segmentation of ICA.

METHODS

Two selective ensemble methods proposed in this work integrated the weighted ensemble approach with per-image quality estimation. The segmentation outcomes from five base models with different loss functions were ranked either by mask morphology or estimated dice similarity coefficient (DSC). The final output was determined by imposing different weights according to the ranks. The ranking criteria based on mask morphology were formulated from empirical insight to avoid frequent types of segmentation errors (MSEN), while the estimation of DSCs was performed by comparing the pseudo-ground truth generated from a meta-learner (ESEN). Five-fold cross-validation was performed with the internal dataset of 7426 coronary angiograms from 2924 patients, and prediction model was externally validated with 556 images of 226 patients.

RESULTS

The selective ensemble methods improved the segmentation performance with DSCs up to 93.07% and provided a better delineation of coronary lesion with local DSCs of up to 93.93%, outperforming all individual models. Proposed methods also minimized the chances of mask disconnection in the most narrowed regions to 2.10%. The robustness of the proposed methods was also evident in the external validation. Inference time for major vessel segmentation was approximately one-sixth of a second.

CONCLUSION

Proposed methods successfully reduced morphological errors in the predicted masks and were able to enhance the robustness of the automatic segmentation. The results suggest better applicability of real-time QCA-based diagnostic methods in routine clinical settings.

One-Stage Detection without Segmentation for Multi-Type Coronary Lesions in Angiography Images Using Deep Learning

It is rare to use the one-stage model without segmentation for the automatic detection of coronary lesions. This study sequentially enrolled 200 patients with significant stenoses and occlusions of the right coronary and categorized their angiography images into two angle views: The CRA (cranial) view of 98 patients with 2453 images and the LAO (left anterior oblique) view of 176 patients with 3338 images. Randomization was performed at the patient level to the training set and test set using a 7:3 ratio. YOLOv5 was adopted as the key model for direct detection. Four types of lesions were studied: Local Stenosis (LS), Diffuse Stenosis (DS), Bifurcation Stenosis (BS), and Chronic Total Occlusion (CTO). At the image level, the precision, recall, mAP@0.1, and mAP@0.5 predicted by the model were 0.64, 0.68, 0.66, and 0.49 in the CRA view and 0.68, 0.73, 0.70, and 0.56 in the LAO view, respectively. At the patient level, the precision, recall, and F1scores predicted by the model were 0.52, 0.91, and 0.65 in the CRA view and 0.50, 0.94, and 0.64 in the LAO view, respectively. YOLOv5 performed the best for lesions of CTO and LS at both the image level and the patient level. In conclusion, the one-stage model without segmentation as YOLOv5 is feasible to be used in automatic coronary lesion detection, with the most suitable types of lesions as LS and CTO.

Magnetically assisted soft milli-tools for occluded lumen morphology detection

Methodologies based on intravascular imaging have revolutionized the diagnosis and treatment of endovascular diseases. However, current methods are limited in detecting, i.e., visualizing and crossing, complicated occluded vessels. Therefore, we propose a miniature soft tool comprising a magnet-assisted active deformation segment (ADS) and a fluid drag-driven segment (FDS) to visualize and cross the occlusions with various morphologies. First, via soft-bodied deformation and interaction, the ADS could visualize the structure details of partial occlusions with features as small as 0.5 millimeters. Then, by leveraging the fluidic drag from the pulsatile flow, the FDS could automatically detect an entry point selectively from severe occlusions with complicated microchannels whose diameters are down to 0.2 millimeters. The functions have been validated in both biologically relevant phantoms and organs ex vivo. This soft tool could help enhance the efficacy of minimally invasive medicine for the diagnosis and treatment of occlusions in various circulatory systems.

CathAI: fully automated coronary angiography interpretation and stenosis estimation

Coronary angiography is the primary procedure for diagnosis and management decisions in coronary artery disease (CAD), but ad-hoc visual assessment of angiograms has high variability. Here we report a fully automated approach to interpret angiographic coronary artery stenosis from standard coronary angiograms. Using 13,843 angiographic studies from 11,972 adult patients at University of California, San Francisco (UCSF), between April 1, 2008 and December 31, 2019, we train neural networks to accomplish four sequential necessary tasks for automatic coronary artery stenosis localization and estimation. Algorithms are internally validated against criterion-standard labels for each task in hold-out test datasets. Algorithms are then externally validated in real-world angiograms from the University of Ottawa Heart Institute (UOHI) and also retrained using quantitative coronary angiography (QCA) data from the Montreal Heart Institute (MHI) core lab. The CathAI system achieves state-of-the-art performance across all tasks on unselected, real-world angiograms. Positive predictive value, sensitivity and F1 score are all ≥90% to identify projection angle and ≥93% for left/right coronary artery angiogram detection. To predict obstructive CAD stenosis (≥70%), CathAI exhibits an AUC of 0.862 (95% CI: 0.843-0.880). In UOHI external validation, CathAI achieves AUC 0.869 (95% CI: 0.830-0.907) to predict obstructive CAD. In the MHI QCA dataset, CathAI achieves an AUC of 0.775 (95%. CI: 0.594-0.955) after retraining. In conclusion, multiple purpose-built neural networks can function in sequence to accomplish automated analysis of real-world angiograms, which could increase standardization and reproducibility in angiographic coronary stenosis assessment.

Deep Learning Model for Coronary Angiography

The visual inspection of coronary artery stenosis is known to be significantly affected by variation, due to the presence of other tissues, camera movements, and uneven illumination. More accurate and intelligent coronary angiography diagnostic models are necessary for improving the above problems. In this study, 2980 medical images from 949 patients are collected and a novel deep learning-based coronary angiography (DLCAG) diagnose system is proposed. Firstly, we design a module of coronary classification. Then, we introduce RetinaNet to balance positive and negative samples and improve the recognition accuracy. Additionally, DLCAG adopts instance segmentation to segment the stenosis of vessels and depict the degree of the stenosis vessels. Our DLCAG is available at http://101.132.120.184:8077/ . When doctors use our system, all they need to do is login to the system, upload the coronary angiography videos. Then, a diagnose report is automatically generated.

Developing a Deep-Learning-Based Coronary Artery Disease Detection Technique Using Computer Tomography Images

Coronary artery disease (CAD) is one of the major causes of fatalities across the globe. The recent developments in convolutional neural networks (CNN) allow researchers to detect CAD from computed tomography (CT) images. The CAD detection model assists physicians in identifying cardiac disease at earlier stages. The recent CAD detection models demand a high computational cost and a more significant number of images. Therefore, this study intends to develop a CNN-based CAD detection model. The researchers apply an image enhancement technique to improve the CT image quality. The authors employed You look only once (YOLO) V7 for extracting the features. Aquila optimization is used for optimizing the hyperparameters of the UNet++ model to predict CAD. The proposed feature extraction technique and hyperparameter tuning approach reduces the computational costs and improves the performance of the UNet++ model. Two datasets are utilized for evaluating the performance of the proposed CAD detection model. The experimental outcomes suggest that the proposed method achieves an accuracy, recall, precision, F1-score, Matthews correlation coefficient, and Kappa of 99.4, 98.5, 98.65, 98.6, 95.35, and 95 and 99.5, 98.95, 98.95, 98.95, 96.35, and 96.25 for datasets 1 and 2, respectively. In addition, the proposed model outperforms the recent techniques by obtaining the area under the receiver operating characteristic and precision-recall curve of 0.97 and 0.95, and 0.96 and 0.94 for datasets 1 and 2, respectively. Moreover, the proposed model obtained a better confidence interval and standard deviation of [98.64–98.72] and 0.0014, and [97.41–97.49] and 0.0019 for datasets 1 and 2, respectively. The study’s findings suggest that the proposed model can support physicians in identifying CAD with limited resources.

Deep learning-based end-to-end automated stenosis classification and localization on catheter coronary angiography

Background

Automatic coronary angiography (CAG) assessment may help in faster screening and diagnosis of stenosis in patients with atherosclerotic disease. We aimed to provide an end-to-end workflow that separates cases with normal or mild stenoses from those with higher stenosis severities to facilitate safety screening of a large volume of the CAG images.

Methods

A deep learning-based end-to-end workflow was employed as follows: (1) Candidate frame selection from CAG videograms with Convolutional Neural Network (CNN) + Long Short Term Memory (LSTM) network, (2) Stenosis classification with Inception-v3 using 2 or 3 categories (<25%, >25%, and/or total occlusion) with and without redundancy training, and (3) Stenosis localization with two methods of class activation map (CAM) and anchor-based feature pyramid network (FPN). Overall 13,744 frames from 230 studies were used for the stenosis classification training and fourfold cross-validation for image-, artery-, and per-patient-level. For the stenosis localization training and fourfold cross-validation, 690 images with > 25% stenosis were used.

Results

Our model achieved an accuracy of 0.85, sensitivity of 0.96, and AUC of 0.86 in per-patient level stenosis classification. Redundancy training was effective to improve classification performance. Stenosis position localization was adequate with better quantitative results in anchor-based FPN model, achieving global-sensitivity for left coronary artery (LCA) and right coronary artery (RCA) of 0.68 and 0.70.

Conclusion

We demonstrated a fully automatic end-to-end deep learning-based workflow that eliminates the vessel extraction and segmentation step in coronary artery stenosis classification and localization on CAG images. This tool may be useful to facilitate safety screening in high-volume centers and in clinical trial settings.

Advances in Diagnosis, Therapy, and Prognosis of Coronary Artery Disease Powered by Deep Learning Algorithms

Percutaneous coronary intervention has been a standard treatment strategy for patients with coronary artery disease with continuous ebullient progress in technology and techniques. The application of artificial intelligence and deep learning in particular is currently boosting the development of interventional solutions, improving the efficiency and objectivity of diagnosis and treatment. The ever-growing amount of data and computing power together with cutting-edge algorithms pave the way for the integration of deep learning into clinical practice, which has revolutionized the interventional workflow in imaging processing, interpretation, and navigation. This review discusses the development of deep learning algorithms and their corresponding evaluation metrics together with their clinical applications. Advanced deep learning algorithms create new opportunities for precise diagnosis and tailored treatment with a high degree of automation, reduced radiation, and enhanced risk stratification. Generalization, interpretability, and regulatory issues are remaining challenges that need to be addressed through joint efforts from multidisciplinary community.

Coronary artery stenosis detection via proposal-shifted spatial-temporal transformer in X-ray angiography

Accurate detection of coronary artery stenosis in X-ray angiography (XRA) images is crucial for the diagnosis and treatment of coronary artery disease. However, stenosis detection remains a challenging task due to complicated vascular structures, poor imaging quality, and fickle lesions. While devoted to accurate stenosis detection, most methods are inefficient in the exploitation of spatio-temporal information of XRA sequences, leading to a limited performance on the task. To overcome the problem, we propose a new stenosis detection framework based on a Transformer-based module to aggregate proposal-level spatio-temporal features. In the module, proposal-shifted spatio-temporal tokenization (PSSTT) scheme is devised to gather spatio-temporal region-of-interest (RoI) features for obtaining visual tokens within a local window. Then, the Transformer-based feature aggregation (TFA) network takes the tokens as the inputs to enhance the RoI features by learning the long-range spatio-temporal context for final stenosis prediction. The effectiveness of our method was validated by conducting qualitative and quantitative experiments on 233 XRA sequences of coronary artery. Our method achieves a high F1 score of 90.88%, outperforming other 15 state-of-the-art detection methods. It demonstrates that our method can perform accurate stenosis detection from XRA images due to the strong ability to aggregate spatio-temporal features.

Automatic extraction of coronary arteries using deep learning in invasive coronary angiograms

BACKGROUND

Accurate extraction of coronary arteries from invasive coronary angiography (ICA) images is essential for the diagnosis and risk stratification of coronary artery disease (CAD).

OBJECTIVE

In this study, a novel deep learning (DL) method is proposed for automatically extracting coronary arteries from ICA images.

METHODS

A convolutional neural network (CNN) was developed with full-scale skip connections and full-scale deep supervisions. The encoder architecture was based on the residual and inception modules to obtain multi-scale features from multiple convolutional layers with different window shapes. Transfer learning was utilized to improve both the initial performance and learning efficiency. A hybrid loss function was employed to further optimize the segmentation model.

RESULTS

The model was tested on a data set of 616 ICAs obtained from 210 patients, composed of 437 images for training, 49 images for validation, and 130 images for testing. The segmentation model achieved a Dice score of 0.8942, a sensitivity of 0.8735, a specificity of 0.9954, and a Hausdorff distance of 6.0794 mm; it could predict arteries for a single ICA frame in 0.2114 seconds.

CONCLUSIONS

The results showed that our model outperformed the state-of-the-art deep-learning models. Our new method has great potential for clinical use.

Tailored mechanosensitive nanogels release drugs upon exposure to different levels of stenosis

Owing to the unhealthy lifestyle and genetic susceptibility of today's population, atherosclerosis is one of the global leading causes of life-threatening cardiovascular diseases. Although a rapid intervention is required for severe blood vessel constrictions, a systemic administration of anticoagulant drugs is not the preferred method of choice as the associated risk of bleeding complications is high. In this study, we present mechanosensitive nanogels that exhibit tunable degrees of disintegration upon exposure to different levels of stenosis. Those nanogels can be further functionalized to encapsulate charged drug molecules such as heparin, and they efficiently release their cargo when passing stenotic constrictions; however, passive drug leakage in the absence of mechanical shear stress is very low. Furthermore, heparin molecules liberated from those mechanosensitive nanogels show a similar blood clot lysis efficiency as the free drug molecules, which demonstrates that drug encapsulation into those nanogels does not interfere with the functionality of the cargo. Thus, the hemocompatible and mechanoresponsive nanogels developed here represent a smart and efficient drug delivery platform that can offer safer solutions for vascular therapy.

Age prediction from coronary angiography using a deep neural network: Age as a potential label to extract prognosis-related imaging features

Coronary angiography (CAG) is still considered the reference standard for coronary artery assessment, especially in the treatment of acute coronary syndrome (ACS). Although aging causes changes in coronary arteries, the age-related imaging features on CAG and their prognostic relevance have not been fully characterized. We hypothesized that a deep neural network (DNN) model could be trained to estimate vascular age only using CAG and that this age prediction from CAG could show significant associations with clinical outcomes of ACS. A DNN was trained to estimate vascular age using ten separate frames from each of 5,923 CAG videos from 572 patients. It was then tested on 1,437 CAG videos from 144 patients. Subsequently, 298 ACS patients who underwent percutaneous coronary intervention (PCI) were analysed to assess whether predicted age by DNN was associated with clinical outcomes. Age predicted as a continuous variable showed mean absolute error of 4 years with R squared of 0.72 (r = 0.856). Among the ACS patients stratified by predicted age from CAG images before PCI, major adverse cardiovascular events (MACE) were more frequently observed in the older vascular age group than in the younger vascular age group (p = 0.017). Furthermore, after controlling for actual age, gender, peak creatine kinase, and history of heart failure, the older vascular age group independently suffered from more MACE (hazard ratio 2.14, 95% CI 1.07 to 4.29, p = 0.032). The vascular age estimated based on CAG imaging by DNN showed high predictive value. The age predicted from CAG images by DNN could have significant associations with clinical outcomes in patients with ACS.

Classification of Left and Right Coronary Arteries in Coronary Angiographies Using Deep Learning

Multi-frame X-ray images (videos) of the coronary arteries obtained using coronary angiography (CAG) provide detailed information about the anatomy and blood flow in the coronary arteries and play a pivotal role in diagnosing and treating ischemic heart disease. Deep learning has the potential to quickly and accurately quantify narrowings and blockages of the arteries from CAG videos. A CAG consists of videos acquired separately for the left coronary artery and the right coronary artery (LCA and RCA, respectively). The pathology for LCA and RCA is typically only reported for the entire CAG, and not for the individual videos. However, training of stenosis quantification models is difficult when the RCA and LCA information of the videos are unknown. Here, we present a deep learning-based approach for classifying LCA and RCA in CAG videos. Our approach enables linkage of videos with the reported pathological findings. We manually labeled 3545 and 520 videos (approximately seven videos per CAG) to enable training and testing of the models, respectively. We obtained F1 scores of 0.99 on the test set for LCA and RCA classification LCA and RCA classification on the test set. The classification performance was further investigated with extensive experiments across different model architectures (R(2+1)D, X3D, and MVIT), model input sizes, data augmentations, and the number of videos used for training. Our results showed that CAG videos could be accurately curated using deep learning, which is an essential preprocessing step for a downstream application in diagnostics of coronary artery disease.

Improving convolutional neural network learning based on a hierarchical bezier generative model for stenosis detection in X-ray images

BACKGROUND AND OBJECTIVE

Automatic detection of stenosis on X-ray Coronary Angiography (XCA) images may help diagnose early coronary artery disease. Stenosis is manifested by a buildup of plaque in the arteries, decreasing the blood flow to the heart, increasing the risk of a heart attack. Convolutional Neural Networks (CNNs) have been successfully applied to identify pathological, regular, and featured tissues on rich and diverse medical image datasets. Nevertheless, CNNs find operative and performing limitations while working with small and poorly diversified databases. Transfer learning from large natural image datasets (such as ImageNet) has become a de-facto method to improve neural networks performance in the medical image domain.

METHODS

This paper proposes a novel Hierarchical Bezier-based Generative Model (HBGM) to improve the CNNs training process to detect stenosis. Herein, artificial image patches are generated to enlarge the original database, speeding up network convergence. The artificial dataset consists of 10,000 images containing 50% stenosis and 50% non-stenosis cases. Besides, a reliable Fréchet Inception Distance (FID) is used to evaluate the generated data quantitatively. Therefore, by using the proposed framework, the network is pre-trained with the artificial datasets and subsequently fine-tuned using the real XCA training dataset. The real dataset consists of 250 XCA image patches, selecting 125 images for stenosis and the remainder for non-stenosis cases. Furthermore, a Convolutional Block Attention Module (CBAM) was included in the network architecture as a self-attention mechanism to improve the efficiency of the network.

RESULTS

The results showed that the pre-trained networks using the proposed generative model outperformed the results concerning training from scratch. Particularly, an accuracy, precision, sensitivity, and F1-score of 0.8934, 0.9031, 0.8746, 0.8880, 0.9111, respectively, were achieved. The generated artificial dataset obtains a mean FID of 84.0886, with more realistic visual XCA images.

CONCLUSIONS

Different ResNet architectures for stenosis detection have been evaluated, including attention modules into the network. Numerical results demonstrated that by using the HBGM is obtained a higher performance than training from scratch, even outperforming the ImageNet pre-trained models.

Automatic coronary artery segmentation and diagnosis of stenosis by deep learning based on computed tomographic coronary angiography

OBJECTIVES

Coronary computed tomography angiography (CCTA) has rapidly developed in the coronary artery disease (CAD) field. However, manual coronary artery tree segmentation and reconstruction are time-consuming and tedious. Deep learning algorithms have been successfully developed for medical image analysis to process extensive data. Thus, we aimed to develop a deep learning tool for automatic coronary artery reconstruction and an automated CAD diagnosis model based on a large, single-centre retrospective CCTA cohort.

METHODS

Automatic CAD diagnosis consists of two subtasks. One is a segmentation task, which aims to extract the region of interest (ROI) from original images with U-Net. The second task is an identification task, which we implemented using 3DNet. The coronary artery tree images and clinical parameters were input into 3DNet, and the CAD diagnosis result was output.

RESULTS

We built a coronary artery segmentation model based on CCTA images with the corresponding labelling. The segmentation model had a mean Dice value of 0.771 ± 0.021. Based on this model, we built an automated diagnosis model (classification model) for CAD. The average accuracy and area under the receiver operating characteristic curve (AUC) were 0.750 ± 0.056 and 0.737, respectively.

CONCLUSION

Herein, using a deep learning algorithm, we realized the rapid classification and diagnosis of CAD from CCTA images in two steps. Our deep learning model can automatically segment the coronary artery quickly and accurately and can deliver a diagnosis of ≥ 50% coronary artery stenosis. Artificial intelligence methods such as deep learning have the potential to elevate the efficiency in CCTA image analysis considerably.

KEY POINTS

• The deep learning model rapidly achieved a high Dice value (0.771 ± 0.0210) in the autosegmentation of coronary arteries using CCTA images. • Based on the segmentation model, we built a CAD autoclassifier with the 3DNet algorithm, which achieved a good diagnostic performance (AUC) of 0.737. • The deep neural network could be used in the image postprocessing of coronary computed tomography angiography to achieve a quick and accurate diagnosis of CAD.

Current State and Future Perspectives of Artificial Intelligence for Automated Coronary Angiography Imaging Analysis in Patients with Ischemic Heart Disease

PURPOSE OF REVIEW

Artificial intelligence (AI) applications in (interventional) cardiology continue to emerge. This review summarizes the current state and future perspectives of AI for automated imaging analysis in invasive coronary angiography (ICA).

RECENT FINDINGS

Recently, 12 studies on AI for automated imaging analysis In ICA have been published. In these studies, machine learning (ML) models have been developed for frame selection, segmentation, lesion assessment, and functional assessment of coronary flow. These ML models have been developed on monocenter datasets (in range 31-14,509 patients) and showed moderate to good performance. However, only three ML models were externally validated. Given the current pace of AI developments for the analysis of ICA, less-invasive, objective, and automated diagnosis of CAD can be expected in the near future. Further research on this technology in the catheterization laboratory may assist and improve treatment allocation, risk stratification, and cath lab logistics by integrating ICA analysis with other clinical characteristics.

Artificial Intelligence—A Good Assistant to Multi-Modality Imaging in Managing Acute Coronary Syndrome

Acute coronary syndrome is the leading cause of cardiac death and has a significant impact on patient prognosis. Early identification and proper management are key to ensuring better outcomes and have improved significantly with the development of various cardiovascular imaging modalities. Recently, the use of artificial intelligence as a method of enhancing the capability of cardiovascular imaging has grown. AI can inform the decision-making process, as it enables existing modalities to perform more efficiently and make more accurate diagnoses. This review demonstrates recent applications of AI in cardiovascular imaging to facilitate better patient care.

FCM-DNN: diagnosing coronary artery disease by deep accuracy fuzzy C-means clustering model

Cardiovascular disease is one of the most challenging diseases in middle-aged and older people, which causes high mortality. Coronary artery disease (CAD) is known as a common cardiovascular disease. A standard clinical tool for diagnosing CAD is angiography. The main challenges are dangerous side effects and high angiography costs. Today, the development of artificial intelligence-based methods is a valuable achievement for diagnosing disease. Hence, in this paper, artificial intelligence methods such as neural network (NN), deep neural network (DNN), and fuzzy C-means clustering combined with deep neural network (FCM-DNN) are developed for diagnosing CAD on a cardiac magnetic resonance imaging (CMRI) dataset. The original dataset is used in two different approaches. First, the labeled dataset is applied to the NN and DNN to create the NN and DNN models. Second, the labels are removed, and the unlabeled dataset is clustered via the FCM method, and then, the clustered dataset is fed to the DNN to create the FCM-DNN model. By utilizing the second clustering and modeling, the training process is improved, and consequently, the accuracy is increased. As a result, the proposed FCM-DNN model achieves the best performance with a 99.91% accuracy specifying 10 clusters, i.e., 5 clusters for healthy subjects and 5 clusters for sick subjects, through the 10-fold cross-validation technique compared to the NN and DNN models reaching the accuracies of 92.18% and 99.63%, respectively. To the best of our knowledge, no study has been conducted for CAD diagnosis on the CMRI dataset using artificial intelligence methods. The results confirm that the proposed FCM-DNN model can be helpful for scientific and research centers.

Automatic construction of coronary artery tree structure based on vessel blood flow tracking

We sought to propose an innovative vessel blood flow tracking (VBFT) method to extract coronary artery tree (CAT) and to assess the effectiveness of this VBFT versus the single-frame method. Construction of a CAT from a segmented artery is the basis of artificial intelligence-aided angiographic diagnosis. However, construction of a CAT using a single frame remains challenging, due to bifurcations and overlaps in two-dimensional angiograms. Overall, 13,222 angiograms, including 28,539 vessels, were retrospectively collected from 3275 patients and were then annotated. Coronary arteries were automatically segmented by a previously established deep neural networks (DNNs), and the skeleton lines were then extracted from segmentation images to construct CAT using the single-frame method and the VBFT method. Additionally, 1322 angiograms with 2201 vessels were used to test these two methods. Compared to the single-frame method, the VBFT method can significantly improve the accuracy of CAT as (84.3% vs. 72.3%; p < 0.001). Overlap (OV) was higher in the VBFT group than that in the Single-Frame group (91.1% vs. 87.5%; p < 0.001). The VBFT method significantly reduced the incidence of the lack of branching (7.30% vs. 13.9%, p < 0.001), insufficient length (6.70% vs. 11.0%, p < 0.001), and redundant branches (1.60% vs. 3.10%, p < 0.001). The VBFT method improved the extraction of a CAT structure, which will facilitate the development of artificial intelligence-aided angiographic diagnosis. Cardiologists can efficiently diagnose CAD using this method.

MBNM: Multi-branch network based on memory features for long-tailed medical image recognition

BACKGROUND AND OBJECTIVES

Deep learning algorithms show revolutionary potential in computer-aided diagnosis. These computer-aided diagnosis techniques often rely on large-scale, balanced standard datasets. However, there are many rare diseases in real clinical scenarios, which makes the medical datasets present a highly imbalanced long-tailed distribution, leading to the poor generalization ability of deep learning models. Currently, most algorithms to solve this problem involve more complex modules and loss functions. But for complicated tasks in the medical domain, usually suffer from issues such as increased inference time and unstable performance. Therefore, it is a great challenge to develop a computer-aided diagnosis algorithm for long-tailed medical data.

METHODS

We proposed the Multi-Branch Network based on Memory Features (MBNM) for Long-Tailed Medical Image Recognition. MBNM includes three branches, where each branch focuses on a different learning task: 1) the regular learning branch learns the generalizable feature representations; 2) the tail learning branch gains extra intra-class diversity for the tail classes through the feature memory module and the improved reverse sampler to improve the classification performance of the tail classes; 3) the fusion balance branch integrates various decision-making advantages and introduces an adaptive loss function to re-balance the classification performance of easy and difficult samples.

RESULTS

We conducted experiments on the multi-disease Ophthalmic OCT datasets with imbalance factors of 98.48 and skin image datasets Skin-7 with imbalance factors of 58.3. The Accuracy, MCR, F1-Score, Precision, and AUC of our model were significantly improved over the strong baselines in the auxiliary diagnosis scenario where the clinical medical data is extremely imbalanced. Furthermore, we demonstrated that MBNM outperforms the state-of-the-art models on the publicly available natural image datasets (CIFAR-10 and CIFAR-100).

CONCLUSIONS

The proposed algorithm can solve the problem of imbalanced data distribution with little added cost. In addition, the memory module does not act in the inference phase, thereby saving inference time. And it shows outstanding performance on medical images and natural images with a variety of imbalance factors.

Current Status and Future Perspective of Artificial Intelligence in the Management of Peptic Ulcer Bleeding: A Review of Recent Literature

With the decreasing incidence of peptic ulcer bleeding (PUB) over the past two decades, the clinician experience of managing patients with PUB has also declined, especially for young endoscopists. A patient with PUB management requires collaborative care involving the emergency department, gastroenterologist, radiologist, and surgeon, from initial assessment to hospital discharge. The application of artificial intelligence (AI) methods has remarkably improved people's lives. In particular, AI systems have shown great potential in many areas of gastroenterology to increase human performance. Colonoscopy polyp detection or diagnosis by an AI system was recently introduced for commercial use to improve endoscopist performance. Although PUB is a longstanding health problem, these newly introduced AI technologies may soon impact endoscopists' clinical practice by improving the quality of care for these patients. To update the current status of AI application in PUB, we reviewed recent relevant literature and provided future perspectives that are required to integrate such AI tools into real-world practice.

Selective Prediction LSTM for Time Series Health Datasets using Unit-wise Batch Standardization: Algorithm Development and Validation (Preprint)

Background

In any health care system, both the classification of data and the confidence level of such classifications are important. Therefore, a selective prediction model is required to classify time series health data according to confidence levels of prediction.

Objective

This study aims to develop a method using long short-term memory (LSTM) models with a reject option for time series health data classification.

Methods

An existing selective prediction method was adopted to implement an option for rejecting a classification output in LSTM models. However, a conventional selection function approach to LSTM does not achieve acceptable performance during learning stages. To tackle this problem, we proposed a unit-wise batch standardization that attempts to normalize each hidden unit in LSTM to apply the structural characteristics of LSTM models that concern the selection function.

Results

The ability of our method to approximate the target confidence level was compared by coverage violations for 2 time series of health data sets consisting of human activity and arrhythmia. For both data sets, our approach yielded lower average coverage violations (0.98% and 1.79% for each data set) than those of the conventional approach. In addition, the classification performance when using the reject option was compared with that of other normalization methods. Our method demonstrated superior performance for selective risk (12.63% and 17.82% for each data set), false-positive rates (2.09% and 5.8% for each data set), and false-negative rates (10.58% and 17.24% for each data set).

Conclusions

Our normalization approach can help make selective predictions for time series health data. We expect this technique to enhance the confidence of users in classification systems and improve collaborative efforts between humans and artificial intelligence in the medical field through the use of classification that considers confidence.