Automatic extraction and stenosis evaluation of coronary arteries in invasive coronary angiograms

Revolutionizing Acute Cardiac Care with Artificial Intelligence: Opportunities and Challenges

This manuscript reviews the application of artificial intelligence (AI) in acute cardiac care, highlighting its potential to transform patient outcomes in the face of the global burden of cardiovascular diseases. It explores how AI algorithms can rapidly and accurately process data for the prediction and diagnosis of acute cardiac conditions. The paper examines AI's impact on patient health across various diagnostic tools such as echocardiography, electrocardiography, coronary angiography, cardiac CT, and MRI and discusses the regulatory landscape for AI in healthcare, categorizes AI algorithms by their risk levels. Furthermore, it addresses the challenges of data quality, generalizability, bias, transparency, and regulatory considerations, underscoring the necessity for inclusive data and robust validation processes. The review concludes with future perspectives on integrating AI into clinical workflows and the ongoing need for research, regulation, and innovation to harness AI's full potential in improving acute cardiac care.

Artificial Intelligence for Cardiovascular Care-Part 1: Advances: JACC Review Topic of the Week

Recent artificial intelligence (AI) advancements in cardiovascular care offer potential enhancements in diagnosis, treatment, and outcomes. Innovations to date focus on automating measurements, enhancing image quality, and detecting diseases using novel methods. Applications span wearables, electrocardiograms, echocardiography, angiography, genetics, and more. AI models detect diseases from electrocardiograms at accuracy not previously achieved by technology or human experts, including reduced ejection fraction, valvular heart disease, and other cardiomyopathies. However, AI's unique characteristics necessitate rigorous validation by addressing training methods, real-world efficacy, equity concerns, and long-term reliability. Despite an exponentially growing number of studies in cardiovascular AI, trials showing improvement in outcomes remain lacking. A number are currently underway. Embracing this rapidly evolving technology while setting a high evaluation benchmark will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.

Evaluation of stenoses using AI video models applied to coronary angiography

The coronary angiogram is the gold standard for evaluating the severity of coronary artery disease stenoses. Presently, the assessment is conducted visually by cardiologists, a method that lacks standardization. This study introduces DeepCoro, a ground-breaking AI-driven pipeline that integrates advanced vessel tracking and a video-based Swin3D model that was trained and validated on a dataset comprised of 182,418 coronary angiography videos spanning 5 years. DeepCoro achieved a notable precision of 71.89% in identifying coronary artery segments and demonstrated a mean absolute error of 20.15% (95% CI: 19.88-20.40) and a classification AUROC of 0.8294 (95% CI: 0.8215-0.8373) in stenosis percentage prediction compared to traditional cardiologist assessments. When compared to two expert interventional cardiologists, DeepCoro achieved lower variability than the clinical reports (19.09%; 95% CI: 18.55-19.58 vs 21.00%; 95% CI: 20.20-21.76, respectively). In addition, DeepCoro can be fine-tuned to a different modality type. When fine-tuned on quantitative coronary angiography assessments, DeepCoro attained an even lower mean absolute error of 7.75% (95% CI: 7.37-8.07), underscoring the reduced variability inherent to this method. This study establishes DeepCoro as an innovative video-based, adaptable tool in coronary artery disease analysis, significantly enhancing the precision and reliability of stenosis assessment.

3D/2D Vessel Registration Based on Monte Carlo Tree Search and Manifold Regularization

The augmented intra-operative real-time imaging in vascular interventional surgery, which is generally performed by projecting preoperative computed tomography angiography images onto intraoperative digital subtraction angiography (DSA) images, can compensate for the deficiencies of DSA-based navigation, such as lack of depth information and excessive use of toxic contrast agents. 3D/2D vessel registration is the critical step in image augmentation. A 3D/2D registration method based on vessel graph matching is proposed in this study. For rigid registration, the matching of vessel graphs can be decomposed into continuous states, thus 3D/2D vascular registration is formulated as a search tree problem. The Monte Carlo tree search method is applied to find the optimal vessel matching associated with the highest rigid registration score. For nonrigid registration, we propose a novel vessel deformation model based on manifold regularization. This model incorporates the smoothness constraint of vessel topology into the objective function. Furthermore, we derive simplified gradient formulas that enable fast registration. The proposed technique undergoes evaluation against seven rigid and three nonrigid methods using a variety of data - simulated, algorithmically generated, and manually annotated - across three vascular anatomies: the hepatic artery, coronary artery, and aorta. Our findings show the proposed method's resistance to pose variations, noise, and deformations, outperforming existing methods in terms of registration accuracy and computational efficiency. The proposed method demonstrates average registration errors of 2.14 mm and 0.34 mm for rigid and nonrigid registration, and an average computation time of 0.51 s.

Coronary vessel segmentation in coronary angiography with a multi-scale U-shaped transformer incorporating boundary aggregation and topology preservation

Coronary vessel segmentation plays a pivotal role in automating the auxiliary diagnosis of coronary heart disease. The continuity and boundary accuracy of the segmented vessels directly affect the subsequent processing. Notably, during segmentation, vessels with severe stenosis can easily cause boundary errors and breakage, resulting in isolated islands. To address these issues, we propose a novel multi-scale U-shaped transformer with boundary aggregation and topology preservation (UT-BTNet) for coronary vessel segmentation in coronary angiography. Specifically, considering the characteristics of coronary vessels, we first develop the UT-BTNet for coronary vessels segmentation, which combines the advantages of a convolutional neural networks (CNN) and a transformer, and is able to effectively extract the local and global features of angiographic images. Secondly, we innovatively employ boundary loss and topological loss in two stages, in addition to the traditional losses. In the first stage, boundary loss is adopted, which has the effect of boundary aggregation. In the second stage, topological loss is applied to preserve the topology of the vessels, after the network converges. In the experiment, in addition to the two metrics of Dice and intersection over union (IoU), we specifically propose two metrics of boundary intersection over union (BIoU) and Betti error to evaluate boundary accuracy and the continuity of segmentation results. The results show that the Dice is 0.9291, the IoU is 0.8687, the BIoU is 0.5094, and the Betti error is 0.3400. Compared with the other state-of-the-art methods, UT-BTNet achieves better segmentation results, while ensuring the continuity and boundary accuracy of the vessels, indicating its potential clinical value.

A novel approach for diabetic foot diagnosis: Deep learning-based detection of lower extremity arterial stenosis

PURPOSE OF THE STUDY

Assessing the lower extremity arterial stenosis scores (LEASS) in patients with diabetic foot ulcer (DFU) is a challenging task that requires considerable time and efforts from physicians, and it may yield varying results. The presence of vascular wall calcification and other irrelevant tissue information surrounding the vessel can further compound the difficulties of this evaluation. Automatic detection of lower extremity arterial stenosis (LEAS) is expected to help doctors develop treatment plans for patients faster.

METHODS

In this paper, we first reconstructed the 3D model of blood vessels by medical digital image processing and then utilized it as the training data for deep learning (DL) in conjunction with the non-calcified part of blood vessels in the original data. We proposed an improved model of vascular stenosis small target detection based on YOLOv5. We added Convolutional Block Attention Module (CBAM) in backbone, replaced Path Aggregation Network (PANET) with Bidirectional Feature Pyramid Network (BiFPN) and replaced C3 with GhostC3 in neck to improve the recognition of three types of stenosis targets (I: <50 %, II: 51 % - 99 %, III: completely occluded). Additionally, we utilized K-Means++ instead of K-Means for better algorithm convergence performance, and enhanced the Complete-IoU (CIoU) loss function to Alpha-Scylla-IoU (ASIoU) loss for faster reasoning and convergence. Lastly, we conducted comparisons between our approach and five other prominent models.

RESULT

Our method had the best average ability to detect three types of stenosis with 85.40% mean Average Precision (mAP) and 74.60 Frames Per Second (FPS) and explored the possibility of applying DL to the detection of LEAS in diabetic foot. The code is available at github.com/wuchongxin/yolov5_LEAS.git.

Selective ensemble methods for deep learning segmentation of major vessels in invasive coronary angiography

BACKGROUND

Invasive coronary angiography (ICA) is a primary imaging modality that visualizes the lumen area of coronary arteries for diagnosis and interventional guidance. In the current practice of quantitative coronary analysis (QCA), semi-automatic segmentation tools require labor-intensive and time-consuming manual correction, limiting their application in the catheterization room.

PURPOSE

This study aims to propose rank-based selective ensemble methods that improve the segmentation performance and reduce morphological errors that limit fully automated quantification of coronary artery using deep-learning segmentation of ICA.

METHODS

Two selective ensemble methods proposed in this work integrated the weighted ensemble approach with per-image quality estimation. The segmentation outcomes from five base models with different loss functions were ranked either by mask morphology or estimated dice similarity coefficient (DSC). The final output was determined by imposing different weights according to the ranks. The ranking criteria based on mask morphology were formulated from empirical insight to avoid frequent types of segmentation errors (MSEN), while the estimation of DSCs was performed by comparing the pseudo-ground truth generated from a meta-learner (ESEN). Five-fold cross-validation was performed with the internal dataset of 7426 coronary angiograms from 2924 patients, and prediction model was externally validated with 556 images of 226 patients.

RESULTS

The selective ensemble methods improved the segmentation performance with DSCs up to 93.07% and provided a better delineation of coronary lesion with local DSCs of up to 93.93%, outperforming all individual models. Proposed methods also minimized the chances of mask disconnection in the most narrowed regions to 2.10%. The robustness of the proposed methods was also evident in the external validation. Inference time for major vessel segmentation was approximately one-sixth of a second.

CONCLUSION

Proposed methods successfully reduced morphological errors in the predicted masks and were able to enhance the robustness of the automatic segmentation. The results suggest better applicability of real-time QCA-based diagnostic methods in routine clinical settings.

AGMN: Association Graph-based Graph Matching Network for Coronary Artery Semantic Labeling on Invasive Coronary Angiograms

Semantic labeling of coronary arterial segments in invasive coronary angiography (ICA) is important for automated assessment and report generation of coronary artery stenosis in computer-aided coronary artery disease (CAD) diagnosis. However, separating and identifying individual coronary arterial segments is challenging because morphological similarities of different branches on the coronary arterial tree and human-to-human variabilities exist. Inspired by the training procedure of interventional cardiologists for interpreting the structure of coronary arteries, we propose an association graph-based graph matching network (AGMN) for coronary arterial semantic labeling. We first extract the vascular tree from invasive coronary angiography (ICA) and convert it into multiple individual graphs. Then, an association graph is constructed from two individual graphs where each vertex represents the relationship between two arterial segments. Thus, we convert the arterial segment labeling task into a vertex classification task; ultimately, the semantic artery labeling becomes equivalent to identifying the artery-to-artery correspondence on graphs. More specifically, the AGMN extracts the vertex features by the embedding module using the association graph, aggregates the features from adjacent vertices and edges by graph convolution network, and decodes the features to generate the semantic mappings between arteries. By learning the mapping of arterial branches between two individual graphs, the unlabeled arterial segments are classified by the labeled segments to achieve semantic labeling. A dataset containing 263 ICAs was employed to train and validate the proposed model, and a five-fold cross-validation scheme was performed. Our AGMN model achieved an average accuracy of 0.8264, an average precision of 0.8276, an average recall of 0.8264, and an average F1-score of 0.8262, which significantly outperformed existing coronary artery semantic labeling methods. In conclusion, we have developed and validated a new algorithm with high accuracy, interpretability, and robustness for coronary artery semantic labeling on ICAs.

EAGMN: Coronary artery semantic labeling using edge attention graph matching network

Coronary artery disease (CAD) is one of the primary causes leading deaths worldwide. The presence of atherosclerotic lesions in coronary arteries is the underlying pathophysiological basis of CAD, and accurate extraction of individual arterial branches using invasive coronary angiography (ICA) is crucial for stenosis detection and CAD diagnosis. However, deep-learning-based models face challenges in generating semantic segmentation for coronary arteries due to the morphological similarity among different types of arteries. To address this challenge, we propose an innovative approach called the Edge Attention Graph Matching Network (EAGMN) for coronary artery semantic labeling. Inspired by the learning process of interventional cardiologists in interpreting ICA images, our model compares arterial branches between two individual graphs generated from different ICAs. We begin with extracting individual graphs based on the vascular tree obtained from the ICA. Each node in the individual graph represents an arterial segment, and the EAGMN aims to learn the similarity between nodes from the two individual graphs. By converting the coronary artery semantic segmentation task into a graph node similarity comparison task, identifying the node-to-node correspondence would assign semantic labels for each arterial branch. More specifically, the EAGMN utilizes the association graph constructed from the two individual graphs as input. A graph attention module is employed for feature embedding and aggregation, while a decoder generates the linear assignment for node-to-node semantic mapping. Based on the learned node-to-node relationships, unlabeled coronary arterial segments are classified using the labeled coronary arterial segments, thereby achieving semantic labeling. A dataset with 263 labeled ICAs is used to train and validate the EAGMN. Experimental results indicate the EAGMN achieved a weighted accuracy of 0.8653, a weighted precision of 0.8656, a weighted recall of 0.8653 and a weighted F1-score of 0.8643. Furthermore, we employ ZORRO to provide interpretability and explainability of the graph matching for artery semantic labeling. These findings highlight the potential of the EAGMN for accurate and efficient coronary artery semantic labeling using ICAs. By leveraging the inherent characteristics of ICAs and incorporating graph matching techniques, our proposed model provides a promising solution for improving CAD diagnosis and treatment.

A framework of myocardial bridge detection with x-ray angiography sequence

BACKGROUND

Myocardial bridges are congenital anatomical abnormalities in which myocardium covers a segment of coronary arteries, leading to stenocardia, myocardial ischemia, and sudden cardiac death in severe cases. However, automatic diagnosis of myocardial bridge presents significant challenges.

METHOD

A novel framework of myocardial bridge detection with x-ray angiography sequence is proposed, which can realize automatic detection of vessel stenosis and myocardial bridge. Firstly, we employ a novel neural network model for coronary vessel segmentation, which consists of both CNNs and transformer structures to effectively extract both local and global information of the vessels. Secondly, we describe the vessel segment information, establish the vessel tree in the image, and fuse the vessel tree information between sequences. Finally, based on vessel stenosis detection, we realize automatic detection of the myocardial bridge by querying the blood vessels between the image sequence information.

RESULTS

In experiment, we evaluate the segmentation results using two metrics, Dice and ASD, and achieve scores of 0.917 and 1.39, respectively. In the stenosis detection, we achieve an average accuracy rate of 92.7% in stenosis detection among 262 stenoses. In multi-frame image processing, vessels in different frames can be well-matched, and the accuracy of myocardial bridge detection achieves 75%.

CONCLUSIONS

Our experimental results demonstrate that the algorithm can automatically detect stenosis and myocardial bridge, providing a new idea for subsequent automatic diagnosis of coronary vessels.

Automatic reorientation by deep learning to generate short-axis SPECT myocardial perfusion images

BACKGROUND

Single photon emission computed tomography (SPECT) myocardial perfusion images (MPI) can be displayed both in traditional short-axis (SA) cardiac planes and polar maps for interpretation and quantification. It is essential to reorient the reconstructed transaxial SPECT MPI into standard SA slices. This study is aimed to develop a deep-learning-based approach for automatic reorientation of MPI.

METHODS

A total of 254 patients were enrolled, including 226 stress SPECT MPIs and 247 rest SPECT MPIs. Fivefold cross-validation with 180 stress and 201 rest MPIs was used for training and internal validation; the remaining images were used for testing. The rigid transformation parameters (translation and rotation) from manual reorientation were annotated by an experienced nuclear cardiologist and used as the reference standard. A convolutional neural network (CNN) was designed to predict the transformation parameters. Then, the derived transform was applied to the grid generator and sampler in spatial transformer network (STN) to generate the reoriented image. A loss function containing mean absolute errors for translation and mean square errors for rotation was employed. A three-stage optimization strategy was adopted for model optimization: (1) optimize the translation parameters while fixing the rotation parameters; (2) optimize rotation parameters while fixing the translation parameters; (3) optimize both translation and rotation parameters together.

RESULTS

In the test set, the Spearman determination coefficients of the translation distances and rotation angles between the model prediction and the reference standard were 0.993 in X axis, 0.992 in Y axis, 0.994 in Z axis, 0.987 along X axis, 0.990 along Y axis and 0.996 along Z axis, respectively. For the 46 stress MPIs in the test set, the Spearman determination coefficients were 0.858 in percentage of profusion defect (PPD) and 0.858 in summed stress score (SSS); for the 46 rest MPIs in the test set, the Spearman determination coefficients were 0.9 in PPD and 0.9 in summed rest score (SRS).

CONCLUSIONS

Our deep learning-based LV reorientation method is able to accurately generate the SA images. Technical validations and subsequent evaluations of measured clinical parameters show that it has great promise for clinical use.

One-Stage Detection without Segmentation for Multi-Type Coronary Lesions in Angiography Images Using Deep Learning

It is rare to use the one-stage model without segmentation for the automatic detection of coronary lesions. This study sequentially enrolled 200 patients with significant stenoses and occlusions of the right coronary and categorized their angiography images into two angle views: The CRA (cranial) view of 98 patients with 2453 images and the LAO (left anterior oblique) view of 176 patients with 3338 images. Randomization was performed at the patient level to the training set and test set using a 7:3 ratio. YOLOv5 was adopted as the key model for direct detection. Four types of lesions were studied: Local Stenosis (LS), Diffuse Stenosis (DS), Bifurcation Stenosis (BS), and Chronic Total Occlusion (CTO). At the image level, the precision, recall, mAP@0.1, and mAP@0.5 predicted by the model were 0.64, 0.68, 0.66, and 0.49 in the CRA view and 0.68, 0.73, 0.70, and 0.56 in the LAO view, respectively. At the patient level, the precision, recall, and F1scores predicted by the model were 0.52, 0.91, and 0.65 in the CRA view and 0.50, 0.94, and 0.64 in the LAO view, respectively. YOLOv5 performed the best for lesions of CTO and LS at both the image level and the patient level. In conclusion, the one-stage model without segmentation as YOLOv5 is feasible to be used in automatic coronary lesion detection, with the most suitable types of lesions as LS and CTO.

TS-GCN: A novel tumor segmentation method integrating transformer and GCN

As one of the critical branches of medical image processing, the task of segmentation of breast cancer tumors is of great importance for planning surgical interventions, radiotherapy and chemotherapy. Breast cancer tumor segmentation faces several challenges, including the inherent complexity and heterogeneity of breast tissue, the presence of various imaging artifacts and noise in medical images, low contrast between the tumor region and healthy tissue, and inconsistent size of the tumor region. Furthermore, the existing segmentation methods may not fully capture the rich spatial and contextual information in small-sized regions in breast images, leading to suboptimal performance. In this paper, we propose a novel breast tumor segmentation method, called the transformer and graph convolutional neural (TS-GCN) network, for medical imaging analysis. Specifically, we designed a feature aggregation network to fuse the features extracted from the transformer, GCN and convolutional neural network (CNN) networks. The CNN extract network is designed for the image's local deep feature, and the transformer and GCN networks can better capture the spatial and context dependencies among pixels in images. By leveraging the strengths of three feature extraction networks, our method achieved superior segmentation performance on the BUSI dataset and dataset B. The TS-GCN showed the best performance on several indexes, with Acc of 0.9373, Dice of 0.9058, IoU of 0.7634, F1 score of 0.9338, and AUC of 0.9692, which outperforms other state-of-the-art methods. The research of this segmentation method provides a promising future for medical image analysis and diagnosis of other diseases.

Contrast U-Net driven by sufficient texture extraction for carotid plaque detection

Ischemic heart disease or stroke caused by the rupture or dislodgement of a carotid plaque poses a huge risk to human health. To obtain accurate information on the carotid plaque characteristics of patients and to assist clinicians in the determination and identification of atherosclerotic areas, which is one significant foundation work. Existing work in this field has not deliberately extracted texture information of carotid from the ultrasound images. However, texture information is a very important part of carotid ultrasound images. To make full use of the texture information in carotid ultrasound images, a novel network based on U-Net called Contrast U-Net is designed in this paper. First, the proposed network mainly relies on a contrast block to extract accurate texture information. Moreover, to make the network better learn the texture information of each channel, the squeeze-and-excitation block is introduced to assist in the jump connection from encoding to decoding. Experimental results from intravascular ultrasound image datasets show that the proposed network can achieve superior performance compared with other popular models in carotid plaque detection.

LcmUNet: A Lightweight Network Combining CNN and MLP for Real-Time Medical Image Segmentation

In recent years, UNet and its improved variants have become the main methods for medical image segmentation. Although these models have achieved excellent results in segmentation accuracy, their large number of network parameters and high computational complexity make it difficult to achieve medical image segmentation in real-time therapy and diagnosis rapidly. To address this problem, we introduce a lightweight medical image segmentation network (LcmUNet) based on CNN and MLP. We designed LcmUNet's structure in terms of model performance, parameters, and computational complexity. The first three layers are convolutional layers, and the last two layers are MLP layers. In the convolution part, we propose an LDA module that combines asymmetric convolution, depth-wise separable convolution, and an attention mechanism to reduce the number of network parameters while maintaining a strong feature-extraction capability. In the MLP part, we propose an LMLP module that helps enhance contextual information while focusing on local information and improves segmentation accuracy while maintaining high inference speed. This network also covers skip connections between the encoder and decoder at various levels. Our network achieves real-time segmentation results accurately in extensive experiments. With only 1.49 million model parameters and without pre-training, LcmUNet demonstrated impressive performance on different datasets. On the ISIC2018 dataset, it achieved an IoU of 85.19%, 92.07% recall, and 92.99% precision. On the BUSI dataset, it achieved an IoU of 63.99%, 79.96% recall, and 76.69% precision. Lastly, on the Kvasir-SEG dataset, LcmUNet achieved an IoU of 81.89%, 88.93% recall, and 91.79% precision.

A lightweight dual-path cascaded network for vessel segmentation in fundus image

Automatic and fast segmentation of retinal vessels in fundus images is a prerequisite in clinical ophthalmic diseases; however, the high model complexity and low segmentation accuracy still limit its application. This paper proposes a lightweight dual-path cascaded network (LDPC-Net) for automatic and fast vessel segmentation. We designed a dual-path cascaded network via two U-shaped structures. Firstly, we employed a structured discarding (SD) convolution module to alleviate the over-fitting problem in both codec parts. Secondly, we introduced the depthwise separable convolution (DSC) technique to reduce the parameter amount of the model. Thirdly, a residual atrous spatial pyramid pooling (ResASPP) model is constructed in the connection layer to aggregate multi-scale information effectively. Finally, we performed comparative experiments on three public datasets. Experimental results show that the proposed method achieved superior performance on the accuracy, connectivity, and parameter quantity, thus proving that it can be a promising lightweight assisted tool for ophthalmic diseases.

GL-FusionNet: Fusing global and local features to classify deep and superficial partial thickness burn

Burns constitute one of the most common injuries in the world, and they can be very painful for the patient. Especially in the judgment of superficial partial thickness burns and deep partial thickness burns, many inexperienced clinicians are easily confused. Therefore, in order to make burn depth classification automated as well as accurate, we have introduced the deep learning method. This methodology uses a U-Net to segment burn wounds. On this basis, a new thickness burn classification model that fuses global and local features (GL-FusionNet) is proposed. For the thickness burn classification model, we use a ResNet50 to extract local features, use a ResNet101 to extract global features, and finally implement the add method to perform feature fusion and obtain the deep partial or superficial partial thickness burn classification results. Burns images are collected clinically, and they are segmented and labeled by professional physicians. Among the segmentation methods, the U-Net used achieved a Dice score of 85.352 and IoU score of 83.916, which are the best results among all of the comparative experiments. In the classification model, different existing classification networks are mainly used, as well as a fusion strategy and feature extraction method that are adjusted to conduct experiments; the proposed fusion network model also achieved the best results. Our method yielded the following: accuracy of 93.523, recall of 93.67, precision of 93.51, and F1-score of 93.513. In addition, the proposed method can quickly complete the auxiliary diagnosis of the wound in the clinic, which can greatly improve the efficiency of the initial diagnosis of burns and the nursing care of clinical medical staff.

Progressive Perception Learning for Main Coronary Segmentation in X-Ray Angiography

Main coronary segmentation from the X-ray angiography images is important for the computer-aided diagnosis and treatment of coronary disease. However, it confronts the challenge at three different image granularities (the semantic, surrounding, and local levels). The challenge includes the semantic confusion between the main and collateral vessels, low contrast between the foreground vessel and background surroundings, and local ambiguity near the vessel boundaries. The traditional hand-crafted feature-based methods may be insufficient because they may lack the semantic relationship information and may not distinguish the main and collateral vessels. The existing deep learning-based methods seem to have issues due to the deficiency in the long-distance semantic relationship capture, the foreground and background interference adaptability, and the boundary detail information preservation. To solve the main coronary segmentation challenge, we propose the progressive perception learning (PPL) framework to inspect these three different image granularities. Specifically, the PPL contains the context, interference, and boundary perception modules. The context perception is designed to focus on the main coronary vessel based on the semantic dependence capture among different coronary segments. The interference perception is designed to purify the feature maps based on the foreground vessel enhancement and background artifact suppression. The boundary perception is designed to highlight the boundary details based on boundary feature extraction through the intersection between the foreground and background predictions. Extensive experiments on 1085 subjects show that the PPL is effective (e.g., the overall Dice is greater than 95%), and superior to thirteen state-of-the-art coronary segmentation methods.

Advances in Diagnosis, Therapy, and Prognosis of Coronary Artery Disease Powered by Deep Learning Algorithms

Percutaneous coronary intervention has been a standard treatment strategy for patients with coronary artery disease with continuous ebullient progress in technology and techniques. The application of artificial intelligence and deep learning in particular is currently boosting the development of interventional solutions, improving the efficiency and objectivity of diagnosis and treatment. The ever-growing amount of data and computing power together with cutting-edge algorithms pave the way for the integration of deep learning into clinical practice, which has revolutionized the interventional workflow in imaging processing, interpretation, and navigation. This review discusses the development of deep learning algorithms and their corresponding evaluation metrics together with their clinical applications. Advanced deep learning algorithms create new opportunities for precise diagnosis and tailored treatment with a high degree of automation, reduced radiation, and enhanced risk stratification. Generalization, interpretability, and regulatory issues are remaining challenges that need to be addressed through joint efforts from multidisciplinary community.

Coronary artery stenosis detection via proposal-shifted spatial-temporal transformer in X-ray angiography

Accurate detection of coronary artery stenosis in X-ray angiography (XRA) images is crucial for the diagnosis and treatment of coronary artery disease. However, stenosis detection remains a challenging task due to complicated vascular structures, poor imaging quality, and fickle lesions. While devoted to accurate stenosis detection, most methods are inefficient in the exploitation of spatio-temporal information of XRA sequences, leading to a limited performance on the task. To overcome the problem, we propose a new stenosis detection framework based on a Transformer-based module to aggregate proposal-level spatio-temporal features. In the module, proposal-shifted spatio-temporal tokenization (PSSTT) scheme is devised to gather spatio-temporal region-of-interest (RoI) features for obtaining visual tokens within a local window. Then, the Transformer-based feature aggregation (TFA) network takes the tokens as the inputs to enhance the RoI features by learning the long-range spatio-temporal context for final stenosis prediction. The effectiveness of our method was validated by conducting qualitative and quantitative experiments on 233 XRA sequences of coronary artery. Our method achieves a high F1 score of 90.88%, outperforming other 15 state-of-the-art detection methods. It demonstrates that our method can perform accurate stenosis detection from XRA images due to the strong ability to aggregate spatio-temporal features.

An Effective Framework for Deep-Learning-Enhanced Quantitative Microwave Imaging and Its Potential for Medical Applications

Microwave imaging is emerging as an alternative modality to conventional medical diagnostics technologies. However, its adoption is hindered by the intrinsic difficulties faced in the solution of the underlying inverse scattering problem, namely non-linearity and ill-posedness. In this paper, an innovative approach for a reliable and automated solution of the inverse scattering problem is presented, which combines a qualitative imaging technique and deep learning in a two-step framework. In the first step, the orthogonality sampling method is employed to process measurements of the scattered field into an image, which explicitly provides an estimate of the targets shapes and implicitly encodes information in their contrast values. In the second step, the images obtained in the previous step are fed into a neural network (U-Net), whose duty is retrieving the exact shape of the target and its contrast value. This task is cast as an image segmentation one, where each pixel is classified into a discrete set of permittivity values within a given range. The use of a reduced number of possible permittivities facilitates the training stage by limiting its scope. The approach was tested with synthetic data and validated with experimental data taken from the Fresnel database to allow a fair comparison with the literature. Finally, its potential for biomedical imaging is demonstrated with a numerical example related to microwave brain stroke diagnosis.

Automatic extraction of coronary arteries using deep learning in invasive coronary angiograms

BACKGROUND

Accurate extraction of coronary arteries from invasive coronary angiography (ICA) images is essential for the diagnosis and risk stratification of coronary artery disease (CAD).

OBJECTIVE

In this study, a novel deep learning (DL) method is proposed for automatically extracting coronary arteries from ICA images.

METHODS

A convolutional neural network (CNN) was developed with full-scale skip connections and full-scale deep supervisions. The encoder architecture was based on the residual and inception modules to obtain multi-scale features from multiple convolutional layers with different window shapes. Transfer learning was utilized to improve both the initial performance and learning efficiency. A hybrid loss function was employed to further optimize the segmentation model.

RESULTS

The model was tested on a data set of 616 ICAs obtained from 210 patients, composed of 437 images for training, 49 images for validation, and 130 images for testing. The segmentation model achieved a Dice score of 0.8942, a sensitivity of 0.8735, a specificity of 0.9954, and a Hausdorff distance of 6.0794 mm; it could predict arteries for a single ICA frame in 0.2114 seconds.

CONCLUSIONS

The results showed that our model outperformed the state-of-the-art deep-learning models. Our new method has great potential for clinical use.

The application of the nnU-Net-based automatic segmentation model in assisting carotid artery stenosis and carotid atherosclerotic plaque evaluation

Objective: No new U-net (nnU-Net) is a newly-developed deep learning neural network, whose advantages in medical image segmentation have been noticed recently. This study aimed to investigate the value of the nnU-Net-based model for computed tomography angiography (CTA) imaging in assisting the evaluation of carotid artery stenosis (CAS) and atherosclerotic plaque. Methods: This study retrospectively enrolled 93 CAS-suspected patients who underwent head and neck CTA examination, then randomly divided them into the training set (N = 70) and the validation set (N = 23) in a 3:1 ratio. The radiologist-marked images in the training set were used for the development of the nnU-Net model, which was subsequently tested in the validation set. Results: In the training set, the nnU-Net had already displayed a good performance for CAS diagnosis and atherosclerotic plaque segmentation. Then, its utility was further confirmed in the validation set: the Dice similarity coefficient value of the nnU-Net model in segmenting background, blood vessels, calcification plaques, and dark spots reached 0.975, 0.974 0.795, and 0.498, accordingly. Besides, the nnU-Net model displayed a good consistency with physicians in assessing CAS (Kappa = 0.893), stenosis degree (Kappa = 0.930), the number of calcification plaque (Kappa = 0.922), non-calcification (Kappa = 0.768) and mixed plaque (Kappa = 0.793), as well as the max thickness of calcification plaque (intraclass correlation coefficient = 0.972). Additionally, the evaluation time of the nnU-Net model was shortened compared with the physicians (27.3 ± 4.4 s vs. 296.8 ± 81.1 s, p < 0.001). Conclusion: The automatic segmentation model based on nnU-Net shows good accuracy, reliability, and efficiency in assisting CTA to evaluate CAS and carotid atherosclerotic plaques.

Three-dimensional Fusion of Myocardial Perfusion SPECT and Invasive Coronary Angiography Guides Coronary Revascularization

BACKGROUND

SPECT myocardial perfusion imaging (SPECT MPI) and invasive coronary angiography (ICA) provide complementary clinical information in the diagnosis of coronary artery disease (CAD). We have developed an approach for 3D fusion of perfusion data from SPECT MPI and coronary anatomy from ICA. In this study, we aimed to evaluate its clinical value when compared to the traditional side-by-side readings.

METHODS

Thirty-six CAD patients who had at least one stenosis ≥ 50% were retrospectively enrolled. Based on the presence of a perfusion defect in a territory subtended by a coronary vessel, all vessels were classified as matched, unmatched, or normal groups via both the fusion and side-by-side analysis. The treatments recommended by the fusion and side-by-side analysis were compared with those that the patients received. Major adverse cardiac events (MACE), defined as all-cause death, myocardial infarction, unstable angina requiring hospitalization, and unplanned revascularization, were assessed.

RESULTS

The overall vessel-based concordance was 78.7% between the fusion and side-by-side analysis. Compared with the side-by-side analysis, 23 coronary arteries (29 equivocal segments) of 19 patients were reclassified via fusion of data. In the matched, unmatched, and normal groups, the numbers of vessels with hemodynamically significant stenosis which caused reversible defect were 37 vs 53, 28 vs 14, and 43 vs 41 (P < .01) when comparing the side-by-side analysis with the fusion, and the revascularization ratios per vessel were 69% vs 88%, 29% vs 10%, and 2% vs 2% between them. During the five-year follow-up, 8 patients (22.2%) experienced MACE. Patients who received the same treatment as the guidance of 3D fusion results (n = 22) had superior outcomes when compared with those who did not (n = 14) (P < .01).

CONCLUSIONS

Compared with the side-by-side analysis, the 3D fusion of SPECT MPI and ICA provided incremental diagnostic and prognostic value.

Estimation of Caenorhabditis Elegans Lifespan Stages Using a Dual-Path Network Combining Biomarkers and Physiological Changes

Assessing individual aging has always been an important topic in aging research. Caenorhabditis elegans (C. elegans) has a short lifespan and is a popular model organism widely utilized in aging research. Studying the differences in C. elegans life stages is of great significance for human health and aging. In order to study the differences in C. elegans lifespan stages, the classification of lifespan stages is the first task to be performed. In the past, biomarkers and physiological changes captured with imaging were commonly used to assess aging in isogenic C. elegans individuals. However, all of the current research has focused only on physiological changes or biomarkers for the assessment of aging, which affects the accuracy of assessment. In this paper, we combine two types of features for the assessment of lifespan stages to improve assessment accuracy. To fuse the two types of features, an improved high-efficiency network (Att-EfficientNet) is proposed. In the new EfficientNet, attention mechanisms are introduced so that accuracy can be further improved. In addition, in contrast to previous research, which divided the lifespan into three stages, we divide the lifespan into six stages. We compared the classification method with other CNN-based methods as well as other classic machine learning methods. The results indicate that the classification method has a higher accuracy rate (72%) than other CNN-based methods and some machine learning methods.

Automatic identification of end-diastolic and end-systolic cardiac frames from invasive coronary angiography videos

BACKGROUND

Automatic identification of proper image frames at the end-diastolic (ED) and end-systolic (ES) frames during the review of invasive coronary angiograms (ICA) is important to assess blood flow during a cardiac cycle, reconstruct the 3D arterial anatomy from bi-planar views, and generate the complementary fusion map with myocardial images. The current identification method primarily relies on visual interpretation, making it not only time-consuming but also less reproducible.

OBJECITVE

In this paper, we propose a new method to automatically identify angiographic image frames associated with the ED and ES cardiac phases.

METHOD

A detection algorithm is first used to detect the key points (i.e. landmarks) of coronary arteries, and then an optical flow method is employed to track the trajectories of the selected key points. The ED and ES frames are identified based on all these trajectories. Our method was tested with 62 ICA videos from two separate medical centers.

RESULTS

Comparing consensus interpretations by two human expert readers, excellent agreement was achieved by the proposed algorithm: the agreement rates within a one-frame range were 92.99% and 92.73% for the automatic identification of the ED and ES image frames, respectively.

CONCLUSION

The proposed automated method showed great potential for being an integral part of automated ICA image analysis.

Current State and Future Perspectives of Artificial Intelligence for Automated Coronary Angiography Imaging Analysis in Patients with Ischemic Heart Disease

Purpose of Review

Artificial intelligence (AI) applications in (interventional) cardiology continue to emerge. This review summarizes the current state and future perspectives of AI for automated imaging analysis in invasive coronary angiography (ICA).

Recent Findings

Recently, 12 studies on AI for automated imaging analysis In ICA have been published. In these studies, machine learning (ML) models have been developed for frame selection, segmentation, lesion assessment, and functional assessment of coronary flow. These ML models have been developed on monocenter datasets (in range 31–14,509 patients) and showed moderate to good performance. However, only three ML models were externally validated.

Summary

Given the current pace of AI developments for the analysis of ICA, less-invasive, objective, and automated diagnosis of CAD can be expected in the near future. Further research on this technology in the catheterization laboratory may assist and improve treatment allocation, risk stratification, and cath lab logistics by integrating ICA analysis with other clinical characteristics.

Multiscale U-Net with Spatial Positional Attention for Retinal Vessel Segmentation

Retinal vessel segmentation is essential for the detection and diagnosis of eye diseases. However, it is difficult to accurately identify the vessel boundary due to the large variations of scale in the retinal vessels and the low contrast between the vessel and the background. Deep learning has a good effect on retinal vessel segmentation since it can capture representative and distinguishing features for retinal vessels. An improved U-Net algorithm for retinal vessel segmentation is proposed in this paper. To better identify vessel boundaries, the traditional convolutional operation CNN is replaced by a global convolutional network and boundary refinement in the coding part. To better divide the blood vessel and background, the improved position attention module and channel attention module are introduced in the jumping connection part. Multiscale input and multiscale dense feature pyramid cascade modules are used to better obtain feature information. In the decoding part, convolutional long and short memory networks and deep dilated convolution are used to extract features. In public datasets, DRIVE and CHASE_DB1, the accuracy reached 96.99% and 97.51%. The average performance of the proposed algorithm is better than that of existing algorithms.

An improved residual U-Net with morphological-based loss function for automatic liver segmentation in computed tomography

This paper proposes an improved ResU-Net framework for automatic liver CT segmentation. By employing a new loss function and data augmentation strategy, the accuracy of liver segmentation is improved, and the performance is verified on two public datasets LiTS17 and SLiver07. Firstly, to speed up the convergence of the model, the residual module is used to replace the original convolution module of U-Net. Secondly, to suppress the problem of pixel imbalance, the opposite number of Dice is proposed to replace the cross-entropy loss function, and the morphological method is introduced to weigh the pixels. Finally, to improve the generalization ability of the model, random affine transformation and random elastic deformation are employed for data augmentation. From 20 training datasets of Sliver07, 16 sets were selected as the training set, two sets were used for verification, and two sets were used for the test; meanwhile, from 131 training datasets of LiTS2017, eight sets were selected as the test set. In the experiment, four evaluation metrics, including DICE global, DICE per case, VOE, and RVD, were calculated, with the accuracies of 94.28, 94.24 ± 2.07, 10.83 ± 3.70, and -0.25 ± 2.74, respectively. Compared with U-Net and ResU-Net, the performance of the proposed method is significantly improved. The experimental results show that, although the method's complexity is high, it has a faster convergence speed and stronger generalization ability. The segmentation effect on the 2D image is significantly improved, and the scalability on 3D data is also robust. In addition, the proposed method performs well in the case of low-contrast neighboring organs, which proves the robustness of the proposed method.