1
|
Mineo R, Salanitri FP, Bellitto G, Kavasidis I, Filippo OD, Millesimo M, Ferrari GMD, Aldinucci M, Giordano D, Palazzo S, D'Ascenzo F, Spampinato C. A Convolutional-Transformer Model for FFR and iFR Assessment From Coronary Angiography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2866-2877. [PMID: 38954582 DOI: 10.1109/tmi.2024.3383283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/04/2024]
Abstract
The quantification of stenosis severity from X-ray catheter angiography is a challenging task. Indeed, this requires to fully understand the lesion's geometry by analyzing dynamics of the contrast material, only relying on visual observation by clinicians. To support decision making for cardiac intervention, we propose a hybrid CNN-Transformer model for the assessment of angiography-based non-invasive fractional flow-reserve (FFR) and instantaneous wave-free ratio (iFR) of intermediate coronary stenosis. Our approach predicts whether a coronary artery stenosis is hemodynamically significant and provides direct FFR and iFR estimates. This is achieved through a combination of regression and classification branches that forces the model to focus on the cut-off region of FFR (around 0.8 FFR value), which is highly critical for decision-making. We also propose a spatio-temporal factorization mechanisms that redesigns the transformer's self-attention mechanism to capture both local spatial and temporal interactions between vessel geometry, blood flow dynamics, and lesion morphology. The proposed method achieves state-of-the-art performance on a dataset of 778 exams from 389 patients. Unlike existing methods, our approach employs a single angiography view and does not require knowledge of the key frame; supervision at training time is provided by a classification loss (based on a threshold of the FFR/iFR values) and a regression loss for direct estimation. Finally, the analysis of model interpretability and calibration shows that, in spite of the complexity of angiographic imaging data, our method can robustly identify the location of the stenosis and correlate prediction uncertainty to the provided output scores.
Collapse
|
2
|
Zhang J, Fang J, Xu Y, Si G. How AI and Robotics Will Advance Interventional Radiology: Narrative Review and Future Perspectives. Diagnostics (Basel) 2024; 14:1393. [PMID: 39001283 PMCID: PMC11241154 DOI: 10.3390/diagnostics14131393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2024] [Revised: 06/20/2024] [Accepted: 06/26/2024] [Indexed: 07/16/2024] Open
Abstract
The rapid advancement of artificial intelligence (AI) and robotics has led to significant progress in various medical fields including interventional radiology (IR). This review focuses on the research progress and applications of AI and robotics in IR, including deep learning (DL), machine learning (ML), and convolutional neural networks (CNNs) across specialties such as oncology, neurology, and cardiology, aiming to explore potential directions in future interventional treatments. To ensure the breadth and depth of this review, we implemented a systematic literature search strategy, selecting research published within the last five years. We conducted searches in databases such as PubMed and Google Scholar to find relevant literature. Special emphasis was placed on selecting large-scale studies to ensure the comprehensiveness and reliability of the results. This review summarizes the latest research directions and developments, ultimately analyzing their corresponding potential and limitations. It furnishes essential information and insights for researchers, clinicians, and policymakers, potentially propelling advancements and innovations within the domains of AI and IR. Finally, our findings indicate that although AI and robotics technologies are not yet widely applied in clinical settings, they are evolving across multiple aspects and are expected to significantly improve the processes and efficacy of interventional treatments.
Collapse
Affiliation(s)
- Jiaming Zhang
- Department of Radiology, Clinical Medical College, Southwest Medical University, Luzhou 646699, China; (J.Z.); (J.F.)
| | - Jiayi Fang
- Department of Radiology, Clinical Medical College, Southwest Medical University, Luzhou 646699, China; (J.Z.); (J.F.)
| | - Yanneng Xu
- Department of Radiology, Affiliated Traditional Chinese Medicine Hospital, Southwest Medical University, Luzhou 646699, China;
| | - Guangyan Si
- Department of Radiology, Affiliated Traditional Chinese Medicine Hospital, Southwest Medical University, Luzhou 646699, China;
| |
Collapse
|
3
|
Doolub G, Khurshid S, Theriault-Lauzier P, Nolin Lapalme A, Tastet O, So D, Labrecque Langlais E, Cobin D, Avram R. Revolutionising Acute Cardiac Care With Artificial Intelligence: Opportunities and Challenges. Can J Cardiol 2024:S0828-282X(24)00443-4. [PMID: 38901544 DOI: 10.1016/j.cjca.2024.06.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Revised: 05/29/2024] [Accepted: 06/12/2024] [Indexed: 06/22/2024] Open
Abstract
This article reviews the application of artificial intelligence (AI) in acute cardiac care, highlighting its potential to transform patient outcomes in the face of the global burden of cardiovascular diseases. It explores how AI algorithms can rapidly and accurately process data for the prediction and diagnosis of acute cardiac conditions. The review examines AI's impact on patient health across various diagnostic tools such as echocardiography, electrocardiography, coronary angiography, cardiac computed tomography, and magnetic resonance imaging, discusses the regulatory landscape for AI in health care, and categorises AI algorithms by their risk levels. Furthermore, it addresses the challenges of data quality, generalisability, bias, transparency, and regulatory considerations, underscoring the necessity for inclusive data and robust validation processes. The review concludes with future perspectives on integrating AI into clinical workflows and the ongoing need for research, regulation, and innovation to harness AI's full potential in improving acute cardiac care.
Collapse
Affiliation(s)
- Gemina Doolub
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Québec, Canada
| | - Shaan Khurshid
- Cardiovascular Disease Initiative, Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA; Demoulas Center for Cardiac Arrhythmias, Massachusetts General Hospital, Boston, Massachusetts, USA
| | | | - Alexis Nolin Lapalme
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Québec, Canada; Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Québec, Canada; Mila-Québec AI Institute, Montréal, Québec, Canada
| | - Olivier Tastet
- Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Québec, Canada
| | - Derek So
- University of Ottawa, Heart Institute, Ottawa, Ontario, Canada
| | | | - Denis Cobin
- Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Québec, Canada
| | - Robert Avram
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Québec, Canada; Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Québec, Canada.
| |
Collapse
|
4
|
Elias P, Jain SS, Poterucha T, Randazzo M, Lopez Jimenez F, Khera R, Perez M, Ouyang D, Pirruccello J, Salerno M, Einstein AJ, Avram R, Tison GH, Nadkarni G, Natarajan V, Pierson E, Beecy A, Kumaraiah D, Haggerty C, Avari Silva JN, Maddox TM. Artificial Intelligence for Cardiovascular Care-Part 1: Advances: JACC Review Topic of the Week. J Am Coll Cardiol 2024; 83:2472-2486. [PMID: 38593946 DOI: 10.1016/j.jacc.2024.03.400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 03/14/2024] [Indexed: 04/11/2024]
Abstract
Recent artificial intelligence (AI) advancements in cardiovascular care offer potential enhancements in diagnosis, treatment, and outcomes. Innovations to date focus on automating measurements, enhancing image quality, and detecting diseases using novel methods. Applications span wearables, electrocardiograms, echocardiography, angiography, genetics, and more. AI models detect diseases from electrocardiograms at accuracy not previously achieved by technology or human experts, including reduced ejection fraction, valvular heart disease, and other cardiomyopathies. However, AI's unique characteristics necessitate rigorous validation by addressing training methods, real-world efficacy, equity concerns, and long-term reliability. Despite an exponentially growing number of studies in cardiovascular AI, trials showing improvement in outcomes remain lacking. A number are currently underway. Embracing this rapidly evolving technology while setting a high evaluation benchmark will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.
Collapse
Affiliation(s)
- Pierre Elias
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA; Department of Biomedical Informatics Columbia University Irving Medical Center, New York, New York, USA
| | - Sneha S Jain
- Division of Cardiology, Stanford University School of Medicine, Palo Alto, California, USA
| | - Timothy Poterucha
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA
| | - Michael Randazzo
- Division of Cardiology, University of Chicago Medical Center, Chicago, Illinois, USA
| | | | - Rohan Khera
- Division of Cardiology, Yale School of Medicine, New Haven, Connecticut, USA
| | - Marco Perez
- Division of Cardiology, Stanford University School of Medicine, Palo Alto, California, USA
| | - David Ouyang
- Division of Cardiology, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | - James Pirruccello
- Division of Cardiology, University of California-San Francisco, San Francisco, California, USA
| | - Michael Salerno
- Division of Cardiology, Stanford University School of Medicine, Palo Alto, California, USA
| | - Andrew J Einstein
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA
| | - Robert Avram
- Division of Cardiology, Montreal Heart Institute, Montreal, Quebec, Canada
| | - Geoffrey H Tison
- Division of Cardiology, University of California-San Francisco, San Francisco, California, USA
| | - Girish Nadkarni
- Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | | | - Emma Pierson
- Department of Computer Science, Cornell Tech, New York, New York, USA
| | - Ashley Beecy
- NewYork-Presbyterian Health System, New York, New York, USA; Division of Cardiology, Weill Cornell Medical College, New York, New York, USA
| | - Deepa Kumaraiah
- Seymour, Paul and Gloria Milstein Division of Cardiology, Columbia University Irving Medical Center, New York, New York, USA; NewYork-Presbyterian Health System, New York, New York, USA
| | - Chris Haggerty
- Department of Biomedical Informatics Columbia University Irving Medical Center, New York, New York, USA; NewYork-Presbyterian Health System, New York, New York, USA
| | - Jennifer N Avari Silva
- Division of Cardiology, Washington University School of Medicine, St Louis, Missouri, USA
| | - Thomas M Maddox
- Division of Cardiology, Washington University School of Medicine, St Louis, Missouri, USA.
| |
Collapse
|
5
|
Wu C, Xu C, Ou S, Wu X, Guo J, Qi Y, Cai S. A novel approach for diabetic foot diagnosis: Deep learning-based detection of lower extremity arterial stenosis. Diabetes Res Clin Pract 2024; 207:111032. [PMID: 38049035 DOI: 10.1016/j.diabres.2023.111032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 11/23/2023] [Accepted: 11/30/2023] [Indexed: 12/06/2023]
Abstract
PURPOSE OF THE STUDY Assessing the lower extremity arterial stenosis scores (LEASS) in patients with diabetic foot ulcer (DFU) is a challenging task that requires considerable time and efforts from physicians, and it may yield varying results. The presence of vascular wall calcification and other irrelevant tissue information surrounding the vessel can further compound the difficulties of this evaluation. Automatic detection of lower extremity arterial stenosis (LEAS) is expected to help doctors develop treatment plans for patients faster. METHODS In this paper, we first reconstructed the 3D model of blood vessels by medical digital image processing and then utilized it as the training data for deep learning (DL) in conjunction with the non-calcified part of blood vessels in the original data. We proposed an improved model of vascular stenosis small target detection based on YOLOv5. We added Convolutional Block Attention Module (CBAM) in backbone, replaced Path Aggregation Network (PANET) with Bidirectional Feature Pyramid Network (BiFPN) and replaced C3 with GhostC3 in neck to improve the recognition of three types of stenosis targets (I: <50 %, II: 51 % - 99 %, III: completely occluded). Additionally, we utilized K-Means++ instead of K-Means for better algorithm convergence performance, and enhanced the Complete-IoU (CIoU) loss function to Alpha-Scylla-IoU (ASIoU) loss for faster reasoning and convergence. Lastly, we conducted comparisons between our approach and five other prominent models. RESULT Our method had the best average ability to detect three types of stenosis with 85.40% mean Average Precision (mAP) and 74.60 Frames Per Second (FPS) and explored the possibility of applying DL to the detection of LEAS in diabetic foot. The code is available at github.com/wuchongxin/yolov5_LEAS.git.
Collapse
Affiliation(s)
- Chongxin Wu
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China
| | - Changpeng Xu
- Department of Orthopaedics, Guangdong Second Provincial General Hospital, Guangzhou 510317, China
| | - Shuanji Ou
- Department of Orthopaedics, Guangdong Second Provincial General Hospital, Guangzhou 510317, China
| | - Xiaodong Wu
- Department of Orthopaedics, Guangdong Second Provincial General Hospital, Guangzhou 510317, China
| | - Jing Guo
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China
| | - Yong Qi
- Department of Orthopaedics, Guangdong Second Provincial General Hospital, Guangzhou 510317, China; Guangdong Traditional Medical and Sports Injury Rehabilitation Research Institute, Guangdong Second Provincial General Hospital, Guangzhou 510317, China.
| | - Shuting Cai
- School of Integrated Circuits, Guangdong University of Technology, Guangzhou 510006, China.
| |
Collapse
|
6
|
Wu H, Zhao J, Li J, Zeng Y, Wu W, Zhou Z, Wu S, Xu L, Song M, Yu Q, Song Z, Chen L. One-Stage Detection without Segmentation for Multi-Type Coronary Lesions in Angiography Images Using Deep Learning. Diagnostics (Basel) 2023; 13:3011. [PMID: 37761378 PMCID: PMC10528585 DOI: 10.3390/diagnostics13183011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 09/12/2023] [Accepted: 09/18/2023] [Indexed: 09/29/2023] Open
Abstract
It is rare to use the one-stage model without segmentation for the automatic detection of coronary lesions. This study sequentially enrolled 200 patients with significant stenoses and occlusions of the right coronary and categorized their angiography images into two angle views: The CRA (cranial) view of 98 patients with 2453 images and the LAO (left anterior oblique) view of 176 patients with 3338 images. Randomization was performed at the patient level to the training set and test set using a 7:3 ratio. YOLOv5 was adopted as the key model for direct detection. Four types of lesions were studied: Local Stenosis (LS), Diffuse Stenosis (DS), Bifurcation Stenosis (BS), and Chronic Total Occlusion (CTO). At the image level, the precision, recall, mAP@0.1, and mAP@0.5 predicted by the model were 0.64, 0.68, 0.66, and 0.49 in the CRA view and 0.68, 0.73, 0.70, and 0.56 in the LAO view, respectively. At the patient level, the precision, recall, and F1scores predicted by the model were 0.52, 0.91, and 0.65 in the CRA view and 0.50, 0.94, and 0.64 in the LAO view, respectively. YOLOv5 performed the best for lesions of CTO and LS at both the image level and the patient level. In conclusion, the one-stage model without segmentation as YOLOv5 is feasible to be used in automatic coronary lesion detection, with the most suitable types of lesions as LS and CTO.
Collapse
Affiliation(s)
- Hui Wu
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Jing Zhao
- Department of Geriatrics, The Third Medical Center of Chinese PLA General Hospital, Beijing 100039, China
| | - Jiehui Li
- State Key Laboratory of Cardiovascular Disease, Department of Cardiac Surgery, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Yan Zeng
- Department of Research Center, Shanghai United Imaging Intelligence Co., Ltd., Shanghai 201807, China
| | - Weiwei Wu
- College of Biomedical Engineering, Capital Medical University, Beijing 100069, China
| | - Zhuhuang Zhou
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Shuicai Wu
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Liang Xu
- State Key Laboratory of Cardiovascular Disease, Department of Structural Heart Disease, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Min Song
- State Key Laboratory of Cardiovascular Disease, Department of Cardiac Surgery, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Qibin Yu
- State Key Laboratory of Cardiovascular Disease, Department of Cardiac Surgery, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Ziwei Song
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Lin Chen
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| |
Collapse
|
7
|
Cong C, Kato Y, Vasconcellos HDD, Ostovaneh MR, Lima JAC, Ambale-Venkatesh B. Deep learning-based end-to-end automated stenosis classification and localization on catheter coronary angiography. Front Cardiovasc Med 2023; 10:944135. [PMID: 36824452 PMCID: PMC9941145 DOI: 10.3389/fcvm.2023.944135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2022] [Accepted: 01/16/2023] [Indexed: 02/09/2023] Open
Abstract
Background Automatic coronary angiography (CAG) assessment may help in faster screening and diagnosis of stenosis in patients with atherosclerotic disease. We aimed to provide an end-to-end workflow that separates cases with normal or mild stenoses from those with higher stenosis severities to facilitate safety screening of a large volume of the CAG images. Methods A deep learning-based end-to-end workflow was employed as follows: (1) Candidate frame selection from CAG videograms with Convolutional Neural Network (CNN) + Long Short Term Memory (LSTM) network, (2) Stenosis classification with Inception-v3 using 2 or 3 categories (<25%, >25%, and/or total occlusion) with and without redundancy training, and (3) Stenosis localization with two methods of class activation map (CAM) and anchor-based feature pyramid network (FPN). Overall 13,744 frames from 230 studies were used for the stenosis classification training and fourfold cross-validation for image-, artery-, and per-patient-level. For the stenosis localization training and fourfold cross-validation, 690 images with > 25% stenosis were used. Results Our model achieved an accuracy of 0.85, sensitivity of 0.96, and AUC of 0.86 in per-patient level stenosis classification. Redundancy training was effective to improve classification performance. Stenosis position localization was adequate with better quantitative results in anchor-based FPN model, achieving global-sensitivity for left coronary artery (LCA) and right coronary artery (RCA) of 0.68 and 0.70. Conclusion We demonstrated a fully automatic end-to-end deep learning-based workflow that eliminates the vessel extraction and segmentation step in coronary artery stenosis classification and localization on CAG images. This tool may be useful to facilitate safety screening in high-volume centers and in clinical trial settings.
Collapse
Affiliation(s)
- Chao Cong
- Division of Cardiology, Johns Hopkins University, Baltimore, MD, United States
- School of Electrical and Electronic Engineering, Chongqing University of Technology, Chongqing, China
| | - Yoko Kato
- Division of Cardiology, Johns Hopkins University, Baltimore, MD, United States
| | | | | | - Joao A. C. Lima
- Division of Cardiology, Johns Hopkins University, Baltimore, MD, United States
| | | |
Collapse
|
8
|
Chu M, Wu P, Li G, Yang W, Gutiérrez-Chico JL, Tu S. Advances in Diagnosis, Therapy, and Prognosis of Coronary Artery Disease Powered by Deep Learning Algorithms. JACC. ASIA 2023; 3:1-14. [PMID: 36873752 PMCID: PMC9982227 DOI: 10.1016/j.jacasi.2022.12.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 11/29/2022] [Accepted: 12/01/2022] [Indexed: 02/17/2023]
Abstract
Percutaneous coronary intervention has been a standard treatment strategy for patients with coronary artery disease with continuous ebullient progress in technology and techniques. The application of artificial intelligence and deep learning in particular is currently boosting the development of interventional solutions, improving the efficiency and objectivity of diagnosis and treatment. The ever-growing amount of data and computing power together with cutting-edge algorithms pave the way for the integration of deep learning into clinical practice, which has revolutionized the interventional workflow in imaging processing, interpretation, and navigation. This review discusses the development of deep learning algorithms and their corresponding evaluation metrics together with their clinical applications. Advanced deep learning algorithms create new opportunities for precise diagnosis and tailored treatment with a high degree of automation, reduced radiation, and enhanced risk stratification. Generalization, interpretability, and regulatory issues are remaining challenges that need to be addressed through joint efforts from multidisciplinary community.
Collapse
Affiliation(s)
- Miao Chu
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Peng Wu
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Guanyu Li
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.,Department of Cardiology, Aarhus University Hospital, Aarhus, Denmark
| | - Wei Yang
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, China
| | | | - Shengxian Tu
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
9
|
Han T, Ai D, Li X, Fan J, Song H, Wang Y, Yang J. Coronary artery stenosis detection via proposal-shifted spatial-temporal transformer in X-ray angiography. Comput Biol Med 2023; 153:106546. [PMID: 36641935 DOI: 10.1016/j.compbiomed.2023.106546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 01/03/2023] [Accepted: 01/10/2023] [Indexed: 01/13/2023]
Abstract
Accurate detection of coronary artery stenosis in X-ray angiography (XRA) images is crucial for the diagnosis and treatment of coronary artery disease. However, stenosis detection remains a challenging task due to complicated vascular structures, poor imaging quality, and fickle lesions. While devoted to accurate stenosis detection, most methods are inefficient in the exploitation of spatio-temporal information of XRA sequences, leading to a limited performance on the task. To overcome the problem, we propose a new stenosis detection framework based on a Transformer-based module to aggregate proposal-level spatio-temporal features. In the module, proposal-shifted spatio-temporal tokenization (PSSTT) scheme is devised to gather spatio-temporal region-of-interest (RoI) features for obtaining visual tokens within a local window. Then, the Transformer-based feature aggregation (TFA) network takes the tokens as the inputs to enhance the RoI features by learning the long-range spatio-temporal context for final stenosis prediction. The effectiveness of our method was validated by conducting qualitative and quantitative experiments on 233 XRA sequences of coronary artery. Our method achieves a high F1 score of 90.88%, outperforming other 15 state-of-the-art detection methods. It demonstrates that our method can perform accurate stenosis detection from XRA images due to the strong ability to aggregate spatio-temporal features.
Collapse
Affiliation(s)
- Tao Han
- Laboratory of Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China
| | - Danni Ai
- Laboratory of Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China.
| | - Xinyu Li
- Laboratory of Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China
| | - Jingfan Fan
- Laboratory of Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China
| | - Hong Song
- School of Computer Science and Technology, Beijing Institute of Technology, Beijing, 100081, China
| | - Yining Wang
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100730, China.
| | - Jian Yang
- Laboratory of Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China.
| |
Collapse
|
10
|
Zhang X, Du H, Song G, Bao F, Zhang Y, Wu W, Liu P. X-ray coronary centerline extraction based on C-UNet and a multifactor reconnection algorithm. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 226:107114. [PMID: 36116399 DOI: 10.1016/j.cmpb.2022.107114] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Revised: 08/31/2022] [Accepted: 09/04/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Accurate extraction of the coronary artery centerline is crucial in the processes of coronary artery reconstruction, coronary artery stenosis or lesion detection, and surgical navigation. Furthermore, in clinical medicine, the complex background of angiography, low signal-to-noise ratio, and complex vascular structure make coronary artery centerline extraction challenging. In this study, a direct centerline extraction method is proposed that automatically and accurately extracts vascular centerlines from X-ray coronary angiography images based on deep learning and conventional methods. METHODS In this study, a coronary artery centerline extraction method is proposed that comprises two parts: the preliminary centerline extraction network based on U-Net with a residual network, called C-UNet, and the multifactor centerline reconnection algorithm based on the geometric characteristics of blood vessels. RESULTS The qualitative and quantitative results demonstrate the effectiveness of the presented method. In this study, three widely used evaluation indices were adopted to evaluate the performance of the method: precision, recall, and F1_Score. The experimental results show that this method can accurately extract coronary artery centerlines. CONCLUSIONS The proposed centerline extraction method accurately extracts centerlines from X-ray coronary angiography images and improves both the accuracy and continuity of centerline extraction.
Collapse
Affiliation(s)
- Xinyue Zhang
- School of Mathematics, Shandong University, Jinan, Shandong 250100, China
| | - Hongwei Du
- School of Mathematics, Shandong University, Jinan, Shandong 250100, China
| | - Gang Song
- School of Mathematics, Shandong University, Jinan, Shandong 250100, China
| | - Fangxun Bao
- School of Mathematics, Shandong University, Jinan, Shandong 250100, China.
| | - Yunfeng Zhang
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, Shandong 250014, China
| | - Wei Wu
- Department of Neurology, Qi-Lu Hospital of Shandong University, Jinan, Shandong 250012, China
| | - Peide Liu
- School of Management Science and Engineering, Shandong University of Finance and Economics, Jinan 250014, China
| |
Collapse
|
11
|
Ovalle-Magallanes E, Avina-Cervantes JG, Cruz-Aceves I, Ruiz-Pinales J. Improving convolutional neural network learning based on a hierarchical bezier generative model for stenosis detection in X-ray images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106767. [PMID: 35364481 DOI: 10.1016/j.cmpb.2022.106767] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/17/2021] [Revised: 03/09/2022] [Accepted: 03/19/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic detection of stenosis on X-ray Coronary Angiography (XCA) images may help diagnose early coronary artery disease. Stenosis is manifested by a buildup of plaque in the arteries, decreasing the blood flow to the heart, increasing the risk of a heart attack. Convolutional Neural Networks (CNNs) have been successfully applied to identify pathological, regular, and featured tissues on rich and diverse medical image datasets. Nevertheless, CNNs find operative and performing limitations while working with small and poorly diversified databases. Transfer learning from large natural image datasets (such as ImageNet) has become a de-facto method to improve neural networks performance in the medical image domain. METHODS This paper proposes a novel Hierarchical Bezier-based Generative Model (HBGM) to improve the CNNs training process to detect stenosis. Herein, artificial image patches are generated to enlarge the original database, speeding up network convergence. The artificial dataset consists of 10,000 images containing 50% stenosis and 50% non-stenosis cases. Besides, a reliable Fréchet Inception Distance (FID) is used to evaluate the generated data quantitatively. Therefore, by using the proposed framework, the network is pre-trained with the artificial datasets and subsequently fine-tuned using the real XCA training dataset. The real dataset consists of 250 XCA image patches, selecting 125 images for stenosis and the remainder for non-stenosis cases. Furthermore, a Convolutional Block Attention Module (CBAM) was included in the network architecture as a self-attention mechanism to improve the efficiency of the network. RESULTS The results showed that the pre-trained networks using the proposed generative model outperformed the results concerning training from scratch. Particularly, an accuracy, precision, sensitivity, and F1-score of 0.8934, 0.9031, 0.8746, 0.8880, 0.9111, respectively, were achieved. The generated artificial dataset obtains a mean FID of 84.0886, with more realistic visual XCA images. CONCLUSIONS Different ResNet architectures for stenosis detection have been evaluated, including attention modules into the network. Numerical results demonstrated that by using the HBGM is obtained a higher performance than training from scratch, even outperforming the ImageNet pre-trained models.
Collapse
Affiliation(s)
- Emmanuel Ovalle-Magallanes
- Telematics and Digital Signal Processing Research groups (CAs), Engineering Division, Campus Irapuato-Salamanca, University of Guanajuato, Carretera Salamanca-Valle de Santiago km 3.5 + 1.8km, Comunidad de Palo Blanco, Salamanca, 36885 Guanajuato, Mexico.
| | - Juan Gabriel Avina-Cervantes
- Telematics and Digital Signal Processing Research groups (CAs), Engineering Division, Campus Irapuato-Salamanca, University of Guanajuato, Carretera Salamanca-Valle de Santiago km 3.5 + 1.8km, Comunidad de Palo Blanco, Salamanca, 36885 Guanajuato, Mexico.
| | - Ivan Cruz-Aceves
- CONACYT, Center for Research in Mathematics (CIMAT), A.C., Jalisco S/N, Col. Valenciana, Guanajuato, 36000 Guanajuato, Mexico.
| | - Jose Ruiz-Pinales
- Telematics and Digital Signal Processing Research groups (CAs), Engineering Division, Campus Irapuato-Salamanca, University of Guanajuato, Carretera Salamanca-Valle de Santiago km 3.5 + 1.8km, Comunidad de Palo Blanco, Salamanca, 36885 Guanajuato, Mexico.
| |
Collapse
|
12
|
Molenaar MA, Selder JL, Nicolas J, Claessen BE, Mehran R, Bescós JO, Schuuring MJ, Bouma BJ, Verouden NJ, Chamuleau SAJ. Current State and Future Perspectives of Artificial Intelligence for Automated Coronary Angiography Imaging Analysis in Patients with Ischemic Heart Disease. Curr Cardiol Rep 2022; 24:365-376. [PMID: 35347566 PMCID: PMC8979928 DOI: 10.1007/s11886-022-01655-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/03/2022] [Indexed: 12/17/2022]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) applications in (interventional) cardiology continue to emerge. This review summarizes the current state and future perspectives of AI for automated imaging analysis in invasive coronary angiography (ICA). RECENT FINDINGS Recently, 12 studies on AI for automated imaging analysis In ICA have been published. In these studies, machine learning (ML) models have been developed for frame selection, segmentation, lesion assessment, and functional assessment of coronary flow. These ML models have been developed on monocenter datasets (in range 31-14,509 patients) and showed moderate to good performance. However, only three ML models were externally validated. Given the current pace of AI developments for the analysis of ICA, less-invasive, objective, and automated diagnosis of CAD can be expected in the near future. Further research on this technology in the catheterization laboratory may assist and improve treatment allocation, risk stratification, and cath lab logistics by integrating ICA analysis with other clinical characteristics.
Collapse
Affiliation(s)
- Mitchel A Molenaar
- Amsterdam University Medical Centers-Location VU Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands.
- Amsterdam University Medical Centers-Location Academic Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands.
| | - Jasper L Selder
- Amsterdam University Medical Centers-Location VU Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
| | - Johny Nicolas
- The Zena and Michael A. Wiener Cardiovascular Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1030, New York, NY, 10029-6574, USA
| | - Bimmer E Claessen
- Amsterdam University Medical Centers-Location VU Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
| | - Roxana Mehran
- The Zena and Michael A. Wiener Cardiovascular Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1030, New York, NY, 10029-6574, USA
| | | | - Mark J Schuuring
- Amsterdam University Medical Centers-Location Academic Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
| | - Berto J Bouma
- Amsterdam University Medical Centers-Location Academic Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
| | - Niels J Verouden
- Amsterdam University Medical Centers-Location VU Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
| | - Steven A J Chamuleau
- Amsterdam University Medical Centers-Location VU Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam University Medical Centers-Location Academic Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Cardiovascular Sciences, Amsterdam University Medical Centers-Location Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
13
|
Freitas SA, Nienow D, da Costa CA, Ramos GDO. Functional Coronary Artery Assessment: a Systematic Literature Review. Wien Klin Wochenschr 2021; 134:302-318. [PMID: 34870740 DOI: 10.1007/s00508-021-01970-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Accepted: 10/11/2021] [Indexed: 11/28/2022]
Abstract
Cardiovascular diseases represent the number one cause of death in the world, including the most common disorders in the heart's health, namely coronary artery disease (CAD). CAD is mainly caused by fat accumulated in the arteries' internal walls, creating an atherosclerotic plaque that impacts the blood flow functional behavior. Anatomical plaque characteristics are essential but not sufficient for a complete functional assessment of CAD. In fact, plaque analysis and visual inspection alone have proven insufficient to determine the lesion severity and hemodynamic repercussion. Furthermore, the fractional flow reserve (FFR) exam, which is considered the gold standard for stenosis functional impair determination, is invasive and contains several limitations. Such a panorama evidences the need for new techniques applied to image exams to improve CAD functional assessment. In this article, we perform a systematic literature review on emerging methods determining CAD significance, thus delivering a unique base for comparing these methods, qualitatively and quantitatively. Our goal is to guide further studies with evidence from the most promising methods, highlighting the benefits from both areas. We summarize benchmarks, metrics for evaluation, and challenges already faced, thus shedding light on the requirements for a valid, meaningful, and accepted technique for functional assessment evaluation. We create a base of comparison based on quantitative and qualitative indicators and highlight the most relevant geometrical metrics that correlate with lesion significance. Finally, we point out future benchmarks based on recent literature.
Collapse
Affiliation(s)
- Samuel A Freitas
- Software Innovation Laboratory, Graduate Program in Applied Computing, Universidade do Vale do Rio dos Sinos, São Leopoldo, Brazil
| | - Débora Nienow
- Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil
| | - Cristiano A da Costa
- Software Innovation Laboratory, Graduate Program in Applied Computing, Universidade do Vale do Rio dos Sinos, São Leopoldo, Brazil
| | - Gabriel de O Ramos
- Software Innovation Laboratory, Graduate Program in Applied Computing, Universidade do Vale do Rio dos Sinos, São Leopoldo, Brazil.
| |
Collapse
|