1
|
Bao F, Zhao Y, Zhang X, Zhang Y, Ning Y. SARC-UNet: A coronary artery segmentation method based on spatial attention and residual convolution. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 255:108353. [PMID: 39096572 DOI: 10.1016/j.cmpb.2024.108353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 07/18/2024] [Accepted: 07/22/2024] [Indexed: 08/05/2024]
Abstract
BACKGROUND AND OBJECTIVE Coronary artery segmentation is a pivotal field that has received increasing attention in recent years. However, this task remains challenging because of the inhomogeneous distributions of the contrast agent and dim light, resulting in noise, vascular breakages and small vessel losses in the obtained segmentation results. METHODS To acquire better automatic blood vessel segmentation results for coronary angiography images, a UNet-based segmentation network (SARC-UNet) is constructed for coronary artery segmentation; this approach is based on residual convolution and spatial attention. First, we use the low-light image enhancement (LIME) approach to increase the contrast and clarity levels of coronary angiography images. Then, we design two residual convolution fusion modules (RCFM1 and RCFM2) that can successfully fuse the local and global information of coronary images while also capturing the characteristics of finer-grained blood vessels, hence preventing the loss of tiny blood vessels in the segmentation findings. Finally, using a cascaded waterfall structure, we create a new location-enhanced spatial attention (LESA) mechanism that can efficiently improve the long-distance dependencies between coronary vascular pixel features, eradicating vascular ruptures and noise in the segmentation results. RESULTS This article subjectively and objectively evaluates the experimental results. This method has performed well on five general indicators. Furthermore, it outperforms the connectivity indicators proposed in this article. This method can effectively segment blood vessels and obtain higher accuracy results. CONCLUSIONS Numerous experiments have shown that the suggested method outperforms the state-of-the-art approaches, particularly in terms of vessel connectivity and small blood vessel segmentation.
Collapse
Affiliation(s)
- Fangxun Bao
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China.
| | - Yongqi Zhao
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China
| | - Xinyue Zhang
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China
| | - Yunfeng Zhang
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, Shandong, 250014, China
| | - Yang Ning
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, Shandong, 250101, China
| |
Collapse
|
2
|
Yagis E, Aslani S, Jain Y, Zhou Y, Rahmani S, Brunet J, Bellier A, Werlein C, Ackermann M, Jonigk D, Tafforeau P, Lee PD, Walsh C. Deep Learning for 3D Vascular Segmentation in Phase Contrast Tomography. RESEARCH SQUARE 2024:rs.3.rs-4613439. [PMID: 39070623 PMCID: PMC11276017 DOI: 10.21203/rs.3.rs-4613439/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/30/2024]
Abstract
Automated blood vessel segmentation is critical for biomedical image analysis, as vessel morphology changes are associated with numerous pathologies. Still, precise segmentation is difficult due to the complexity of vascular structures, anatomical variations across patients, the scarcity of annotated public datasets, and the quality of images. Our goal is to provide a foundation on the topic and identify a robust baseline model for application to vascular segmentation using a new imaging modality, Hierarchical Phase-Contrast Tomography (HiP-CT). We begin with an extensive review of current machine learning approaches for vascular segmentation across various organs. Our work introduces a meticulously curated training dataset, verified by double annotators, consisting of vascular data from three kidneys imaged using Hierarchical Phase-Contrast Tomography (HiP-CT) as part of the Human Organ Atlas Project. HiP-CT, pioneered at the European Synchrotron Radiation Facility in 2020, revolutionizes 3D organ imaging by offering resolution around 20μm/voxel, and enabling highly detailed localized zooms up to 1μm/voxel without physical sectioning. We leverage the nnU-Net framework to evaluate model performance on this high-resolution dataset, using both known and novel samples, and implementing metrics tailored for vascular structures. Our comprehensive review and empirical analysis on HiP-CT data sets a new standard for evaluating machine learning models in high-resolution organ imaging. Our three experiments yielded Dice scores of 0.9523 and 0.9410, and 0.8585, respectively. Nevertheless, DSC primarily assesses voxel-to-voxel concordance, overlooking several crucial characteristics of the vessels and should not be the sole metric for deciding the performance of vascular segmentation. Our results show that while segmentations yielded reasonably high scores-such as centerline Dice values ranging from 0.82 to 0.88, certain errors persisted. Specifically, large vessels that collapsed due to the lack of hydro-static pressure (HiP-CT is an ex vivo technique) were segmented poorly. Moreover, decreased connectivity in finer vessels and higher segmentation errors at vessel boundaries were observed. Such errors, particularly in significant vessels, obstruct the understanding of the structures by interrupting vascular tree connectivity. Through our review and outputs, we aim to set a benchmark for subsequent model evaluations using various modalities, especially with the HiP-CT imaging database.
Collapse
Affiliation(s)
- Ekin Yagis
- Department of Mechanical Engineering, University College London, London, UK
| | - Shahab Aslani
- Department of Mechanical Engineering, University College London, London, UK
- Centre for Medical Image Computing, University College London, London UK
| | - Yashvardhan Jain
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, USA
| | - Yang Zhou
- Department of Mechanical Engineering, University College London, London, UK
| | - Shahrokh Rahmani
- Department of Mechanical Engineering, University College London, London, UK
| | - Joseph Brunet
- Department of Mechanical Engineering, University College London, London, UK
- European Synchrotron Radiation Facility, Grenoble, France
| | | | - Christopher Werlein
- Institute of Pathology, Hannover Medical School, Carl-Neuberg-Straße 1, 30625, Hannover, Germany
| | | | - Danny Jonigk
- Member of the German Center for Lung Research (DZL), Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Hannover, Germany
| | - Paul Tafforeau
- European Synchrotron Radiation Facility, Grenoble, France
| | - Peter D. Lee
- Department of Mechanical Engineering, University College London, London, UK
| | - Claire Walsh
- Department of Mechanical Engineering, University College London, London, UK
| |
Collapse
|
3
|
Lashgari M, Choudhury RP, Banerjee A. Patient-specific in silico 3D coronary model in cardiac catheterisation laboratories. Front Cardiovasc Med 2024; 11:1398290. [PMID: 39036504 PMCID: PMC11257904 DOI: 10.3389/fcvm.2024.1398290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2024] [Accepted: 06/06/2024] [Indexed: 07/23/2024] Open
Abstract
Coronary artery disease is caused by the buildup of atherosclerotic plaque in the coronary arteries, affecting the blood supply to the heart, one of the leading causes of death around the world. X-ray coronary angiography is the most common procedure for diagnosing coronary artery disease, which uses contrast material and x-rays to observe vascular lesions. With this type of procedure, blood flow in coronary arteries is viewed in real-time, making it possible to detect stenoses precisely and control percutaneous coronary interventions and stent insertions. Angiograms of coronary arteries are used to plan the necessary revascularisation procedures based on the calculation of occlusions and the affected segments. However, their interpretation in cardiac catheterisation laboratories presently relies on sequentially evaluating multiple 2D image projections, which limits measuring lesion severity, identifying the true shape of vessels, and analysing quantitative data. In silico modelling, which involves computational simulations of patient-specific data, can revolutionise interventional cardiology by providing valuable insights and optimising treatment methods. This paper explores the challenges and future directions associated with applying patient-specific in silico models in catheterisation laboratories. We discuss the implications of the lack of patient-specific in silico models and how their absence hinders the ability to accurately predict and assess the behaviour of individual patients during interventional procedures. Then, we introduce the different components of a typical patient-specific in silico model and explore the potential future directions to bridge this gap and promote the development and utilisation of patient-specific in silico models in the catheterisation laboratories.
Collapse
Affiliation(s)
- Mojtaba Lashgari
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom
| | - Robin P. Choudhury
- Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, Oxford, United Kingdom
| | - Abhirup Banerjee
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom
- Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
4
|
Chang SS, Lin CT, Wang WC, Hsu KC, Wu YL, Liu CH, Fann YC. Optimizing ensemble U-Net architectures for robust coronary vessel segmentation in angiographic images. Sci Rep 2024; 14:6640. [PMID: 38503839 PMCID: PMC10951254 DOI: 10.1038/s41598-024-57198-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 03/15/2024] [Indexed: 03/21/2024] Open
Abstract
Automated coronary angiography assessment requires precise vessel segmentation, a task complicated by uneven contrast filling and background noise. Our research introduces an ensemble U-Net model, SE-RegUNet, designed to accurately segment coronary vessels using 100 labeled angiographies from angiographic images. SE-RegUNet incorporates RegNet encoders and squeeze-and-excitation blocks to enhance feature extraction. A dual-phase image preprocessing strategy further improves the model's performance, employing unsharp masking and contrast-limited adaptive histogram equalization. Following fivefold cross-validation and Ranger21 optimization, the SE-RegUNet 4GF model emerged as the most effective, evidenced by performance metrics such as a Dice score of 0.72 and an accuracy of 0.97. Its potential for real-world application is highlighted by its ability to process images at 41.6 frames per second. External validation on the DCA1 dataset demonstrated the model's consistent robustness, achieving a Dice score of 0.76 and an accuracy of 0.97. The SE-RegUNet 4GF model's precision in segmenting blood vessels in coronary angiographies showcases its remarkable efficiency and accuracy. However, further development and clinical testing are necessary before it can be routinely implemented in medical practice.
Collapse
Affiliation(s)
- Shih-Sheng Chang
- Division of Cardiovascular Medicine, China Medical University Hospital, Taichung, Taiwan
- School of Medicine, College of Medicine, China Medical University, Taichung, Taiwan
| | - Ching-Ting Lin
- Artificial Intelligence Center, China Medical University Hospital, Taichung, Taiwan
| | - Wei-Chun Wang
- Department of Neurology, China Medical University Hospital, Taichung, Taiwan
- Neuroscience and Brain Disease Center, China Medical University, Taichung, Taiwan
- Artificial Intelligence Center, China Medical University Hospital, Taichung, Taiwan
| | - Kai-Cheng Hsu
- Artificial Intelligence Center, China Medical University Hospital, Taichung, Taiwan
- Department of Neurology, China Medical University Hospital, Taichung, Taiwan
- Neuroscience and Brain Disease Center, China Medical University, Taichung, Taiwan
- School of Medicine, College of Medicine, China Medical University, Taichung, Taiwan
| | - Ya-Lun Wu
- Artificial Intelligence Center, China Medical University Hospital, Taichung, Taiwan
| | - Chia-Hao Liu
- Division of Cardiovascular Medicine, China Medical University Hospital, Taichung, Taiwan
| | - Yang C Fann
- Division of Intramural Research, National Institute of Neurological Disorders and Stroke, National Institutes of Health, 35 Convent Dr., Bethesda, MD, 20892, USA.
| |
Collapse
|
5
|
Arefinia F, Aria M, Rabiei R, Hosseini A, Ghaemian A, Roshanpoor A. Non-invasive fractional flow reserve estimation using deep learning on intermediate left anterior descending coronary artery lesion angiography images. Sci Rep 2024; 14:1818. [PMID: 38245614 PMCID: PMC10799954 DOI: 10.1038/s41598-024-52360-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Accepted: 01/17/2024] [Indexed: 01/22/2024] Open
Abstract
This study aimed to design an end-to-end deep learning model for estimating the value of fractional flow reserve (FFR) using angiography images to classify left anterior descending (LAD) branch angiography images with average stenosis between 50 and 70% into two categories: FFR > 80 and FFR ≤ 80. In this study 3625 images were extracted from 41 patients' angiography films. Nine pre-trained convolutional neural networks (CNN), including DenseNet121, InceptionResNetV2, VGG16, VGG19, ResNet50V2, Xception, MobileNetV3Large, DenseNet201, and DenseNet169, were used to extract the features of images. DenseNet169 indicated higher performance compared to other networks. AUC, Accuracy, Sensitivity, Specificity, Precision, and F1-score of the proposed DenseNet169 network were 0.81, 0.81, 0.86, 0.75, 0.82, and 0.84, respectively. The deep learning-based method proposed in this study can non-invasively and consistently estimate FFR from angiographic images, offering significant clinical potential for diagnosing and treating coronary artery disease by combining anatomical and physiological parameters.
Collapse
Affiliation(s)
- Farhad Arefinia
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mehrad Aria
- Cancer Research Center, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Reza Rabiei
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Azamossadat Hosseini
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Ali Ghaemian
- Department of Cardiology, Faculty of Medicine, Cardiovascular Research Center, Mazandaran University of Medical Sciences, Sari, Iran
| | - Arash Roshanpoor
- Department of Computer, Yadegar-e-Imam Khomeini (RAH), Islamic Azad University, Janat-Abad Branch, Tehran, Iran
| |
Collapse
|
6
|
Wang G, Zhou P, Gao H, Qin Z, Wang S, Sun J, Yu H. Coronary vessel segmentation in coronary angiography with a multi-scale U-shaped transformer incorporating boundary aggregation and topology preservation. Phys Med Biol 2024; 69:025012. [PMID: 38200403 DOI: 10.1088/1361-6560/ad0b63] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 11/09/2023] [Indexed: 01/12/2024]
Abstract
Coronary vessel segmentation plays a pivotal role in automating the auxiliary diagnosis of coronary heart disease. The continuity and boundary accuracy of the segmented vessels directly affect the subsequent processing. Notably, during segmentation, vessels with severe stenosis can easily cause boundary errors and breakage, resulting in isolated islands. To address these issues, we propose a novel multi-scale U-shaped transformer with boundary aggregation and topology preservation (UT-BTNet) for coronary vessel segmentation in coronary angiography. Specifically, considering the characteristics of coronary vessels, we first develop the UT-BTNet for coronary vessels segmentation, which combines the advantages of a convolutional neural networks (CNN) and a transformer, and is able to effectively extract the local and global features of angiographic images. Secondly, we innovatively employ boundary loss and topological loss in two stages, in addition to the traditional losses. In the first stage, boundary loss is adopted, which has the effect of boundary aggregation. In the second stage, topological loss is applied to preserve the topology of the vessels, after the network converges. In the experiment, in addition to the two metrics of Dice and intersection over union (IoU), we specifically propose two metrics of boundary intersection over union (BIoU) and Betti error to evaluate boundary accuracy and the continuity of segmentation results. The results show that the Dice is 0.9291, the IoU is 0.8687, the BIoU is 0.5094, and the Betti error is 0.3400. Compared with the other state-of-the-art methods, UT-BTNet achieves better segmentation results, while ensuring the continuity and boundary accuracy of the vessels, indicating its potential clinical value.
Collapse
Affiliation(s)
- Guangpu Wang
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, People's Republic of China
| | - Peng Zhou
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, People's Republic of China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, People's Republic of China
| | - Hui Gao
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, People's Republic of China
| | - Zewei Qin
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, People's Republic of China
| | - Shuo Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, People's Republic of China
| | - Jinglai Sun
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, People's Republic of China
| | - Hui Yu
- Department of Biomedical Engineering, School of Precision Instrument and Opto-Electronics Engineering, Tianjin University, People's Republic of China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, People's Republic of China
| |
Collapse
|
7
|
Liao S, Wang B, Lin S. Optimizing cardiovascular image segmentation through integrated hierarchical features and attention mechanisms. Technol Health Care 2024; 32:403-413. [PMID: 38759064 PMCID: PMC11191477 DOI: 10.3233/thc-248035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/19/2024]
Abstract
BACKGROUND Cardiovascular diseases are the top cause of death in China. Manual segmentation of cardiovascular images, prone to errors, demands an automated, rapid, and precise solution for clinical diagnosis. OBJECTIVE The paper highlights deep learning in automatic cardiovascular image segmentation, efficiently identifying pixel regions of interest for auxiliary diagnosis and research in cardiovascular diseases. METHODS In our study, we introduce innovative Region Weighted Fusion (RWF) and Shape Feature Refinement (SFR) modules, utilizing polarized self-attention for significant performance improvement in multiscale feature integration and shape fine-tuning. The RWF module includes reshaping, weight computation, and feature fusion, enhancing high-resolution attention computation and reducing information loss. Model optimization through loss functions offers a more reliable solution for cardiovascular medical image processing. RESULTS Our method excels in segmentation accuracy, emphasizing the vital role of the RWF module. It demonstrates outstanding performance in cardiovascular image segmentation, potentially raising clinical practice standards. CONCLUSIONS Our method ensures reliable medical image processing, guiding cardiovascular segmentation for future advancements in practical healthcare and contributing scientifically to enhanced disease diagnosis and treatment.
Collapse
Affiliation(s)
- Shijia Liao
- School of Informatics, Xiamen University, Xiamen, Fujian, China
| | - Bin Wang
- School of Informatics, Xiamen University, Xiamen, Fujian, China
- Emergency Department, Xiamen Cardiovascular Hospital of Xiamen University, Xiamen University, Xiamen, Fujian, China
| | - Shiming Lin
- School of Informatics, Xiamen University, Xiamen, Fujian, China
- School of Information Engineering, Changji University, Changji, Xinjiang Uygur Autonomous Region, China
| |
Collapse
|
8
|
Park J, Kweon J, Kim YI, Back I, Chae J, Roh JH, Kang DY, Lee PH, Ahn JM, Kang SJ, Park DW, Lee SW, Lee CW, Park SW, Park SJ, Kim YH. Selective ensemble methods for deep learning segmentation of major vessels in invasive coronary angiography. Med Phys 2023; 50:7822-7839. [PMID: 37310802 DOI: 10.1002/mp.16554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2022] [Revised: 03/29/2023] [Accepted: 05/26/2023] [Indexed: 06/15/2023] Open
Abstract
BACKGROUND Invasive coronary angiography (ICA) is a primary imaging modality that visualizes the lumen area of coronary arteries for diagnosis and interventional guidance. In the current practice of quantitative coronary analysis (QCA), semi-automatic segmentation tools require labor-intensive and time-consuming manual correction, limiting their application in the catheterization room. PURPOSE This study aims to propose rank-based selective ensemble methods that improve the segmentation performance and reduce morphological errors that limit fully automated quantification of coronary artery using deep-learning segmentation of ICA. METHODS Two selective ensemble methods proposed in this work integrated the weighted ensemble approach with per-image quality estimation. The segmentation outcomes from five base models with different loss functions were ranked either by mask morphology or estimated dice similarity coefficient (DSC). The final output was determined by imposing different weights according to the ranks. The ranking criteria based on mask morphology were formulated from empirical insight to avoid frequent types of segmentation errors (MSEN), while the estimation of DSCs was performed by comparing the pseudo-ground truth generated from a meta-learner (ESEN). Five-fold cross-validation was performed with the internal dataset of 7426 coronary angiograms from 2924 patients, and prediction model was externally validated with 556 images of 226 patients. RESULTS The selective ensemble methods improved the segmentation performance with DSCs up to 93.07% and provided a better delineation of coronary lesion with local DSCs of up to 93.93%, outperforming all individual models. Proposed methods also minimized the chances of mask disconnection in the most narrowed regions to 2.10%. The robustness of the proposed methods was also evident in the external validation. Inference time for major vessel segmentation was approximately one-sixth of a second. CONCLUSION Proposed methods successfully reduced morphological errors in the predicted masks and were able to enhance the robustness of the automatic segmentation. The results suggest better applicability of real-time QCA-based diagnostic methods in routine clinical settings.
Collapse
Affiliation(s)
- Jeeone Park
- Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
| | - Jihoon Kweon
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
| | - Young In Kim
- Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
| | - Inwook Back
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Jihye Chae
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Jae-Hyung Roh
- Department of Cardiology, Chungnam National University Sejong Hospital, Chungnam National University School of Medicine, Daejeon, South Korea
| | - Do-Yoon Kang
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Pil Hyung Lee
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Jung-Min Ahn
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Soo-Jin Kang
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Duk-Woo Park
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Seung-Whan Lee
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Cheol Whan Lee
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Seong-Wook Park
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Seung-Jung Park
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| | - Young-Hak Kim
- Division of Cardiology, Department of Internal Medicine, Medical Center, University of Ulsan College of Medicine, Asan, Seoul, South Korea
| |
Collapse
|
9
|
Iyer K, Nallamothu BK, Figueroa CA, Nadakuditi RR. A multi-stage neural network approach for coronary 3D reconstruction from uncalibrated X-ray angiography images. Sci Rep 2023; 13:17603. [PMID: 37845232 PMCID: PMC10579444 DOI: 10.1038/s41598-023-44633-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 10/10/2023] [Indexed: 10/18/2023] Open
Abstract
We present a multi-stage neural network approach for 3D reconstruction of coronary artery trees from uncalibrated 2D X-ray angiography images. This method uses several binarized images from different angles to reconstruct a 3D coronary tree without any knowledge of image acquisition parameters. The method consists of a single backbone network and separate stages for vessel centerline and radius reconstruction. The output is an analytical matrix representation of the coronary tree suitable for downstream applications such as hemodynamic modeling of local vessel narrowing (i.e., stenosis). The network was trained using a dataset of synthetic coronary trees from a vessel generator informed by both clinical image data and literature values on coronary anatomy. Our multi-stage network achieved sub-pixel accuracy in reconstructing vessel radius (RMSE = 0.16 ± 0.07 mm) and stenosis radius (MAE = 0.27 ± 0.18 mm), the most important feature used to inform diagnostic decisions. The network also led to 52% and 38% reduction in vessel centerline reconstruction errors compared to a single-stage network and projective geometry-based methods, respectively. Our method demonstrated robustness to overcome challenges such as vessel foreshortening or overlap in the input images. This work is an important step towards automated analysis of anatomic and functional disease severity in the coronary arteries.
Collapse
Affiliation(s)
- Kritika Iyer
- University of Michigan, 2800 Plymouth Road Building 20-210W, Ann Arbor, MI, 48109, USA.
| | | | - C Alberto Figueroa
- University of Michigan, 2800 Plymouth Road Building 20-210W, Ann Arbor, MI, 48109, USA
| | - Raj R Nadakuditi
- University of Michigan, 2800 Plymouth Road Building 20-210W, Ann Arbor, MI, 48109, USA
| |
Collapse
|
10
|
Truskey GA. The Potential of Deep Learning to Advance Clinical Applications of Computational Biomechanics. Bioengineering (Basel) 2023; 10:1066. [PMID: 37760168 PMCID: PMC10525821 DOI: 10.3390/bioengineering10091066] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 09/06/2023] [Accepted: 09/07/2023] [Indexed: 09/29/2023] Open
Abstract
When combined with patient information provided by advanced imaging techniques, computational biomechanics can provide detailed patient-specific information about stresses and strains acting on tissues that can be useful in diagnosing and assessing treatments for diseases and injuries. This approach is most advanced in cardiovascular applications but can be applied to other tissues. The challenges for advancing computational biomechanics for real-time patient diagnostics and treatment include errors and missing information in the patient data, the large computational requirements for the numerical solutions to multiscale biomechanical equations, and the uncertainty over boundary conditions and constitutive relations. This review summarizes current efforts to use deep learning to address these challenges and integrate large data sets and computational methods to enable real-time clinical information. Examples are drawn from cardiovascular fluid mechanics, soft-tissue mechanics, and bone biomechanics. The application of deep-learning convolutional neural networks can reduce the time taken to complete image segmentation, and meshing and solution of finite element models, as well as improving the accuracy of inlet and outlet conditions. Such advances are likely to facilitate the adoption of these models to aid in the assessment of the severity of cardiovascular disease and the development of new surgical treatments.
Collapse
Affiliation(s)
- George A Truskey
- Department of Biomedical Engineering, Duke University, Durham, NC 27701, USA
| |
Collapse
|
11
|
Avram R, Olgin JE, Ahmed Z, Verreault-Julien L, Wan A, Barrios J, Abreau S, Wan D, Gonzalez JE, Tardif JC, So DY, Soni K, Tison GH. CathAI: fully automated coronary angiography interpretation and stenosis estimation. NPJ Digit Med 2023; 6:142. [PMID: 37568050 PMCID: PMC10421915 DOI: 10.1038/s41746-023-00880-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Accepted: 07/21/2023] [Indexed: 08/13/2023] Open
Abstract
Coronary angiography is the primary procedure for diagnosis and management decisions in coronary artery disease (CAD), but ad-hoc visual assessment of angiograms has high variability. Here we report a fully automated approach to interpret angiographic coronary artery stenosis from standard coronary angiograms. Using 13,843 angiographic studies from 11,972 adult patients at University of California, San Francisco (UCSF), between April 1, 2008 and December 31, 2019, we train neural networks to accomplish four sequential necessary tasks for automatic coronary artery stenosis localization and estimation. Algorithms are internally validated against criterion-standard labels for each task in hold-out test datasets. Algorithms are then externally validated in real-world angiograms from the University of Ottawa Heart Institute (UOHI) and also retrained using quantitative coronary angiography (QCA) data from the Montreal Heart Institute (MHI) core lab. The CathAI system achieves state-of-the-art performance across all tasks on unselected, real-world angiograms. Positive predictive value, sensitivity and F1 score are all ≥90% to identify projection angle and ≥93% for left/right coronary artery angiogram detection. To predict obstructive CAD stenosis (≥70%), CathAI exhibits an AUC of 0.862 (95% CI: 0.843-0.880). In UOHI external validation, CathAI achieves AUC 0.869 (95% CI: 0.830-0.907) to predict obstructive CAD. In the MHI QCA dataset, CathAI achieves an AUC of 0.775 (95%. CI: 0.594-0.955) after retraining. In conclusion, multiple purpose-built neural networks can function in sequence to accomplish automated analysis of real-world angiograms, which could increase standardization and reproducibility in angiographic coronary stenosis assessment.
Collapse
Affiliation(s)
- Robert Avram
- Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology, 505 Parnassus Avenue, San Francisco, CA, 94143, USA
- Division of Cardiology, Department of Medicine, Montreal Heart Institute - Université de Montréal, 5000 Rue Belanger, Montreal, QC, H1T 1C8, Canada
| | - Jeffrey E Olgin
- Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology, 505 Parnassus Avenue, San Francisco, CA, 94143, USA
- Cardiovascular Research Institute, University of California, San Francisco, CA, 94143, USA
| | - Zeeshan Ahmed
- Division of Cardiology, Department of Medicine, University of Ottawa Heart Institute, University of Ottawa, 40 Ruskin Street, Ottawa, ON, K1Y 4W7, Canada
| | - Louis Verreault-Julien
- Division of Cardiology, Department of Medicine, University of Ottawa Heart Institute, University of Ottawa, 40 Ruskin Street, Ottawa, ON, K1Y 4W7, Canada
| | - Alvin Wan
- Cardiovascular Research Institute, University of California, San Francisco, CA, 94143, USA
| | - Joshua Barrios
- Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology, 505 Parnassus Avenue, San Francisco, CA, 94143, USA
| | - Sean Abreau
- Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology, 505 Parnassus Avenue, San Francisco, CA, 94143, USA
| | - Derek Wan
- Department of Electrical Engineering and Computer Science, RISE Lab, University of California, Berkeley, Soda Hall, Berkeley, CA, 94720-1770, USA
| | - Joseph E Gonzalez
- Department of Electrical Engineering and Computer Science, RISE Lab, University of California, Berkeley, Soda Hall, Berkeley, CA, 94720-1770, USA
| | - Jean-Claude Tardif
- Division of Cardiology, Department of Medicine, Montreal Heart Institute - Université de Montréal, 5000 Rue Belanger, Montreal, QC, H1T 1C8, Canada
| | - Derek Y So
- Division of Cardiology, Department of Medicine, University of Ottawa Heart Institute, University of Ottawa, 40 Ruskin Street, Ottawa, ON, K1Y 4W7, Canada
| | - Krishan Soni
- Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology, 505 Parnassus Avenue, San Francisco, CA, 94143, USA
| | - Geoffrey H Tison
- Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology, 505 Parnassus Avenue, San Francisco, CA, 94143, USA.
- Cardiovascular Research Institute, University of California, San Francisco, CA, 94143, USA.
- Department of Electrical Engineering and Computer Science, RISE Lab, University of California, Berkeley, Soda Hall, Berkeley, CA, 94720-1770, USA.
- Bakar Computational Health Sciences Institute, University of California, San Francisco, 94158, USA.
| |
Collapse
|
12
|
Kaba Ş, Haci H, Isin A, Ilhan A, Conkbayir C. The Application of Deep Learning for the Segmentation and Classification of Coronary Arteries. Diagnostics (Basel) 2023; 13:2274. [PMID: 37443668 DOI: 10.3390/diagnostics13132274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 06/30/2023] [Accepted: 07/03/2023] [Indexed: 07/15/2023] Open
Abstract
In recent years, the prevalence of coronary artery disease (CAD) has become one of the leading causes of death around the world. Accurate stenosis detection of coronary arteries is crucial for timely treatment. Cardiologists use visual estimations when reading coronary angiography images to diagnose stenosis. As a result, they face various challenges which include high workloads, long processing times and human error. Computer-aided segmentation and classification of coronary arteries, as to whether stenosis is present or not, significantly reduces the workload of cardiologists and human errors caused by manual processes. Moreover, deep learning techniques have been shown to aid medical experts in diagnosing diseases using biomedical imaging. Thus, this study proposes the use of automatic segmentation of coronary arteries using U-Net, ResUNet-a, UNet++, models and classification using DenseNet201, EfficientNet-B0, Mobilenet-v2, ResNet101 and Xception models. In the case of segmentation, the comparative analysis of the three models has shown that U-Net achieved the highest score with a 0.8467 Dice score and 0.7454 Jaccard Index in comparison with UNet++ and ResUnet-a. Evaluation of the classification model's performances has shown that DenseNet201 performed better than other pretrained models with 0.9000 accuracy, 0.9833 specificity, 0.9556 PPV, 0.7746 Cohen's Kappa and 0.9694 Area Under the Curve (AUC).
Collapse
Affiliation(s)
- Şerife Kaba
- Department of Biomedical Engineering, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Huseyin Haci
- Department of Electrical-Electronic Engineering, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Ali Isin
- Department of Biomedical Engineering, Cyprus International University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Ahmet Ilhan
- Department of Computer Engineering, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Cenk Conkbayir
- Department of Cardiology, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| |
Collapse
|
13
|
Zhang H, Gao Z, Zhang D, Hau WK, Zhang H. Progressive Perception Learning for Main Coronary Segmentation in X-Ray Angiography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:864-879. [PMID: 36327189 DOI: 10.1109/tmi.2022.3219126] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Main coronary segmentation from the X-ray angiography images is important for the computer-aided diagnosis and treatment of coronary disease. However, it confronts the challenge at three different image granularities (the semantic, surrounding, and local levels). The challenge includes the semantic confusion between the main and collateral vessels, low contrast between the foreground vessel and background surroundings, and local ambiguity near the vessel boundaries. The traditional hand-crafted feature-based methods may be insufficient because they may lack the semantic relationship information and may not distinguish the main and collateral vessels. The existing deep learning-based methods seem to have issues due to the deficiency in the long-distance semantic relationship capture, the foreground and background interference adaptability, and the boundary detail information preservation. To solve the main coronary segmentation challenge, we propose the progressive perception learning (PPL) framework to inspect these three different image granularities. Specifically, the PPL contains the context, interference, and boundary perception modules. The context perception is designed to focus on the main coronary vessel based on the semantic dependence capture among different coronary segments. The interference perception is designed to purify the feature maps based on the foreground vessel enhancement and background artifact suppression. The boundary perception is designed to highlight the boundary details based on boundary feature extraction through the intersection between the foreground and background predictions. Extensive experiments on 1085 subjects show that the PPL is effective (e.g., the overall Dice is greater than 95%), and superior to thirteen state-of-the-art coronary segmentation methods.
Collapse
|
14
|
Schwarz EL, Pegolotti L, Pfaller MR, Marsden AL. Beyond CFD: Emerging methodologies for predictive simulation in cardiovascular health and disease. BIOPHYSICS REVIEWS 2023; 4:011301. [PMID: 36686891 PMCID: PMC9846834 DOI: 10.1063/5.0109400] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 12/12/2022] [Indexed: 01/15/2023]
Abstract
Physics-based computational models of the cardiovascular system are increasingly used to simulate hemodynamics, tissue mechanics, and physiology in evolving healthy and diseased states. While predictive models using computational fluid dynamics (CFD) originated primarily for use in surgical planning, their application now extends well beyond this purpose. In this review, we describe an increasingly wide range of modeling applications aimed at uncovering fundamental mechanisms of disease progression and development, performing model-guided design, and generating testable hypotheses to drive targeted experiments. Increasingly, models are incorporating multiple physical processes spanning a wide range of time and length scales in the heart and vasculature. With these expanded capabilities, clinical adoption of patient-specific modeling in congenital and acquired cardiovascular disease is also increasing, impacting clinical care and treatment decisions in complex congenital heart disease, coronary artery disease, vascular surgery, pulmonary artery disease, and medical device design. In support of these efforts, we discuss recent advances in modeling methodology, which are most impactful when driven by clinical needs. We describe pivotal recent developments in image processing, fluid-structure interaction, modeling under uncertainty, and reduced order modeling to enable simulations in clinically relevant timeframes. In all these areas, we argue that traditional CFD alone is insufficient to tackle increasingly complex clinical and biological problems across scales and systems. Rather, CFD should be coupled with appropriate multiscale biological, physical, and physiological models needed to produce comprehensive, impactful models of mechanobiological systems and complex clinical scenarios. With this perspective, we finally outline open problems and future challenges in the field.
Collapse
Affiliation(s)
- Erica L. Schwarz
- Departments of Pediatrics and Bioengineering, Stanford University, Stanford, California 94305, USA
| | - Luca Pegolotti
- Departments of Pediatrics and Bioengineering, Stanford University, Stanford, California 94305, USA
| | - Martin R. Pfaller
- Departments of Pediatrics and Bioengineering, Stanford University, Stanford, California 94305, USA
| | - Alison L. Marsden
- Departments of Pediatrics and Bioengineering, Stanford University, Stanford, California 94305, USA
| |
Collapse
|
15
|
Li X, Liu X, Deng X, Fan Y. Interplay between Artificial Intelligence and Biomechanics Modeling in the Cardiovascular Disease Prediction. Biomedicines 2022; 10:2157. [PMID: 36140258 PMCID: PMC9495955 DOI: 10.3390/biomedicines10092157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Revised: 08/26/2022] [Accepted: 08/28/2022] [Indexed: 11/16/2022] Open
Abstract
Cardiovascular disease (CVD) is the most common cause of morbidity and mortality worldwide, and early accurate diagnosis is the key point for improving and optimizing the prognosis of CVD. Recent progress in artificial intelligence (AI), especially machine learning (ML) technology, makes it possible to predict CVD. In this review, we first briefly introduced the overview development of artificial intelligence. Then we summarized some ML applications in cardiovascular diseases, including ML-based models to directly predict CVD based on risk factors or medical imaging findings and the ML-based hemodynamics with vascular geometries, equations, and methods for indirect assessment of CVD. We also discussed case studies where ML could be used as the surrogate for computational fluid dynamics in data-driven models and physics-driven models. ML models could be a surrogate for computational fluid dynamics, accelerate the process of disease prediction, and reduce manual intervention. Lastly, we briefly summarized the research difficulties and prospected the future development of AI technology in cardiovascular diseases.
Collapse
Affiliation(s)
- Xiaoyin Li
- Beijing Advanced Innovation Centre for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Chinese Education Ministry, School of Biological Science and Medical Engineering, Beihang University, Beijing 100083, China
| | - Xiao Liu
- Beijing Advanced Innovation Centre for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Chinese Education Ministry, School of Biological Science and Medical Engineering, Beihang University, Beijing 100083, China
| | - Xiaoyan Deng
- Beijing Advanced Innovation Centre for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Chinese Education Ministry, School of Biological Science and Medical Engineering, Beihang University, Beijing 100083, China
| | - Yubo Fan
- Beijing Advanced Innovation Centre for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Chinese Education Ministry, School of Biological Science and Medical Engineering, Beihang University, Beijing 100083, China
- School of Engineering Medicine, Beihang University, Beijing 100083, China
| |
Collapse
|
16
|
Su B, Wang Z, Gong Y, Li M, Teng Y, Yu S, Zong Y, Yao W, Wang J. Anal center detection and classification of perianal healthy condition. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
17
|
Classification of Left and Right Coronary Arteries in Coronary Angiographies Using Deep Learning. ELECTRONICS 2022. [DOI: 10.3390/electronics11132087] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
Multi-frame X-ray images (videos) of the coronary arteries obtained using coronary angiography (CAG) provide detailed information about the anatomy and blood flow in the coronary arteries and play a pivotal role in diagnosing and treating ischemic heart disease. Deep learning has the potential to quickly and accurately quantify narrowings and blockages of the arteries from CAG videos. A CAG consists of videos acquired separately for the left coronary artery and the right coronary artery (LCA and RCA, respectively). The pathology for LCA and RCA is typically only reported for the entire CAG, and not for the individual videos. However, training of stenosis quantification models is difficult when the RCA and LCA information of the videos are unknown. Here, we present a deep learning-based approach for classifying LCA and RCA in CAG videos. Our approach enables linkage of videos with the reported pathological findings. We manually labeled 3545 and 520 videos (approximately seven videos per CAG) to enable training and testing of the models, respectively. We obtained F1 scores of 0.99 on the test set for LCA and RCA classification LCA and RCA classification on the test set. The classification performance was further investigated with extensive experiments across different model architectures (R(2+1)D, X3D, and MVIT), model input sizes, data augmentations, and the number of videos used for training. Our results showed that CAG videos could be accurately curated using deep learning, which is an essential preprocessing step for a downstream application in diagnostics of coronary artery disease.
Collapse
|
18
|
Hillen B, Lopez DA, Schomer E, Nagele M, Simon P. Towards Exercise Radiomics: Deep Neural Network-Based Automatic Analysis of Thermal Images Captured During Exercise. IEEE J Biomed Health Inform 2022; 26:4530-4540. [PMID: 35759601 DOI: 10.1109/jbhi.2022.3186530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Infrared thermography is increasingly applied in sports science due to promising observations regarding changes in skin's surface radiation temperature ( Tsr) before, during, and after exercise. The common manual thermogram analysis limits an objective and reproducible measurement of Tsr. Previous analysis approaches depend highly on expert knowledge and have not been applied during movement. We aimed to develop a deep neural network (DNN) capable of automatically and objectively segmenting body parts, recognizing blood vessel-associated Tsr distributions, and continuously measuring Tsr during exercise. We conducted 38 cardiopulmonary exercise tests on a treadmill. We developed two DNNs: body part network and vessel network, to perform semantic segmentation of 1 107 855 captured thermal images. Both DNNs were trained with 263 training and 75 validation images. Additionally, we compare the results of a common manual thermogram analysis with these of the DNNs. Performance analysis identified a mean IoU of 0.8 for body part network and 0.6 for vessel network. There is a high agreement between manual and automatic analysis (r = 0.999; p 0.001; T-test: p = 0.116), with a mean difference of 0.01 C (0.08). Non-parametric Bland Altman's analysis showed that the 95% agreement ranges between -0.086 C and 0.228 C. The developed DNNs enable automatic, objective, and continuous measurement of Tsr and recognition of blood vessel-associated Tsr distributions in resting and moving legs. Hence, the DNNs surpass previous algorithms by eliminating manual region of interest selection and form the currently needed foundation to extensively investigate Tsr distributions related to non-invasive diagnostics of (patho-)physiological traits in means of exercise radiomics.
Collapse
|
19
|
Machine Learning for Cardiovascular Biomechanics Modeling: Challenges and Beyond. Ann Biomed Eng 2022; 50:615-627. [PMID: 35445297 DOI: 10.1007/s10439-022-02967-4] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 04/07/2022] [Indexed: 12/13/2022]
Abstract
Recent progress in machine learning (ML), together with advanced computational power, have provided new research opportunities in cardiovascular modeling. While classifying patient outcomes and medical image segmentation with ML have already shown significant promising results, ML for the prediction of biomechanics such as blood flow or tissue dynamics is in its infancy. This perspective article discusses some of the challenges in using ML for replacing well-established physics-based models in cardiovascular biomechanics. Specifically, we discuss the large landscape of input features in 3D patient-specific modeling as well as the high-dimensional output space of field variables that vary in space and time. We argue that the end purpose of such ML models needs to be clearly defined and the tradeoff between the loss in accuracy and the gained speedup carefully interpreted in the context of translational modeling. We also discuss several exciting venues where ML could be strategically used to augment traditional physics-based modeling in cardiovascular biomechanics. In these applications, ML is not replacing physics-based modeling, but providing opportunities to solve ill-defined problems, improve measurement data quality, enable a solution to computationally expensive problems, and interpret complex spatiotemporal data by extracting hidden patterns. In summary, we suggest a strategic integration of ML in cardiovascular biomechanics modeling where the ML model is not the end goal but rather a tool to facilitate enhanced modeling.
Collapse
|
20
|
Jiang TY, Ju FL, Dai YX, Li J, Li YF, Bai YJ, Cui ZQ, Xu ZH, Zhang ZQ. Real-Time Tracking of Object Melting Based on Enhanced DeepLab v3+ Network. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:2309317. [PMID: 35401724 PMCID: PMC8986418 DOI: 10.1155/2022/2309317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 02/13/2022] [Accepted: 02/24/2022] [Indexed: 11/23/2022]
Abstract
In order to reveal the dissolution behavior of iron tailings in blast furnace slag, the main component of iron tailings, SiO2, was used for research. Aiming at the problem of information loss and inaccurate extraction of tracking molten SiO2 particles in high temperature, a method based on the improved DeepLab v3+ network was proposed to track, segment, and extract small object particles in real time. First, by improving the decoding layer of the DeepLab v3+ network, construct dense ASPP (atrous spatial pyramid pooling) modules with different dilation rates to optimize feature extraction, increase the shallow convolution of the backbone network, and merge it into the upper convolution decoding part to increase detailed capture. Secondly, integrate the lightweight network MobileNet v3 to reduce network parameters, further speed up image detection, and reduce the memory usage to achieve real-time image segmentation and adapt to low-level configuration hardware. Finally, improve the expression of the loss function for the binary classification model of small object in this paper, combining the advantages of the Dice Loss binary classification segmentation and the Focal Loss balance of positive and negative samples, solving the problem of unbalanced dataset caused by the small proportion of positive samples. Experimental results show that MIoU (mean intersection over union) of the proposed model for small object segmentation is 6% higher than that of the original model, the overall MIoU is increased by 3%, and the execution time and memory consumption are only half of the original model, which can be well applied to real-time tracking and segmentation of small particles.
Collapse
Affiliation(s)
- Tian-yu Jiang
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Feng-lan Ju
- College of Metallurgy and Energy, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Ya-xun Dai
- College of Metallurgy and Energy, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Jie Li
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Yi-fan Li
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Yun-jie Bai
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Ze-qian Cui
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Zheng-han Xu
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| | - Zun-Qian Zhang
- Hebei Engineering Research Center for the Intelligentization of Iron Ore Optimization and Ironmaking Raw Materials Preparation Processes, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Hebei Key Laboratory of Data Science and Application, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Key Laboratory of Engineering Computing, North China University of Science and Technology, Tangshan, Hebei 063210, China
- Tangshan Intelligent Industry and Image Processing Technology Innovation Center, North China University of Science and Technology, Tangshan, Hebei 063210, China
| |
Collapse
|