1
|
Wang J, Yu F, Zhang M, Lu J, Qian Z. A 3D framework for segmentation of carotid artery vessel wall and identification of plaque compositions in multi-sequence MR images. Comput Med Imaging Graph 2024; 116:102402. [PMID: 38810486 DOI: 10.1016/j.compmedimag.2024.102402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 04/30/2024] [Accepted: 05/14/2024] [Indexed: 05/31/2024]
Abstract
Accurately assessing carotid artery wall thickening and identifying risky plaque components are critical for early diagnosis and risk management of carotid atherosclerosis. In this paper, we present a 3D framework for automated segmentation of the carotid artery vessel wall and identification of the compositions of carotid plaque in multi-sequence magnetic resonance (MR) images under the challenge of imperfect manual labeling. Manual labeling is commonly done in 2D slices of these multi-sequence MR images and often lacks perfect alignment across 2D slices and the multiple MR sequences, leading to labeling inaccuracies. To address such challenges, our framework is split into two parts: a segmentation subnetwork and a plaque component identification subnetwork. Initially, a 2D localization network pinpoints the carotid artery's position, extracting the region of interest (ROI) from the input images. Following that, a signed-distance-map-enabled 3D U-net (Çiçek etal, 2016)an adaptation of the nnU-net (Ronneberger and Fischer, 2015) segments the carotid artery vessel wall. This method allows for the concurrent segmentation of the vessel wall area using the signed distance map (SDM) loss (Xue et al., 2020) which regularizes the segmentation surfaces in 3D and reduces erroneous segmentation caused by imperfect manual labels. Subsequently, the ROI of the input images and the obtained vessel wall masks are extracted and combined to obtain the identification results of plaque components in the identification subnetwork. Tailored data augmentation operations are introduced into the framework to reduce the false positive rate of calcification and hemorrhage identification. We trained and tested our proposed method on a dataset consisting of 115 patients, and it achieves an accurate segmentation result of carotid artery wall (0.8459 Dice), which is superior to the best result in published studies (0.7885 Dice). Our approach yielded accuracies of 0.82, 0.73 and 0.88 for the identification of calcification, lipid-rich core and hemorrhage components. Our proposed framework can be potentially used in clinical and research settings to help radiologists perform cumbersome reading tasks and evaluate the risk of carotid plaques.
Collapse
Affiliation(s)
- Jian Wang
- Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging, Building 3-4F, 9 Yongteng N. Road, Beijing 100080, China.
| | - Fan Yu
- Department of Radiology and Nuclear medicine, Xuanwu Hospital, Capital Medical University, Changchun Street, No. 45, Beijing 100053, China; Beijing Key Laboratory of Magnetic Resonance Imaging and Brain Informatics, Beijing 100053, China.
| | - Mengze Zhang
- Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging, Building 3-4F, 9 Yongteng N. Road, Beijing 100080, China.
| | - Jie Lu
- Department of Radiology and Nuclear medicine, Xuanwu Hospital, Capital Medical University, Changchun Street, No. 45, Beijing 100053, China; Beijing Key Laboratory of Magnetic Resonance Imaging and Brain Informatics, Beijing 100053, China.
| | - Zhen Qian
- Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging, Building 3-4F, 9 Yongteng N. Road, Beijing 100080, China.
| |
Collapse
|
2
|
Martelli E, Capoccia L, Di Francesco M, Cavallo E, Pezzulla MG, Giudice G, Bauleo A, Coppola G, Panagrosso M. Current Applications and Future Perspectives of Artificial and Biomimetic Intelligence in Vascular Surgery and Peripheral Artery Disease. Biomimetics (Basel) 2024; 9:465. [PMID: 39194444 DOI: 10.3390/biomimetics9080465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2024] [Revised: 07/05/2024] [Accepted: 07/24/2024] [Indexed: 08/29/2024] Open
Abstract
Artificial Intelligence (AI) made its first appearance in 1956, and since then it has progressively introduced itself in healthcare systems and patients' information and care. AI functions can be grouped under the following headings: Machine Learning (ML), Deep Learning (DL), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Computer Vision (CV). Biomimetic intelligence (BI) applies the principles of systems of nature to create biological algorithms, such as genetic and neural network, to be used in different scenarios. Chronic limb-threatening ischemia (CLTI) represents the last stage of peripheral artery disease (PAD) and has increased over recent years, together with the rise in prevalence of diabetes and population ageing. Nowadays, AI and BI grant the possibility of developing new diagnostic and treatment solutions in the vascular field, given the possibility of accessing clinical, biological, and imaging data. By assessing the vascular anatomy in every patient, as well as the burden of atherosclerosis, and classifying the level and degree of disease, sizing and planning the best endovascular treatment, defining the perioperative complications risk, integrating experiences and resources between different specialties, identifying latent PAD, thus offering evidence-based solutions and guiding surgeons in the choice of the best surgical technique, AI and BI challenge the role of the physician's experience in PAD treatment.
Collapse
Affiliation(s)
- Eugenio Martelli
- Division of Vascular Surgery, Department of Surgery, S Maria Goretti Hospital, 81100 Latina, Italy
- Department of General and Specialist Surgery, Sapienza University of Rome, 00161 Rome, Italy
- Faculty of Medicine, Saint Camillus International University of Health Sciences, 00131 Rome, Italy
| | - Laura Capoccia
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Marco Di Francesco
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Eduardo Cavallo
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Maria Giulia Pezzulla
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Giorgio Giudice
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Antonio Bauleo
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Giuseppe Coppola
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| | - Marco Panagrosso
- Division of Vascular and Endovascular Surgery, Department of Cardiovascular Sciences, S. Anna and S. Sebastiano Hospital, 81100 Caserta, Italy
| |
Collapse
|
3
|
Wu J, Xin J, Yang X, Matkovic LA, Zhao X, Zheng N, Li R. Segmentation of carotid artery vessel wall and diagnosis of carotid atherosclerosis on black blood magnetic resonance imaging with multi-task learning. Med Phys 2024; 51:1775-1797. [PMID: 37681965 DOI: 10.1002/mp.16728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 07/04/2023] [Accepted: 07/29/2023] [Indexed: 09/09/2023] Open
Abstract
BACKGROUND Atherosclerotic cardiovascular disease is the leading cause of death worldwide. Early detection of carotid atherosclerosis can prevent the progression of cardiovascular disease. Many (semi-) automatic methods have been designed for the segmentation of carotid vessel wall and the diagnosis of carotid atherosclerosis (i.e., the lumen segmentation, the outer wall segmentation, and the carotid atherosclerosis diagnosis) on black blood magnetic resonance imaging (BB-MRI). However, most of these methods ignore the intrinsic correlation among different tasks on BB-MRI, leading to limited performance. PURPOSE Thus, we model the intrinsic correlation among the lumen segmentation, the outer wall segmentation, and the carotid atherosclerosis diagnosis tasks on BB-MRI by using the multi-task learning technique and propose a gated multi-task network (GMT-Net) to perform three related tasks in a neural network (i.e., carotid artery lumen segmentation, outer wall segmentation, and carotid atherosclerosis diagnosis). METHODS In the proposed method, the GMT-Net is composed of three modules, including the sharing module, the segmentation module, and the diagnosis module, which interact with each other to achieve better learning performance. At the same time, two new adaptive layers, namely, the gated exchange layer and the gated fusion layer, are presented to exchange and merge branch features. RESULTS The proposed method is applied to the CAREII dataset (i.e., 1057 scans) for the lumen segmentation, the outer wall segmentation, and the carotid atherosclerosis diagnosis. The proposed method can achieve promising segmentation performances (0.9677 Dice for the lumen and 0.9669 Dice for the outer wall) and better diagnosis accuracy of carotid atherosclerosis (0.9516 AUC and 0.9024 Accuracy) in the "CAREII test" dataset (i.e., 106 scans). The results show that the proposed method has statistically significant accuracy and efficiency. CONCLUSIONS Even without the intervention of reviewers required for the previous works, the proposed method automatically segments the lumen and outer wall together and diagnoses carotid atherosclerosis with high performance. The proposed method can be used in clinical trials to help radiologists get rid of tedious reading tasks, such as screening review to separate normal carotid arteries from atherosclerotic arteries and to outline vessel wall contours.
Collapse
Affiliation(s)
- Jiayi Wu
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University
| | - Jingmin Xin
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University
| | - Xiaofeng Yang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Luke A Matkovic
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Xihai Zhao
- Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Nanning Zheng
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University
| | - Rui Li
- Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| |
Collapse
|
4
|
Camarasa R, Kervadec H, Kooi ME, Hendrikse J, Nederkoorn PJ, Bos D, de Bruijne M. Nested star-shaped objects segmentation using diameter annotations. Med Image Anal 2023; 90:102934. [PMID: 37688981 DOI: 10.1016/j.media.2023.102934] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 08/02/2023] [Accepted: 08/10/2023] [Indexed: 09/11/2023]
Abstract
Most current deep learning based approaches for image segmentation require annotations of large datasets, which limits their application in clinical practice. We observe a mismatch between the voxelwise ground-truth that is required to optimize an objective at a voxel level and the commonly used, less time-consuming clinical annotations seeking to characterize the most important information about the patient (diameters, counts, etc.). In this study, we propose to bridge this gap for the case of multiple nested star-shaped objects (e.g., a blood vessel lumen and its outer wall) by optimizing a deep learning model based on diameter annotations. This is achieved by extracting in a differentiable manner the boundary points of the objects at training time, and by using this extraction during the backpropagation. We evaluate the proposed approach on segmentation of the carotid artery lumen and wall from multisequence MR images, thus reducing the annotation burden to only four annotated landmarks required to measure the diameters in the direction of the vessel's maximum narrowing. Our experiments show that training based on diameter annotations produces state-of-the-art weakly supervised segmentations and performs reasonably compared to full supervision. We made our code publicly available at https://gitlab.com/radiology/aim/carotid-artery-image-analysis/nested-star-shaped-objects.
Collapse
Affiliation(s)
- Robin Camarasa
- Biomedical Imaging Group Rotterdam, Erasmus MC, Rotterdam, The Netherlands; Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands.
| | - Hoel Kervadec
- Biomedical Imaging Group Rotterdam, Erasmus MC, Rotterdam, The Netherlands; Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - M Eline Kooi
- Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, Maastricht, The Netherlands; Cardiovascular Research Institute Maastricht (CARIM), Maastricht University, Maastricht, The Netherlands
| | - Jeroen Hendrikse
- Department of Radiology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Paul J Nederkoorn
- Department of Neurology, Academic Medical Center University of Amsterdam, Amsterdam, The Netherlands
| | - Daniel Bos
- Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands
| | - Marleen de Bruijne
- Biomedical Imaging Group Rotterdam, Erasmus MC, Rotterdam, The Netherlands; Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; Department of Computer Science, University of Copenhagen, Denmark.
| |
Collapse
|
5
|
Lei Y, Wang T, Roper J, Tian S, Patel P, Bradley JD, Jani AB, Liu T, Yang X. Automatic segmentation of neurovascular bundle on mri using deep learning based topological modulated network. Med Phys 2023; 50:5479-5488. [PMID: 36939189 PMCID: PMC10509305 DOI: 10.1002/mp.16378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 01/20/2023] [Accepted: 03/09/2023] [Indexed: 03/21/2023] Open
Abstract
PURPOSE Radiation damage on neurovascular bundles (NVBs) may be the cause of sexual dysfunction after radiotherapy for prostate cancer. However, it is challenging to delineate NVBs as organ-at-risks from planning CTs during radiotherapy. Recently, the integration of MR into radiotherapy made NVBs contour delineating possible. In this study, we aim to develop an MRI-based deep learning method for automatic NVB segmentation. METHODS The proposed method, named topological modulated network, consists of three subnetworks, that is, a focal modulation, a hierarchical block and a topological fully convolutional network (FCN). The focal modulation is used to derive the location and bounds of left and right NVBs', namely the candidate volume-of-interests (VOIs). The hierarchical block aims to highlight the NVB boundaries information on derived feature map. The topological FCN then segments the NVBs inside the VOIs by considering the topological consistency nature of the vascular delineating. Based on the location information of candidate VOIs, the segmentations of NVBs can then be brought back to the input MRI's coordinate system. RESULTS A five-fold cross-validation study was performed on 60 patient cases to evaluate the performance of the proposed method. The segmented results were compared with manual contours. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95 ) are (left NVB) 0.81 ± 0.10, 1.49 ± 0.88 mm, and (right NVB) 0.80 ± 0.15, 1.54 ± 1.22 mm, respectively. CONCLUSION We proposed a novel deep learning-based segmentation method for NVBs on pelvic MR images. The good segmentation agreement of our method with the manually drawn ground truth contours supports the feasibility of the proposed method, which can be potentially used to spare NVBs during proton and photon radiotherapy and thereby improve the quality of life for prostate cancer patients.
Collapse
Affiliation(s)
- Yang Lei
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Tonghe Wang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, USA
| | - Justin Roper
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Sibo Tian
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Pretesh Patel
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Jeffrey D Bradley
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Ashesh B Jani
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| | - Tian Liu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
- Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Xiaofeng Yang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, Georgia, USA
| |
Collapse
|
6
|
Zhou H, Xiao J, Ganesh S, Lerner A, Ruan D, Fan Z. VWI-APP: Vessel wall imaging-dedicated automated processing pipeline for intracranial atherosclerotic plaque quantification. Med Phys 2023; 50:1496-1506. [PMID: 36345580 PMCID: PMC10033308 DOI: 10.1002/mp.16074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Revised: 10/16/2022] [Accepted: 10/30/2022] [Indexed: 11/09/2022] Open
Abstract
BACKGROUND Quantitative plaque assessment based on 3D magnetic resonance (MR) vessel wall imaging (VWI) has been shown to provide valuable numerical markers of the burden and risk of intracranial atherosclerotic disease (ICAD). However, plaque quantification is currently time-consuming and observer-dependent due to the demand for heavy manual effort. A VWI-dedicated automated processing pipeline (VWI-APP) is desirable. PURPOSE To develop and evaluate a VWI-APP for end-to-end quantitative analysis of intracranial atherosclerotic plaque. METHODS We retrospectively enrolled 91 subjects with ICAD (80 for pipeline development, 10 for an end-to-end pipeline evaluation, and 1 for demonstrating longitudinal plaque assessment) who had undergone VWI and MR angiography. In an end-to-end evaluation, diameter stenosis (DS), normalized wall index (NWI), remodeling ratio (RR), plaque wall contrast ratio (CR), and total plaque volume (TPV) were quantified at each culprit lesion using the developed VWI-APP and a computer-aided manual approach by a neuroradiologist, respectively. The time consumed in each quantification approach was recorded. Two-sided paired t-tests and intraclass correlation coefficient (ICC) were used to determine the difference and agreement in each plaque metric between VWI-APP and manual quantification approaches. RESULTS There was no significant difference between VWI-APP and manual quantification in each plaque metric. The ICC was 0.890, 0.813, 0.827, 0.891, and 0.991 for DS, NWI, RR, CR, and TPV, respectively, suggesting good to excellent accuracy of the pipeline method in plaque quantification. Quantitative analysis of each culprit lesion on average took 675.7 s using the manual approach but shortened to 238.3 s with the aid of VWI-APP. CONCLUSIONS VWI-APP is an accurate and efficient approach to intracranial atherosclerotic plaque quantification. Further clinical assessment of this automated tool is warranted to establish its utility in the risk assessment of ICAD lesions.
Collapse
Affiliation(s)
- Hanyue Zhou
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Jiayu Xiao
- Department of Radiology, University of Southern California, Los Angeles, CA 90033, USA
| | - Siddarth Ganesh
- Department of Biomedical Engineering, University of Southern California, Los Angeles, CA 90089, USA
| | - Alexander Lerner
- Department of Radiology, University of Southern California, Los Angeles, CA 90033, USA
| | - Dan Ruan
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA 90095, USA
- Department of Radiation Oncology, University of California, Los Angeles, CA 90095, USA
| | - Zhaoyang Fan
- Department of Radiology, University of Southern California, Los Angeles, CA 90033, USA
- Department of Biomedical Engineering, University of Southern California, Los Angeles, CA 90089, USA
- Department of Radiation Oncology, University of Southern California, Los Angeles, CA 90033, USA
| |
Collapse
|
7
|
Zhou H, Xiao J, Li D, Fan Z, Ruan D. Intracranial vessel wall segmentation with deep learning using a novel tiered loss function incorporating class inclusion. Med Phys 2022; 49:6975-6985. [PMID: 35815927 PMCID: PMC9742123 DOI: 10.1002/mp.15860] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Revised: 05/30/2022] [Accepted: 06/30/2022] [Indexed: 01/01/2023] Open
Abstract
PURPOSE To develop an automated vessel wall segmentation method on T1-weighted intracranial vessel wall magnetic resonance images, with a focus on modeling the inclusion relation between the inner and outer boundaries of the vessel wall. METHODS We propose a novel method that estimates the inner and outer vessel wall boundaries simultaneously, using a network with a single output channel resembling the level-set function height. The network is driven by a unique tiered loss that accounts for data fidelity of the lumen and vessel wall classes and a length regularization to encourage boundary smoothness. RESULTS Implemented with a 2.5D UNet with a ResNet backbone, the proposed method achieved Dice similarity coefficients (DSC) in 2D of 0.925 ± 0.048, 0.786 ± 0.084, Hausdorff distance (HD) of 0.286 ± 0.436, 0.345 ± 0.419 mm, and mean surface distance (MSD) of 0.083 ± 0.037 and 0.103 ± 0.032 mm for the lumen and vessel wall, respectively, on a test set; compared favorably to a baseline UNet model that achieved DSC 0.924 ± 0.047, 0.794 ± 0.082, HD 0.298 ± 0.477, 0.394 ± 0.431 mm, and MSD 0.087 ± 0.056, 0.119 ± 0.059 mm. Our vessel wall segmentation method achieved substantial improvement in morphological integrity and accuracy compared to benchmark methods. CONCLUSIONS The proposed method provides a systematic approach to model the inclusion morphology and incorporate it into an optimization infrastructure. It can be applied to any application where inclusion exists among a (sub)set of classes to be segmented. Improved feasibility in result morphology promises better support for clinical quantification and decision.
Collapse
Affiliation(s)
- Hanyue Zhou
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, California, USA
| | - Jiayu Xiao
- Department of Radiology, University of Southern California, Los Angeles, California, USA
| | - Debiao Li
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, California, USA
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | - Zhaoyang Fan
- Department of Radiology, University of Southern California, Los Angeles, California, USA
- Department of Radiation Oncology, University of Southern California, Los Angeles, California, USA
- Department of Biomedical Engineering, University of Southern California, Los Angeles, California, USA
| | - Dan Ruan
- Department of Bioengineering, University of California, Los Angeles, Los Angeles, California, USA
- Department of Radiation Oncology, University of California, Los Angeles, California, USA
| |
Collapse
|
8
|
Sandali O, Tahiri JHR, Armia Balamoun A, Duliere C, El Sanharawi M, Borderie V. Use of Black-and-White Digital Filters to Optimize Visualization in Cataract Surgery. J Clin Med 2022; 11:4056. [PMID: 35887820 PMCID: PMC9316540 DOI: 10.3390/jcm11144056] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 07/06/2022] [Accepted: 07/11/2022] [Indexed: 02/05/2023] Open
Abstract
PURPOSE To evaluate the effect of a black-and-white (BW) filter on the optimization of visualization at each stage of cataract surgery. METHODS Prospective, single-center, single-surgeon, consecutive case series of 40 patients undergoing cataract surgery with BW filter. Surgical images and videos were recorded with and without the BW filter at each stage of cataract surgery. Contrast measurements of surgical images and subjective analysis of video sequences were performed. RESULTS The surgeons assessed the BW filter to optimize the tissue visibility of capsulorhexis contours, hydrodissection fluid wave perception, the contrast of instruments through a nucleus during phaco-chop, and subincisional cortex contrast through the corneal edema. Despite the higher contrasts' value obtained with BW filter images during nucleus removal, posterior capsular polishing and viscous removal, the surgeons subjectively reported no significant advantage of using a BW filter. Standard color images were found to be better for localizing the limbal area during incision and for nucleus sculpture to assess groove depth. CONCLUSIONS In conclusion, we describe here the potential indications for BW filter use at particular stages in cataract surgery. A BW filter could be used, with caution, in cases of poor visualization.
Collapse
Affiliation(s)
- Otman Sandali
- Centre Hospitalier National d’Ophtalmologie des XV-XX, Research Team 968, Institut de la Vision, Pierre & Marie Curie University Paris 06, 75012 Paris, France;
- Service de Chirurgie Ambulatoire, Hôpital Guillaume-de-Varye, 18230 Bourges, France
| | | | - Ashraf Armia Balamoun
- Watany Eye Hospital (WEH), Cairo 11775, Egypt;
- Watany Research and Development Centre, Cairo 11775, Egypt
- Ashraf Armia Eye Clinic, Giza 12655, Egypt
| | | | - Mohamed El Sanharawi
- Service d’Ophtalmologie, Centre Hospitalier de Châteaudun, 28200 Châteaudun, France;
| | - Vincent Borderie
- Centre Hospitalier National d’Ophtalmologie des XV-XX, Research Team 968, Institut de la Vision, Pierre & Marie Curie University Paris 06, 75012 Paris, France;
| |
Collapse
|
9
|
A Semantic Segmentation Method with Emphasis on the Edges for Automatic Vessel Wall Analysis. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12147012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
To develop a precise semantic segmentation method with an emphasis on the edges for automated segmentation of the arterial vessel wall and plaque based on the convolutional neural network (CNN) in order to facilitate the quantitative assessment of plaque in patients with ischemic stroke. A total of 124 subjects’ MR vessel wall images were used to train, validate, and test the model using deep learning. An end-to-end architecture network that can emphasize the edge information, namely the Edge Vessel Segmentation Network (EVSegNet) for automated segmentation of the arterial vessel wall, is proposed. The EVSegNet network consists of two workflows: one is implemented to achieve finely and multiscale segmentation by combining Dense Upsampling Convolution (DUC) and Hybrid Dilated Convolution (HDC) with different dilation rates modules, and the other utilizes edge information and is fused with another workflow to finally segment the vessel wall. The proposed network demonstrates robust segmentation of the vessel wall and better performance with a Dice (%) of 87.5, compared with the traditional U-net that has a Dice (%) of 81.0 and other U-net-based models on the test dataset. The results suggest that the proposed segmentation method with an emphasis on the edges improves segmentation accuracy effectively and will facilitate the quantitative assessment of atherosclerosis.
Collapse
|
10
|
Saba L, Antignani PL, Gupta A, Cau R, Paraskevas KI, Poredos P, Wasserman B, Kamel H, Avgerinos ED, Salgado R, Caobelli F, Aluigi L, Savastano L, Brown M, Hatsukami T, Hussein E, Suri JS, Mansilha A, Wintermark M, Staub D, Montequin JF, Rodriguez RTT, Balu N, Pitha J, Kooi ME, Lal BK, Spence JD, Lanzino G, Marcus HS, Mancini M, Chaturvedi S, Blinc A. International Union of Angiology (IUA) consensus paper on imaging strategies in atherosclerotic carotid artery imaging: From basic strategies to advanced approaches. Atherosclerosis 2022; 354:23-40. [DOI: 10.1016/j.atherosclerosis.2022.06.1014] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Revised: 06/10/2022] [Accepted: 06/14/2022] [Indexed: 12/24/2022]
|
11
|
Yu AC, Mohajer B, Eng J. External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. Radiol Artif Intell 2022; 4:e210064. [PMID: 35652114 DOI: 10.1148/ryai.210064] [Citation(s) in RCA: 94] [Impact Index Per Article: 47.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 03/09/2022] [Accepted: 04/12/2022] [Indexed: 01/17/2023]
Abstract
Purpose To assess generalizability of published deep learning (DL) algorithms for radiologic diagnosis. Materials and Methods In this systematic review, the PubMed database was searched for peer-reviewed studies of DL algorithms for image-based radiologic diagnosis that included external validation, published from January 1, 2015, through April 1, 2021. Studies using nonimaging features or incorporating non-DL methods for feature extraction or classification were excluded. Two reviewers independently evaluated studies for inclusion, and any discrepancies were resolved by consensus. Internal and external performance measures and pertinent study characteristics were extracted, and relationships among these data were examined using nonparametric statistics. Results Eighty-three studies reporting 86 algorithms were included. The vast majority (70 of 86, 81%) reported at least some decrease in external performance compared with internal performance, with nearly half (42 of 86, 49%) reporting at least a modest decrease (≥0.05 on the unit scale) and nearly a quarter (21 of 86, 24%) reporting a substantial decrease (≥0.10 on the unit scale). No study characteristics were found to be associated with the difference between internal and external performance. Conclusion Among published external validation studies of DL algorithms for image-based radiologic diagnosis, the vast majority demonstrated diminished algorithm performance on the external dataset, with some reporting a substantial performance decrease.Keywords: Meta-Analysis, Computer Applications-Detection/Diagnosis, Neural Networks, Computer Applications-General (Informatics), Epidemiology, Technology Assessment, Diagnosis, Informatics Supplemental material is available for this article. © RSNA, 2022.
Collapse
Affiliation(s)
- Alice C Yu
- Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 1800 Orleans St, Baltimore, MD 21287
| | - Bahram Mohajer
- Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 1800 Orleans St, Baltimore, MD 21287
| | - John Eng
- Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 1800 Orleans St, Baltimore, MD 21287
| |
Collapse
|
12
|
Gimnich OA, Zil-E-Ali A, Brunner G. Imaging Approaches to the Diagnosis of Vascular Diseases. Curr Atheroscler Rep 2022; 24:85-96. [PMID: 35080717 DOI: 10.1007/s11883-022-00988-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/13/2021] [Indexed: 12/12/2022]
Abstract
PURPOSE OF REVIEW Vascular imaging is a complex field including numerous modalities and imaging markers. This review is focused on important and recent findings in atherosclerotic carotid artery plaque imaging with an emphasis on developments in magnetic resonance imaging (MRI) and computed tomography (CT). RECENT FINDINGS Recent evidence shows that carotid plaque characteristics and not only established measures of carotid plaque burden and stenosis are associated independently with cardiovascular outcomes. On carotid MRI, the presence of a lipid-rich necrotic core (LRNC) has been associated with incident cardiovascular disease (CVD) events independent of wall thickness, a traditional measure of plaque burden. On carotid MRI, intraplaque hemorrhage (IPH) presence has been identified as an independent predictor of stroke. The presence of a fissured carotid fibrous cap has been associated with contrast enhancement on CT angiography imaging. Carotid artery plaque characteristics have been associated with incident CVD events, and advanced plaque imaging techniques may gain additional prominence in the clinical treatment decision process.
Collapse
Affiliation(s)
- Olga A Gimnich
- Penn State Heart and Vascular Institute, Pennsylvania State University College of Medicine, Hershey, PA, USA
| | - Ahsan Zil-E-Ali
- Penn State Heart and Vascular Institute, Pennsylvania State University College of Medicine, Hershey, PA, USA
| | - Gerd Brunner
- Penn State Heart and Vascular Institute, Pennsylvania State University College of Medicine, Hershey, PA, USA.
| |
Collapse
|
13
|
Zhou H, Xiao J, Fan Z, Ruan D. INTRACRANIAL VESSEL WALL SEGMENTATION FOR ATHEROSCLEROTIC PLAQUE QUANTIFICATION. PROCEEDINGS. IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING 2021; 2021:1416-1419. [PMID: 34405036 DOI: 10.1109/isbi48211.2021.9434018] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Intracranial vessel wall segmentation is critical for the quantitative assessment of intracranial atherosclerosis based on magnetic resonance vessel wall imaging. This work further improves on a previous 2D deep learning segmentation network by the utilization of 1) a 2.5D structure to balance network complexity and regularizing geometry continuity; 2) a UNET++ model to achieve structure adaptation; 3) an additional approximated Hausdorff distance (HD) loss into the objective to enhance geometry conformality; and 4) landing in a commonly used morphological measure of plaque burden - the normalized wall index (NWI) - to match the clinical endpoint. The modified network achieved Dice similarity coefficient of 0.9172 ± 0.0598 and 0.7833 ± 0.0867, HD of 0.3252 ± 0.5071 mm and 0.4914 ± 0.5743 mm, mean surface distance of 0.0940 ± 0.0781 mm and 0.1408 ± 0.0917 mm for the lumen and vessel wall, respectively. These results compare favorably to those obtained by the original 2D UNET on all segmentation metrics. Additionally, the proposed segmentation network reduced the mean absolute error in NWI from 0.0732 ± 0.0294 to 0.0725 ± 0.0333.
Collapse
Affiliation(s)
- Hanyue Zhou
- Department of Bioengineering, University of California, Los Angeles, CA 90095, US
| | - Jiayu Xiao
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, US
| | - Zhaoyang Fan
- Department of Bioengineering, University of California, Los Angeles, CA 90095, US.,Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, US.,Department of Radiology, University of Southern California, Los Angeles, CA 90033, US
| | - Dan Ruan
- Department of Bioengineering, University of California, Los Angeles, CA 90095, US.,Department of Radiation Oncology, University of California, Los Angeles, CA 90095, US
| |
Collapse
|
14
|
Jiang M, Zhao Y, Chiu B. Segmentation of common and internal carotid arteries from 3D ultrasound images based on adaptive triple loss. Med Phys 2021; 48:5096-5114. [PMID: 34309866 DOI: 10.1002/mp.15127] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2021] [Revised: 07/09/2021] [Accepted: 07/12/2021] [Indexed: 11/07/2022] Open
Abstract
PURPOSE Vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT) measured from three-dimensional (3D) ultrasound (US) carotid images are sensitive to anti-atherosclerotic effects of medical/dietary treatments. VWV and VWT measurements require the lumen-intima (LIB) and media-adventitia boundaries (MAB) at the common and internal carotid arteries (CCA and ICA). However, most existing segmentation techniques were capable of segmenting the CCA only. An approach capable of segmenting the MAB and LIB from the CCA and ICA was required to accelerate VWV and VWT quantification. METHODS Segmentation for CCA and ICA was performed independently using the proposed two-channel U-Net, which was driven by a novel loss function known as the adaptive triple Dice loss (ADTL) function. The training set was augmented by interpolating manual segmentation along the longitudinal direction, thereby taking continuity of the artery into account. A test-time augmentation (TTA) approach was applied, in which segmentation was performed three times based on the input axial images and its flipped versions; the final segmentation was generated by pixel-wise majority voting. RESULTS Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.1% ± 4.1% and 91.6% ± 6.6% for the MAB and LIB, in the CCA, respectively, and 94.2% ± 3.3% and 89.0% ± 8.1% for the MAB and LIB, in the ICA, respectively. TTA and ATDL independently contributed to a statistically significant improvement to all boundaries except the LIB in ICA. CONCLUSIONS The proposed two-channel U-Net with ADTL and TTA can segment the CCA and ICA accurately and efficiently from the 3DUS volume. Our approach has the potential to accelerate the transition of 3DUS measurements of carotid atherosclerosis to clinical research.
Collapse
Affiliation(s)
- Mingjie Jiang
- Department of Electrical Engineering, City University of Hong Kong, Hong Kong, Hong Kong
| | - Yuan Zhao
- Department of Electrical Engineering, City University of Hong Kong, Hong Kong, Hong Kong
| | - Bernard Chiu
- Department of Electrical Engineering, City University of Hong Kong, Hong Kong, Hong Kong
| |
Collapse
|
15
|
Saba L, Sanagala SS, Gupta SK, Koppula VK, Johri AM, Khanna NN, Mavrogeni S, Laird JR, Pareek G, Miner M, Sfikakis PP, Protogerou A, Misra DP, Agarwal V, Sharma AM, Viswanathan V, Rathore VS, Turk M, Kolluri R, Viskovic K, Cuadrado-Godia E, Kitas GD, Sharma N, Nicolaides A, Suri JS. Multimodality carotid plaque tissue characterization and classification in the artificial intelligence paradigm: a narrative review for stroke application. ANNALS OF TRANSLATIONAL MEDICINE 2021; 9:1206. [PMID: 34430647 PMCID: PMC8350643 DOI: 10.21037/atm-20-7676] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Accepted: 02/25/2021] [Indexed: 12/12/2022]
Abstract
Cardiovascular disease (CVD) is one of the leading causes of morbidity and mortality in the United States of America and globally. Carotid arterial plaque, a cause and also a marker of such CVD, can be detected by various non-invasive imaging modalities such as magnetic resonance imaging (MRI), computer tomography (CT), and ultrasound (US). Characterization and classification of carotid plaque-type in these imaging modalities, especially into symptomatic and asymptomatic plaque, helps in the planning of carotid endarterectomy or stenting. It can be challenging to characterize plaque components due to (I) partial volume effect in magnetic resonance imaging (MRI) or (II) varying Hausdorff values in plaque regions in CT, and (III) attenuation of echoes reflected by the plaque during US causing acoustic shadowing. Artificial intelligence (AI) methods have become an indispensable part of healthcare and their applications to the non-invasive imaging technologies such as MRI, CT, and the US. In this narrative review, three main types of AI models (machine learning, deep learning, and transfer learning) are analyzed when applied to MRI, CT, and the US. A link between carotid plaque characteristics and the risk of coronary artery disease is presented. With regard to characterization, we review tools and techniques that use AI models to distinguish carotid plaque types based on signal processing and feature strengths. We conclude that AI-based solutions offer an accurate and robust path for tissue characterization and classification for carotid artery plaque imaging in all three imaging modalities. Due to cost, user-friendliness, and clinical effectiveness, AI in the US has dominated the most.
Collapse
Affiliation(s)
- Luca Saba
- Department of Radiology, Azienda Ospedaliero Universitaria (AOU), Cagliari, Italy
| | - Skandha S Sanagala
- CSE Department, CMR College of Engineering & Technology, Hyderabad, India.,CSE Department, Bennett University, Greater Noida, UP, India
| | - Suneet K Gupta
- CSE Department, Bennett University, Greater Noida, UP, India
| | - Vijaya K Koppula
- CSE Department, CMR College of Engineering & Technology, Hyderabad, India
| | - Amer M Johri
- Department of Medicine, Division of Cardiology, Queen's University, Kingston, Ontario, Canada
| | - Narendra N Khanna
- Department of Cardiology, Indraprastha APOLLO Hospitals, New Delhi, India
| | - Sophie Mavrogeni
- Cardiology Clinic, Onassis Cardiac Surgery Center, Athens, Greece
| | - John R Laird
- Heart and Vascular Institute, Adventist Health St. Helena, St Helena, CA, USA
| | - Gyan Pareek
- Minimally Invasive Urology Institute, Brown University, Providence, Rhode Island, USA
| | - Martin Miner
- Men's Health Center, Miriam Hospital Providence, Rhode Island, USA
| | - Petros P Sfikakis
- Rheumatology Unit, National Kapodistrian University of Athens, Greece
| | - Athanasios Protogerou
- Department of Cardiovascular Prevention, National and Kapodistrian University of Athens, Athens, Greece
| | - Durga P Misra
- Department of Clinical Immunology and Rheumatology, SGPGIMS, Lucknow, India
| | - Vikas Agarwal
- Department of Clinical Immunology and Rheumatology, SGPGIMS, Lucknow, India
| | - Aditya M Sharma
- Division of Cardiovascular Medicine, University of Virginia, VA, USA
| | - Vijay Viswanathan
- MV Hospital for Diabetes & Professor M Viswanathan Diabetes Research Centre, Chennai, India
| | - Vijay S Rathore
- Nephrology Department, Kaiser Permanente, Sacramento, CA, USA
| | - Monika Turk
- The Hanse-Wissenschaftskolleg Institute for Advanced Study, Delmenhorst, Germany
| | | | | | | | - George D Kitas
- R & D Academic Affairs, Dudley Group NHS Foundation Trust, Dudley, UK
| | - Neeraj Sharma
- Department of Biomedical Engineering, IIT-BHU, Banaras, UP, India
| | - Andrew Nicolaides
- Vascular Screening and Diagnostic Centre, University of Nicosia, Nicosia, Cyprus
| | - Jasjit S Suri
- Stroke Monitoring and Diagnostic Division, AtheroPoint™, Roseville, CA, USA
| |
Collapse
|
16
|
Flores AM, Demsas F, Leeper NJ, Ross EG. Leveraging Machine Learning and Artificial Intelligence to Improve Peripheral Artery Disease Detection, Treatment, and Outcomes. Circ Res 2021; 128:1833-1850. [PMID: 34110911 PMCID: PMC8285054 DOI: 10.1161/circresaha.121.318224] [Citation(s) in RCA: 50] [Impact Index Per Article: 16.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Peripheral artery disease is an atherosclerotic disorder which, when present, portends poor patient outcomes. Low diagnosis rates perpetuate poor management, leading to limb loss and excess rates of cardiovascular morbidity and death. Machine learning algorithms and artificially intelligent systems have shown great promise in application to many areas in health care, such as accurately detecting disease, predicting patient outcomes, and automating image interpretation. Although the application of these technologies to peripheral artery disease are in their infancy, their promises are tremendous. In this review, we provide an introduction to important concepts in the fields of machine learning and artificial intelligence, detail the current state of how these technologies have been applied to peripheral artery disease, and discuss potential areas for future care enhancement with advanced analytics.
Collapse
Affiliation(s)
- Alyssa M Flores
- Department of Surgery, Division of Vascular Surgery (A.M.F., F.D., N.J.L., E.G.R.), Stanford University School of Medicine, CA
| | - Falen Demsas
- Department of Surgery, Division of Vascular Surgery (A.M.F., F.D., N.J.L., E.G.R.), Stanford University School of Medicine, CA
| | - Nicholas J Leeper
- Department of Surgery, Division of Vascular Surgery (A.M.F., F.D., N.J.L., E.G.R.), Stanford University School of Medicine, CA
- Department of Medicine, Division of Cardiovascular Medicine (N.J.L.), Stanford University School of Medicine, CA
- Stanford Cardiovascular Institute, CA (N.J.L., E.G.R.)
| | - Elsie Gyang Ross
- Department of Surgery, Division of Vascular Surgery (A.M.F., F.D., N.J.L., E.G.R.), Stanford University School of Medicine, CA
- Stanford Center for Biomedical Informatics Research, Department of Medicine, Stanford University, CA. (E.G.R.)
- Stanford Cardiovascular Institute, CA (N.J.L., E.G.R.)
| |
Collapse
|
17
|
Fu Y, Lei Y, Wang T, Curran WJ, Liu T, Yang X. A review of deep learning based methods for medical image multi-organ segmentation. Phys Med 2021; 85:107-122. [PMID: 33992856 PMCID: PMC8217246 DOI: 10.1016/j.ejmp.2021.05.003] [Citation(s) in RCA: 73] [Impact Index Per Article: 24.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Revised: 03/12/2021] [Accepted: 05/03/2021] [Indexed: 12/12/2022] Open
Abstract
Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.
Collapse
Affiliation(s)
- Yabo Fu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Yang Lei
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Tonghe Wang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Walter J Curran
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Tian Liu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Xiaofeng Yang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA.
| |
Collapse
|
18
|
Chen L, Zhao H, Jiang H, Balu N, Geleri DB, Chu B, Watase H, Zhao X, Li R, Xu J, Hatsukami TS, Xu D, Hwang JN, Yuan C. Domain adaptive and fully automated carotid artery atherosclerotic lesion detection using an artificial intelligence approach (LATTE) on 3D MRI. Magn Reson Med 2021; 86:1662-1673. [PMID: 33885165 DOI: 10.1002/mrm.28794] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 03/07/2021] [Accepted: 03/18/2021] [Indexed: 01/17/2023]
Abstract
PURPOSE To develop and evaluate a domain adaptive and fully automated review workflow (lesion assessment through tracklet evaluation, LATTE) for assessment of atherosclerotic disease in 3D carotid MR vessel wall imaging (MR VWI). METHODS VWI of 279 subjects with carotid atherosclerosis were used to develop LATTE, mainly convolutional neural network (CNN)-based domain adaptive lesion classification after image quality assessment and artery of interest localization. Heterogeneity in test sets from various sites usually causes inferior CNN performance. With our novel unsupervised domain adaptation (DA), LATTE was designed to accurately classify arteries into normal arteries and early and advanced lesions without additional annotations on new datasets. VWI of 271 subjects from four datasets (eight sites) with slightly different imaging parameters/signal patterns were collected to assess the effectiveness of DA of LATTE using the area under the receiver operating characteristic curve (AUC) on all lesions and advanced lesions before and after DA. RESULTS LATTE had good performance with advanced/all lesion classification, with the AUC of >0.88/0.83, significant improvements from >0.82/0.80 if without DA. CONCLUSIONS LATTE can locate target arteries and distinguish carotid atherosclerotic lesions with consistently improved performance with DA on new datasets. It may be useful for carotid atherosclerosis detection and assessment on various clinical sites.
Collapse
Affiliation(s)
- Li Chen
- Department of Electrical and Computer Engineering, University of Washington, Seattle, Washington, USA
| | - Huilin Zhao
- Renji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Hongjian Jiang
- Department of Electrical and Computer Engineering, University of Washington, Seattle, Washington, USA
| | - Niranjan Balu
- Department of Radiology, University of Washington, Seattle, Washington, USA
| | | | - Baocheng Chu
- Department of Radiology, University of Washington, Seattle, Washington, USA
| | - Hiroko Watase
- Department of Surgery, University of Washington, Seattle, Washington, USA
| | - Xihai Zhao
- Department of Biomedical Engineering, Tsinghua University School of Medicine, Beijing, China
| | - Rui Li
- Department of Biomedical Engineering, Tsinghua University School of Medicine, Beijing, China
| | - Jianrong Xu
- Renji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Thomas S Hatsukami
- Department of Surgery, University of Washington, Seattle, Washington, USA
| | - Dongxiang Xu
- Department of Radiology, University of Washington, Seattle, Washington, USA
| | - Jenq-Neng Hwang
- Department of Electrical and Computer Engineering, University of Washington, Seattle, Washington, USA
| | - Chun Yuan
- Department of Radiology, University of Washington, Seattle, Washington, USA
| |
Collapse
|
19
|
Machine Learning Quantitation of Cardiovascular and Cerebrovascular Disease: A Systematic Review of Clinical Applications. Diagnostics (Basel) 2021; 11:diagnostics11030551. [PMID: 33808677 PMCID: PMC8003459 DOI: 10.3390/diagnostics11030551] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2021] [Revised: 03/12/2021] [Accepted: 03/16/2021] [Indexed: 01/10/2023] Open
Abstract
Research into machine learning (ML) for clinical vascular analysis, such as those useful for stroke and coronary artery disease, varies greatly between imaging modalities and vascular regions. Limited accessibility to large diverse patient imaging datasets, as well as a lack of transparency in specific methods, are obstacles to further development. This paper reviews the current status of quantitative vascular ML, identifying advantages and disadvantages common to all imaging modalities. Literature from the past 8 years was systematically collected from MEDLINE® and Scopus database searches in January 2021. Papers satisfying all search criteria, including a minimum of 50 patients, were further analysed and extracted of relevant data, for a total of 47 publications. Current ML image segmentation, disease risk prediction, and pathology quantitation methods have shown sensitivities and specificities over 70%, compared to expert manual analysis or invasive quantitation. Despite this, inconsistencies in methodology and the reporting of results have prevented inter-model comparison, impeding the identification of approaches with the greatest potential. The clinical potential of this technology has been well demonstrated in Computed Tomography of coronary artery disease, but remains practically limited in other modalities and body regions, particularly due to a lack of routine invasive reference measurements and patient datasets.
Collapse
|
20
|
Zhang L, Zhang J, Li Z, Song Y. A multiple-channel and atrous convolution network for ultrasound image segmentation. Med Phys 2020; 47:6270-6285. [PMID: 33007105 DOI: 10.1002/mp.14512] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 09/12/2020] [Accepted: 09/22/2020] [Indexed: 11/08/2022] Open
Abstract
PURPOSE Ultrasound image segmentation is a challenging task due to a low signal-to-noise ratio and poor image quality. Although several approaches based on the convolutional neural network (CNN) have been applied to ultrasound image segmentation, they have weak generalization ability. We propose an end-to-end, multiple-channel and atrous CNN designed to extract a greater amount of semantic information for segmentation of ultrasound images. METHOD A multiple-channel and atrous convolution network is developed, referred to as MA-Net. Similar to U-Net, MA-Net is based on an encoder-decoder architecture and includes five modules: the encoder, atrous convolution, pyramid pooling, decoder, and residual skip pathway modules. In the encoder module, we aim to capture more information with multiple-channel convolution and use large kernel convolution instead of small filters in each convolution operation. In the last layer, atrous convolution and pyramid pooling are used to extract multi-scale features. The architecture of the decoder is similar to that of the encoder module, except that up-sampling is used instead of down-sampling. Furthermore, the residual skip pathway module connects the subnetworks of the encoder and decoder to optimize learning from the deeper layer and improve the accuracy of segmentation. During the learning process, we adopt multi-task learning to enhance segmentation performance. Five types of datasets are used in our experiments. Because the original training data are limited, we apply data augmentation (e.g., horizontal and vertical flipping, random rotations, and random scaling) to our training data. We use the Dice score, precision, recall, Hausdorff distance (HD), average symmetric surface distance (ASD), and root mean square symmetric surface distance (RMSD) as the metrics for segmentation evaluation. Meanwhile, Friedman test was performed as the nonparametric statistical analysis to evaluate the algorithms. RESULTS For the datasets of brachia plexus (BP), fetal head, and lymph node segmentations, MA-Net achieved average Dice scores of 0.776, 0.973, and 0.858, respectively; with average precisions of 0.787, 0.968, and 0.854, respectively; average recalls of 0.788, 0.978, and 0.885, respectively; average HDs (mm) of 13.591, 10.924, and 19.245, respectively; average ASDs (mm) of 4.822, 4.152, and 4.312, respectively; and average RMSDs (mm) of 4.979, 4.161, and 4.930, respectively. Compared with U-Net, U-Net++, M-Net, and Dilated U-Net, the average performance of the MA-Net increased by approximately 5.68%, 2.85%, 6.59%, 36.03%, 23.64%, and 31.71% for Dice, precision, recall, HD, ASD, and RMSD, respectively. Moreover, we verified the generalization of MA-Net segmentation to lower grade brain glioma MRI and lung CT images. In addition, the MA-Net achieved the highest mean rank in the Friedman test. CONCLUSION The proposed MA-Net accurately segments ultrasound images with high generalization, and therefore, it offers a useful tool for diagnostic application in ultrasound images.
Collapse
Affiliation(s)
- Lun Zhang
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China.,Yunnan Vocational Institute of Energy Technology, Qujing, Yunnan, 655001, China
| | - Junhua Zhang
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China
| | - Zonggui Li
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China
| | - Yingchao Song
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China
| |
Collapse
|
21
|
Rapid vessel segmentation and reconstruction of head and neck angiograms using 3D convolutional neural network. Nat Commun 2020; 11:4829. [PMID: 32973154 PMCID: PMC7518426 DOI: 10.1038/s41467-020-18606-2] [Citation(s) in RCA: 43] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2020] [Accepted: 08/30/2020] [Indexed: 11/24/2022] Open
Abstract
The computed tomography angiography (CTA) postprocessing manually recognized by technologists is extremely labor intensive and error prone. We propose an artificial intelligence reconstruction system supported by an optimized physiological anatomical-based 3D convolutional neural network that can automatically achieve CTA reconstruction in healthcare services. This system is trained and tested with 18,766 head and neck CTA scans from 5 tertiary hospitals in China collected between June 2017 and November 2018. The overall reconstruction accuracy of the independent testing dataset is 0.931. It is clinically applicable due to its consistency with manually processed images, which achieves a qualification rate of 92.1%. This system reduces the time consumed from 14.22 ± 3.64 min to 4.94 ± 0.36 min, the number of clicks from 115.87 ± 25.9 to 4 and the labor force from 3 to 1 technologist after five months application. Thus, the system facilitates clinical workflows and provides an opportunity for clinical technologists to improve humanistic patient care. Manual postprocessing of computed tomography angiography (CTA) images is extremely labor intensive and error prone. Here, the authors propose an artificial intelligence reconstruction system that can automatically achieve CTA reconstruction in healthcare services.
Collapse
|
22
|
Dai X, Lei Y, Zhang Y, Qiu RLJ, Wang T, Dresser SA, Curran WJ, Patel P, Liu T, Yang X. Automatic multi-catheter detection using deeply supervised convolutional neural network in MRI-guided HDR prostate brachytherapy. Med Phys 2020; 47:4115-4124. [PMID: 32484573 DOI: 10.1002/mp.14307] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Revised: 05/19/2020] [Accepted: 05/24/2020] [Indexed: 12/19/2022] Open
Abstract
PURPOSE High-dose-rate (HDR) brachytherapy is an established technique to be used as monotherapy option or focal boost in conjunction with external beam radiation therapy (EBRT) for treating prostate cancer. Radiation source path reconstruction is a critical procedure in HDR treatment planning. Manually identifying the source path is labor intensive and time inefficient. In recent years, magnetic resonance imaging (MRI) has become a valuable imaging modality for image-guided HDR prostate brachytherapy due to its superb soft-tissue contrast for target delineation and normal tissue contouring. The purpose of this study is to investigate a deep-learning-based method to automatically reconstruct multiple catheters in MRI for prostate cancer HDR brachytherapy treatment planning. METHODS Attention gated U-Net incorporated with total variation (TV) regularization model was developed for multi-catheter segmentation in MRI. The attention gates were used to improve the accuracy of identifying small catheter points, while TV regularization was adopted to encode the natural spatial continuity of catheters into the model. The model was trained using the binary catheter annotation images offered by experienced physicists as ground truth paired with original MRI images. After the network was trained, MR images of a new prostate cancer patient receiving HDR brachytherapy were fed into the model to predict the locations and shapes of all the catheters. Quantitative assessments of our proposed method were based on catheter shaft and tip errors compared to the ground truth. RESULTS Our method detected 299 catheters from 20 patients receiving HDR prostate brachytherapy with a catheter tip error of 0.37 ± 1.68 mm and a catheter shaft error of 0.93 ± 0.50 mm. For detection of catheter tips, our method resulted in 87% of the catheter tips within an error of less than ± 2.0 mm, and more than 71% of the tips can be localized within an absolute error of no >1.0 mm. For catheter shaft localization, 97% of catheters were detected with an error of <2.0 mm, while 63% were within 1.0 mm. CONCLUSIONS In this study, we proposed a novel multi-catheter detection method to precisely localize the tips and shafts of catheters in three-dimensional MRI images of HDR prostate brachytherapy. It paves the way for elevating the quality and outcome of MRI-guided HDR prostate brachytherapy.
Collapse
Affiliation(s)
- Xianjin Dai
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Yang Lei
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Yupei Zhang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Richard L J Qiu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Tonghe Wang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Sean A Dresser
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Walter J Curran
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Pretesh Patel
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Tian Liu
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| | - Xiaofeng Yang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, 30332, USA
| |
Collapse
|
23
|
Erratum: “Deep Morphology Aided Diagnosis Network for Segmentation of Carotid Artery Vessel Wall and Diagnosis of Carotid Atherosclerosis on Black‐Blood Vessel Wall MRI” [Med. Phys. 46, 5544–5561 (2019)]. Med Phys 2020; 47:2575. [DOI: 10.1002/mp.14092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
|
24
|
Jin H, Geng J, Yin Y, Hu M, Yang G, Xiang S, Zhai X, Ji Z, Fan X, Hu P, He C, Qin L, Zhang H. Fully automated intracranial aneurysm detection and segmentation from digital subtraction angiography series using an end-to-end spatiotemporal deep neural network. J Neurointerv Surg 2020; 12:1023-1027. [DOI: 10.1136/neurintsurg-2020-015824] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2020] [Revised: 03/10/2020] [Accepted: 03/16/2020] [Indexed: 12/29/2022]
Abstract
BackgroundIntracranial aneurysms (IAs) are common in the population and may cause death.ObjectiveTo develop a new fully automated detection and segmentation deep neural network based framework to assist neurologists in evaluating and contouring intracranial aneurysms from 2D+time digital subtraction angiography (DSA) sequences during diagnosis.MethodsThe network structure is based on a general U-shaped design for medical image segmentation and detection. The network includes a fully convolutional technique to detect aneurysms in high-resolution DSA frames. In addition, a bidirectional convolutional long short-term memory module is introduced at each level of the network to capture the change in contrast medium flow across the 2D DSA frames. The resulting network incorporates both spatial and temporal information from DSA sequences and can be trained end-to-end. Furthermore, deep supervision was implemented to help the network converge. The proposed network structure was trained with 2269 DSA sequences from 347 patients with IAs. After that, the system was evaluated on a blind test set with 947 DSA sequences from 146 patients.ResultsOf the 354 aneurysms, 316 (89.3%) were successfully detected, corresponding to a patient level sensitivity of 97.7% at an average false positive number of 3.77 per sequence. The system runs for less than one second per sequence with an average dice coefficient score of 0.533.ConclusionsThis deep neural network assists in successfully detecting and segmenting aneurysms from 2D DSA sequences, and can be used in clinical practice.
Collapse
|