1
|
Kv R, Prasad K, Peralam Yegneswaran P. Segmentation and Classification Approaches of Clinically Relevant Curvilinear Structures: A Review. J Med Syst 2023; 47:40. [PMID: 36971852 PMCID: PMC10042761 DOI: 10.1007/s10916-023-01927-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 02/25/2023] [Indexed: 03/29/2023]
Abstract
Detection of curvilinear structures from microscopic images, which help the clinicians to make an unambiguous diagnosis is assuming paramount importance in recent clinical practice. Appearance and size of dermatophytic hyphae, keratitic fungi, corneal and retinal vessels vary widely making their automated detection cumbersome. Automated deep learning methods, endowed with superior self-learning capacity, have superseded the traditional machine learning methods, especially in complex images with challenging background. Automatic feature learning ability using large input data with better generalization and recognition capability, but devoid of human interference and excessive pre-processing, is highly beneficial in the above context. Varied attempts have been made by researchers to overcome challenges such as thin vessels, bifurcations and obstructive lesions in retinal vessel detection as revealed through several publications reviewed here. Revelations of diabetic neuropathic complications such as tortuosity, changes in the density and angles of the corneal fibers have been successfully sorted in many publications reviewed here. Since artifacts complicate the images and affect the quality of analysis, methods addressing these challenges have been described. Traditional and deep learning methods, that have been adapted and published between 2015 and 2021 covering retinal vessels, corneal nerves and filamentous fungi have been summarized in this review. We find several novel and meritorious ideas and techniques being put to use in the case of retinal vessel segmentation and classification, which by way of cross-domain adaptation can be utilized in the case of corneal and filamentous fungi also, making suitable adaptations to the challenges to be addressed.
Collapse
Affiliation(s)
- Rajitha Kv
- Department of Biomedical Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, 576104, Karnataka, India
| | - Keerthana Prasad
- Manipal School of Information Sciences, Manipal Academy of Higher Education, Manipal, 576104, Karnataka, India.
| | - Prakash Peralam Yegneswaran
- Department of Microbiology, Kasturba Medical College, Manipal Academy of Higher Education, Manipal, 576104, Karnataka, India
| |
Collapse
|
2
|
Su J, Li S, Wolff L, van Zwam W, Niessen WJ, van der Lugt A, van Walsum T. Deep reinforcement learning for cerebral anterior vessel tree extraction from 3D CTA images. Med Image Anal 2023; 84:102724. [PMID: 36525842 DOI: 10.1016/j.media.2022.102724] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 11/24/2022] [Accepted: 12/02/2022] [Indexed: 12/13/2022]
Abstract
Extracting the cerebral anterior vessel tree of patients with an intracranial large vessel occlusion (LVO) is relevant to investigate potential biomarkers that can contribute to treatment decision making. The purpose of our work is to develop a method that can achieve this from routinely acquired computed tomography angiography (CTA) and computed tomography perfusion (CTP) images. To this end, we regard the anterior vessel tree as a set of bifurcations and connected centerlines. The method consists of a proximal policy optimization (PPO) based deep reinforcement learning (DRL) approach for tracking centerlines, a convolutional neural network based bifurcation detector, and a breadth-first vessel tree construction approach taking the tracking and bifurcation detection results as input. We experimentally determine the added values of various components of the tracker. Both DRL vessel tracking and CNN bifurcation detection were assessed in a cross validation experiment using 115 subjects. The anterior vessel tree formation was evaluated on an independent test set of 25 subjects, and compared to interobserver variation on a small subset of images. The DRL tracking result achieves a median overlapping rate until the first error (1.8 mm off the reference standard) of 100, [46, 100] % on 8032 vessels over 115 subjects. The bifurcation detector reaches an average recall and precision of 76% and 87% respectively during the vessel tree formation process. The final vessel tree formation achieves a median recall of 68% and precision of 70%, which is in line with the interobserver agreement.
Collapse
Affiliation(s)
- Jiahang Su
- Department of Radiology & Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands.
| | - Shuai Li
- Department of Radiology & Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands
| | - Lennard Wolff
- Department of Radiology & Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands
| | - Wim van Zwam
- Department of Radiology & Nuclear Medicine, Maastricht UMC, Cardiovascular Research Institute Maastricht, The Netherlands
| | - Wiro J Niessen
- Department of Radiology & Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands; Faculty of Applied Sciences, Delft University of Technology, The Netherlands
| | - Aad van der Lugt
- Department of Radiology & Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands
| | - Theo van Walsum
- Department of Radiology & Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands
| |
Collapse
|
3
|
Liu W, Yang H, Tian T, Cao Z, Pan X, Xu W, Jin Y, Gao F. Full-Resolution Network and Dual-Threshold Iteration for Retinal Vessel and Coronary Angiograph Segmentation. IEEE J Biomed Health Inform 2022; 26:4623-4634. [PMID: 35788455 DOI: 10.1109/jbhi.2022.3188710] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Vessel segmentation is critical for disease diagnosis and surgical planning. Recently, the vessel segmentation method based on deep learning has achieved outstanding performance. However, vessel segmentation remains challenging due to thin vessels with low contrast that easily lose spatial information in the traditional U-shaped segmentation network. To alleviate this problem, we propose a novel and straightforward full-resolution network (FR-UNet) that expands horizontally and vertically through a multiresolution convolution interactive mechanism while retaining full image resolution. In FR-UNet, the feature aggregation module integrates multiscale feature maps from adjacent stages to supplement high-level contextual information. The modified residual blocks continuously learn multiresolution representations to obtain a pixel-level accuracy prediction map. Moreover, we propose the dual-threshold iterative algorithm (DTI) to extract weak vessel pixels for improving vessel connectivity. The proposed method was evaluated on retinal vessel datasets (DRIVE, CHASE_DB1, and STARE) and coronary angiography datasets (DCA1 and CHUAC). The results demonstrate that FR-UNet outperforms state-of-the-art methods by achieving the highest Sen, AUC, F1, and IOU on most of the above-mentioned datasets with fewer parameters, and that DTI enhances vessel connectivity while greatly improving sensitivity. The code is available at: https://github.com/lseventeen/FR-UNet.
Collapse
|
4
|
Jeon B. Deep Recursive Bayesian Tracking for Fully Automatic Centerline Extraction of Coronary Arteries in CT Images. SENSORS (BASEL, SWITZERLAND) 2021; 21:6087. [PMID: 34577293 PMCID: PMC8471768 DOI: 10.3390/s21186087] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Revised: 09/07/2021] [Accepted: 09/08/2021] [Indexed: 11/17/2022]
Abstract
Extraction of coronary arteries in coronary computed tomography (CT) angiography is a prerequisite for the quantification of coronary lesions. In this study, we propose a tracking method combining a deep convolutional neural network (DNN) and particle filtering method to identify the trajectories from the coronary ostium to each distal end from 3D CT images. The particle filter, as a non-linear approximator, is an appropriate tracking framework for such thin and elongated structures; however, the robust 'vesselness' measurement is essential for extracting coronary centerlines. Importantly, we employed the DNN to robustly measure the vesselness using patch images, and we integrated softmax values to the likelihood function in our particle filtering framework. Tangent patches represent cross-sections of coronary arteries of circular shapes. Thus, 2D tangent patches are assumed to include enough features of coronary arteries, and the use of 2D patches significantly reduces computational complexity. Because coronary vasculature has multiple bifurcations, we also modeled a method to detect branching sites by clustering the particle locations. The proposed method is compared with three commercial workstations and two conventional methods from the academic literature.
Collapse
Affiliation(s)
- Byunghwan Jeon
- School of Computer Science, Kyungil University, Gyeongsan 38428, Korea
| |
Collapse
|
5
|
Jia D, Zhuang X. Learning-based algorithms for vessel tracking: A review. Comput Med Imaging Graph 2021; 89:101840. [PMID: 33548822 DOI: 10.1016/j.compmedimag.2020.101840] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Revised: 10/07/2020] [Accepted: 12/03/2020] [Indexed: 11/24/2022]
Abstract
Developing efficient vessel-tracking algorithms is crucial for imaging-based diagnosis and treatment of vascular diseases. Vessel tracking aims to solve recognition problems such as key (seed) point detection, centerline extraction, and vascular segmentation. Extensive image-processing techniques have been developed to overcome the problems of vessel tracking that are mainly attributed to the complex morphologies of vessels and image characteristics of angiography. This paper presents a literature review on vessel-tracking methods, focusing on machine-learning-based methods. First, the conventional machine-learning-based algorithms are reviewed, and then, a general survey of deep-learning-based frameworks is provided. On the basis of the reviewed methods, the evaluation issues are introduced. The paper is concluded with discussions about the remaining exigencies and future research.
Collapse
Affiliation(s)
- Dengqiang Jia
- School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Xiahai Zhuang
- School of Data Science, Fudan University, Shanghai, China.
| |
Collapse
|
6
|
Mookiah MRK, Hogg S, MacGillivray TJ, Prathiba V, Pradeepa R, Mohan V, Anjana RM, Doney AS, Palmer CNA, Trucco E. A review of machine learning methods for retinal blood vessel segmentation and artery/vein classification. Med Image Anal 2020; 68:101905. [PMID: 33385700 DOI: 10.1016/j.media.2020.101905] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2019] [Revised: 11/10/2020] [Accepted: 11/11/2020] [Indexed: 12/20/2022]
Abstract
The eye affords a unique opportunity to inspect a rich part of the human microvasculature non-invasively via retinal imaging. Retinal blood vessel segmentation and classification are prime steps for the diagnosis and risk assessment of microvascular and systemic diseases. A high volume of techniques based on deep learning have been published in recent years. In this context, we review 158 papers published between 2012 and 2020, focussing on methods based on machine and deep learning (DL) for automatic vessel segmentation and classification for fundus camera images. We divide the methods into various classes by task (segmentation or artery-vein classification), technique (supervised or unsupervised, deep and non-deep learning, hand-crafted methods) and more specific algorithms (e.g. multiscale, morphology). We discuss advantages and limitations, and include tables summarising results at-a-glance. Finally, we attempt to assess the quantitative merit of DL methods in terms of accuracy improvement compared to other methods. The results allow us to offer our views on the outlook for vessel segmentation and classification for fundus camera images.
Collapse
Affiliation(s)
| | - Stephen Hogg
- VAMPIRE project, Computing (SSEN), University of Dundee, Dundee DD1 4HN, UK
| | - Tom J MacGillivray
- VAMPIRE project, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh EH16 4SB, UK
| | - Vijayaraghavan Prathiba
- Madras Diabetes Research Foundation and Dr. Mohan's Diabetes Specialities Centre, Gopalapuram, Chennai 600086, India
| | - Rajendra Pradeepa
- Madras Diabetes Research Foundation and Dr. Mohan's Diabetes Specialities Centre, Gopalapuram, Chennai 600086, India
| | - Viswanathan Mohan
- Madras Diabetes Research Foundation and Dr. Mohan's Diabetes Specialities Centre, Gopalapuram, Chennai 600086, India
| | - Ranjit Mohan Anjana
- Madras Diabetes Research Foundation and Dr. Mohan's Diabetes Specialities Centre, Gopalapuram, Chennai 600086, India
| | - Alexander S Doney
- Division of Population Health and Genomics, Ninewells Hospital and Medical School, University of Dundee, Dundee, DD1 9SY, UK
| | - Colin N A Palmer
- Division of Population Health and Genomics, Ninewells Hospital and Medical School, University of Dundee, Dundee, DD1 9SY, UK
| | - Emanuele Trucco
- VAMPIRE project, Computing (SSEN), University of Dundee, Dundee DD1 4HN, UK
| |
Collapse
|
7
|
Shoba SG, Therese AB. Detection of glaucoma disease in fundus images based on morphological operation and finite element method. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.101986] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
8
|
Hervella ÁS, Rouco J, Novo J, Penedo MG, Ortega M. Deep multi-instance heatmap regression for the detection of retinal vessel crossings and bifurcations in eye fundus images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 186:105201. [PMID: 31783244 DOI: 10.1016/j.cmpb.2019.105201] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2019] [Revised: 10/09/2019] [Accepted: 11/11/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVES The analysis of the retinal vasculature plays an important role in the diagnosis of many ocular and systemic diseases. In this context, the accurate detection of the vessel crossings and bifurcations is an important requirement for the automated extraction of relevant biomarkers. In that regard, we propose a novel approach that addresses the simultaneous detection of vessel crossings and bifurcations in eye fundus images. METHOD We propose to formulate the detection of vessel crossings and bifurcations in eye fundus images as a multi-instance heatmap regression. In particular, a deep neural network is trained in the prediction of multi-instance heatmaps that model the likelihood of a pixel being a landmark location. This novel approach allows to make predictions using full images and integrates into a single step the detection and distinction of the vascular landmarks. RESULTS The proposed method is validated on two public datasets of reference that include detailed annotations for vessel crossings and bifurcations in eye fundus images. The conducted experiments evidence that the proposed method offers a satisfactory performance. In particular, the proposed method achieves 74.23% and 70.90% F-score for the detection of crossings and bifurcations, respectively, in color fundus images. Furthermore, the proposed method outperforms previous works by a significant margin. CONCLUSIONS The proposed multi-instance heatmap regression allows to successfully exploit the potential of modern deep learning algorithms for the simultaneous detection of retinal vessel crossings and bifurcations. Consequently, this results in a significant improvement over previous methods, which will further facilitate the automated analysis of the retinal vasculature in many pathological conditions.
Collapse
Affiliation(s)
- Álvaro S Hervella
- CITIC-Research Center of Information and Communication Technologies, Universidade da Coruña, A Coruña, Spain; Department of Computer Science, Universidade da Coruña, A Coruña, Spain.
| | - José Rouco
- CITIC-Research Center of Information and Communication Technologies, Universidade da Coruña, A Coruña, Spain; Department of Computer Science, Universidade da Coruña, A Coruña, Spain
| | - Jorge Novo
- CITIC-Research Center of Information and Communication Technologies, Universidade da Coruña, A Coruña, Spain; Department of Computer Science, Universidade da Coruña, A Coruña, Spain
| | - Manuel G Penedo
- CITIC-Research Center of Information and Communication Technologies, Universidade da Coruña, A Coruña, Spain; Department of Computer Science, Universidade da Coruña, A Coruña, Spain
| | - Marcos Ortega
- CITIC-Research Center of Information and Communication Technologies, Universidade da Coruña, A Coruña, Spain; Department of Computer Science, Universidade da Coruña, A Coruña, Spain
| |
Collapse
|
9
|
Zhao H, Sun Y, Li H. Retinal vascular junction detection and classification via deep neural networks. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 183:105096. [PMID: 31586789 DOI: 10.1016/j.cmpb.2019.105096] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2019] [Revised: 09/09/2019] [Accepted: 09/25/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVES The retinal fundus contains intricate vascular trees, some of which are mutually intersected and overlapped. The intersection and overlapping of retinal vessels represent vascular junctions (i.e. bifurcation and crossover) in 2D retinal images. These junctions are important for analyzing vascular diseases and tracking the morphology of vessels. In this paper, we propose a two-stage pipeline to detect and classify the junction points. METHODS In the detection stage, a RCNN-based Junction Proposal Network is utilized to search the potential bifurcation and crossover locations directly on color retinal images, which is followed by a Junction Refinement Network to eliminate the false detections. In the classification stage, the detected junction points are identified as crossover or bifurcation using the proposed Junction Classification Network that shares the same model structure with the refinement network. RESULTS Our approach achieves 70% and 60% F1-score on DRIVE and IOSTAR dataset respectively which outperform the state-of-the-art methods by 4.5% and 1.7%, with a high and balanced precision and recall values. CONCLUSIONS This paper proposes a new junction detection and classification method which performs directly on color retinal images without any vessel segmentation nor skeleton preprocessing. The superior performance demonstrates that the effectiveness of our approach.
Collapse
Affiliation(s)
- He Zhao
- School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
| | - Yun Sun
- School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
| | - Huiqi Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China.
| |
Collapse
|
10
|
Nazir T, Irtaza A, Shabbir Z, Javed A, Akram U, Mahmood MT. Diabetic retinopathy detection through novel tetragonal local octa patterns and extreme learning machines. Artif Intell Med 2019; 99:101695. [PMID: 31606114 DOI: 10.1016/j.artmed.2019.07.003] [Citation(s) in RCA: 39] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2018] [Revised: 01/17/2019] [Accepted: 07/15/2019] [Indexed: 12/13/2022]
Abstract
Diabetic retinopathy (DR) is an eye disease that victimize the people suffering from diabetes from many years. The severe form of DR results in form of the blindness that can initially be controlled by the DR-screening oriented treatment. The effective screening programs require the trained human resource that manually grade the fundus images to understand the severity of the disease. But due to the complexity of this process, and the insufficient number of the trained workers, the precise manual grading is an expensive process. The CAD-based solutions try to address these limitations but most of the existing DR detection systems are as evaluated over small sets and become ineffective when applied in real scenarios. Therefore, in this paper we proposed a novel technique to precisely detect the various stages of the DR by extending the research of the content-based image retrieval domain. To achieve the human-level performance over the large-scale DR-datasets (i.e. Kaggle-DR), the fundus images are represented by the novel tetragonal local octa pattern (T-LOP) features, that are then classified through the extreme learning machine (ELM). To justify the significance of the method, the proposed scheme is compared against several state-of-the-art methods including the deep learning-based methods over four DR-datasets of variational lengths (i.e. Kaggle-DR, DRIVE, Review-DB, STARE). The experimental results confirm the significance of the DR-detection scheme to serve as a stand-alone solution for providing the precise information of the severity of the DR in an efficient manner.
Collapse
Affiliation(s)
- Tahira Nazir
- Department of Computer Science, University of Engineering and Technology Taxila, Pakistan
| | - Aun Irtaza
- Department of Computer Science, University of Engineering and Technology Taxila, Pakistan
| | - Zain Shabbir
- Department of Electrical, Electronics and Communication Engineering, University of Engineering and Technology, Lahore (Faisalabad Campus), Pakistan
| | - Ali Javed
- Department of Software Engineering, University of Engineering and Technology Taxila, Pakistan
| | - Usman Akram
- Department of Computer and Software Engineering, National University of Science and Technology, Islamabad, Pakistan
| | - Muhammad Tariq Mahmood
- School of Computer Science and Engineering, Korea University of Technology and Education, 1600 Chungjeolno, Byeogchunmyun, 31253 Cheonan, South Korea.
| |
Collapse
|