1
|
Xu J, Yang K, Chen Y, Dai L, Zhang D, Shuai P, Shi R, Yang Z. Reliable and stable fundus image registration based on brain-inspired spatially-varying adaptive pyramid context aggregation network. Front Neurosci 2023; 16:1117134. [PMID: 36726854 PMCID: PMC9884961 DOI: 10.3389/fnins.2022.1117134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 12/28/2022] [Indexed: 01/18/2023] Open
Abstract
The task of fundus image registration aims to find matching keypoints between an image pair. Traditional methods detect the keypoint by hand-designed features, which fail to cope with complex application scenarios. Due to the strong feature learning ability of deep neural network, current image registration methods based on deep learning directly learn to align the geometric transformation between the reference image and test image in an end-to-end manner. Another mainstream of this task aims to learn the displacement vector field between the image pair. In this way, the image registration has achieved significant advances. However, due to the complicated vascular morphology of retinal image, such as texture and shape, current widely used image registration methods based on deep learning fail to achieve reliable and stable keypoint detection and registration results. To this end, in this paper, we aim to bridge this gap. Concretely, since the vessel crossing and branching points can reliably and stably characterize the key components of fundus image, we propose to learn to detect and match all the crossing and branching points of the input images based on a single deep neural network. Moreover, in order to accurately locate the keypoints and learn discriminative feature embedding, a brain-inspired spatially-varying adaptive pyramid context aggregation network is proposed to incorporate the contextual cues under the supervision of structured triplet ranking loss. Experimental results show that the proposed method achieves more accurate registration results with significant speed advantage.
Collapse
Affiliation(s)
- Jie Xu
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing Key Laboratory of Ophthalmology and Visual Sciences, Beijing, China
| | - Kang Yang
- Beijing Zhizhen Internet Technology Co. Ltd.,Beijing, China
| | - Youxin Chen
- Department of Ophthalmology, Peking Union Medical College Hospital, Beijing, China
| | - Liming Dai
- Beijing Zhizhen Internet Technology Co. Ltd.,Beijing, China
| | - Dongdong Zhang
- Beijing Zhizhen Internet Technology Co. Ltd.,Beijing, China
| | - Ping Shuai
- Department of Health Management and Physical Examination, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
- School of Medicine, University of Electronic Science and Technology of China, Chengdu, China
| | - Rongjie Shi
- Beijing Zhizhen Internet Technology Co. Ltd.,Beijing, China
| | - Zhanbo Yang
- Beijing Zhizhen Internet Technology Co. Ltd.,Beijing, China
| |
Collapse
|
2
|
Networks behind the morphology and structural design of living systems. Phys Life Rev 2022; 41:1-21. [DOI: 10.1016/j.plrev.2022.03.001] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 03/04/2022] [Indexed: 01/06/2023]
|
3
|
At the Pulse of Time: Machine Vision in Retinal Videos. ACTA NEUROCHIRURGICA. SUPPLEMENT 2021; 134:303-311. [PMID: 34862554 DOI: 10.1007/978-3-030-85292-4_34] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
Spontaneous venous pulsations (SVP) are a common finding in healthy people. The absence of SVP is associated with rapid progression in glaucoma and increased intracranial pressure. Traditionally, SVP has been documented qualitatively by clinicians during biomicroscopy. Nowadays numerous imaging devices recording the fundus exist. Hence, video data for objectification of SVP is readily available. Still, these clinical datasets are afflicted with various quality issues and artifacts. In this machine vision based study, we explore methods to overcome challenges in identifying SVP in fundus videos of varying quality and provide a detailed protocol thereof. Hereby, we aim to lower the burden of access of implementing machine vision in clinical video datasets and quantification of SVP.
Collapse
|
4
|
Jiang H, Gao M, Yang K, Zhang D, Ma H, Qian W. Neonatal Fundus Image Registration and Mosaic Using Improved Speeded Up Robust Features Based on Shannon Entropy. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3004-3007. [PMID: 34891876 DOI: 10.1109/embc46164.2021.9630593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Fundus examination of the newborn is quite important, which needs to be done timely so as to avoid irreversible blindness. Ophthalmologists have to review at least five images of each eye during one examination, which is a time-consuming task. To improve the diagnosis efficiency, this paper proposed a stable and robust fundus image mosaic method based on improved Speeded Up Robust Features (SURF) with Shannon entropy and make real assessment when ophthalmologists used it clinically. Our method is characterized by avoiding the useless detection and extraction of the feature points in the non-overlapping region of the paired images during registration process. The experiments showed that the proposed method successfully registered 90.91% of 110 different field of view (FOV) image pairs from 22 eyes of 13 screening newborns and acquired 93.51% normalized correlation coefficient and 1.2557 normalized mutual information. Also, the total fusion success rate reached 86.36% and a subjective visual assessment approach was adopted to measure the fusion performance by three experts, which obtained 84.85% acceptance rate. The performance of our proposed method demonstrated its accuracy and effectiveness in the clinical application, which can help ophthalmologists a lot during their diagnosis.
Collapse
|
5
|
Oyarzun Laura C, Wesarg S, Sakas G. Graph matching survey for medical imaging: On the way to deep learning. Methods 2021; 202:3-13. [PMID: 34216788 DOI: 10.1016/j.ymeth.2021.06.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2021] [Revised: 05/18/2021] [Accepted: 06/11/2021] [Indexed: 11/18/2022] Open
Abstract
The interest on graph matching has not stopped growing since the late seventies. The basic idea of graph matching consists of generating graph representations of different data or structures and compare those representations by searching correspondences between them. There are manifold techniques that have been developed to find those correspondences and the choice of one or another depends on the characteristics of the application of interest. These applications range from pattern recognition (e.g. biometric identification) to signal processing or artificial intelligence. One of the aspects that make graph matching so attractive is its ability to facilitate data analysis, and medical imaging is one of the fields that can benefit from this in a greater extent. The potential of graph matching to find similarities and differences between data acquired at different points in time shows its potential to improve diagnosis, follow-up of human diseases or any other of the clinical scenarios that require comparison between different datasets. In spite of the large amount of papers that were published in this field to the date there is no survey paper of graph matching for clinical applications. This survey aims to fill this gap.
Collapse
Affiliation(s)
- Cristina Oyarzun Laura
- Visual Healthcare Technologies, Fraunhofer Institute for Computer Graphics Research IGD, Germany.
| | - Stefan Wesarg
- Visual Healthcare Technologies, Fraunhofer Institute for Computer Graphics Research IGD, Germany
| | - Georgios Sakas
- Interactive Graphics Systems Group, Technical University of Darmstadt, Germany
| |
Collapse
|
6
|
Yoon S, Yoon CH, Lee D. Topological recovery for non-rigid 2D/3D registration of coronary artery models. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 200:105922. [PMID: 33440300 DOI: 10.1016/j.cmpb.2020.105922] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/10/2019] [Accepted: 12/23/2020] [Indexed: 06/12/2023]
Abstract
BACKGROUND AND OBJECTIVE Intra-operative X-ray angiography, the current standard method for visualizing and diagnosing cardiovascular disease, is limited in its ability to provide essential 3D information. These limitations are disadvantages in treating patients. For example, it is a cause of lowering the success rate of interventional procedures. Here, we propose a novel 2D-3D non-rigid registration method to understand vascular geometry during percutaneous coronary intervention. METHODS The proposed method uses the local bijection pair distance as a cost function to minimize the effect of inconsistencies from center-line extraction. Moreover, novel cage-based 3D deformation and multi-threaded particle swarm optimization are utilized to implement real-time registration. We evaluated the proposed method for 154 examinations from 10 anonymous patients by coverage percentage, comparing the average distance of the 2D extracted center-line with that of the registered 3D center-line. RESULTS The proposed 2D-3D non-rigid registration method achieved an average distance of 1.98 mm with a 0.54 s computation time. Additionally, in aiming to reduce the uncertainty of XA images, we used the proposed method to retrospectively visualize the connections between 2D vascular segments and the distal part of occlusions. CONCLUSIONS Ultimately, the proposed 2D/3D non-rigid registration method can successfully register the 3D center-line of coronary arteries with corresponding 2D XA images, and is computationally sufficient for online usage. Therefore, this method can improve the success rate of such procedures as a percutaneous coronary intervention and provide the information necessary to diagnose cardiovascular diseases better.
Collapse
Affiliation(s)
- Siyeop Yoon
- 5, Hwarang-ro 14-gil, Seongbuk-gu, Seoul,Center for Healthcare Robotics, Korea Institute of Science and Technology, South Korea; 5, Hwarang-ro 14-gil, Seongbuk-gu, Seoul, KIST School, Korea University of Science and Technology, South Korea.
| | - Chang Hwan Yoon
- Gumi-ro, 82-gil 173, Bundang-gu, Seongnam, Seoul national university Bundang Hospital, South Korea.
| | - Deukhee Lee
- 5, Hwarang-ro 14-gil, Seongbuk-gu, Seoul,Center for Healthcare Robotics, Korea Institute of Science and Technology, South Korea; 5, Hwarang-ro 14-gil, Seongbuk-gu, Seoul, KIST School, Korea University of Science and Technology, South Korea.
| |
Collapse
|
7
|
Deep multispectral image registration network. Comput Med Imaging Graph 2021; 87:101815. [PMID: 33418174 DOI: 10.1016/j.compmedimag.2020.101815] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2020] [Revised: 09/27/2020] [Accepted: 10/30/2020] [Indexed: 11/24/2022]
Abstract
Multispectral imaging (MSI) of the ocular fundus provides a sequence of narrow-band images to show the different depths in the retina and choroid. One challenge in analyzing MSI images comes from the image-to-image spatial misalignment, which occurs because the acquisition time of eye MSI images is commonly longer than the natural time scale of the eye's saccadic movement. It is necessary to align images because ophthalmologists usually overlay two of the images to analyze specific features when analyzing MSI images. In this paper, we propose a weakly supervised MSI image registration network, called MSI-R-NET, for multispectral fundus image registration. Compared to other deep-learning-based registration methods, MSI-R-NET utilizes the blood vessel segmentation label to provide spatial correspondence. In addition, we employ a feature equilibrium module to connect the aggregating layers better, and propose a multiresolution auto-context structure to adapt the registration task. In the testing stage, given a new pair of MSI images, the trained model can predict the pixelwise spatial correspondence without labeled blood vessel information. The experimental results demonstrate that the proposed segmentation-driven registration method is highly accurate.
Collapse
|
8
|
Hernandez-Matas C, Zabulis X, Argyros AA. REMPE: Registration of Retinal Images Through Eye Modelling and Pose Estimation. IEEE J Biomed Health Inform 2020; 24:3362-3373. [DOI: 10.1109/jbhi.2020.2984483] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
9
|
Kleefeldt N, Bermond K, Tarau IS, Hillenkamp J, Berlin A, Sloan KR, Ach T. Quantitative Fundus Autofluorescence: Advanced Analysis Tools. Transl Vis Sci Technol 2020; 9:2. [PMID: 32855849 PMCID: PMC7422829 DOI: 10.1167/tvst.9.8.2] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2019] [Accepted: 05/18/2020] [Indexed: 12/31/2022] Open
Abstract
Purpose To use multimodal retinal images (including quantitative fundus autofluorescence [QAF]) for spectral-domain optical coherence tomography (SD-OCT)-based image registration and alignment. For each age decade of healthy adults, normative fine-grained QAF retinal maps are generated and advanced methods for QAF image analysis are applied. Methods Multimodal retinal images were obtained from 103 healthy subjects (age 19–77 years; unremarkable retina/macula, age-appropriate clear optic media). Custom written FIJI plugins enabled: (1) determination of the fovea in SD-OCT and the edge of the optic disc in infrared (IR) images; (2) alignment and superimposition of multimodal retinal images based on foveal and optic disc position; (3) plotting of normative QAF retinal maps for each decade; and (4) comparison of individual retinas with normative retinas of different decades using newly introduced analysis patterns (QAF97, freehand tool). Results SD-OCT based image registration enables easy image registration, alignment, and analysis of different modalities (QAF, IR, and SD-OCT here reported). In QAF, intensities significantly increase with age with two major inclines between the third/fourth and seventh/eighth decades. With aging, the parafoveal area of maximum QAF intensity slightly shifts from temporal-superior to temporal. Compared with standard QAF analysis, refined QAF analysis patterns reveal a more detailed analysis of QAF, especially in the diseased retina. Conclusions Age-related QAF normative retinal maps can be used to directly compare and classify individual's QAF intensities. Advanced QAF analysis tools will further help to interpret autofluorescence changes in normal aging and in the diseased retina in a multimodal imaging setting. Translational Relevance Advanced methods for QAF analysis link basic findings with clinical observations in normal aging and in the diseased macula.
Collapse
Affiliation(s)
- Nikolai Kleefeldt
- Department of Ophthalmology, University Hospital Würzburg, Würzburg, Germany
| | - Katharina Bermond
- Department of Ophthalmology, University Hospital Würzburg, Würzburg, Germany
| | - Ioana-Sandra Tarau
- Department of Ophthalmology, University Hospital Würzburg, Würzburg, Germany
| | - Jost Hillenkamp
- Department of Ophthalmology, University Hospital Würzburg, Würzburg, Germany
| | - Andreas Berlin
- Department of Ophthalmology, University Hospital Würzburg, Würzburg, Germany
| | - Kenneth R Sloan
- Department of Computer Science, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Thomas Ach
- Department of Ophthalmology, University Hospital Würzburg, Würzburg, Germany
| |
Collapse
|
10
|
Motta D, Casaca W, Paiva A. Vessel Optimal Transport for Automated Alignment of Retinal Fundus Images. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2019; 28:6154-6168. [PMID: 31283507 DOI: 10.1109/tip.2019.2925287] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Optimal transport has emerged as a promising and useful tool for supporting modern image processing applications such as medical imaging and scientific visualization. Indeed, the optimal transport theory enables great flexibility in modeling problems related to image registration, as different optimization resources can be successfully used as well as the choice of suitable matching models to align the images. In this paper, we introduce an automated framework for fundus image registration which unifies optimal transport theory, image processing tools, and graph matching schemes into a functional and concise methodology. Given two ocular fundus images, we construct representative graphs which embed in their structures spatial and topological information from the eye's blood vessels. The graphs produced are then used as input by our optimal transport model in order to establish a correspondence between their sets of nodes. Finally, geometric transformations are performed between the images so as to accomplish the registration task properly. Our formulation relies on the solid mathematical foundation of optimal transport as a constrained optimization problem, being also robust when dealing with outliers created during the matching stage. We demonstrate the accuracy and effectiveness of the present framework throughout a comprehensive set of qualitative and quantitative comparisons against several influential state-of-the-art methods on various fundus image databases.
Collapse
|
11
|
Pinheiro MA, Kybic J, Fua P. Geometric Graph Matching Using Monte Carlo Tree Search. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2017; 39:2171-2185. [PMID: 28114003 DOI: 10.1109/tpami.2016.2636200] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
We present an efficient matching method for generalized geometric graphs. Such graphs consist of vertices in space connected by curves and can represent many real world structures such as road networks in remote sensing, or vessel networks in medical imaging. Graph matching can be used for very fast and possibly multimodal registration of images of these structures. We formulate the matching problem as a single player game solved using Monte Carlo Tree Search, which automatically balances exploring new possible matches and extending existing matches. Our method can handle partial matches, topological differences, geometrical distortion, does not use appearance information and does not require an initial alignment. Moreover, our method is very efficient-it can match graphs with thousands of nodes, which is an order of magnitude better than the best competing method, and the matching only takes a few seconds.
Collapse
|
12
|
Chen L, Lian Y, Guo Y, Wang Y, Hatsukami TS, Pimentel K, Balu N, Yuan C. A vascular image registration method based on network structure and circuit simulation. BMC Bioinformatics 2017; 18:229. [PMID: 28464789 PMCID: PMC5414324 DOI: 10.1186/s12859-017-1649-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2016] [Accepted: 04/24/2017] [Indexed: 12/01/2022] Open
Abstract
Background Image registration is an important research topic in the field of image processing. Applying image registration to vascular image allows multiple images to be strengthened and fused, which has practical value in disease detection, clinical assisted therapy, etc. However, it is hard to register vascular structures with high noise and large difference in an efficient and effective method. Results Different from common image registration methods based on area or features, which were sensitive to distortion and uncertainty in vascular structure, we proposed a novel registration method based on network structure and circuit simulation. Vessel images were transformed to graph networks and segmented to branches to reduce the calculation complexity. Weighted graph networks were then converted to circuits, in which node voltages of the circuit reflecting the vessel structures were used for node registration. The experiments in the two-dimensional and three-dimensional simulation and clinical image sets showed the success of our proposed method in registration. Conclusions The proposed vascular image registration method based on network structure and circuit simulation is stable, fault tolerant and efficient, which is a useful complement to the current mainstream image registration methods.
Collapse
Affiliation(s)
- Li Chen
- Department of Electronic Engineering, Fudan University, Shanghai, 200433, China.,Department of Electrical Engineering, University of Washington, Seattle, WA, 98195, USA
| | - Yuxi Lian
- Department of Electronic Engineering, Fudan University, Shanghai, 200433, China
| | - Yi Guo
- Department of Electronic Engineering, Fudan University, Shanghai, 200433, China.
| | - Yuanyuan Wang
- Department of Electronic Engineering, Fudan University, Shanghai, 200433, China
| | - Thomas S Hatsukami
- Department of Surgery, University of Washington, Seattle, WA, 98195, USA
| | - Kristi Pimentel
- Department of Radiology, University of Washington, Seattle, WA, 98195-7115, USA
| | - Niranjan Balu
- Department of Radiology, University of Washington, Seattle, WA, 98195-7115, USA
| | - Chun Yuan
- Department of Radiology, University of Washington, Seattle, WA, 98195-7115, USA
| |
Collapse
|
13
|
Khakzar M, Pourghassem H. A retinal image authentication framework based on a graph-based representation algorithm in a two-stage matching structure. Biocybern Biomed Eng 2017. [DOI: 10.1016/j.bbe.2017.09.001] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
14
|
Gellrich MM. A simple method for panretinal imaging with the slit lamp. Int Ophthalmol 2016; 36:775-780. [PMID: 26879088 DOI: 10.1007/s10792-016-0193-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2015] [Accepted: 01/28/2016] [Indexed: 11/25/2022]
|
15
|
Brophy M, Chaudhury A, Beauchemin SS, Barron JL. A Method for Global Non-rigid Registration of Multiple Thin Structures. 2015 12TH CONFERENCE ON COMPUTER AND ROBOT VISION 2015. [DOI: 10.1109/crv.2015.35] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/19/2023]
|
16
|
Fua P, Knott GW. Modeling brain circuitry over a wide range of scales. Front Neuroanat 2015; 9:42. [PMID: 25904852 PMCID: PMC4387921 DOI: 10.3389/fnana.2015.00042] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2015] [Accepted: 03/17/2015] [Indexed: 11/13/2022] Open
Abstract
If we are ever to unravel the mysteries of brain function at its most fundamental level, we will need a precise understanding of how its component neurons connect to each other. Electron Microscopes (EM) can now provide the nanometer resolution that is needed to image synapses, and therefore connections, while Light Microscopes (LM) see at the micrometer resolution required to model the 3D structure of the dendritic network. Since both the topology and the connection strength are integral parts of the brain's wiring diagram, being able to combine these two modalities is critically important. In fact, these microscopes now routinely produce high-resolution imagery in such large quantities that the bottleneck becomes automated processing and interpretation, which is needed for such data to be exploited to its full potential. In this paper, we briefly review the Computer Vision techniques we have developed at EPFL to address this need. They include delineating dendritic arbors from LM imagery, segmenting organelles from EM, and combining the two into a consistent representation.
Collapse
Affiliation(s)
- Pascal Fua
- Computer Vision Lab, I&C School, École Polytechnique Fédérale de Lausanne Lausanne, Switzerland
| | - Graham W Knott
- Bioelectron Microscopy Core Facility, École Polytechnique Fédérale de Lausanne Lausanne, Switzerland
| |
Collapse
|
17
|
Serradell E, Pinheiro MA, Sznitman R, Kybic J, Moreno-Noguer F, Fua P. Non-Rigid Graph Registration Using Active Testing Search. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2015; 37:625-638. [PMID: 26353266 DOI: 10.1109/tpami.2014.2343235] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
We present a new approach for matching sets of branching curvilinear structures that form graphs embedded in R2 or R3 and may be subject to deformations. Unlike earlier methods, ours does not rely on local appearance similarity nor does require a good initial alignment. Furthermore, it can cope with non-linear deformations, topological differences, and partial graphs. To handle arbitrary non-linear deformations, we use Gaussian process regressions to represent the geometrical mapping relating the two graphs. In the absence of appearance information, we iteratively establish correspondences between points, update the mapping accordingly, and use it to estimate where to find the most likely correspondences that will be used in the next step. To make the computation tractable for large graphs, the set of new potential matches considered at each iteration is not selected at random as with many RANSAC-based algorithms. Instead, we introduce a so-called Active Testing Search strategy that performs a priority search to favor the most likely matches and speed-up the process. We demonstrate the effectiveness of our approach first on synthetic cases and then on angiography data, retinal fundus images, and microscopy image stacks acquired at very different resolutions.
Collapse
|
18
|
Liu S, Datta A, Ho D, Dwelle J, Wang D, Milner TE, Rylander HG, Markey MK. Effect of image registration on longitudinal analysis of retinal nerve fiber layer thickness of non-human primates using Optical Coherence Tomography (OCT). EYE AND VISION (LONDON, ENGLAND) 2015; 2:3. [PMID: 26605359 PMCID: PMC4657366 DOI: 10.1186/s40662-015-0013-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/13/2014] [Accepted: 01/27/2015] [Indexed: 11/18/2022]
Abstract
BACKGROUND In this paper we determined the benefits of image registration on estimating longitudinal retinal nerve fiber layer thickness (RNFLT) changes. METHODS RNFLT maps around the optic nerve head (ONH) of healthy primate eyes were measured using Optical Coherence Tomography (OCT) weekly for 30 weeks. One automatic algorithm based on mutual information (MI) and the other semi-automatic algorithm based on log-polar transform cross-correlation using manually segmented blood vessels (LPCC_MSBV), were used to register retinal maps longitudinally. We compared the precision and recall between manually segmented image pairs for the two algorithms using a linear mixed effects model. RESULTS We found that the precision calculated between manually segmented image pairs following registration by LPCC_MSBV algorithm is significantly better than the one following registration by MI algorithm (p < <0.0001). Trend of the all-rings and temporal, superior, nasal and inferior (TSNI) quadrants average of RNFLT over time in healthy primate eyes are not affected by registration. RNFLT of clock hours 1, 2, and 10 showed significant change over 30 weeks (p = 0.0058, 0.0054, and 0.0298 for clock hours 1, 2 and 10 respectively) without registration, but stayed constant over time with registration. CONCLUSIONS The LPCC_MSBV provides better registration of RNFLT maps recorded on different dates than the automatic MI algorithm. Registration of RNFLT maps can improve clinical analysis of glaucoma progression.
Collapse
Affiliation(s)
- Shuang Liu
- />Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
- />Present address: Clinical Neuroscience Imaging Center (CNIC), Department of Neurology, Yale School of Medicine, New Haven, CT 06510 USA
| | - Anjali Datta
- />Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Derek Ho
- />Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Jordan Dwelle
- />Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Daifeng Wang
- />Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Thomas E Milner
- />Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Henry Grady Rylander
- />Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Mia K Markey
- />Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
- />Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030 USA
| |
Collapse
|
19
|
Richa R, Linhares R, Comunello E, von Wangenheim A, Schnitzler JY, Wassmer B, Guillemot C, Thuret G, Gain P, Hager G, Taylor R. Fundus image mosaicking for information augmentation in computer-assisted slit-lamp imaging. IEEE TRANSACTIONS ON MEDICAL IMAGING 2014; 33:1304-1312. [PMID: 24718569 DOI: 10.1109/tmi.2014.2309440] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Laser photocoagulation is currently the standard treatment for sight-threatening diseases worldwide, namely diabetic retinopathy and retinal vein occlusions. The slit lamp biomicroscope is the most commonly used device for this procedure, specially for the treatment of the eye periphery. However, only a small portion of the retina can be visualized through the biomicroscope, complicating the task of localizing and identifying surgical targets, increasing treatment duration and patient discomfort. In order to assist surgeons, we propose a method for creating intraoperative retina maps for view expansion using a slit-lamp device. Based on the mosaicking method described by Richa et al, 2012, the proposed method is a combination of direct and feature-based methods, suitable for the textured nature of the human retina. In this paper, we describe three major enhancements to the original formulation. The first is a visual tracking method using local illumination compensation to cope with the challenging visualization conditions. The second is an efficient pixel selection scheme for increased computational efficiency. The third is an entropy-based mosaic update method to dynamically improve the retina map during exploration. To evaluate the performance of the proposed method, we conducted several experiments on human subjects with a computer-assisted slit-lamp prototype. We also demonstrate the practical value of the system for photo documentation, diagnosis and intraoperative navigation.
Collapse
|
20
|
Dashtbozorg B, Mendonça AM, Campilho A. An automatic graph-based approach for artery/vein classification in retinal images. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2014; 23:1073-83. [PMID: 23693131 DOI: 10.1109/tip.2013.2263809] [Citation(s) in RCA: 77] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
The classification of retinal vessels into artery/vein (A/V) is an important phase for automating the detection of vascular changes, and for the calculation of characteristic signs associated with several systemic diseases such as diabetes, hypertension, and other cardiovascular conditions. This paper presents an automatic approach for A/V classification based on the analysis of a graph extracted from the retinal vasculature. The proposed method classifies the entire vascular tree deciding on the type of each intersection point (graph nodes) and assigning one of two labels to each vessel segment (graph links). Final classification of a vessel segment as A/V is performed through the combination of the graph-based labeling results with a set of intensity features. The results of this proposed method are compared with manual labeling for three public databases. Accuracy values of 88.3%, 87.4%, and 89.8% are obtained for the images of the INSPIRE-AVR, DRIVE, and VICAVR databases, respectively. These results demonstrate that our method outperforms recent approaches for A/V classification.
Collapse
|
21
|
Kolar R, Harabis V, Odstrcilik J. Hybrid retinal image registration using phase correlation. IMAGING SCIENCE JOURNAL 2013. [DOI: 10.1179/1743131x11y.0000000065] [Citation(s) in RCA: 35] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/31/2022]
|
22
|
Li B, Li HK. Automated analysis of diabetic retinopathy images: principles, recent developments, and emerging trends. Curr Diab Rep 2013; 13:453-9. [PMID: 23686810 DOI: 10.1007/s11892-013-0393-9] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
Diabetic retinopathy (DR) is a vision-threatening complication of diabetes. Timely diagnosis and intervention are essential for treatment that reduces the risk of vision loss. A good color retinal (fundus) photograph can be used as a surrogate for face-to-face evaluation of DR. The use of computers to assist or fully automate DR evaluation from retinal images has been studied for many years. Early work showed promising results for algorithms in detecting and classifying DR pathology. Newer techniques include those that adapt machine learning technology to DR image analysis. Challenges remain, however, that must be overcome before fully automatic DR detection and analysis systems become practical clinical tools.
Collapse
Affiliation(s)
- Baoxin Li
- School of Computing, Informatics & Decision Systems Engineering, Arizona State University, Tempe, AZ 85281, USA.
| | | |
Collapse
|
23
|
Retinal image matching using hierarchical vascular features. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2011; 2011:749054. [PMID: 22013434 PMCID: PMC3195514 DOI: 10.1155/2011/749054] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2011] [Revised: 07/29/2011] [Accepted: 08/01/2011] [Indexed: 11/17/2022]
Abstract
We propose a method for retinal image matching that can be used in image matching for person identification or patient longitudinal study. Vascular invariant features are extracted from the retinal image, and a feature vector is constructed for each of the vessel segments in the retinal blood vessels. The feature vectors are represented in a tree structure with maintaining the vessel segments actual hierarchical positions. Using these feature vectors, corresponding images are matched. The method identifies the same vessel in the corresponding images for comparing the desired feature(s). Initial results are encouraging and demonstrate that the proposed method is suitable for image matching and patient longitudinal study.
Collapse
|