Color fundus image registration techniques and applications for automated analysis of diabetic retinopathy progression: A review

Research on registration method for enface image using multi-feature fusion

Objective.The purpose of this work is to accurately and quickly register the Optical coherence tomography (OCT) projection (enface) images at adjacent time points, and to solve the problem of interference caused by CNV lesions on the registration features.Approach.In this work, a multi-feature registration strategy was proposed, in which a combined feature (com-feature) containing 3D information, intersection information and SURF feature was designed. Firstly, the coordinates of all feature points were extracted as combined features, and then these feature coordinates were added to the initial vascular coordinate set simplified by the Douglas-Peucker algorithm as the point set for registration. Finally, the coherent point drift registration algorithm was used to register the enface coordinate point sets of adjacent time series.Main results.The newly designed features significantly improve the success rate of global registration of vascular networks in enface images, while the simplification step greatly improves the registration speed on the basis of preserving vascular features. The MSE, DSC and time complexity of the proposed method are 0.07993, 0.9693 and 42.7016 s, respectively.Significance.CNV is a serious retinal disease in ophthalmology. The registration of OCT enface images at adjacent time points can timely monitor the progress of the disease and assist doctors in making diagnoses. The proposed method not only improves the accuracy of OCT enface image registration, but also significantly reduces the time complexity. It has good registration results in clinical routine and provides a more efficient method for clinical diagnosis and treatment.

Medical image registration and its application in retinal images: a review

Medical image registration is vital for disease diagnosis and treatment with its ability to merge diverse information of images, which may be captured under different times, angles, or modalities. Although several surveys have reviewed the development of medical image registration, they have not systematically summarized the existing medical image registration methods. To this end, a comprehensive review of these methods is provided from traditional and deep-learning-based perspectives, aiming to help audiences quickly understand the development of medical image registration. In particular, we review recent advances in retinal image registration, which has not attracted much attention. In addition, current challenges in retinal image registration are discussed and insights and prospects for future research provided.

A Review of Medical Image Registration for Different Modalities

Medical image registration has become pivotal in recent years with the integration of various imaging modalities like X-ray, ultrasound, MRI, and CT scans, enabling comprehensive analysis and diagnosis of biological structures. This paper provides a comprehensive review of registration techniques for medical images, with an in-depth focus on 2D-2D image registration methods. While 3D registration is briefly touched upon, the primary emphasis remains on 2D techniques and their applications. This review covers registration techniques for diverse modalities, including unimodal, multimodal, interpatient, and intra-patient. The paper explores the challenges encountered in medical image registration, including geometric distortion, differences in image properties, outliers, and optimization convergence, and discusses their impact on registration accuracy and reliability. Strategies for addressing these challenges are highlighted, emphasizing the need for continual innovation and refinement of techniques to enhance the accuracy and reliability of medical image registration systems. The paper concludes by emphasizing the importance of accurate medical image registration in improving diagnosis.

Resilient back-propagation machine learning-based classification on fundus images for retinal microaneurysm detection

BACKGROUND

The timely diagnosis of medical conditions, particularly diabetic retinopathy, relies on the identification of retinal microaneurysms. However, the commonly used retinography method poses a challenge due to the diminutive dimensions and limited differentiation of microaneurysms in images.

PROBLEM STATEMENT

Automated identification of microaneurysms becomes crucial, necessitating the use of comprehensive ad-hoc processing techniques. Although fluorescein angiography enhances detectability, its invasiveness limits its suitability for routine preventative screening.

OBJECTIVE

This study proposes a novel approach for detecting retinal microaneurysms using a fundus scan, leveraging circular reference-based shape features (CR-SF) and radial gradient-based texture features (RG-TF).

METHODOLOGY

The proposed technique involves extracting CR-SF and RG-TF for each candidate microaneurysm, employing a robust back-propagation machine learning method for training. During testing, extracted features from test images are compared with training features to categorize microaneurysm presence.

RESULTS

The experimental assessment utilized four datasets (MESSIDOR, Diaretdb1, e-ophtha-MA, and ROC), employing various measures. The proposed approach demonstrated high accuracy (98.01%), sensitivity (98.74%), specificity (97.12%), and area under the curve (91.72%).

CONCLUSION

The presented approach showcases a successful method for detecting retinal microaneurysms using a fundus scan, providing promising accuracy and sensitivity. This non-invasive technique holds potential for effective screening in diabetic retinopathy and other related medical conditions.

Discriminative-Region Multi-Label Classification of Ultra-Widefield Fundus Images

Ultra-widefield fundus image (UFI) has become a crucial tool for ophthalmologists in diagnosing ocular diseases because of its ability to capture a wide field of the retina. Nevertheless, detecting and classifying multiple diseases within this imaging modality continues to pose a significant challenge for ophthalmologists. An automated disease classification system for UFI can support ophthalmologists in making faster and more precise diagnoses. However, existing works for UFI classification often focus on a single disease or assume each image only contains one disease when tackling multi-disease issues. Furthermore, the distinctive characteristics of each disease are typically not utilized to improve the performance of the classification systems. To address these limitations, we propose a novel approach that leverages disease-specific regions of interest for the multi-label classification of UFI. Our method uses three regions, including the optic disc area, the macula area, and the entire UFI, which serve as the most informative regions for diagnosing one or multiple ocular diseases. Experimental results on a dataset comprising 5930 UFIs with six common ocular diseases showcase that our proposed approach attains exceptional performance, with the area under the receiver operating characteristic curve scores for each class spanning from 95.07% to 99.14%. These results not only surpass existing state-of-the-art methods but also exhibit significant enhancements, with improvements of up to 5.29%. These results demonstrate the potential of our method to provide ophthalmologists with valuable information for early and accurate diagnosis of ocular diseases, ultimately leading to improved patient outcomes.

Deep Learning and Medical Image Processing Techniques for Diabetic Retinopathy: A Survey of Applications, Challenges, and Future Trends

Diabetic retinopathy (DR) is a common eye retinal disease that is widely spread all over the world. It leads to the complete loss of vision based on the level of severity. It damages both retinal blood vessels and the eye's microscopic interior layers. To avoid such issues, early detection of DR is essential in association with routine screening methods to discover mild causes in manual initiation. But these diagnostic procedures are extremely difficult and expensive. The unique contributions of the study include the following: first, providing detailed background of the DR disease and the traditional detection techniques. Second, the various imaging techniques and deep learning applications in DR are presented. Third, the different use cases and real-life scenarios are explored relevant to DR detection wherein deep learning techniques have been implemented. The study finally highlights the potential research opportunities for researchers to explore and deliver effective performance results in diabetic retinopathy detection.

A Fully Unsupervised Deep Learning Framework for Non-Rigid Fundus Image Registration

In ophthalmology, the registration problem consists of finding a geometric transformation that aligns a pair of images, supporting eye-care specialists who need to record and compare images of the same patient. Considering the registration methods for handling eye fundus images, the literature offers only a limited number of proposals based on deep learning (DL), whose implementations use the supervised learning paradigm to train a model. Additionally, ensuring high-quality registrations while still being flexible enough to tackle a broad range of fundus images is another drawback faced by most existing methods in the literature. Therefore, in this paper, we address the above-mentioned issues by introducing a new DL-based framework for eye fundus registration. Our methodology combines a U-shaped fully convolutional neural network with a spatial transformation learning scheme, where a reference-free similarity metric allows the registration without assuming any pre-annotated or artificially created data. Once trained, the model is able to accurately align pairs of images captured under several conditions, which include the presence of anatomical differences and low-quality photographs. Compared to other registration methods, our approach achieves better registration outcomes by just passing as input the desired pair of fundus images.

Robust Detection Model of Vascular Landmarks for Retinal Image Registration: A Two-Stage Convolutional Neural Network

Registration is useful for image processing in computer vision. It can be applied to retinal images and provide support for ophthalmologists in tracking disease progression and monitoring therapeutic responses. This study proposed a robust detection model of vascular landmarks to improve the performance of retinal image registration. The proposed model consists of a two-stage convolutional neural network, in which one segments the retinal vessels on a pair of images, and the other detects junction points from the vessel segmentation image. Information obtained from the model was utilized for the registration. The keypoints were extracted based on the acquired vascular landmark points, and the orientation features were calculated as descriptors. Then, the reference and sensed images were registered by matching keypoints using a homography matrix and random sample consensus algorithm. The proposed method was evaluated on five databases and seven evaluation metrics to verify both clinical effectiveness and robustness. The results established that the proposed method showed outstanding performance for registration compared with other state-of-the-art methods. In particular, the high and significantly improved registration results were identified on FIRE database with area under the curve (AUC) of 0.988, 0.511, and 0.803 in S, P, and A classes. Furthermore, the proposed method worked well on poor quality and multimodal datasets demonstrating an ability to achieve high AUC above 0.8.

The Longitudinal Assessment of Vascular Parameters of the Retina and Their Correlations with Systemic Characteristics in Type 2 Diabetes-A Pilot Study

The aim of the study was to assess various retinal vessel parameters of diabetes mellitus (DM) patients and their correlations with systemic factors in type 2 DM. A retrospective exploratory study in which 21 pairs of baseline and follow-up images of patients affected by DM were randomly chosen from the Sankara Nethralaya−Diabetic Retinopathy Study (SN DREAMS) I and II datasets. Patients’ fundus was photographed, and the diagnosis was made based on Klein classification. Vessel thickness parameters were generated using a web-based retinal vascular analysis platform called VASP. The thickness changes between the baseline and follow-up images were computed and normalized with the actual thicknesses of baseline images. The majority of parameters showed 10~20% changes over time. Vessel width in zone C for the second vein was significantly reduced from baseline to follow-up, which showed positive correlations with systolic blood pressure and serum high-density lipoproteins. Fractal dimension for all vessels in zones B and C and fractal dimension for vein in zones A, B and C showed a minimal increase from baseline to follow-up, which had a linear relationship with diastolic pressure, mean arterial pressure, serum triglycerides (p < 0.05). Lacunarity for all vessels and veins in zones A, B and C showed a minimal decrease from baseline to follow-up which had a negative correlation with pulse pressure and positive correlation with serum triglycerides (p < 0.05). The vessel widths for the first and second arteries significantly increased from baseline to follow-up and had an association with high-density lipoproteins, glycated haemoglobin A1C, serum low-density lipoproteins and total serum cholesterol. The central reflex intensity ratio for the second artery was significantly decreased from baseline to follow-up, and positive correlations were noted with serum triglyceride, serum low-density lipoproteins and total serum cholesterol. The coefficients for branches in zones B and C artery and the junctional exponent deviation for the artery in zone A decreased from baseline to follow-up showed positive correlations with serum triglycerides, serum low-density lipoproteins and total serum cholesterol. Identifying early microvascular changes in diabetic patients will allow for earlier intervention, improve visual outcomes and prevent vision loss.

Registration of computed tomography images of a lung infected with COVID-19 based in the new meta-heuristic algorithm HPSGWO

Computed tomography (CT) helps the radiologist in the rapid and correct detection of a person infected with the coronavirus disease 2019 (COVID-19), and this by showing the presence of the ground-glass opacity in the lung of with the virus. Tracking the evolution of the spread of the ground-glass opacity (GGO) in the lung of the person infected with the virus needs to study more than one image in different times. The various CT images must be registration to identify the evolution of the ground glass in the lung and to facilitate the study and identification of the virus. Due to the process of registration images is essentially an improvement problem, we present in this paper a new HPSGWO algorithm for registration CT images of a lung infected with the COVID-19. This algorithm is a hybridization of the two algorithms Particle swarm optimization (PSO) and Grey wolf optimizer (GWO). The simulation results obtained after applying the algorithm to the test images show that the proposed approach achieved high-precision and robust registration compared to other methods such as GWO, PSO, Firefly Algorithm (FA), and Crow Searcha Algorithms (CSA).

Color fundus image registration using a learning-based domain-specific landmark detection methodology

Medical imaging, and particularly retinal imaging, allows to accurately diagnose many eye pathologies as well as some systemic diseases such as hypertension or diabetes. Registering these images is crucial to correctly compare key structures, not only within patients, but also to contrast data with a model or among a population. Currently, this field is dominated by complex classical methods because the novel deep learning methods cannot compete yet in terms of results and commonly used methods are difficult to adapt to the retinal domain. In this work, we propose a novel method to register color fundus images based on previous works which employed classical approaches to detect domain-specific landmarks. Instead, we propose to use deep learning methods for the detection of these highly-specific domain-related landmarks. Our method uses a neural network to detect the bifurcations and crossovers of the retinal blood vessels, whose arrangement and location are unique to each eye and person. This proposal is the first deep learning feature-based registration method in fundus imaging. These keypoints are matched using a method based on RANSAC (Random Sample Consensus) without the requirement to calculate complex descriptors. Our method was tested using the public FIRE dataset, although the landmark detection network was trained using the DRIVE dataset. Our method provides accurate results, a registration score of 0.657 for the whole FIRE dataset (0.908 for category S, 0.293 for category P and 0.660 for category A). Therefore, our proposal can compete with complex classical methods and beat the deep learning methods in the state of the art.

Video-based eye tracking performance for computer-assisted diagnostic support of diabetic neuropathy

Diabetes is currently one of the major public health threats. The essential components for effective treatment of diabetes include early diagnosis and regular monitoring. However, health-care providers are often short of human resources to closely monitor populations at risk. In this work, a video-based eye-tracking method is proposed as a low-cost alternative for detection of diabetic neuropathy. The method is based on the tracking of the eye-trajectories recorded on videos while the subject follows a target on a screen, forcing saccadic movements. Upon extraction of the eye trajectories, representation of the obtained time-series is made with the help of heteroscedastic ARX (H-ARX) models, which capture the dynamics and latency on the subject's response, while features based on the H-ARX model's predictive ability are subsequently used for classification. The methodology is evaluated on a population constituted by 11 control and 20 insulin-treated diabetic individuals suffering from diverse diabetic complications including neuropathy and retinopathy. Results show significant differences on latency and eye movement precision between the populations of control subjects and diabetics, while simultaneously demonstrating that both groups can be classified with an accuracy of 95%. Although this study is limited by the small sample size, the results align with other findings in the literature and encourage further research.

Hybrid registration of retinal fluorescein angiography and optical coherence tomography images of patients with diabetic retinopathy

Diabetic retinopathy (DR) is a common ophthalmic disease among diabetic patients. It is essential to diagnose DR in the early stages of treatment. Various imaging systems have been proposed to detect and visualize retina diseases. The fluorescein angiography (FA) imaging technique is now widely used as a gold standard technique to evaluate the clinical manifestations of DR. Optical coherence tomography (OCT) imaging is another technique that provides 3D information of the retinal structure. The FA and OCT images are captured in two different phases and field of views and image fusion of these modalities are of interest to clinicians. This paper proposes a hybrid registration framework based on the extraction and refinement of segmented major blood vessels of retinal images. The newly extracted features significantly improve the success rate of global registration results in the complex blood vessel network of retinal images. Afterward, intensity-based and deformable transformations are utilized to further compensate the motion magnitude between the FA and OCT images. Experimental results of 26 images of the various stages of DR patients indicate that this algorithm yields promising registration and fusion results for clinical routine.

Automated diabetic retinopathy screening for primary care settings using deep learning

Diabetic Retinopathy (DR) is one of the leading causes of blindness in the United States and other high-income countries. Early detection is key to prevention, which could be achieved effectively with a fully automated screening tool performing well on clinically relevant measures in primary care settings. We have built an artificial intelligence-based tool on a cloud-based platform for large-scale screening of DR as referable or non-referable. In this paper, we aim to validate this tool built using deep learning based techniques. The cloud-based screening model was developed and tested using deep learning techniques with 88702 images from the Kaggle dataset and externally validated using 1748 high-resolution images of the retina (or fundus images) from the Messidor-2 dataset. For validation in the primary care settings, 264 images were taken prospectively from two diabetes clinics in Queens, New York. The images were uploaded to the cloud-based software for testing the automated system as compared to expert ophthalmologists’ evaluations of referable DR. Measures used were area under the curve (AUC), sensitivity, and specificity of the screening model with respect to professional graders. The screening system achieved a high sensitivity of 99.21% and a specificity of 97.59% on the Kaggle test dataset with an AUC of 0.9992. The system was also externally validated in Messidor-2, where it achieved a sensitivity of 97.63% and a specificity of 99.49% (AUC, 0.9985). On primary care data, the sensitivity was 92.3% overall (12/13 referable images are correctly identified), and overall specificity was 94.8% (233/251 non-referable images). The proposed DR screening tool achieves state-of-the-art performance among the publicly available datasets: Kaggle and Messidor-2 to the best of our knowledge. The performance on various clinically relevant measures demonstrates that the tool is suitable for screening and early diagnosis of DR in primary care settings.

Visualizing and understanding inherent features in SD-OCT for the progression of age-related macular degeneration using deconvolutional neural networks

To develop a convolutional neural network visualization strategy so that optical coherence tomography (OCT) features contributing to the evolution of age-related macular degeneration (AMD) can be better determined. We have trained a U-Net model to utilize baseline OCT to predict the progression of geographic atrophy (GA), a late stage manifestation of AMD. We have augmented the U-Net architecture by attaching deconvolutional neural networks (deconvnets). Deconvnets produce the reconstructed feature maps and provide an indication regarding the inherent baseline OCT features contributing to GA progression. Experiments were conducted on longitudinal spectral domain (SD)-OCT and fundus autofluorescence images collected from 70 eyes with GA. The intensity of Bruch's membrane-outer choroid (BMChoroid) retinal junction exhibited a relative importance of 24%, in the GA progression. The intensity of the inner retinal pigment epithelium (RPE) and BM junction (InRPEBM) showed a relative importance of 22%. BMChoroid (where the AMD feature/damage of choriocapillaris was included) followed by InRPEBM (where the AMD feature/damage of RPE was included) are the layers which appear to be most relevant in predicting the progression of AMD.

Microaneurysms detection in color fundus images using machine learning based on directional local contrast

Background

As one of the major complications of diabetes, diabetic retinopathy (DR) is a leading cause of visual impairment and blindness due to delayed diagnosis and intervention. Microaneurysms appear as the earliest symptom of DR. Accurate and reliable detection of microaneurysms in color fundus images has great importance for DR screening.

Methods

A microaneurysms' detection method using machine learning based on directional local contrast (DLC) is proposed for the early diagnosis of DR. First, blood vessels were enhanced and segmented using improved enhancement function based on analyzing eigenvalues of Hessian matrix. Next, with blood vessels excluded, microaneurysm candidate regions were obtained using shape characteristics and connected components analysis. After image segmented to patches, the features of each microaneurysm candidate patch were extracted, and each candidate patch was classified into microaneurysm or non-microaneurysm. The main contributions of our study are (1) making use of directional local contrast in microaneurysms' detection for the first time, which does make sense for better microaneurysms' classification. (2) Applying three different machine learning techniques for classification and comparing their performance for microaneurysms' detection. The proposed algorithm was trained and tested on e-ophtha MA database, and further tested on another independent DIARETDB1 database. Results of microaneurysms' detection on the two databases were evaluated on lesion level and compared with existing algorithms.

Results

The proposed method has achieved better performance compared with existing algorithms on accuracy and computation time. On e-ophtha MA and DIARETDB1 databases, the area under curve (AUC) of receiver operating characteristic (ROC) curve was 0.87 and 0.86, respectively. The free-response ROC (FROC) score on the two databases was 0.374 and 0.210, respectively. The computation time per image with resolution of 2544×1969, 1400×960 and 1500×1152 is 29 s, 3 s and 2.6 s, respectively.

Conclusions

The proposed method using machine learning based on directional local contrast of image patches can effectively detect microaneurysms in color fundus images and provide an effective scientific basis for early clinical DR diagnosis.