Integrated Analysis of Vascular and Nonvascular Changes From Color Retinal Fundus Image Sequences

Retinal image registration method for myopia development

OBJECTIVE

The development of myopia is usually accompanied by changes in retinal vessels, optic disc, optic cup, fovea, and other retinal structures as well as the length of the ocular axis. And the accurate registration of retinal images is very important for the extraction and analysis of retinal structural changes. However, the registration of retinal images with myopia development faces a series of challenges, due to the unique curved surface of the retina, as well as the changes in fundus curvature caused by ocular axis elongation. Therefore, our goal is to improve the registration accuracy of the retinal images with myopia development.

METHOD

In this study, we propose a 3D spatial model for the pair of retinal images with myopia development. In this model, we introduce a novel myopia development model that simulates the changes in the length of ocular axis and fundus curvature due to the development of myopia. We also consider the distortion model of the fundus camera during the imaging process. Based on the 3D spatial model, we further implement a registration framework, which utilizes corresponding points in the pair of retinal images to achieve registration in the way of 3D pose estimation.

RESULTS

The proposed method is quantitatively evaluated on the publicly available dataset without myopia development and our Fundus Image Myopia Development (FIMD) dataset. The proposed method is shown to perform more accurate and stable registration than state-of-the-art methods, especially for retinal images with myopia development.

SIGNIFICANCE

To the best of our knowledge, this is the first retinal image registration method for the study of myopia development. This method significantly improves the registration accuracy of retinal images which have myopia development. The FIMD dataset we constructed has been made publicly available to promote the study in related fields.

ConKeD: multiview contrastive descriptor learning for keypoint-based retinal image registration

Retinal image registration is of utmost importance due to its wide applications in medical practice. In this context, we propose ConKeD, a novel deep learning approach to learn descriptors for retinal image registration. In contrast to current registration methods, our approach employs a novel multi-positive multi-negative contrastive learning strategy that enables the utilization of additional information from the available training samples. This makes it possible to learn high-quality descriptors from limited training data. To train and evaluate ConKeD, we combine these descriptors with domain-specific keypoints, particularly blood vessel bifurcations and crossovers, that are detected using a deep neural network. Our experimental results demonstrate the benefits of the novel multi-positive multi-negative strategy, as it outperforms the widely used triplet loss technique (single-positive and single-negative) as well as the single-positive multi-negative alternative. Additionally, the combination of ConKeD with the domain-specific keypoints produces comparable results to the state-of-the-art methods for retinal image registration, while offering important advantages such as avoiding pre-processing, utilizing fewer training samples, and requiring fewer detected keypoints, among others. Therefore, ConKeD shows a promising potential towards facilitating the development and application of deep learning-based methods for retinal image registration.

Red lesion in fundus image with hexagonal pattern feature and two-level segmentation

Red lesion identification at its early stage is very essential for the treatment of diabetic retinopathy to prevent loss of vision. This work proposes a red lesion detection algorithm that uses Hexagonal pattern-based features with two-level segmentation that can detect hemorrhage and microaneurysms in the fundus image. The proposed scheme initially pre-processes the fundus image followed by a two-level segmentation. The level 1 segmentation eliminates the background whereas the level 2 segmentation eliminates the blood vessels that introduce more false positives. A hexagonal pattern-based feature is extracted from the red lesion candidates which can highly differentiate the lesion from non-lesion regions. The hexagonal pattern features are then trained using the recurrent neural network and are classified to eliminate the false negatives. For the evaluation of the proposed red lesion algorithm, the datasets namely ROC challenge, e-ophtha, DiaretDB1, and Messidor are used with the metrics such as Accuracy, Recall, Precision, F1 score, Specificity, and AUC. The scheme provides an average Accuracy, Recall (Sensitivity), Precision, F1 score, Specificity, and AUC of 95.48%, 84.54%, 97.3%, 90.47%, 86.81% and 93.43% respectively.

Automated feature-based grading and progression analysis of diabetic retinopathy

Background

In diabetic retinopathy (DR) screening programmes feature-based grading guidelines are used by human graders. However, recent deep learning approaches have focused on end to end learning, based on labelled data at the whole image level. Most predictions from such software offer a direct grading output without information about the retinal features responsible for the grade. In this work, we demonstrate a feature based retinal image analysis system, which aims to support flexible grading and monitor progression.

Methods

The system was evaluated against images that had been graded according to two different grading systems; The International Clinical Diabetic Retinopathy and Diabetic Macular Oedema Severity Scale and the UK’s National Screening Committee guidelines.

Results

External evaluation on large datasets collected from three nations (Kenya, Saudi Arabia and China) was carried out. On a DR referable level, sensitivity did not vary significantly between different DR grading schemes (91.2–94.2.0%) and there were excellent specificity values above 93% in all image sets. More importantly, no cases of severe non-proliferative DR, proliferative DR or DMO were missed.

Conclusions

We demonstrate the potential of an AI feature-based DR grading system that is not constrained to any specific grading scheme.

Retinal image registration as a tool for supporting clinical applications

BACKGROUND AND OBJECTIVE

The study of small vessels allows for the analysis and diagnosis of diseases with strong vasculopathy. This type of vessels can be observed non-invasively in the retina via fundoscopy. The analysis of these vessels can be facilitated by applications built upon Retinal Image Registration (RIR), such as mosaicing, Super Resolution (SR) or eye shape estimation. RIR is challenging due to possible changes in the retina across time, the utilization of diverse acquisition devices with varying properties, or the curved shape of the retina.

METHODS

We employ the Retinal Image Registration through Eye Modelling and Pose Estimation (REMPE) framework, which simultaneously estimates the cameras' relative poses, as well as eye shape and orientation to develop RIR applications and to study their effectiveness.

RESULTS

We assess quantitatively the suitability of the REMPE framework towards achieving SR and eye shape estimation. Additionally, we provide indicative results demonstrating qualitatively its usefulness in the context of longitudinal studies, mosaicing, and multiple image registration. Besides the improvement over registration accuracy, demonstrated via registration applications, the most important novelty presented in this work is the eye shape estimation and the generation of 3D point meshes. This has the potential for allowing clinicians to perform measurements on 3D representations of the eye, instead of doing so in 2D images that contain distortions induced because of the projection on the image space.

CONCLUSIONS

RIR is very effective in supporting applications such as SR, eye shape estimation, longitudinal studies, mosaicing and multiple image registration. Its improved registration accuracy compared to the state of the art translates directly in improved performance when supporting the aforementioned applications.

A review on computer-aided recent developments for automatic detection of diabetic retinopathy

Diabetic retinopathy is a serious microvascular disorder that might result in loss of vision and blindness. It seriously damages the retinal blood vessels and reduces the light-sensitive inner layer of the eye. Due to the manual inspection of retinal fundus images on diabetic retinopathy to detect the morphological abnormalities in Microaneurysms (MAs), Exudates (EXs), Haemorrhages (HMs), and Inter retinal microvascular abnormalities (IRMA) is very difficult and time consuming process. In order to avoid this, the regular follow-up screening process, and early automatic Diabetic Retinopathy detection are necessary. This paper discusses various methods of analysing automatic retinopathy detection and classification of different grading based on the severity levels. In addition, retinal blood vessel detection techniques are also discussed for the ultimate detection and diagnostic procedure of proliferative diabetic retinopathy. Furthermore, the paper elaborately discussed the systematic review accessed by authors on various publicly available databases collected from different medical sources. In the survey, meta-analysis of several methods for diabetic feature extraction, segmentation and various types of classifiers have been used to evaluate the system performance metrics for the diagnosis of DR. This survey will be helpful for the technical persons and researchers who want to focus on enhancing the diagnosis of a system that would be more powerful in real life.

Remote examination of exudates-impact of macular oedema

One of the major causes of eye blindness is identified to be as diabetic retinopathy, which if not detected in earlier stage would cause a serious issue. Long-term diabetes causes diabetic retinopathy. The significant key factor leading to diabetic retinopathy is exudates which affect the retina part and causes eye defects. Thus the first and foremost task in the automated detection of macular oedema is to detect the presence of these exudates. The authors use image processing techniques to detect the optic disc, exudates and the presence of macular oedema. Their method has the sensitivity 96.07%, selectivity 97.36%, and accuracy 96.62% for the exudates detection and in the case of macular oedema detection the sensitivity 97.75%, selectivity 100%, and accuracy 98.86% is achieved. The performance comparison with other methods reveals that their method can be used as a screening process for diabetic retinopathy. In addition to that, the algorithm can help to detect macular oedema.

An experimental evaluation of the accuracy of keypoints-based retinal image registration

This work regards an investigation of the accuracy of a state-of-the-art, keypoint-based retinal image registration approach, as to the type of keypoint features used to guide the registration process. The employed registration approach is a local method that incorporates the notion of a 3D retinal surface imaged from different viewpoints and has been shown, experimentally, to be more accurate than competing approaches. The correspondences obtained between SIFT, SURF, Harris-PIIFD and vessel bifurcations are studied, either individually or in combinations. The combination of SIFT features with vessel bifurcations was found to perform better than other combinations or any individual feature type, alone. The registration approach is also comparatively evaluated against representative methods of the state-of-the-art in retinal image registration, using a benchmark dataset that covers a broad range of cases regarding the overlap of the acquired images and the anatomical characteristics of the imaged retinas.

Retinal image registration through simultaneous camera pose and eye shape estimation

In this paper, a retinal image registration method is proposed. The approach utilizes keypoint correspondences and assumes that the human eye has a spherical or ellipsoidal shape. The image registration problem amounts to solving a camera 3D pose estimation problem and, simultaneously, an eye 3D shape estimation problem. The camera pose estimation problem is solved by estimating the relative pose between the views from which the images were acquired. The eye shape estimation problem parameterizes the shape and orientation of an ellipsoidal model for the eye. Experimental evaluation shows 17.91% reduction of registration error and 47.52% reduction of the error standard deviation over state of the art methods.

Retinal image registration under the assumption of a spherical eye

We propose a method for registering a pair of retinal images. The proposed approach employs point correspondences and assumes that the human eye has a spherical shape. The image registration problem is formulated as a 3D pose estimation problem, solved by estimating the rigid transformation that relates the views from which the two images were acquired. Given this estimate, each image can be warped upon the other so that pixels with the same coordinates image the same retinal point. Extensive experimental evaluation shows improved accuracy over state of the art methods, as well as robustness to noise and spurious keypoint matches. Experiments also indicate the method's applicability to the comparative analysis of images from different examinations that may exhibit changes and its applicability to diagnostic support.

Computer-aided diagnosis of diabetic retinopathy: a review

Diabetes mellitus may cause alterations in the retinal microvasculature leading to diabetic retinopathy. Unchecked, advanced diabetic retinopathy may lead to blindness. It can be tedious and time consuming to decipher subtle morphological changes in optic disk, microaneurysms, hemorrhage, blood vessels, macula, and exudates through manual inspection of fundus images. A computer aided diagnosis system can significantly reduce the burden on the ophthalmologists and may alleviate the inter and intra observer variability. This review discusses the available methods of various retinal feature extractions and automated analysis.

Classification of time series of multispectral images with limited training data

Image classification usually requires the availability of reliable reference data collected for the considered image to train supervised classifiers. Unfortunately when time series of images are considered, this is seldom possible because of the costs associated with reference data collection. In most of the applications it is realistic to have reference data available for one or few images of a time series acquired on the area of interest. In this paper, we present a novel system for automatically classifying image time series that takes advantage of image(s) with an associated reference information (i.e., the source domain) to classify image(s) for which reference information is not available (i.e., the target domain). The proposed system exploits the already available knowledge on the source domain and, when possible, integrates it with a minimum amount of new labeled data for the target domain. In addition, it is able to handle possible significant differences between statistical distributions of the source and target domains. Here, the method is presented in the context of classification of remote sensing image time series, where ground reference data collection is a highly critical and demanding task. Experimental results show the effectiveness of the proposed technique. The method can work on multimodal (e.g., multispectral) images.

Digital ocular fundus imaging: a review

Ocular fundus imaging plays a key role in monitoring the health status of the human eye. Currently, a large number of imaging modalities allow the assessment and/or quantification of ocular changes from a healthy status. This review focuses on the main digital fundus imaging modality, color fundus photography, with a brief overview of complementary techniques, such as fluorescein angiography. While focusing on two-dimensional color fundus photography, the authors address the evolution from nondigital to digital imaging and its impact on diagnosis. They also compare several studies performed along the transitional path of this technology. Retinal image processing and analysis, automated disease detection and identification of the stage of diabetic retinopathy (DR) are addressed as well. The authors emphasize the problems of image segmentation, focusing on the major landmark structures of the ocular fundus: the vascular network, optic disk and the fovea. Several proposed approaches for the automatic detection of signs of disease onset and progression, such as microaneurysms, are surveyed. A thorough comparison is conducted among different studies with regard to the number of eyes/subjects, imaging modality, fundus camera used, field of view and image resolution to identify the large variation in characteristics from one study to another. Similarly, the main features of the proposed classifications and algorithms for the automatic detection of DR are compared, thereby addressing computer-aided diagnosis and computer-aided detection for use in screening programs.

Automatic summarization of changes in biological image sequences using algorithmic information theory

An algorithmic information-theoretic method is presented for object-level summarization of meaningful changes in image sequences. Object extraction and tracking data are represented as an attributed tracking graph (ATG). Time courses of object states are compared using an adaptive information distance measure, aided by a closed-form multidimensional quantization. The notion of meaningful summarization is captured by using the gap statistic to estimate the randomness deficiency from algorithmic statistics. The summary is the clustering result and feature subset that maximize the gap statistic. This approach was validated on four bioimaging applications: 1) It was applied to a synthetic data set containing two populations of cells differing in the rate of growth, for which it correctly identified the two populations and the single feature out of 23 that separated them; 2) it was applied to 59 movies of three types of neuroprosthetic devices being inserted in the brain tissue at three speeds each, for which it correctly identified insertion speed as the primary factor affecting tissue strain; 3) when applied to movies of cultured neural progenitor cells, it correctly distinguished neurons from progenitors without requiring the use of a fixative stain; and 4) when analyzing intracellular molecular transport in cultured neurons undergoing axon specification, it automatically confirmed the role of kinesins in axon specification.