The use of radial symmetry to localize retinal landmarks

Unpaired fundus image enhancement based on constrained generative adversarial networks

Fundus photography (FP) is a crucial technique for diagnosing the progression of ocular and systemic diseases in clinical studies, with wide applications in early clinical screening and diagnosis. However, due to the nonuniform illumination and imbalanced intensity caused by various reasons, the quality of fundus images is often severely weakened, brings challenges for automated screening, analysis, and diagnosis of diseases. To resolve this problem, we developed strongly constrained generative adversarial networks (SCGAN). The results demonstrate that the quality of various datasets were more significantly enhanced based on SCGAN, simultaneously more effectively retaining tissue and vascular information under various experimental conditions. Furthermore, the clinical effectiveness and robustness of this model were validated by showing its improved ability in vascular segmentation as well as disease diagnosis. Our study provides a new comprehensive approach for FP and also possesses the potential capacity to advance artificial intelligence-assisted ophthalmic examination.

End-to-End Fovea Localisation in Colour Fundus Images With a Hierarchical Deep Regression Network

Accurately locating the fovea is a prerequisite for developing computer aided diagnosis (CAD) of retinal diseases. In colour fundus images of the retina, the fovea is a fuzzy region lacking prominent visual features and this makes it difficult to directly locate the fovea. While traditional methods rely on explicitly extracting image features from the surrounding structures such as the optic disc and various vessels to infer the position of the fovea, deep learning based regression technique can implicitly model the relation between the fovea and other nearby anatomical structures to determine the location of the fovea in an end-to-end fashion. Although promising, using deep learning for fovea localisation also has many unsolved challenges. In this paper, we present a new end-to-end fovea localisation method based on a hierarchical coarse-to-fine deep regression neural network. The innovative features of the new method include a multi-scale feature fusion technique and a self-attention technique to exploit location, semantic, and contextual information in an integrated framework, a multi-field-of-view (multi-FOV) feature fusion technique for context-aware feature learning and a Gaussian-shift-cropping method for augmenting effective training data. We present extensive experimental results on two public databases and show that our new method achieved state-of-the-art performances. We also present a comprehensive ablation study and analysis to demonstrate the technical soundness and effectiveness of the overall framework and its various constituent components.

A robust method for the automatic location of the optic disc and the fovea in fundus images

BACKGROUND AND OBJECTIVE

The location of the optic disc (OD) and the fovea is usually crucial in automatic screening systems for diabetic retinopathy. Previous methods aimed at their location often fail when these structures do not have the standard appearance. The purpose of this work is to propose novel, robust methods for the automatic detection of the OD and the fovea.

METHODS

The proposed method comprises a preprocessing stage, a method for retinal background extraction, a vasculature segmentation phase and the computation of various novel saliency maps. The main novelty of this work is the combination of the proposed saliency maps, which represent the spatial relationships between some structures of the retina and the visual appearance of the OD and fovea. Another contribution is the method to extract the retinal background, based on region-growing.

RESULTS

The proposed methods were evaluated over a proprietary database and three public databases: DRIVE, DiaretDB1 and Messidor. For the OD, we achieved 100% accuracy for all databases except Messidor (99.50%). As for the fovea location, we also reached 100% accuracy for all databases except Messidor (99.67%).

CONCLUSIONS

Our results suggest that the proposed methods are robust and effective to automatically detect the OD and the fovea. This way, they can be useful in automatic screening systems for diabetic retinopathy as well as other retinal diseases.

Convexity shape constraints for retinal blood vessel segmentation and foveal avascular zone detection

Diabetic retinopathy (DR) has become a major worldwide health problem due to the increase in blindness among diabetics at early ages. The detection of DR pathologies such as microaneurysms, hemorrhages and exudates through advanced computational techniques is of utmost importance in patient health care. New computer vision techniques are needed to improve upon traditional screening of color fundus images. The segmentation of the entire anatomical structure of the retina is a crucial phase in detecting these pathologies. This work proposes a novel framework for fast and fully automatic blood vessel segmentation and fovea detection. The preprocessing method involved both contrast limited adaptive histogram equalization and the brightness preserving dynamic fuzzy histogram equalization algorithms to enhance image contrast and eliminate noise artifacts. Afterwards, the color spaces and their intrinsic components were examined to identify the most suitable color model to reveal the foreground pixels against the entire background. Several samples were then collected and used by the renowned convexity shape prior segmentation algorithm. The proposed methodology achieved an average vasculature segmentation accuracy exceeding 96%, 95%, 98% and 94% for the DRIVE, STARE, HRF and Messidor publicly available datasets, respectively. An additional validation step reached an average accuracy of 94.30% using an in-house dataset provided by the Hospital Sant Joan of Reus (Spain). Moreover, an outstanding detection accuracy of over 98% was achieved for the foveal avascular zone. An extensive state-of-the-art comparison was also conducted. The proposed approach can thus be integrated into daily clinical practice to assist medical experts in the diagnosis of DR.

Mathematical Mirroring for Identification of Local Symmetry Centers in Microscopic Images Local Symmetry Detection in FIJI

Symmetry is omnipresent in nature and we encounter symmetry routinely in our everyday life. It is also common on the microscopic level, where symmetry is often key to the proper function of core biological processes. The human brain is exquisitely well suited to recognize such symmetrical features with ease. In contrast, computational recognition of such patterns in images is still surprisingly challenging. In this paper we describe a mathematical approach to identifying smaller local symmetrical structures within larger images. Our algorithm attributes a local symmetry score to each image pixel, which subsequently allows the identification of the symmetrical centers of an object. Though there are already many methods available to detect symmetry in images, to the best of our knowledge, our algorithm is the first that is easily applicable in ImageJ/FIJI. We have created an interactive plugin in FIJI that allows the detection and thresholding of local symmetry values. The plugin combines the different reflection symmetry axis of a square to get a good coverage of reflection symmetry in all directions. To demonstrate the plugins potential, we analyzed images of bacterial chemoreceptor arrays and intracellular vesicle trafficking events, which are two prominent examples of biological systems with symmetrical patterns.

Robust Fovea Localization Based on Symmetry Measure

Automatic fovea localization is a challenging issue. In this article, we focus on the study of fovea localization and propose a robust fovea localization method. We propose concentric circular sectional symmetry measure (CCSSM) for symmetry axis detection, and region of interest (ROI) determination, which is a global feature descriptor robust against local feature changes, to solve the lesion interference issue, i.e., fovea visibility interference from lesions, using both structure features and morphological features. We propose the index of convexity and concavity (ICC) as the convexity-concavity measure of the surface and provide a quantitative evaluation tool for ophthalmologists to learn whether the occurrence of lesion within the ROI. We propose the weighted gradient accumulation map, which is insensitive to local intensity changes and can overcome the influence of noise and contamination, to perform refined localization. The advantages of the proposed method lies in two aspects. First, the accuracy and robustness can be achieved without typical sophisticated manner, i.e., blood vessel segmentation and parabola fitting. Second, the lesion interference is considered in our plan of fovea localization. Our proposed symmetry-based method is innovative in the solution of fovea detection, and it is simple, practical, and controllable. Experiment results show that the proposed method can resist the interference of unbalanced illumination and lesions, and achieve high accuracy rate in five datasets. Compared to the state-of-the-art methods, high robustness and accuracy of the proposed method guarantees its reliability.

Macula segmentation and fovea localization employing image processing and heuristic based clustering for automated retinal screening

BACKGROUND AND OBJECTIVE

Macula segmentation and fovea localization is one of the primary tasks in retinal analysis as they are responsible for detailed vision. Existing approaches required segmentation of retinal structures viz. optic disc and blood vessels for this purpose.

METHOD

This work avoids knowledge of other retinal structures and attempts data mining techniques to segment macula. Unsupervised clustering algorithm is exploited for this purpose. Selection of initial cluster centres has a great impact on performance of clustering algorithms. A heuristic based clustering in which initial centres are selected based on measures defining statistical distribution of data is incorporated in the proposed methodology. The initial phase of proposed framework includes image cropping, green channel extraction, contrast enhancement and application of mathematical closing. Then, the pre-processed image is subjected to heuristic based clustering yielding a binary map. The binary image is post-processed to eliminate unwanted components. Finally, the component which possessed the minimum intensity is finalized as macula and its centre constitutes the fovea.

RESULTS

The proposed approach outperforms existing works by reporting that 100%,of HRF, 100% of DRIVE, 96.92% of DIARETDB0, 97.75% of DIARETDB1, 98.81% of HEI-MED, 90% of STARE and 99.33% of MESSIDOR images satisfy the 1R criterion, a standard adopted for evaluating performance of macula and fovea identification.

CONCLUSION

The proposed system thus helps the ophthalmologists in identifying the macula thereby facilitating to identify if any abnormality is present within the macula region.

Template Matching via Densities on the Roto-Translation Group

We propose a template matching method for the detection of 2D image objects that are characterized by orientation patterns. Our method is based on data representations via orientation scores, which are functions on the space of positions and orientations, and which are obtained via a wavelet-type transform. This new representation allows us to detect orientation patterns in an intuitive and direct way, namely via cross-correlations. Additionally, we propose a generalized linear regression framework for the construction of suitable templates using smoothing splines. Here, it is important to recognize a curved geometry on the position-orientation domain, which we identify with the Lie group SE(2): the roto-translation group. Templates are then optimized in a B-spline basis, and smoothness is defined with respect to the curved geometry. We achieve state-of-the-art results on three different applications: detection of the optic nerve head in the retina (99.83 percent success rate on 1,737 images), of the fovea in the retina (99.32 percent success rate on 1,616 images), and of the pupil in regular camera images (95.86 percent on 1,521 images). The high performance is due to inclusion of both intensity and orientation features with effective geometric priors in the template matching. Moreover, our method is fast due to a cross-correlation based matching approach.

Fast detection of the main anatomical structures in digital retinal images based on intra- and inter-structure relational knowledge

BACKGROUND AND OBJECTIVE

The anatomical structure detection in retinal images is an open problem. However, most of the works in the related literature are oriented to the detection of each structure individually or assume the previous detection of a structure which is used as a reference. The objective of this paper is to obtain simultaneous detection of the main retinal structures (optic disc, macula, network of vessels and vascular bundle) in a fast and robust way.

METHODS

We propose a new methodology oriented to accomplish the mentioned objective. It consists of two stages. In an initial stage, a set of operators is applied to the retinal image. Each operator uses intra-structure relational knowledge in order to produce a set of candidate blobs that belongs to the desired structure. In a second stage, a set of tuples is created, each of which contains a different combination of the candidate blobs. Next, filtering operators, using inter-structure relational knowledge, are used in order to find the winner tuple. A method using template matching and mathematical morphology is implemented following the proposed methodology.

RESULTS

A success is achieved if the distance between the automatically detected blob center and the actual structure center is less than or equal to one optic disc radius. The success rates obtained in the different public databases analyzed were: MESSIDOR (99.33%, 98.58%, 97.92%), DIARETDB1 (96.63%, 100%, 97.75%), DRIONS (100%, n/a, 100%) and ONHSD (100%, 98.85%, 97.70%) for optic disc (OD), macula (M) and vascular bundle (VB), respectively. Finally, the overall success rate obtained in this study for each structure was: 99.26% (OD), 98.69% (M) and 98.95% (VB). The average time of processing per image was 4.16 ± 0.72 s.

CONCLUSIONS

The main advantage of the use of inter-structure relational knowledge was the reduction of the number of false positives in the detection process. The implemented method is able to simultaneously detect four structures. It is fast, robust and its detection results are competitive in relation to other methods of the recent literature.

A Review on Recent Developments for Detection of Diabetic Retinopathy

Diabetic retinopathy is caused by the retinal micro vasculature which may be formed as a result of diabetes mellitus. Blindness may appear as a result of unchecked and severe cases of diabetic retinopathy. Manual inspection of fundus images to check morphological changes in microaneurysms, exudates, blood vessels, hemorrhages, and macula is a very time-consuming and tedious work. It can be made easily with the help of computer-aided system and intervariability for the observer. In this paper, several techniques for detecting microaneurysms, hemorrhages, and exudates are discussed for ultimate detection of nonproliferative diabetic retinopathy. Blood vessels detection techniques are also discussed for the diagnosis of proliferative diabetic retinopathy. Furthermore, the paper elaborates a discussion on the experiments accessed by authors for the detection of diabetic retinopathy. This work will be helpful for the researchers and technical persons who want to utilize the ongoing research in this area.

Modulation of retinal image vasculature analysis to extend utility and provide secondary value from optical coherence tomography imaging

Retinal image analysis is emerging as a key source of biomarkers of chronic systemic conditions affecting the cardiovascular system and brain. The rapid development and increasing diversity of commercial retinal imaging systems present a challenge to image analysis software providers. In addition, clinicians are looking to extract maximum value from the clinical imaging taking place. We describe how existing and well-established retinal vasculature segmentation and measurement software for fundus camera images has been modulated to analyze scanning laser ophthalmoscope retinal images generated by the dual-modality Heidelberg SPECTRALIS(®) instrument, which also features optical coherence tomography.

Automatic retinal vessel classification using a Least Square-Support Vector Machine in VAMPIRE

It is important to classify retinal blood vessels into arterioles and venules for computerised analysis of the vasculature and to aid discovery of disease biomarkers. For instance, zone B is the standardised region of a retinal image utilised for the measurement of the arteriole to venule width ratio (AVR), a parameter indicative of microvascular health and systemic disease. We introduce a Least Square-Support Vector Machine (LS-SVM) classifier for the first time (to the best of our knowledge) to label automatically arterioles and venules. We use only 4 image features and consider vessels inside zone B (802 vessels from 70 fundus camera images) and in an extended zone (1,207 vessels, 70 fundus camera images). We achieve an accuracy of 94.88% and 93.96% in zone B and the extended zone, respectively, with a training set of 10 images and a testing set of 60 images. With a smaller training set of only 5 images and the same testing set we achieve an accuracy of 94.16% and 93.95%, respectively. This experiment was repeated five times by randomly choosing 10 and 5 images for the training set. Mean classification accuracy are close to the above mentioned result. We conclude that the performance of our system is very promising and outperforms most recently reported systems. Our approach requires smaller training data sets compared to others but still results in a similar or higher classification rate.

Optic Disc Boundary and Vessel Origin Segmentation of Fundus Images

This paper presents a novel classification-based optic disc (OD) segmentation algorithm that detects the OD boundary and the location of vessel origin (VO) pixel. First, the green plane of each fundus image is resized and morphologically reconstructed using a circular structuring element. Bright regions are then extracted from the morphologically reconstructed image that lie in close vicinity of the major blood vessels. Next, the bright regions are classified as bright probable OD regions and non-OD regions using six region-based features and a Gaussian mixture model classifier. The classified bright probable OD region with maximum Vessel-Sum and Solidity is detected as the best candidate region for the OD. Other bright probable OD regions within 1-disc diameter from the centroid of the best candidate OD region are then detected as remaining candidate regions for the OD. A convex hull containing all the candidate OD regions is then estimated, and a best-fit ellipse across the convex hull becomes the segmented OD boundary. Finally, the centroid of major blood vessels within the segmented OD boundary is detected as the VO pixel location. The proposed algorithm has low computation time complexity and it is robust to variations in image illumination, imaging angles, and retinal abnormalities. This algorithm achieves 98.8%-100% OD segmentation success and OD segmentation overlap score in the range of 72%-84% on images from the six public datasets of DRIVE, DIARETDB1, DIARETDB0, CHASE_DB1, MESSIDOR, and STARE in less than 2.14 s per image. Thus, the proposed algorithm can be used for automated detection of retinal pathologies, such as glaucoma, diabetic retinopathy, and maculopathy.

Association between retinal vasculature and muscle mass in older people

Sarcopenia in older people is a major health issue and its early detection could help target interventions and improve health. Evidence suggests that poor muscle mass is associated with greater arterial stiffness and cardiovascular risk. Arterial stiffness in turn is associated with smaller retinal artery width. This study examined the association of muscle mass in older people with retinal vascular width, a non-invasive measure of vascular function.

METHODS

Participants >65 years were recruited to a cross-sectional study.

EXCLUSIONS

Inability to walk independently; diabetes mellitus; stroke (within 6 months), severe macular degeneration, glaucoma, retinal dystrophy; advanced cataract. Digital Retinal images of both eyes were analysed using the VAMPIRE software suite. Central Retinal Artery and Vein Equivalents (CRVE and CRAE) were measured. Body composition was measured using Dual Energy X ray Absorptimetry (DXA). Appendicular Skeletal Muscle Mass/Height(2) was calculated. Physical function was measured: 6-min walk distance, Short Physical performance battery, handgrip strength and quadriceps strength.

RESULTS

79 participants with mean age 72 (SD 6) years were recruited. 44% were female. Digital Retinal images of sufficient quality for measuring CRAE and CRVE were available for 51/75 (68%) of participants. Regression analysis showed significant association between larger ASMM/H(2) and smaller CRAE (β=-0.20, p=0.001) and CRVE (β=-0.12, p=0.05). Handgrip strength, body mass index and sex combined with CRAE explained 88% and with CRVE explained 86% of the variance in ASMM/H(2).

CONCLUSION

Larger muscle mass was significantly associated with smaller retinal artery size in older people. This unexpected finding needs further investigation.

Suitability of UK Biobank Retinal Images for Automatic Analysis of Morphometric Properties of the Vasculature

Purpose

To assess the suitability of retinal images held in the UK Biobank - the largest retinal data repository in a prospective population-based cohort - for computer assisted vascular morphometry, generating measures that are commonly investigated as candidate biomarkers of systemic disease.

Methods

Non-mydriatic fundus images from both eyes of 2,690 participants - people with a self-reported history of myocardial infarction (n=1,345) and a matched control group (n=1,345) - were analysed using VAMPIRE software. These images were drawn from those of 68,554 UK Biobank participants who underwent retinal imaging at recruitment. Four operators were trained in the use of the software to measure retinal vascular tortuosity and bifurcation geometry.

Results

Total operator time was approximately 360 hours (4 minutes per image). 2,252 (84%) of participants had at least one image of sufficient quality for the software to process, i.e. there was sufficient detection of retinal vessels in the image by the software to attempt the measurement of the target parameters. 1,604 (60%) of participants had an image of at least one eye that was adequately analysed by the software, i.e. the measurement protocol was successfully completed. Increasing age was associated with a reduced proportion of images that could be processed (p=0.0004) and analysed (p<0.0001). Cases exhibited more acute arteriolar branching angles (p=0.02) as well as lower arteriolar and venular tortuosity (p<0.0001).

Conclusions

A proportion of the retinal images in UK Biobank are of insufficient quality for automated analysis. However, the large size of the UK Biobank means that tens of thousands of images are available and suitable for computational analysis. Parametric information measured from the retinas of participants with suspected cardiovascular disease was significantly different to that measured from a matched control group.

Retinal imaging as a source of biomarkers for diagnosis, characterization and prognosis of chronic illness or long-term conditions

The black void behind the pupil was optically impenetrable before the invention of the ophthalmoscope by von Helmholtz over 150 years ago. Advances in retinal imaging and image processing, especially over the past decade, have opened a route to another unexplored landscape, the retinal neurovascular architecture and the retinal ganglion pathways linking to the central nervous system beyond. Exploiting these research opportunities requires multidisciplinary teams to explore the interface sitting at the border between ophthalmology, neurology and computing science. It is from the detail and depth of retinal phenotyping that novel metrics and candidate biomarkers are likely to emerge. Confirmation that in vivo retinal neurovascular measures are predictive of microvascular change in the brain and other organs is likely to be a major area of research activity over the next decade. Unlocking this hidden potential within the retina requires integration of structural and functional data sets, that is, multimodal mapping and longitudinal studies spanning the natural history of the disease process. And with further advances in imaging, it is likely that this area of retinal research will remain active and clinically relevant for many years to come. Accordingly, this review looks at state-of-the-art retinal imaging and its application to diagnosis, characterization and prognosis of chronic illness or long-term conditions.

Accurate and reliable segmentation of the optic disc in digital fundus images

We describe a complete pipeline for the detection and accurate automatic segmentation of the optic disc in digital fundus images. This procedure provides separation of vascular information and accurate inpainting of vessel-removed images, symmetry-based optic disc localization, and fitting of incrementally complex contour models at increasing resolutions using information related to inpainted images and vessel masks. Validation experiments, performed on a large dataset of images of healthy and pathological eyes, annotated by experts and partially graded with a quality label, demonstrate the good performances of the proposed approach. The method is able to detect the optic disc and trace its contours better than the other systems presented in the literature and tested on the same data. The average error in the obtained contour masks is reasonably close to the interoperator errors and suitable for practical applications. The optic disc segmentation pipeline is currently integrated in a complete software suite for the semiautomatic quantification of retinal vessel properties from fundus camera images (VAMPIRE).

Ocular fundus reference images from optical coherence tomography

Two-dimensional images computed from three-dimensional optical coherence tomography (OCT) data are intrinsically aligned with it, allowing to accurately position a retinal OCT scan within the ocular fundus. In this work, we aim to compute an OCT fundus reference image with improved retinal vasculature extension and contrast over traditional approaches. Based on the shadow casted by hemoglobin on the outer layers of the retina, we compute three independent images from the OCT volumetric data (including the traditional fundus reference image). Combining these images, a fourth one is created that is able to outperform the other three, both quantitatively and qualitatively (as evaluated by retina specialists). The vascular network extension provided by this method was also compared with widely used fundus imaging modalities, showing that it is similar to that achieved with color fundus photography. In this way, the proposed method is an important starting point to the segmentation of the vascular tree and provides users with a detailed fundus reference image.