Automatic retinal image registration scheme using global optimization techniques

On the interplay between physical and content priors in deep learning for computational imaging

Deep learning (DL) has been applied extensively in many computational imaging problems, often leading to superior performance over traditional iterative approaches. However, two important questions remain largely unanswered: first, how well can the trained neural network generalize to objects very different from the ones in training? This is particularly important in practice, since large-scale annotated examples similar to those of interest are often not available during training. Second, has the trained neural network learnt the underlying (inverse) physics model, or has it merely done something trivial, such as memorizing the examples or point-wise pattern matching? This pertains to the interpretability of machine-learning based algorithms. In this work, we use the Phase Extraction Neural Network (PhENN) [Optica 4, 1117-1125 (2017)], a deep neural network (DNN) for quantitative phase retrieval in a lensless phase imaging system as the standard platform and show that the two questions are related and share a common crux: the choice of the training examples. Moreover, we connect the strength of the regularization effect imposed by a training set to the training process with the Shannon entropy of images in the dataset. That is, the higher the entropy of the training images, the weaker the regularization effect can be imposed. We also discover that weaker regularization effect leads to better learning of the underlying propagation model, i.e. the weak object transfer function, applicable for weakly scattering objects under the weak object approximation. Finally, simulation and experimental results show that better cross-domain generalization performance can be achieved if DNN is trained on a higher-entropy database, e.g. the ImageNet, than if the same DNN is trained on a lower-entropy database, e.g. MNIST, as the former allows the underlying physics model be learned better than the latter.

Phase extraction neural network (PhENN) with coherent modulation imaging (CMI) for phase retrieval at low photon counts

Imaging with low-dose light is of importance in various fields, especially when minimizing radiation-induced damage onto samples is desirable. The raw image captured at the detector plane is then predominantly a Poisson random process with Gaussian noise added due to the quantum nature of photo-electric conversion. Under such noisy conditions, highly ill-posed problems such as phase retrieval from raw intensity measurements become prone to strong artifacts in the reconstructions; a situation that deep neural networks (DNNs) have already been shown to be useful at improving. Here, we demonstrate that random phase modulation on the optical field, also known as coherent modulation imaging (CMI), in conjunction with the phase extraction neural network (PhENN) and a Gerchberg-Saxton-Fienup (GSF) approximant, further improves resilience to noise of the phase-from-intensity imaging problem. We offer design guidelines for implementing the CMI hardware with the proposed computational reconstruction scheme and quantify reconstruction improvement as function of photon count.

Morphological Band Registration of Multispectral Cameras for Water Quality Analysis with Unmanned Aerial Vehicle

Multispectral imagery contains abundant spectral information on terrestrial and oceanic targets, and retrieval of the geophysical variables of the targets is possible when the radiometric integrity of the data is secured. Multispectral cameras typically require the registration of individual band images because their lens locations for individual bands are often displaced from each other, thereby generating images of different viewing angles. Although this type of displacement can be corrected through a geometric transformation of the image coordinates, a mismatch or misregistration between the bands still remains, owing to the image acquisition timing that differs by bands. Even a short time difference is critical for the image quality of fast-moving targets, such as water surfaces, and this type of deformation cannot be compensated for with a geometric transformation between the bands. This study proposes a novel morphological band registration technique, based on the quantile matching method, for which the correspondence between the pixels of different bands is not sought by their geometric relationship, but by the radiometric distribution constructed in the vicinity of the pixel. In this study, a Micasense Rededge-M camera was operated on an unmanned aerial vehicle and multispectral images of coastal areas were acquired at various altitudes to examine the performance of the proposed method for different spatial scales. To assess the impact of the correction on a geophysical variable, the performance of the proposed method was evaluated for the chlorophyll-a concentration estimation. The results showed that the proposed method successfully removed the noisy spatial pattern caused by misregistration while maintaining the original spatial resolution for both homogeneous scenes and an episodic scene with a red tide outbreak.

Vessel Optimal Transport for Automated Alignment of Retinal Fundus Images

Optimal transport has emerged as a promising and useful tool for supporting modern image processing applications such as medical imaging and scientific visualization. Indeed, the optimal transport theory enables great flexibility in modeling problems related to image registration, as different optimization resources can be successfully used as well as the choice of suitable matching models to align the images. In this paper, we introduce an automated framework for fundus image registration which unifies optimal transport theory, image processing tools, and graph matching schemes into a functional and concise methodology. Given two ocular fundus images, we construct representative graphs which embed in their structures spatial and topological information from the eye's blood vessels. The graphs produced are then used as input by our optimal transport model in order to establish a correspondence between their sets of nodes. Finally, geometric transformations are performed between the images so as to accomplish the registration task properly. Our formulation relies on the solid mathematical foundation of optimal transport as a constrained optimization problem, being also robust when dealing with outliers created during the matching stage. We demonstrate the accuracy and effectiveness of the present framework throughout a comprehensive set of qualitative and quantitative comparisons against several influential state-of-the-art methods on various fundus image databases.

A supervised network for fast image-guided radiotherapy (IGRT) registration

3D/3D image registration in IGRT, which aligns planning Computed Tomography (CT) image set with on board Cone Beam CT (CBCT) image set in a short time with high accuracy, is still a challenge due to its high computational cost and complex anatomical structure of medical image. In order to overcome these difficulties, a new method is proposed which contains a coarse registration and a fine registration. For the coarse registration, a supervised regression convolutional neural networks (CNNs) is used to optimize the spatial variation by minimizing the loss when combine the CT images with the CBCT images. For the fine registration, intensity-based image registration is used to calculate the accurate spatial difference of the input image pairs. A coarse registration can get a rough result with a wide capture range in less than 0.5 s. Sequentially a fine registration can get accurate results in a reasonable short time. RSD-111 T chest phantom was used to test our new method. The set-up error was calculated in less than 10s in time scale, and was reduced to sub-millimeter level in spatial scale. The average residual errors in translation and rotation are within ±0.5 mm and ± 0.2°.

Development of a novel noninvasive system for measurement and imaging of the arterial phase oxygen density ratio in the retinal microcirculation

PURPOSE

This study was conducted in order to develop a novel noninvasive system for measurement and imaging of the arterial oxygen density ratio (ODR) in the retinal microcirculation.

METHODS

We developed a system composed of two digital cameras with two different filters, which were attached to a fundus camera capable of simultaneously obtaining two images. Actual measurements were performed on healthy volunteer eyes (n = 61). A new algorithm for ODR measurement and pixel-level imaging of erythrocytes was constructed from these data. The algorithm was based on the morphological closing operation and the line convergence index filter. For system calibration, we compared and verified the ODR values in arterioles and venules that were specified in advance for 56 eyes with reproducibility. In 10 additional volunteers, ODR measurements and imaging of the arterial phase in the retinal microcirculation corresponding to changes in oxygen saturation of the peripheral arteries at normal breathing and breath holding were performed.

RESULTS

Estimation of incident light to erythrocytes and pixel-level ODR calculation were achieved using the algorithm. The mean ODR values of arterioles and venules were 0.77 ± 0.060 and 1.02 ± 0.067, respectively. It was possible to separate these regions, calibrate at the pixel level, and estimate the arterial phase. In each of the 10 volunteers, changes in the arterial phase ODR corresponding to changes in oxygen saturation of the peripheral arteries were observed before and after breath holding on ODR images. The mean ODR in 10 volunteers was increased by breath holding (p < 0.05).

CONCLUSIONS

We developed a basic system for arterial phase ODR measurement and imaging of the retinal microcirculation. With further validation and development, this may provide a useful tool for evaluating retinal oxygen metabolism in the retinal microcirculation.

Retinal image registration based on keypoint correspondences, spherical eye modeling and camera pose estimation

In this work, an image registration method for two retinal images is proposed. The proposed method utilizes keypoint correspondences and assumes a spherical model of the eye. Image registration is treated as a pose estimation problem, which requires estimation of the rigid transformation that relates the two images. Using this estimate, one image can be warped so that it is registered to the coordinate frame of the other. Experimental evaluation shows improved accuracy over state-of-the-art approaches as well as robustness to noise and spurious keypoint correspondences. Experiments also indicate the method's applicability to diagnostic image enhancement and comparative analysis of images from different examinations.

An experimental evaluation of the accuracy of keypoints-based retinal image registration

This work regards an investigation of the accuracy of a state-of-the-art, keypoint-based retinal image registration approach, as to the type of keypoint features used to guide the registration process. The employed registration approach is a local method that incorporates the notion of a 3D retinal surface imaged from different viewpoints and has been shown, experimentally, to be more accurate than competing approaches. The correspondences obtained between SIFT, SURF, Harris-PIIFD and vessel bifurcations are studied, either individually or in combinations. The combination of SIFT features with vessel bifurcations was found to perform better than other combinations or any individual feature type, alone. The registration approach is also comparatively evaluated against representative methods of the state-of-the-art in retinal image registration, using a benchmark dataset that covers a broad range of cases regarding the overlap of the acquired images and the anatomical characteristics of the imaged retinas.

Retinal image registration through simultaneous camera pose and eye shape estimation

In this paper, a retinal image registration method is proposed. The approach utilizes keypoint correspondences and assumes that the human eye has a spherical or ellipsoidal shape. The image registration problem amounts to solving a camera 3D pose estimation problem and, simultaneously, an eye 3D shape estimation problem. The camera pose estimation problem is solved by estimating the relative pose between the views from which the images were acquired. The eye shape estimation problem parameterizes the shape and orientation of an ellipsoidal model for the eye. Experimental evaluation shows 17.91% reduction of registration error and 47.52% reduction of the error standard deviation over state of the art methods.

Automatic Retinal Image Registration Using Blood Vessel Segmentation and SIFT Feature

Automatic retinal image registration is still a great challenge in computer aided diagnosis and screening system. In this paper, a new retinal image registration method is proposed based on the combination of blood vessel segmentation and scale invariant feature transform (SIFT) feature. The algorithm includes two stages: retinal image segmentation and registration. In the segmentation stage, the blood vessel is segmented by using the guided filter to enhance the vessel structure and the bottom-hat transformation to extract blood vessel. In the registration stage, the SIFT algorithm is adopted to detect the feature of vessel segmentation image, complemented by using a random sample consensus (RANSAC) algorithm to eliminate incorrect matches. We evaluate our method from both segmentation and registration aspects. For segmentation evaluation, we test our method on DRIVE database, which provides manually labeled images from two specialists. The experimental results show that our method achieves 0.9562 in accuracy (Acc), which presents competitive performance compare to other existing segmentation methods. For registration evaluation, we test our method on STARE database, and the experimental results demonstrate the superior performance of the proposed method, which makes the algorithm a suitable tool for automated retinal image analysis.

Accurate Joint-Alignment of Indocyanine Green and Fluorescein Angiograph Sequences for Treatment of Subretinal Lesions

In ophthalmology, aligning images in indocyanine green and fluorescein angiograph sequences is important for the treatment of subretinal lesions. This paper introduces an algorithm that is tailored to align jointly in a common reference space all the images in an angiogram sequence containing both modalities. To overcome the issues of low image contrast and low signal-to-noise ratio for late-phase images, the structural similarity between two images is enhanced using Gabor wavelet transform. Image pairs are pairwise registered and the transformations are simultaneously and globally adjusted for a mutually consistent joint alignment. The joint registration process is incremental and the success depends on the correctness of matches from the pairwise registration. To safeguard the joint process, our system performs the consistency test to exclude incorrect pairwise results automatically to ensure correct matches as more images are jointly aligned. Our dataset consists of 60 sequences of polypoidal choroidal vasculopathy collected by the EVEREST Study Group. On average, each sequence contains 20 images. Our algorithm successfully pairwise registered 95.04% of all image pairs, and joint registered 98.7% of all images, with an average alignment error of 1.58 pixels.

Retinal image registration under the assumption of a spherical eye

We propose a method for registering a pair of retinal images. The proposed approach employs point correspondences and assumes that the human eye has a spherical shape. The image registration problem is formulated as a 3D pose estimation problem, solved by estimating the rigid transformation that relates the views from which the two images were acquired. Given this estimate, each image can be warped upon the other so that pixels with the same coordinates image the same retinal point. Extensive experimental evaluation shows improved accuracy over state of the art methods, as well as robustness to noise and spurious keypoint matches. Experiments also indicate the method's applicability to the comparative analysis of images from different examinations that may exhibit changes and its applicability to diagnostic support.

Orthogonal moments for determining correspondence between vessel bifurcations for retinal image registration

Retinal image registration is a necessary step in diagnosis and monitoring of Diabetes Retinopathy (DR), which is one of the leading causes of blindness. Long term diabetes affects the retinal blood vessels and capillaries eventually causing blindness. This progressive damage to retina and subsequent blindness can be prevented by periodic retinal screening. The extent of damage caused by DR can be assessed by comparing retinal images captured during periodic retinal screenings. During image acquisition at the time of periodic screenings translation, rotation and scale (TRS) are introduced in the retinal images. Therefore retinal image registration is an essential step in automated system for screening, diagnosis, treatment and evaluation of DR. This paper presents an algorithm for registration of retinal images using orthogonal moment invariants as features for determining the correspondence between the dominant points (vessel bifurcations) in the reference and test retinal images. As orthogonal moments are invariant to TRS; moment invariants features around a vessel bifurcation are unaltered due to TRS and can be used to determine the correspondence between reference and test retinal images. The vessel bifurcation points are located in segmented, thinned (mono pixel vessel width) retinal images and labeled in corresponding grayscale retinal images. The correspondence between vessel bifurcations in reference and test retinal image is established based on moment invariants features. Further the TRS in test retinal image with respect to reference retinal image is estimated using similarity transformation. The test retinal image is aligned with reference retinal image using the estimated registration parameters. The accuracy of registration is evaluated in terms of mean error and standard deviation of the labeled vessel bifurcation points in the aligned images. The experimentation is carried out on DRIVE database, STARE database, VARIA database and database provided by local government hospital in Pune, India. The experimental results exhibit effectiveness of the proposed algorithm for registration of retinal images.

Landmark matching based retinal image alignment by enforcing sparsity in correspondence matrix

Retinal image alignment is fundamental to many applications in diagnosis of eye diseases. In this paper, we address the problem of landmark matching based retinal image alignment. We propose a novel landmark matching formulation by enforcing sparsity in the correspondence matrix and offer its solutions based on linear programming. The proposed formulation not only enables a joint estimation of the landmark correspondences and a predefined transformation model but also combines the benefits of the softassign strategy (Chui and Rangarajan, 2003) and the combinatorial optimization of linear programming. We also introduced a set of reinforced self-similarities descriptors which can better characterize local photometric and geometric properties of the retinal image. Theoretical analysis and experimental results with both fundus color images and angiogram images show the superior performances of our algorithms to several state-of-the-art techniques.

Salient feature region: a new method for retinal image registration

Retinal image registration is crucial for the diagnoses and treatments of various eye diseases. A great number of methods have been developed to solve this problem; however, fast and accurate registration of low-quality retinal images is still a challenging problem since the low content contrast, large intensity variance as well as deterioration of unhealthy retina caused by various pathologies. This paper provides a new retinal image registration method based on salient feature region (SFR). We first propose a well-defined region saliency measure that consists of both local adaptive variance and gradient field entropy to extract the SFRs in each image. Next, an innovative local feature descriptor that combines gradient field distribution with corresponding geometric information is then computed to match the SFRs accurately. After that, normalized cross-correlation-based local rigid registration is performed on those matched SFRs to refine the accuracy of local alignment. Finally, the two images are registered by adopting high-order global transformation model with locally well-aligned region centers as control points. Experimental results show that our method is quite effective for retinal image registration.

An approach to automatic blood vessel image registration of microcirculation for blood flow analysis on nude mice

Image registration is often a required and a time-consuming step in blood flow analysis of large microscopic video sequences in vivo. In order to obtain stable images for blood flow analysis, frame-to-frame image matching as a preprocessing step is a solution to the problem of movement during image acquisition. In this paper, microscopic system analysis without fluorescent labelling is performed to provide precise and continuous quantitative data of blood flow rate in individual microvessels of nude mice. The performance properties of several matching metrics are evaluated through simulated image registrations. An automatic image registration programme based on Powell's optimisation search method with low calculation redundancy was implemented. The matching method by variance of ratio is computationally efficient and improves the registration robustness and accuracy in practical application of microcirculation registration. The presented registration method shows acceptable results in close requisition to analyse red blood cell velocities, confirming the scientific potential of the system in blood flow analysis.

Automatic model-based tracing algorithm for vessel segmentation and diameter estimation

An automatic algorithm capable of segmenting the whole vessel tree and calculate vessel diameter and orientation in a digital ophthalmologic image is presented in this work. The algorithm is based on a parametric model of a vessel that can assume arbitrarily complex shape and a simple measure of match that quantifies how well the vessel model matches a given angiographic image. An automatic vessel tracing algorithm is described that exploits the geometric model and actively seeks vessel bifurcation, without user intervention. The proposed algorithm uses the geometric vessel model to determine the vessel diameter at each detected central axis pixel. For this reason, the algorithm is fine tuned using a subset of ophthalmologic images of the publically available DRIVE database, by maximizing vessel segmentation accuracy. The proposed algorithm is then applied to the remaining ophthalmological images of the DRIVE database. The segmentation results of the proposed algorithm compare favorably in terms of accuracy with six other well established vessel detection techniques, outperforming three of them in the majority of the available ophthalmologic images. The proposed algorithm achieves subpixel root mean square central axis positioning error that outperforms the non-expert based vessel segmentation, whereas the accuracy of vessel diameter estimation is comparable to that of the non-expert based vessel segmentation.

Retinal Fundus Image Registration via Vascular Structure Graph Matching

Motivated by the observation that a retinal fundus image may contain some unique geometric structures within its vascular trees which can be utilized for feature matching, in this paper, we proposed a graph-based registration framework called GM-ICP to align pairwise retinal images. First, the retinal vessels are automatically detected and represented as vascular structure graphs. A graph matching is then performed to find global correspondences between vascular bifurcations. Finally, a revised ICP algorithm incorporating with quadratic transformation model is used at fine level to register vessel shape models. In order to eliminate the incorrect matches from global correspondence set obtained via graph matching, we proposed a structure-based sample consensus (STRUCT-SAC) algorithm. The advantages of our approach are threefold: (1) global optimum solution can be achieved with graph matching; (2) our method is invariant to linear geometric transformations; and (3) heavy local feature descriptors are not required. The effectiveness of our method is demonstrated by the experiments with 48 pairs retinal images collected from clinical patients.

A partial intensity invariant feature descriptor for multimodal retinal image registration

Detection of vascular bifurcations is a challenging task in multimodal retinal image registration. Existing algorithms based on bifurcations usually fail in correctly aligning poor quality retinal image pairs. To solve this problem, we propose a novel highly distinctive local feature descriptor named partial intensity invariant feature descriptor (PIIFD) and describe a robust automatic retinal image registration framework named Harris-PIIFD. PIIFD is invariant to image rotation, partially invariant to image intensity, affine transformation, and viewpoint/perspective change. Our Harris-PIIFD framework consists of four steps. First, corner points are used as control point candidates instead of bifurcations since corner points are sufficient and uniformly distributed across the image domain. Second, PIIFDs are extracted for all corner points, and a bilateral matching technique is applied to identify corresponding PIIFDs matches between image pairs. Third, incorrect matches are removed and inaccurate matches are refined. Finally, an adaptive transformation is used to register the image pairs. PIIFD is so distinctive that it can be correctly identified even in nonvascular areas. When tested on 168 pairs of multimodal retinal images, the Harris-PIIFD far outperforms existing algorithms in terms of robustness, accuracy, and computational efficiency.

FABC: retinal vessel segmentation using AdaBoost

This paper presents a method for automated vessel segmentation in retinal images. For each pixel in the field of view of the image, a 41-D feature vector is constructed, encoding information on the local intensity structure, spatial properties, and geometry at multiple scales. An AdaBoost classifier is trained on 789 914 gold standard examples of vessel and nonvessel pixels, then used for classifying previously unseen images. The algorithm was tested on the public digital retinal images for vessel extraction (DRIVE) set, frequently used in the literature and consisting of 40 manually labeled images with gold standard. Results were compared experimentally with those of eight algorithms as well as the additional manual segmentation provided by DRIVE. Training was conducted confined to the dedicated training set from the DRIVE database, and feature-based AdaBoost classifier (FABC) was tested on the 20 images from the test set. FABC achieved an area under the receiver operating characteristic (ROC) curve of 0.9561, in line with state-of-the-art approaches, but outperforming their accuracy ( 0.9597 versus 0.9473 for the nearest performer).

Retinal image registration based on salient feature regions

Retinal image registration is essential and crucial for ophthalmologists to diagnose various diseases. A great number of methods have been developed to solve this problem, however, fast and accurate retinal image registration is still a challenging problem since the great content complexity and low image quality of the unhealthy retina. This paper provides a new retinal image registration method based on salient feature regions (SFR). We first extract the SFR in each image based on a well defined region saliency metric. Next, SFR are matched by using an innovative local feature descriptor. Then we register those matched SFR using local rigid transformation. Finally, we register the two images adopting global second order polynomial transformation with locally rigid registered region centers as control points. Experimental results prove that our method is very fast and accurate, especially quite effective for the low quality retinal images registration.

The edge-driven dual-bootstrap iterative closest point algorithm for registration of multimodal fluorescein angiogram sequence

Motivated by the need for multimodal image registration in ophthalmology, this paper introduces an algorithm which is tailored to jointly align in a common reference space all the images in a complete fluorescein angiogram (FA) sequence, which contains both red-free (RF) and FA images. Our work is inspired by Generalized Dual-Bootstrap Iterative Closest Point (GDB-ICP), which rank-orders Lowe keypoint matches and refines the transformation, going from local and low-order to global and higher-order model, computed from each keypoint match in succession. Albeit GDB-ICP has been shown to be robust in registering images taken under different lighting conditions, the performance is not satisfactory for image pairs with substantial, nonlinear intensity differences. Our algorithm, named Edge-Driven DB-ICP, targeting the least reliable component of GDB-ICP, modifies generation of keypoint matches for initialization by extracting the Lowe keypoints from the gradient magnitude image and enriching the keypoint descriptor with global-shape context using the edge points. Our dataset consists of 60 randomly-selected pathological sequences, each on average having up to two RF and 13 FA images. Edge-Driven DB-ICP successfully registered 92.4% of all pairs, and 81.1% multimodal pairs, whereas GDB-ICP registered 80.1% and 40.1%, respectively. For the joint registration of all images in a sequence, Edge-Driven DB-ICP succeeded in 59 sequences, which is a 23% improvement over GDB-ICP.

Automatic registration of retina images based on genetic techniques

The aim of this paper is to develop an automatic method for the registration of multitemporal digital images of the fundus of the human retina. The images are acquired from the same patient at different times by a color fundus camera. The proposed approach is based on the application of global optimization techniques to previously extracted maps of curvilinear structures in the images to be registered (such structures being represented by the vessels in the human retina): in particular, a genetic algorithm is used, in order to estimate the optimum transformation between the input and the base image. The algorithm is tested on two different types of data, gray scale and color images, and for both types, images with small changes and with large changes are used. The comparison between the registered images using the implemented method and a manual one points out that the proposed algorithm provides an accurate registration. The convergence to a solution is not possible only when dealing with images taken from very different view-points.

Nonlinear registration for scanned retinal images: application to ocular polarimetry

Retinal images of approximately 1 degrees of visual field were recorded with a homemade scanning laser ophthalmoscope. The benefit of using a nonlinear registration technique to improve the summation process when averaging frames, rather than a standard approach based on correlation, was assessed. Results suggest that nonlinear methods can surpass linear transformations, allowing improved contrast and more uniform image quality. The importance of this is also demonstrated with specific polarization measurements to determine the degree of polarization across an imaged retinal area. In such a context, where this parameter of polarization is extracted from a combination of registered images, the benefit of the nonlinear method is further increased.

2-D registration and 3-D shape inference of the retinal fundus from fluorescein images

This study presents methods to 2-D registration of retinal image sequences and 3-D shape inference from fluorescein images. The Y-feature is a robust geometric entity that is largely invariant across modalities as well as across the temporal grey level variations induced by the propagation of the dye in the vessels. We first present a Y-feature extraction method that finds a set of Y-feature candidates using local image gradient information. A gradient-based approach is then used to align an articulated model of the Y-feature to the candidates more accurately while optimizing a cost function. Using mutual information, fitted Y-features are subsequently matched across images, including colors and fluorescein angiographic frames, for registration. To reconstruct the retinal fundus in 3-D, the extracted Y-features are used to estimate the epipolar geometry with a plane-and-parallax approach. The proposed solution provides a robust estimation of the fundamental matrix suitable for plane-like surfaces, such as the retinal fundus. The mutual information criterion is used to accurately estimate the dense disparity map. Our experimental results validate the proposed method on a set of difficult fluorescein image pairs.

[Registration of fundus images for generating wide field composite images of the retina ]

The composition of retinal images presents high demands to the applied methods. Substantially different lighting conditions between the images, glarings and fade-outs within one image, large textureless regions and non-linear distortions are the main challenges. We present a fully automatic algorithm for the registration of images of the human retina and their overlay to wide field montage images combining area-based and point-based approaches. The algorithm combines an area-based as well as a point-based approach for determining similarities between images. Various measures of similarity were investigated, where the normalized correlation coefficient was superior compared to the usual definitions of transinformation. The transformation of the images was based on a quadratic model that can be derived from the spherical surface of the retina. This model was compared to four other parameterized transformations and performed best both visually and quantitatively in terms of measured misregistration. Problems may occur if the images are extremely defocused or contain very little relevant structural information.

Registration of retinal angiograms using self organizing maps

In this paper, an automatic method for registering multimodal retinal images is presented. The method consists of three steps: the vessel centerline detection and extraction of bifurcation points only in the reference image, the automatic correspondence of bifurcation points in the two images using a novel implementation of the Self Organized Maps (SOMs) and the extraction of the parameters of the affine transform using the previously obtained correspondences. The proposed registration algorithm was tested on 24 multimodal retinal pairs and the obtained results show an advantageous performance in terms of accuracy with respect to the manual registration.

Robust detection and classification of longitudinal changes in color retinal fundus images for monitoring diabetic retinopathy

A fully automated approach is presented for robust detection and classification of changes in longitudinal time-series of color retinal fundus images of diabetic retinopathy. The method is robust to: 1) spatial variations in illumination resulting from instrument limitations and changes both within, and between patient visits; 2) imaging artifacts such as dust particles; 3) outliers in the training data; 4) segmentation and alignment errors. Robustness to illumination variation is achieved by a novel iterative algorithm to estimate the reflectance of the retina exploiting automatically extracted segmentations of the retinal vasculature, optic disk, fovea, and pathologies. Robustness to dust artifacts is achieved by exploiting their spectral characteristics, enabling application to film-based, as well as digital imaging systems. False changes from alignment errors are minimized by subpixel accuracy registration using a 12-parameter transformation that accounts for unknown retinal curvature and camera parameters. Bayesian detection and classification algorithms are used to generate a color-coded output that is readily inspected. A multiobserver validation on 43 image pairs from 22 eyes involving nonproliferative and proliferative diabetic retinopathies, showed a 97% change detection rate, a 3% miss rate, and a 10% false alarm rate. The performance in correctly classifying the changes was 99.3%. A self-consistency metric, and an error factor were developed to measure performance over more than two periods. The average self consistency was 94% and the error factor was 0.06%. Although this study focuses on diabetic changes, the proposed techniques have broader applicability in ophthalmology.

High-Performance Medical Image Registration Using New Optimization Techniques

Optimization of a similarity metric is an essential component in intensity-based medical image registration. The increasing availability of parallel computers makes parallelizing some registration tasks an attractive option to increase speed. In this paper, two new deterministic, derivative-free, and intrinsically parallel optimization methods are adapted for image registration. DIviding RECTangles (DIRECT) is a global technique for linearly bounded problems, and multidirectional search (MDS) is a recent local method. The performance of DIRECT, MDS, and hybrid methods using a parallel implementation of Powell's method for local refinement, are compared. Experimental results demonstrate that DIRECT and MDS are robust, accurate, and substantially reduce computation time in parallel implementations.

Hybrid retinal image registration

This work studies retinal image registration in the context of the National Institutes of Health (NIH) Early Treatment Diabetic Retinopathy Study (ETDRS) standard. The ETDRS imaging protocol specifies seven fields of each retina and presents three major challenges for the image registration task. First, small overlaps between adjacent fields lead to inadequate landmark points for feature-based methods. Second, the non-uniform contrast/intensity distributions due to imperfect data acquisition will deteriorate the performance of area-based techniques. Third, high-resolution images contain large homogeneous nonvascular/texureless regions that weaken the capabilities of both feature-based and area-based techniques. In this work, we propose a hybrid retinal image registration approach for ETDRS images that effectively combines both area-based and feature-based methods. Four major steps are involved. First, the vascular tree is extracted by using an efficient local entropy-based thresholding technique. Next, zeroth-order translation is estimated by maximizing mutual information based on the binary image pair (area-based). Then image quality assessment regarding the ETDRS field definition is performed based on the translation model. If the image pair is accepted, higher-order transformations will be involved. Specifically, we use two types of features, landmark points and sampling points, for affine/quadratic model estimation. Three empirical conditions are derived experimentally to control the algorithm progress, so that we can achieve the lowest registration error and the highest success rate. Simulation results on 504 pairs of ETDRS images show the effectiveness and robustness of the proposed algorithm.

Retinal image analysis: concepts, applications and potential

As digital imaging and computing power increasingly develop, so too does the potential to use these technologies in ophthalmology. Image processing, analysis and computer vision techniques are increasing in prominence in all fields of medical science, and are especially pertinent to modern ophthalmology, as it is heavily dependent on visually oriented signs. The retinal microvasculature is unique in that it is the only part of the human circulation that can be directly visualised non-invasively in vivo, readily photographed and subject to digital image analysis. Exciting developments in image processing relevant to ophthalmology over the past 15 years includes the progress being made towards developing automated diagnostic systems for conditions, such as diabetic retinopathy, age-related macular degeneration and retinopathy of prematurity. These diagnostic systems offer the potential to be used in large-scale screening programs, with the potential for significant resource savings, as well as being free from observer bias and fatigue. In addition, quantitative measurements of retinal vascular topography using digital image analysis from retinal photography have been used as research tools to better understand the relationship between the retinal microvasculature and cardiovascular disease. Furthermore, advances in electronic media transmission increase the relevance of using image processing in 'teleophthalmology' as an aid in clinical decision-making, with particular relevance to large rural-based communities. In this review, we outline the principles upon which retinal digital image analysis is based. We discuss current techniques used to automatically detect landmark features of the fundus, such as the optic disc, fovea and blood vessels. We review the use of image analysis in the automated diagnosis of pathology (with particular reference to diabetic retinopathy). We also review its role in defining and performing quantitative measurements of vascular topography, how these entities are based on 'optimisation' principles and how they have helped to describe the relationship between systemic cardiovascular disease and retinal vascular changes. We also review the potential future use of fundal image analysis in telemedicine.

Phase II multicenter randomized study of amifostine for prevention of acute radiation rectal toxicity: topical intrarectal versus subcutaneous application

PURPOSE

To investigate the cytoprotective effect of subcutaneous vs. intrarectal administration of amifostine against acute radiation toxicity.

METHODS AND MATERIALS

Patients were randomized to receive amifostine either intrarectally (Group A, n = 27) or a 500-mg flat dose subcutaneously (Group B, n = 26) before irradiation. Therapy was delivered using a four-field technique with three-dimensional conformal planning. In Group A, 1,500 mg of amifostine was administered intrarectally as an aqueous solution in 40 mL of enema. Two different toxicity scales were used: the European Organization for Research and Treatment of Cancer/Radiation Therapy Oncology Group (RTOG) rectal and urologic toxicity criteria and the Subjective-RectoSigmoid scale based on the endoscopic terminology of the World Organization for Digestive Endoscopy. Objective measurements with rectosigmoidoscopy were performed at baseline and 1-2 days after radiotherapy completion. The area under the curve for the time course of mucositis (RTOG criteria) during irradiation represented the mucositis index.

RESULTS

Intrarectal amifostine was feasible and well tolerated without any systemic or local side effects. According to the RTOG toxicity scale, Group A had superior results with a significantly lower incidence of Grades I-II rectal radiation morbidity (11% vs. 42%, p = 0.04) but inferior results concerning urinary toxicity (48% vs. 15%, p = 0.03). The mean rectal mucositis index and Subjective-RectoSigmoid score were significantly lower in Group A (0.44 vs. 2.45 [p = 0.015] and 3.9 vs. 6.0 [p = 0.01], respectively), and the mean urinary mucositis index was lower in Group B (2.39 vs. 0.34, p < 0.028).

CONCLUSIONS

Intrarectal administration of amifostine (1,500 mg) seemed to have a cytoprotective efficacy in acute radiation rectal mucositis but was inferior to subcutaneous administration in terms of urinary toxicity. Additional randomized studies are needed for definitive decisions concerning the cytoprotection of pelvic irradiated areas.