Fully Automated Estimation of Spacing and Density for Retinal Mosaics

Morphology of the normative human cone photoreceptor mosaic and a publicly available adaptive optics montage repository

Adaptive optics ophthalmoscopy has enabled visualization of the in vivo human photoreceptor mosaic in health, disease and its treatment. Despite this, the clinical utility of the imaging technology has been limited by a lack of automated analysis techniques capable of accurately quantifying photoreceptor structure and a lack of an available normative image database. Here, we present a fully automated algorithm for estimating cone spacing and density over a complete adaptive optics montage along with a database of normative images and cone densities. We imaged the cone mosaics surrounding the fovea and along the horizontal and vertical meridians of fifty normal-sighted controls with a custom-built, multimodal adaptive optics scanning light ophthalmoscope. Cone spacing was automatically measured in the frequency domain and spacing measurements were converted to estimates of cone density at all locations across the montage. Consistent with previous reports, cone density measurements were highest near fovea (152,906 ± 53,209 cones/mm2) and decreased exponentially with eccentricity. A 2.5-fold variation was found in cone density estimates at 0.1 mm, this variation decreased to 1.75-fold at 1 mm. We provide all images, mosaic quantifications, and automated software open source. This database will aid investigators in translating adaptive optics ophthalmoscopy to clinical applications.

Revealing speckle obscured living human retinal cells with artificial intelligence assisted adaptive optics optical coherence tomography

BACKGROUND

In vivo imaging of the human retina using adaptive optics optical coherence tomography (AO-OCT) has transformed medical imaging by enabling visualization of 3D retinal structures at cellular-scale resolution, including the retinal pigment epithelial (RPE) cells, which are essential for maintaining visual function. However, because noise inherent to the imaging process (e.g., speckle) makes it difficult to visualize RPE cells from a single volume acquisition, a large number of 3D volumes are typically averaged to improve contrast, substantially increasing the acquisition duration and reducing the overall imaging throughput.

METHODS

Here, we introduce parallel discriminator generative adversarial network (P-GAN), an artificial intelligence (AI) method designed to recover speckle-obscured cellular features from a single AO-OCT volume, circumventing the need for acquiring a large number of volumes for averaging. The combination of two parallel discriminators in P-GAN provides additional feedback to the generator to more faithfully recover both local and global cellular structures. Imaging data from 8 eyes of 7 participants were used in this study.

RESULTS

We show that P-GAN not only improves RPE cell contrast by 3.5-fold, but also improves the end-to-end time required to visualize RPE cells by 99-fold, thereby enabling large-scale imaging of cells in the living human eye. RPE cell spacing measured across a large set of AI recovered images from 3 participants were in agreement with expected normative ranges.

CONCLUSIONS

The results demonstrate the potential of AI assisted imaging in overcoming a key limitation of RPE imaging and making it more accessible in a routine clinical setting.

Spatially-dependent model for rods and cones in the retina

We develop a mathematical model for photoreceptors in the retina. We focus on rod and cone outer segment dynamics and interactions with a nutrient source associated with the retinal pigment epithelium cells. Rod and cone densities (number per unit area of retinal surface) are known to have significant spatial dependence in the retina with cones located primarily near the fovea and the rods located primarily away from the fovea. Our model accounts for this spatial dependence of the rod and cone photoreceptor density as well as for the possibility of nutrient diffusion. We present equilibrium and dynamic solutions, discuss their relation to existing models, and estimate model parameters through comparisons with available experimental measurements of both spatial and temporal photoreceptor characteristics. Our model compares well with existing data on spatially-dependent regrowth of photoreceptor outer segments in the macular region of Rhesus Monkeys. Our predictions are also consistent with existing data on the spatial dependence of photoreceptor outer segment length near the fovea in healthy human subjects. We focus primarily on the healthy eye but our model could be the basis for future efforts designed to explore various retinal pathologies, eye-related injuries, and treatments of these conditions.

Deep learning-enabled volumetric cone photoreceptor segmentation in adaptive optics optical coherence tomography images of normal and diseased eyes

Objective quantification of photoreceptor cell morphology, such as cell diameter and outer segment length, is crucial for early, accurate, and sensitive diagnosis and prognosis of retinal neurodegenerative diseases. Adaptive optics optical coherence tomography (AO-OCT) provides three-dimensional (3-D) visualization of photoreceptor cells in the living human eye. The current gold standard for extracting cell morphology from AO-OCT images involves the tedious process of 2-D manual marking. To automate this process and extend to 3-D analysis of the volumetric data, we propose a comprehensive deep learning framework to segment individual cone cells in AO-OCT scans. Our automated method achieved human-level performance in assessing cone photoreceptors of healthy and diseased participants captured with three different AO-OCT systems representing two different types of point scanning OCT: spectral domain and swept source.

Comprehensive automatic processing and analysis of adaptive optics flood illumination retinal images on healthy subjects

This work presents a novel fully automated method for retinal analysis in images acquired with a flood illuminated adaptive optics retinal camera (AO-FIO). The proposed processing pipeline consists of several steps: First, we register single AO-FIO images in a montage image capturing a larger retinal area. The registration is performed by combination of phase correlation and the scale-invariant feature transform method. A set of 200 AO-FIO images from 10 healthy subjects (10 images from left eye and 10 images from right eye) is processed into 20 montage images and mutually aligned according to the automatically detected fovea center. As a second step, the photoreceptors in the montage images are detected using a method based on regional maxima localization, where the detector parameters were determined with Bayesian optimization according to manually labeled photoreceptors by three evaluators. The detection assessment, based on Dice coefficient, ranges from 0.72 to 0.8. In the next step, the corresponding density maps are generated for each of the montage images. As a final step, representative averaged photoreceptor density maps are created for the left and right eye and thus enabling comprehensive analysis across the montage images and a straightforward comparison with available histological data and other published studies. Our proposed method and software thus enable us to generate AO-based photoreceptor density maps for all measured locations fully automatically, and thus it is suitable for large studies, as those are in pressing need for automated approaches. In addition, the application MATADOR (MATlab ADaptive Optics Retinal Image Analysis) that implements the described pipeline and the dataset with photoreceptor labels are made publicly available.

Short-term Assessment of Subfoveal Injection of Adeno-Associated Virus-Mediated hCHM Gene Augmentation in Choroideremia Using Adaptive Optics Ophthalmoscopy

Importance

Subretinal injection for gene augmentation in retinal degenerations forcefully detaches the neural retina from the retinal pigment epithelium, potentially damaging photoreceptors and/or retinal pigment epithelium cells.

Objective

To use adaptive optics scanning light ophthalmoscopy (AOSLO) to assess the short-term integrity of the cone mosaic following subretinal injections of adeno-associated virus vector designed to deliver a functional version of the CHM gene (AAV2-hCHM) in patients with choroideremia.

Design, Setting, and Participants

This longitudinal case series study enrolled adult patients with choroideremia from February 2015 to January 2016 in the US. To be included in the study, study participants must have received uniocular subfoveal injections of low-dose (5 × 1010 vector genome per eye) or high-dose (1 × 1011 vector genome per eye) AAV2-hCHM. Analysis began February 2015.

Main Outcomes and Measures

The macular regions of both eyes were imaged before and 1 month after injection using a custom-built multimodal AOSLO. Postinjection cone inner segment mosaics were compared with preinjection mosaics at multiple regions of interest. Colocalized spectral-domain optical coherence tomography and dark-adapted cone sensitivity was also acquired at each time point.

Results

Nine study participants ranged in age from 26 to 50 years at the time of enrollment, and all were White men. Postinjection AOSLO images showed preservation of the cone mosaic in all 9 AAV2-hCHM-injected eyes. Mosaics appeared intact and contiguous 1 month postinjection, with the exception of foveal disruption in 1 patient. Optical coherence tomography showed foveal cone outer segment shortening postinjection. Cone-mediated sensitivities were unchanged in 8 of 9 injected and 9 of 9 uninjected eyes. One participant showed acute loss of foveal optical coherence tomography cone outer segment-related signals along with cone sensitivity loss that colocalized with disruption of the mosaic on AOSLO.

Conclusions and Relevance

Integrity of the cone mosaic is maintained following subretinal delivery of AAV2-hCHM, providing strong evidence in support of the safety of the injections. Minor foveal thinning observed following surgery corresponds with short-term cone outer segment shortening rather than cone cell loss. Foveal cone loss in 1 participant raises the possibility of individual vulnerability to the subretinal injection.

Promises and pitfalls of evaluating photoreceptor-based retinal disease with adaptive optics scanning light ophthalmoscopy (AOSLO)

Adaptive optics scanning light ophthalmoscopy (AOSLO) allows visualization of the living human retina with exquisite single-cell resolution. This technology has improved our understanding of normal retinal structure and revealed pathophysiological details of a number of retinal diseases. Despite the remarkable capabilities of AOSLO, it has not seen the widespread commercial adoption and mainstream clinical success of other modalities developed in a similar time frame. Nevertheless, continued advancements in AOSLO hardware and software have expanded use to a broader range of patients. Current devices enable imaging of a number of different retinal cell types, with recent improvements in stimulus and detection schemes enabling monitoring of retinal function, microscopic structural changes, and even subcellular activity. This has positioned AOSLO for use in clinical trials, primarily as exploratory outcome measures or biomarkers that can be used to monitor disease progression or therapeutic response. AOSLO metrics could facilitate patient selection for such trials, to refine inclusion criteria or to guide the choice of therapy, depending on the presence, absence, or functional viability of specific cell types. Here we explore the potential of AOSLO retinal imaging by reviewing clinical applications as well as some of the pitfalls and barriers to more widespread clinical adoption.

Towards distortion-free imaging of the eye

The high power of the eye and optical components used to image it result in "static" distortion, remaining constant across acquired retinal images. In addition, raster-based systems sample points or lines of the image over time, suffering from "dynamic" distortion due to the constant motion of the eye. We recently described an algorithm which corrects for the latter problem but is entirely blind to the former. Here, we describe a new procedure termed "DIOS" (Dewarp Image by Oblique Shift) to remove static distortion of arbitrary type. Much like the dynamic correction method, it relies on locating the same tissue in multiple frames acquired as the eye moves through different gaze positions. Here, the resultant maps of pixel displacement are used to form a sparse system of simultaneous linear equations whose solution gives the common warp seen by all frames. We show that the method successfully handles torsional movement of the eye. We also show that the output of the previously described dynamic correction procedure may be used as input for this new procedure, recovering an image of the tissue that is, in principle, a faithful replica free of any type of distortion. The method could be extended beyond ocular imaging, to any kind of imaging system in which the image can move or be made to move across the detector.

Automated image processing pipeline for adaptive optics scanning light ophthalmoscopy

To mitigate the substantial post-processing burden associated with adaptive optics scanning light ophthalmoscopy (AOSLO), we have developed an open-source, automated AOSLO image processing pipeline with both "live" and "full" modes. The live mode provides feedback during acquisition, while the full mode is intended to automatically integrate the copious disparate modules currently used in generating analyzable montages. The mean (±SD) lag between initiation and montage placement for the live pipeline was 54.6 ± 32.7s. The full pipeline reduced overall human operator time by 54.9 ± 28.4%, with no significant difference in resultant cone density metrics. The reduced overhead decreases both the technical burden and operating cost of AOSLO imaging, increasing overall clinical accessibility.

Integrating adaptive optics-SLO and OCT for multimodal visualization of the human retinal pigment epithelial mosaic

In vivo imaging of human retinal pigment epithelial (RPE) cells has been demonstrated through multiple adaptive optics (AO)-based modalities. However, whether consistent and complete information regarding the cellular structure of the RPE mosaic is obtained across these modalities remains uncertain due to limited comparisons performed in the same eye. Here, an imaging platform combining multimodal AO-scanning light ophthalmoscopy (AO-SLO) with AO-optical coherence tomography (AO-OCT) is developed to make a side-by-side comparison of the same RPE cells imaged across four modalities: AO-darkfield, AO-enhanced indocyanine green (AO-ICG), AO-infrared autofluorescence (AO-IRAF), and AO-OCT. Co-registered images were acquired in five subjects, including one patient with choroideremia. Multimodal imaging provided multiple perspectives of the RPE mosaic that were used to explore variations in RPE cell contrast in a subject-, location-, and even cell-dependent manner. Estimated cell-to-cell spacing and density were found to be consistent both across modalities and with normative data. Multimodal images from a patient with choroideremia illustrate the benefit of using multiple modalities to infer the cellular structure of the RPE mosaic in an affected eye, in which disruptions to the RPE mosaic may locally alter the signal strength, visibility of individual RPE cells, or even source of contrast in unpredictable ways.

Human Eye Optics within a Non-Euclidian Geometrical Approach and Some Implications in Vision Prosthetics Design

An analogy with our previously published theory on the ionospheric auroral gyroscope provides a new perspective in human eye optics. Based on cone cells' real distribution, we model the human eye macula as a pseudospherical surface. This allows the rigorous description of the photoreceptor cell densities in the parafoveal zones modeled further by an optimized paving method. The hexagonal photoreceptors' distribution has been optimally projected on the elliptical pseudosphere, thus designing a prosthetic array counting almost 7000 pixel points. Thanks to the high morphological similarities to a normal human retina, the visual prosthesis performance in camera-free systems might be significantly improved.

Adaptive-glasses time-domain FFOCT for wide-field high-resolution retinal imaging with increased SNR

The highest three-dimensional (3D) resolution possible in in vivo retinal imaging is achieved by combining optical coherence tomography (OCT) and adaptive optics. However, this combination brings important limitations, such as small field-of-view and complex, cumbersome systems, preventing so far the translation of this technology from the research lab to clinics. In this Letter, we mitigate these limitations by combining our compact time-domain full-field OCT (FFOCT) with a multi-actuator adaptive lens positioned just in front of the eye, in a technique we call the adaptive-glasses wavefront sensorless approach. Through this approach, we demonstrate that ocular aberrations can be corrected, increasing the FFOCT signal-to-noise ratio (SNR) and enabling imaging of different retinal layers with a 3D cellular resolution over a 5^∘×5^∘ field-of-view, without apparent anisoplanatism.

Coherence gate shaping for wide field high-resolution in vivo retinal imaging with full-field OCT

Allying high-resolution with a large field-of-view (FOV) is of great importance in the fields of biology and medicine, but it is particularly challenging when imaging non-flat living samples such as the human retina. Indeed, high-resolution is normally achieved with adaptive optics (AO) and scanning methods, which considerably reduce the useful FOV and increase the system complexity. An alternative technique is time-domain full-field optical coherence tomography (FF-OCT), which has already shown its potential for in-vivo high-resolution retinal imaging. Here, we introduce coherence gate shaping for FF-OCT, to optically shape the coherence gate geometry to match the sample curvature, thus achieving a larger FOV than previously possible. Using this instrument, we obtained high-resolution images of living human photoreceptors close to the foveal center without AO and with a 1 mm × 1 mm FOV in a single shot. This novel advance enables the extraction of photoreceptor-based biomarkers with ease and spatiotemporal monitoring of individual photoreceptors. We compare our findings with AO-assisted ophthalmoscopes, highlighting the potential of FF-OCT, as a compact system, to become a routine clinical imaging technique.