Resolving single cone inputs to visual receptive fields

Substrip-based registration and automatic montaging of adaptive optics retinal images

Precise registration and montage are critical for high-resolution adaptive optics retinal image analysis but are challenged by rapid eye movement. We present a substrip-based method to improve image registration and facilitate the automatic montaging of adaptive optics scanning laser ophthalmoscopy (AOSLO). The program first batches the consecutive images into groups based on a translation threshold and selects an image with minimal distortion within each group as the reference. Within each group, the software divides each image into multiple strips and calculates the Normalized Cross-Correlation with the reference frame using two substrips at both ends of the whole strip to estimate the strip translation, producing a registered image. Then, the software aligns the registered images of all groups also using a substrip based registration, thereby generating a montage with cell-for-cell precision in the overlapping areas of adjacent frames. The algorithm was evaluated with AOSLO images acquired in human subjects with normal macular health and patients with age-related macular degeneration (AMD). Images with a motion amplitude of up to 448 pixels in the fast scanner direction over a frame of 512 × 512 pixels can be precisely registered. Automatic montage spanning up to 22.6 degrees on the retina was achieved on a cell-to-cell precision with a low misplacement rate of 0.07% (11/16,501 frames) in normal eyes and 0.51% (149/29,051 frames) in eyes with AMD. Substrip based registration significantly improved AOSLO registration accuracy.

Imaging the Retinal Vascular Mural Cells In Vivo: Elucidating the Timeline of Their Loss in Diabetic Retinopathy

BACKGROUND

Vascular mural cells (VMCs) are integral components of the retinal vasculature with critical homeostatic functions such as maintaining the inner blood-retinal barrier and vascular tone, as well as supporting the endothelial cells. Histopathologic donor eye studies have shown widespread loss of pericytes and smooth muscle cells, the 2 main VMC types, suggesting these cells are critical to the pathogenesis of diabetic retinopathy (DR). There remain, however, critical gaps in our knowledge regarding the timeline of VMC demise in human DR.

METHODS

In this study, we address this gap using adaptive optics scanning laser ophthalmoscopy to quantify retinal VMC density in eyes with no retinal disease (healthy), subjects with diabetes without diabetic retinopathy, and those with clinical DR and diabetic macular edema. We also used optical coherence tomography angiography to quantify capillary density of the superficial and deep capillary plexuses in these eyes.

RESULTS

Our results indicate significant VMC loss in retinal arterioles before the appearance of classic clinical signs of DR (diabetes without diabetic retinopathy versus healthy, 5.0±2.0 versus 6.5±2.0 smooth muscle cells per 100 µm; P<0.05), while a significant reduction in capillary VMC density (5.1±2.3 in diabetic macular edema versus 14.9±6.0 pericytes per 100 µm in diabetes without diabetic retinopathy; P=0.01) and capillary density (superficial capillary plexus vessel density, 37.6±3.8 in diabetic macular edema versus 45.5±2.4 in diabetes without diabetic retinopathy; P<0.0001) is associated with more advanced stages of clinical DR, particularly diabetic macular edema.

CONCLUSIONS

Our results offer a new framework for understanding the pathophysiologic course of VMC compromise in DR, which may facilitate the development and monitoring of therapeutic strategies aimed at VMC preservation and potentially the prevention of clinical DR and its associated morbidity. Imaging retinal VMCs provides an unparalleled opportunity to visualize these cells in vivo and may have wider implications in a range of diseases where these cells are disrupted.

Retinal imaging using adaptive optics optical coherence tomography with fast and accurate real-time tracking

One of the main obstacles in high-resolution 3-D retinal imaging is eye motion, which causes blur and distortion artifacts that require extensive post-processing to be corrected. Here, an adaptive optics optical coherence tomography (AOOCT) system with real-time active eye motion correction is presented. Correction of ocular aberrations and of retinal motion is provided by an adaptive optics scanning laser ophthalmoscope (AOSLO) that is optically and electronically combined with the AOOCT system. We describe the system design and quantify its performance. The AOOCT system features an independent focus adjustment that allows focusing on different retinal layers while maintaining the AOSLO focus on the photoreceptor mosaic for high fidelity active motion correction. The use of a high-quality reference frame for eye tracking increases revisitation accuracy between successive imaging sessions, allowing to collect several volumes from the same area. This system enables spatially targeted retinal imaging as well as volume averaging over multiple imaging sessions with minimal correction of motion in post processing.

Functional specificity of afferent connections in visual thalamus

Cells in mouse visual thalamus receive inputs from both eyes. In this issue of Neuron, Bauer et al. (2021) demonstrate that, as in carnivores and primates, only one eye drives cell firing while inputs from the other eye remain functionally silent.

Human gaze is systematically offset from the center of cone topography

The small physical depression of the human retina, the fovea, is the retinal locus of prime visual resolution, achieved by a peaking topography of the light-sensitive cone photoreceptor outer segments^1-3 and a post-receptor wiring scheme preserving high-density sampling.^4,5 Humans dynamically direct their gaze such that the retinal images of objects of interest fall onto the foveola, the central one-degree diameter of the fovea,^6-8 but it is yet unclear whether a relationship between the individual photoreceptor topography at this location and visual fixation behavior exists.^9,10 By employing adaptive optics in vivo imaging and micro-stimulation,^11-13 we created topographical maps of the complete foveolar cone mosaics in both eyes of 20 healthy participants while simultaneously recording the retinal location of a fixated visual object in a psychophysical experiment with cellular resolution. We found that the locus of fixation was systematically shifted away from the topographical center toward a naso-superior quadrant on the retina, about 5 min of arc of visual angle on average, with a mirror symmetrical trend between fellow eyes. In cyclopean view, the topographical centers were superior to the fixated target, corresponding to areas in the visual field usually more distant^14,15 and thus containing higher spatial frequencies. Given the large variability in foveal topography between individuals, and the surprising precision with which fixation is repeatedly directed to just a small bouquet of cones in the foveola, these findings demonstrate a finely tuned, functionally relevant link between the development of the cellular mosaic of photoreceptors and visual behavior.

Promises and pitfalls of evaluating photoreceptor-based retinal disease with adaptive optics scanning light ophthalmoscopy (AOSLO)

Adaptive optics scanning light ophthalmoscopy (AOSLO) allows visualization of the living human retina with exquisite single-cell resolution. This technology has improved our understanding of normal retinal structure and revealed pathophysiological details of a number of retinal diseases. Despite the remarkable capabilities of AOSLO, it has not seen the widespread commercial adoption and mainstream clinical success of other modalities developed in a similar time frame. Nevertheless, continued advancements in AOSLO hardware and software have expanded use to a broader range of patients. Current devices enable imaging of a number of different retinal cell types, with recent improvements in stimulus and detection schemes enabling monitoring of retinal function, microscopic structural changes, and even subcellular activity. This has positioned AOSLO for use in clinical trials, primarily as exploratory outcome measures or biomarkers that can be used to monitor disease progression or therapeutic response. AOSLO metrics could facilitate patient selection for such trials, to refine inclusion criteria or to guide the choice of therapy, depending on the presence, absence, or functional viability of specific cell types. Here we explore the potential of AOSLO retinal imaging by reviewing clinical applications as well as some of the pitfalls and barriers to more widespread clinical adoption.

Strip-based digital image registration for distortion minimization and robust eye motion measurement from scanned ophthalmic imaging systems

Retinal image-based eye motion measurement from scanned ophthalmic imaging systems, such as scanning laser ophthalmoscopy, has allowed for precise real-time eye tracking at sub-micron resolution. However, the constraints of real-time tracking result in a high error tolerance that is detrimental for some eye motion measurement and imaging applications. We show here that eye motion can be extracted from image sequences when these constraints are lifted, and all data is available at the time of registration. Our approach identifies and discards distorted frames, detects coarse motion to generate a synthetic reference frame and then uses it for fine scale motion tracking with improved sensitivity over a larger area. We demonstrate its application here to tracking scanning laser ophthalmoscopy (TSLO) and adaptive optics scanning light ophthalmoscopy (AOSLO), and show that it can successfully capture most of the eye motion across each image sequence, leaving only between 0.1-3.4% of non-blink frames untracked, while simultaneously minimizing image distortions induced from eye motion. These improvements will facilitate precise measurement of fixational eye movements (FEMs) in TSLO and longitudinal tracking of individual cells in AOSLO.

Persistent Dark Cones in Oligocone Trichromacy Revealed by Multimodal Adaptive Optics Ophthalmoscopy

Dark cone photoreceptors, defined as those with diminished or absent reflectivity when observed with adaptive optics (AO) ophthalmoscopy, are increasingly reported in retinal disorders. However, their structural and functional impact remain unclear. Here, we report a 3-year longitudinal study on a patient with oligocone trichromacy (OT) who presented with persistent, widespread dark cones within and near the macula. Diminished electroretinogram (ERG) cone but normal ERG rod responses together with normal color vision confirmed the OT diagnosis. In addition, the patient had normal to near normal visual acuity and retinal sensitivity. Occasional dark gaps in the photoreceptor layer were observed on optical coherence tomography, in agreement with reflectance AO scanning light ophthalmoscopy, which revealed that over 50% of the cones in the fovea were dark, increasing to 74% at 10° eccentricity. In addition, the cone density was 78% lower than normal histologic value at the fovea, and 20-40% lower at eccentricities of 5-15°. Interestingly, color vision testing was near normal at locations where cones were predominantly dark. These findings illustrate how a retina with predominant dark cones that persist over at least 3 years can support near normal central retinal function. Furthermore, this study adds to the growing evidence that cones can continue to survive under non-ideal conditions.

Adaptive optics: principles and applications in ophthalmology

This is a comprehensive review of the principles and applications of adaptive optics (AO) in ophthalmology. It has been combined with flood illumination ophthalmoscopy, scanning laser ophthalmoscopy, as well as optical coherence tomography to image photoreceptors, retinal pigment epithelium (RPE), retinal ganglion cells, lamina cribrosa and the retinal vasculature. In this review, we highlight the clinical studies that have utilised AO to understand disease mechanisms. However, there are some limitations to using AO in a clinical setting including the cost of running an AO imaging service, the time needed to scan patients, the lack of normative databases and the very small size of area imaged. However, it is undoubtedly an exceptional research tool that enables visualisation of the retina at a cellular level.

High-acuity vision from retinal image motion

A mathematical model and a possible neural mechanism are proposed to account for how fixational drift motion in the retina confers a benefit for the discrimination of high-acuity targets. We show that by simultaneously estimating object shape and eye motion, neurons in visual cortex can compute a higher quality representation of an object by averaging out non-uniformities in the retinal sampling lattice. The model proposes that this is accomplished by two separate populations of cortical neurons - one providing a representation of object shape and another representing eye position or motion - which are coupled through specific multiplicative connections. Combined with recent experimental findings, our model suggests that the visual system may utilize principles not unlike those used in computational imaging for achieving "super-resolution" via camera motion.

Temporal information loss in the macaque early visual system

Stimuli that modulate neuronal activity are not always detectable, indicating a loss of information between the modulated neurons and perception. To identify where in the macaque visual system information about periodic light modulations is lost, signal-to-noise ratios were compared across simulated cone photoreceptors, lateral geniculate nucleus (LGN) neurons, and perceptual judgements. Stimuli were drifting, threshold-contrast Gabor patterns on a photopic background. The sensitivity of LGN neurons, extrapolated to populations, was similar to the monkeys' at low temporal frequencies. At high temporal frequencies, LGN sensitivity exceeded the monkeys' and approached the upper bound set by cone photocurrents. These results confirm a loss of high-frequency information downstream of the LGN. However, this loss accounted for only about 5% of the total. Phototransduction accounted for essentially all of the rest. Together, these results show that low temporal frequency information is lost primarily between the cones and the LGN, whereas high-frequency information is lost primarily within the cones, with a small additional loss downstream of the LGN.

Color, Pattern, and the Retinal Cone Mosaic

Textbook trichromacy accounts for human color vision in terms of spectral sampling by three classes of cone photoreceptors. This account neglects entangling of color and pattern information created by wavelength-dependent optical blur (chromatic aberrations) and interleaved spatial sampling of the retinal image by the three classes of cones. Recent experimental, computational, and neurophysiological work is now considering color and pattern vision at the elementary scale of daylight vison, that is at the scale of individual cones. The results provide insight about rich interactions between color and pattern vision as well as the role of the statistical structure of natural scenes in shaping visual processing.

Two-photon microperimetry: sensitivity of human photoreceptors to infrared light

Microperimetry is a subjective ophthalmologic test used to assess retinal function at various specific and focal locations of the visual field. Historically, visible light has been described as ranging from 400 to 720 nm. However, we previously demonstrated that infra-red light can initiate visual transduction in rod photoreceptors by a mechanism of two-photon absorption by visual pigments. Here we introduce a newly designed and constructed two-photon microperimeter. We provide for the first time evidence of the presence of a nonlinear process occurring in the human retina based on psychophysical tests using newly developed instrumentation. Since infra-red light penetrates the aged front of the eye better than visible light, it has the potential for improved functional diagnostics in patients with age-related visual disorders.

The spectral identity of foveal cones is preserved in hue perception

Organisms are faced with the challenge of making inferences about the physical world from incomplete incoming sensory information. One strategy to combat ambiguity in this process is to combine new information with prior experiences. We investigated the strategy of combining these information sources in color vision. Single cones in human subjects were stimulated and the associated percepts were recorded. Subjects rated each flash for brightness, hue, and saturation. Brightness ratings were proportional to stimulus intensity. Saturation was independent of intensity, but varied between cones. Hue, in contrast, was assigned in a stereotyped manner that was predicted by cone type. These experiments revealed that, near the fovea, long and middle wavelength sensitive cones produce sensations that can be reliably distinguished on the basis of hue, but not saturation or brightness. Taken together, these observations implicate the high-resolution, color-opponent parvocellular pathway in this low-level visual task.

Probing Computation in the Primate Visual System at Single-Cone Resolution

Daylight vision begins when light activates cone photoreceptors in the retina, creating spatial patterns of neural activity. These cone signals are then combined and processed in downstream neural circuits, ultimately producing visual perception. Recent technical advances have made it possible to deliver visual stimuli to the retina that probe this processing by the visual system at its elementary resolution of individual cones. Physiological recordings from nonhuman primate retinas reveal the spatial organization of cone signals in retinal ganglion cells, including how signals from cones of different types are combined to support both spatial and color vision. Psychophysical experiments with human subjects characterize the visual sensations evoked by stimulating a single cone, including the perception of color. Future combined physiological and psychophysical experiments focusing on probing the elementary visual inputs are likely to clarify how neural processing generates our perception of the visual world.

Functional architecture of the foveola revealed in the living primate

The primate foveola, with its high cone density and magnified cortical representation, is exquisitely specialized for high-resolution spatial vision. However, uncovering the wiring of retinal circuitry responsible for this performance has been challenging due to the difficulty in recording receptive fields of foveal retinal ganglion cells (RGCs) in vivo. In this study, we use adaptive optics scanning laser ophthalmoscopy (AOSLO) to image the calcium responses of RGCs in the living primate, with a stable, high precision visual stimulus that allowed us to localize the receptive fields of hundreds of foveal ganglion cells. This approach revealed a precisely radial organization of foveal RGCs, despite the many distortions possible during the extended developmental migration of foveal cells. By back projecting the line connecting RGC somas to their receptive fields, we have been able to define the ‘physiological center’ of the foveola, locating the vertical meridian separating left and right hemifields in vivo.

Light propagation and capture in cone photoreceptors

The light capturing properties of cone photoreceptors create the elementary signals that form the basis of vision. Variation in the amplitude of individual cone signals has been found physiologically as part of normal retinal circuit processing. Less well characterized is how cone signals may vary due to purely optical properties. We present a model of light propagation in cones using a finite difference beam propagation method to simulate how light from a small stimulus travels through a cone plus its immediate neighbors. The model calculates the amount of light absorbed in the cone outer segments, from which an estimate of the photoresponse can be made. We apply the method to adaptive optics microstimulation to find the optimum optical conditions that will confine the most light into a single cone in the human retina. We found that light capture is especially sensitive to beam size at the pupil and to the cone diameter itself, with the two factors having a complex relationship leading to sizable variation in light capture. Model predictions were validated with two types of psychophysical data. The model can be employed with arbitrary stimuli and photoreceptor parameters, making it a useful tool for studying photoreceptor function in normal or diseased conditions.

Clinical Impact of Spontaneous Hyperactivity in Degenerating Retinas: Significance for Diagnosis, Symptoms, and Treatment

Hereditary retinal degenerations result from varied pathophysiologic mechanisms, all ultimately characterized by photoreceptor dysfunction and death. Hence, much research on these diseases has concentrated on the outer retina. Over the past decade or so increasing attention has focused on concomitant changes in complex inner retinal neural circuits that process visual signals for transmission to the brain. One striking abnormality develops before the ultimately profound anatomic disruption of the inner retina. Highly elevated spontaneous activity was first demonstrated in central nervous system visual centers in vivo by Dräger and Hubel (1978), and subsequently has been confirmed in vitro, now in multiple animal models and by multiple investigators (see other contributions to this Research Topic). What evidence exists that this phenomenon occurs in human patients with retinal degeneration, and what is the ultimate effect of spontaneous hyperactivity in the output neurons, the retinal ganglion cells? Here I summarize abnormalities of visual perception among patients with retinal degeneration that may arise from hyperactivity. Next, I consider the disruption of neural encoding and anatomic connectivity that may result within the retina and in downstream visual centers of the brain. I then consider how specific characteristics of hyperactivity may distinguish various forms or stages of retinal degeneration, potentially helping in the near future to refine diagnosis and/or treatment choices for different patients. Finally, I review how consideration of these features may help optimize pharmacologic, gene, stem cell, prosthetic or other therapies to forestall visual loss or restore sight.

Sensations from a single M-cone depend on the activity of surrounding S-cones

Color vision requires the activity of cone photoreceptors to be compared in post-receptoral circuitry. Decades of psychophysical measurements have quantified the nature of these comparative interactions on a coarse scale. How such findings generalize to a cellular scale remains unclear. To answer that question, we quantified the influence of surrounding light on the appearance of spots targeted to individual cones. The eye's aberrations were corrected with adaptive optics and retinal position was precisely tracked in real-time to compensate for natural movement. Subjects reported the color appearance of each spot. A majority of L-and M-cones consistently gave rise to the sensation of white, while a smaller group repeatedly elicited hue sensations. When blue sensations were reported they were more likely mediated by M- than L-cones. Blue sensations were elicited from M-cones against a short-wavelength light that preferentially elevated the quantal catch in surrounding S-cones, while stimulation of the same cones against a white background elicited green sensations. In one of two subjects, proximity to S-cones increased the probability of blue reports when M-cones were probed. We propose that M-cone increments excited both green and blue opponent pathways, but the relative activity of neighboring cones favored one pathway over the other.

Ultra-high contrast retinal display system for single photoreceptor psychophysics

Due to the enormous dynamic range of human photoreceptors in response to light, studying their visual function in the intact retina challenges the stimulation hardware, specifically with regard to the displayable luminance contrast. The adaptive optics scanning laser ophthalmoscope (AOSLO) is an optical platform that focuses light to extremely small retinal extents, approaching the size of single photoreceptor cells. However, the current light modulation techniques produce spurious visible backgrounds which fundamentally limit experimental options. To remove unwanted background light and to improve contrast for high dynamic range visual stimulation in an AOSLO, we cascaded two commercial fiber-coupled acousto-optic modulators (AOMs) and measured their combined optical contrast. By compensating for zero-point differences in the individual AOMs, we demonstrate a multiplicative extinction ratio in the cascade that was in accordance with the extinction ratios of both single AOMs. When latency differences in the AOM response functions were individually corrected, single switch events as short as 50 ns with radiant power contrasts up to 1:10¹⁰ were achieved. This is the highest visual contrast reported for any display system so far. We show psychophysically that this contrast ratio is sufficient to stimulate single foveal photoreceptor cells with small and bright enough visible targets that do not contain a detectable background. Background-free stimulation will enable photoreceptor testing with custom adaptation lights. Furthermore, a larger dynamic range in displayable light levels can drive photoreceptor responses in cones as well as in rods.

Spatiochromatic Interactions between Individual Cone Photoreceptors in the Human Retina

A remarkable feature of human vision is that the retina and brain have evolved circuitry to extract useful spatial and spectral information from signals originating in a photoreceptor mosaic with trichromatic constituents that vary widely in their relative numbers and local spatial configurations. A critical early transformation applied to cone signals is horizontal-cell-mediated lateral inhibition, which imparts a spatially antagonistic surround to individual cone receptive fields, a signature inherited by downstream neurons and implicated in color signaling. In the peripheral retina, the functional connectivity of cone inputs to the circuitry that mediates lateral inhibition is not cone-type specific, but whether these wiring schemes are maintained closer to the fovea remains unsettled, in part because central retinal anatomy is not easily amenable to direct physiological assessment. Here, we demonstrate how the precise topography of the long (L)-, middle (M)-, and short (S)-wavelength-sensitive cones in the human parafovea (1.5° eccentricity) shapes perceptual sensitivity. We used adaptive optics microstimulation to measure psychophysical detection thresholds from individual cones with spectral types that had been classified independently by absorptance imaging. Measured against chromatic adapting backgrounds, the sensitivities of L and M cones were, on average, receptor-type specific, but individual cone thresholds varied systematically with the number of preferentially activated cones in the immediate neighborhood. The spatial and spectral patterns of these interactions suggest that interneurons mediating lateral inhibition in the central retina, likely horizontal cells, establish functional connections with L and M cones indiscriminately, implying that the cone-selective circuitry supporting red-green color vision emerges after the first retinal synapse.SIGNIFICANCE STATEMENT We present evidence for spatially antagonistic interactions between individual, spectrally typed cones in the central retina of human observers using adaptive optics. Using chromatic adapting fields to modulate the relative steady-state activity of long (L)- and middle (M)-wavelength-sensitive cones, we found that single-cone detection thresholds varied predictably with the spectral demographics of the surrounding cones. The spatial scale and spectral pattern of these photoreceptor interactions were consistent with lateral inhibition mediated by retinal horizontal cells that receive nonselective input from L and M cones. These results demonstrate a clear link between the neural architecture of the visual system inputs-cone photoreceptors-and visual perception and have implications for the neural locus of the cone-specific circuitry supporting color vision.

Benefits of retinal image motion at the limits of spatial vision

Even during fixation, our eyes are constantly in motion, creating an ever-changing signal in each photoreceptor. Neuronal processes can exploit such transient signals to serve spatial vision, but it is not known how our finest visual acuity—one that we use for deciphering small letters or identifying distant faces and objects—is maintained when confronted with such change. We used an adaptive optics scanning laser ophthalmoscope to precisely control the spatiotemporal input on a photoreceptor scale in human observers during a visual discrimination task under conditions with habitual, cancelled or otherwise manipulated retinal image motion. We found that when stimuli moved, acuities were about 25% better than when no motion occurred, regardless of whether that motion was self-induced, a playback of similar motion, or an external simulation. We argue that in our particular experimental condition, the visual system is able to synthesize a higher resolution percept from multiple views of a poorly resolved image, a hypothesis that might extend the current understanding of how fixational eye motion serves high acuity vision.

Transverse chromatic aberration across the visual field of the human eye

The purpose of this study was to measure the transverse chromatic aberration (TCA) across the visual field of the human eye objectively. TCA was measured at horizontal and vertical field angles out to ±15° from foveal fixation in the right eye of four subjects. Interleaved retinal images were taken at wavelengths 543 nm and 842 nm in an adaptive optics scanning laser ophthalmoscope (AOSLO). To obtain true measures of the human eye's TCA, the contributions of the AOSLO system's TCA were measured using an on-axis aligned model eye and subtracted from the ocular data. The increase in TCA was found to be linear with eccentricity, with an average slope of 0.21 arcmin/degree of visual field angle (corresponding to 0.41 arcmin/degree for 430 nm to 770 nm). The absolute magnitude of ocular TCA varied between subjects, but was similar to the resolution acuity at 10° in the nasal visual field, encompassing three to four cones. Therefore, TCA can be visually significant. Furthermore, for high-resolution imaging applications, whether visualizing or stimulating cellular features in the retina, it is important to consider the lateral displacements between wavelengths and the variation in blur over the visual field.

Photoreceptor-Based Biomarkers in AOSLO Retinal Imaging

Improved understanding of the mechanisms underlying inherited retinal degenerations has created the possibility of developing much needed treatments for these relentless, blinding diseases. However, standard clinical indicators of retinal health (such as visual acuity and visual field sensitivity) are insensitive measures of photoreceptor survival. In many retinal degenerations, significant photoreceptor loss must occur before measurable differences in visual function are observed. Thus, there is a recognized need for more sensitive outcome measures to assess therapeutic efficacy as numerous clinical trials are getting underway. Adaptive optics (AO) retinal imaging techniques correct for the monochromatic aberrations of the eye and can be used to provide nearly diffraction-limited images of the retina. Many groups routinely are using AO imaging tools to obtain in vivo images of the rod and cone photoreceptor mosaic, and it now is possible to monitor photoreceptor structure over time with single cell resolution. Highlighting recent work using AO scanning light ophthalmoscopy (AOSLO) across a range of patient populations, we review the development of photoreceptor-based metrics (e.g., density/geometry, reflectivity, and size) as candidate biomarkers. Going forward, there is a need for further development of automated tools and normative databases, with the latter facilitating the comparison of data sets across research groups and devices. Ongoing and future clinical trials for inherited retinal diseases will benefit from the improved resolution and sensitivity that multimodal AO retinal imaging affords to evaluate safety and efficacy of emerging therapies.

High temporal resolution aberrometry in a 50-eye population and implications for adaptive optics error budget

We formed a database gathering the wavefront aberrations of 50 healthy eyes measured with an original custom-built Shack-Hartmann aberrometer at a temporal frequency of 236 Hz, with 22 lenslets across a 7-mm diameter pupil, for a duration of 20 s. With this database, we draw statistics on the spatial and temporal behavior of the dynamic aberrations of the eye. Dynamic aberrations were studied on a 5-mm diameter pupil and on a 3.4 s sequence between blinks. We noted that, on average, temporal wavefront variance exhibits a n^-2 power-law with radial order n and temporal spectra follow a f^-1.5 power-law with temporal frequency f. From these statistics, we then extract guidelines for designing an adaptive optics system. For instance, we show the residual wavefront error evolution as a function of the number of corrected modes and of the adaptive optics loop frame rate. In particular, we infer that adaptive optics performance rapidly increases with the loop frequency up to 50 Hz, with gain being more limited at higher rates.

Vision science and adaptive optics, the state of the field

Adaptive optics is a relatively new field, yet it is spreading rapidly and allows new questions to be asked about how the visual system is organized. The editors of this feature issue have posed a series of question to scientists involved in using adaptive optics in vision science. The questions are focused on three main areas. In the first we investigate the use of adaptive optics for psychophysical measurements of visual system function and for improving the optics of the eye. In the second, we look at the applications and impact of adaptive optics on retinal imaging and its promise for basic and applied research. In the third, we explore how adaptive optics is being used to improve our understanding of the neurophysiology of the visual system.

Imaging individual neurons in the retinal ganglion cell layer of the living eye

Although imaging of the living retina with adaptive optics scanning light ophthalmoscopy (AOSLO) provides microscopic access to individual cells, such as photoreceptors, retinal pigment epithelial cells, and blood cells in the retinal vasculature, other important cell classes, such as retinal ganglion cells, have proven much more challenging to image. The near transparency of inner retinal cells is advantageous for vision, as light must pass through them to reach the photoreceptors, but it has prevented them from being directly imaged in vivo. Here we show that the individual somas of neurons within the retinal ganglion cell (RGC) layer can be imaged with a modification of confocal AOSLO, in both monkeys and humans. Human images of RGC layer neurons did not match the quality of monkey images for several reasons, including safety concerns that limited the light levels permissible for human imaging. We also show that the same technique applied to the photoreceptor layer can resolve ambiguity about cone survival in age-related macular degeneration. The capability to noninvasively image RGC layer neurons in the living eye may one day allow for a better understanding of diseases, such as glaucoma, and accelerate the development of therapeutic strategies that aim to protect these cells. This method may also prove useful for imaging other structures, such as neurons in the brain.

How Do Efficient Coding Strategies Depend on Origins of Noise in Neural Circuits?

Neural circuits reliably encode and transmit signals despite the presence of noise at multiple stages of processing. The efficient coding hypothesis, a guiding principle in computational neuroscience, suggests that a neuron or population of neurons allocates its limited range of responses as efficiently as possible to best encode inputs while mitigating the effects of noise. Previous work on this question relies on specific assumptions about where noise enters a circuit, limiting the generality of the resulting conclusions. Here we systematically investigate how noise introduced at different stages of neural processing impacts optimal coding strategies. Using simulations and a flexible analytical approach, we show how these strategies depend on the strength of each noise source, revealing under what conditions the different noise sources have competing or complementary effects. We draw two primary conclusions: (1) differences in encoding strategies between sensory systems-or even adaptational changes in encoding properties within a given system-may be produced by changes in the structure or location of neural noise, and (2) characterization of both circuit nonlinearities as well as noise are necessary to evaluate whether a circuit is performing efficiently.

Untangling the Web between Eye and Brain

How is the picture of the visual scene that the eye encodes represented by neural circuits in the brain? In this issue of Cell, Morgan et al. address this question by forming an ultrastructural "connectome" of the mouse's visual thalamus that depicts individual retinal afferents and every contact these form with target relay cells.

Eye-tracking technology for real-time monitoring of transverse chromatic aberration

Objective measurements of transverse chromatic aberration (TCA) between two or more wavelengths with an adaptive optics scanning laser ophthalmoscope (AOSLO) are very accurate, but frequent measurements are impractical in many experimental settings. Here, we demonstrate a pupil tracker that can accurately measure relative changes in TCA that are caused by small shifts in the pupil relative to the AOSLO imaging beam. Corrections for TCA caused by these shifts improve the measurement of TCA as a function of eccentricity, revealing a strong linear relationship. We propose that pupil tracking be integrated into AOSLO systems, where robust and unobtrusive control of TCA is required.

Characterizing the Human Cone Photoreceptor Mosaic via Dynamic Photopigment Densitometry

Densitometry is a powerful tool for the biophysical assessment of the retina. Until recently, this was restricted to bulk spatial scales in living humans. The application of adaptive optics (AO) to the conventional fundus camera and scanning laser ophthalmoscope (SLO) has begun to translate these studies to cellular scales. Here, we employ an AOSLO to perform dynamic photopigment densitometry in order to characterize the optical properties and spectral types of the human cone photoreceptor mosaic. Cone-resolved estimates of optical density and photosensitivity agree well with bulk estimates, although show smaller variability than previously reported. Photopigment kinetics of individual cones derived from their selective bleaching allowed efficient mapping of cone sub-types in human retina. Estimated uncertainty in identifying a cone as long vs middle wavelength was less than 5%, and the total time taken per subject ranged from 3–9 hours. Short wavelength cones were delineated in every subject with high fidelity. The lack of a third cone-type was confirmed in a protanopic subject. In one color normal subject, cone assignments showed 91% correspondence against a previously reported cone-typing method from more than a decade ago. Combined with cone-targeted stimulation, this brings us closer in studying the visual percept arising from a specific cone type and its implication for color vision circuitry.

High-speed adaptive optics for imaging of the living human eye

The discovery of high frequency temporal fluctuation of human ocular wave aberration dictates the necessity of high speed adaptive optics (AO) correction for high resolution retinal imaging. We present a high speed AO system for an experimental adaptive optics scanning laser ophthalmoscope (AOSLO). We developed a custom high speed Shack-Hartmann wavefront sensor and maximized the wavefront detection speed based upon a trade-off among the wavefront spatial sampling density, the dynamic range, and the measurement sensitivity. We examined the temporal dynamic property of the ocular wavefront under the AOSLO imaging condition and improved the dual-thread AO control strategy. The high speed AO can be operated with a closed-loop frequency up to 110 Hz. Experiment results demonstrated that the high speed AO system can provide improved compensation for the wave aberration up to 30 Hz in the living human eye.

Normal Perceptual Sensitivity Arising From Weakly Reflective Cone Photoreceptors

PURPOSE

To determine the light sensitivity of poorly reflective cones observed in retinas of normal subjects, and to establish a relationship between cone reflectivity and perceptual threshold.

METHODS

Five subjects (four male, one female) with normal vision were imaged longitudinally (7-26 imaging sessions, representing 82-896 days) using adaptive optics scanning laser ophthalmoscopy (AOSLO) to monitor cone reflectance. Ten cones with unusually low reflectivity, as well as 10 normally reflective cones serving as controls, were targeted for perceptual testing. Cone-sized stimuli were delivered to the targeted cones and luminance increment thresholds were quantified. Thresholds were measured three to five times per session for each cone in the 10 pairs, all located 2.2 to 3.3° from the center of gaze.

RESULTS

Compared with other cones in the same retinal area, three of 10 monitored dark cones were persistently poorly reflective, while seven occasionally manifested normal reflectance. Tested psychophysically, all 10 dark cones had thresholds comparable with those from normally reflecting cones measured concurrently (P = 0.49). The variation observed in dark cone thresholds also matched the wide variation seen in a large population (n = 56 cone pairs, six subjects) of normal cones; in the latter, no correlation was found between cone reflectivity and threshold (P = 0.0502).

CONCLUSIONS

Low cone reflectance cannot be used as a reliable indicator of cone sensitivity to light in normal retinas. To improve assessment of early retinal pathology, other diagnostic criteria should be employed along with imaging and cone-based microperimetry.

Active eye-tracking for an adaptive optics scanning laser ophthalmoscope

We demonstrate a system that combines a tracking scanning laser ophthalmoscope (TSLO) and an adaptive optics scanning laser ophthalmoscope (AOSLO) system resulting in both optical (hardware) and digital (software) eye-tracking capabilities. The hybrid system employs the TSLO for active eye-tracking at a rate up to 960 Hz for real-time stabilization of the AOSLO system. AOSLO videos with active eye-tracking signals showed, at most, an amplitude of motion of 0.20 arcminutes for horizontal motion and 0.14 arcminutes for vertical motion. Subsequent real-time digital stabilization limited residual motion to an average of only 0.06 arcminutes (a 95% reduction). By correcting for high amplitude, low frequency drifts of the eye, the active TSLO eye-tracking system enabled the AOSLO system to capture high-resolution retinal images over a larger range of motion than previously possible with just the AOSLO imaging system alone.

Calibration-free sinusoidal rectification and uniform retinal irradiance in scanning light ophthalmoscopy

Sinusoidal rectification (i.e., desinusoiding) is necessary for scanning imaging systems and is typically achieved by calculating a rectification transform from a calibration image such as a regular grid. This approach is susceptible to error due to electronic or mechanical instability that can alter the phase of the imaging window with respect to the calibration transform. Here, we show a calibration-free rectification method implemented from live video of a scanning light ophthalmoscope (SLO) with or without adaptive optics (AO). This approach, which capitalizes on positional differences in the images obtained in the forward and backward scan directions, dynamically keeps the imaging window in phase with the motion of the sinusoidal resonant scanner, preventing errors from signal drift over time. A benefit of this approach is that it allows the light power across the field-of-view (FOV) to be modulated inversely to achieve uniform irradiance on the retina, a feature desirable for functional imaging methods and light safety in SLOs.

Closed-loop optical stabilization and digital image registration in adaptive optics scanning light ophthalmoscopy

Eye motion is a major impediment to the efficient acquisition of high resolution retinal images with the adaptive optics (AO) scanning light ophthalmoscope (AOSLO). Here we demonstrate a solution to this problem by implementing both optical stabilization and digital image registration in an AOSLO. We replaced the slow scanning mirror with a two-axis tip/tilt mirror for the dual functions of slow scanning and optical stabilization. Closed-loop optical stabilization reduced the amplitude of eye-movement related-image motion by a factor of 10-15. The residual RMS error after optical stabilization alone was on the order of the size of foveal cones: ~1.66-2.56 μm or ~0.34-0.53 arcmin with typical fixational eye motion for normal observers. The full implementation, with real-time digital image registration, corrected the residual eye motion after optical stabilization with an accuracy of ~0.20-0.25 μm or ~0.04-0.05 arcmin RMS, which to our knowledge is more accurate than any method previously reported.

Imaging light responses of foveal ganglion cells in the living macaque eye

The fovea dominates primate vision, and its anatomy and perceptual abilities are well studied, but its physiology has been little explored because of limitations of current physiological methods. In this study, we adapted a novel in vivo imaging method, originally developed in mouse retina, to explore foveal physiology in the macaque, which permits the repeated imaging of the functional response of many retinal ganglion cells (RGCs) simultaneously. A genetically encoded calcium indicator, G-CaMP5, was inserted into foveal RGCs, followed by calcium imaging of the displacement of foveal RGCs from their receptive fields, and their intensity-response functions. The spatial offset of foveal RGCs from their cone inputs makes this method especially appropriate for fovea by permitting imaging of RGC responses without excessive light adaptation of cones. This new method will permit the tracking of visual development, progression of retinal disease, or therapeutic interventions, such as insertion of visual prostheses.

Mapping the perceptual grain of the human retina

In humans, experimental access to single sensory receptors is difficult to achieve, yet it is crucial for learning how the signals arising from each receptor are transformed into perception. By combining adaptive optics microstimulation with high-speed eye tracking, we show that retinal function can be probed at the level of the individual cone photoreceptor in living eyes. Classical psychometric functions were obtained from cone-sized microstimuli targeted to single photoreceptors. Revealed psychophysically, the cone mosaic also manifests a variable sensitivity to light across its surface that accords with a simple model of cone light capture. Because this microscopic grain of vision could be detected on the perceptual level, it suggests that photoreceptors can act individually to shape perception, if the normally suboptimal relay of light by the eye's optics is corrected. Thus the precise arrangement of cones and the exact placement of stimuli onto those cones create the initial retinal limits on signals mediating spatial vision.

Color coding in the primate visual pathway: a historical view

The physiology and anatomy of the primate visual pathway are reviewed from a historical perspective, especially in relation to color vision. From the work of the last decades, certain issues have been selected which remain unresolved and still pose a challenge for neurobiologists and psychophysicists. It is suggested that the structure of the primate visual pathway has been colored by the evolution of trichromacy and that many features of the parvocellular pathway represent adaptations to this end.

Adaptive optics without altering visual perception

Adaptive optics combined with visual psychophysics creates the potential to study the relationship between visual function and the retina at the cellular scale. This potential is hampered, however, by visual interference from the wavefront-sensing beacon used during correction. For example, we have previously shown that even a dim, visible beacon can alter stimulus perception (Hofer et al., 2012). Here we describe a simple strategy employing a longer wavelength (980nm) beacon that, in conjunction with appropriate restriction on timing and placement, allowed us to perform psychophysics when dark adapted without altering visual perception. The method was verified by comparing detection and color appearance of foveally presented small spot stimuli with and without the wavefront beacon present in 5 subjects. As an important caution, we found that significant perceptual interference can occur even with a subliminal beacon when additional measures are not taken to limit exposure. Consequently, the lack of perceptual interference should be verified for a given system, and not assumed based on invisibility of the beacon.

Retinal representation of the elementary visual signal

The propagation of visual signals from individual cone photoreceptors through parallel neural circuits was examined in the primate retina. Targeted stimulation of individual cones was combined with simultaneous recording from multiple retinal ganglion cells of identified types. The visual signal initiated by an individual cone produced strong responses with different kinetics in three of the four numerically dominant ganglion cell types. The magnitude and kinetics of light responses in each ganglion cell varied nonlinearly with stimulus strength but in a manner that was independent of the cone of origin after accounting for the overall input strength of each cone. Based on this property of independence, the receptive field profile of an individual ganglion cell could be well estimated from responses to stimulation of each cone individually. Together, these findings provide a quantitative account of how elementary visual inputs form the ganglion cell receptive field.

Distribution and specificity of S-cone (“blue cone”) signals in subcortical visual pathways

We review here the distribution of S-cone signals and properties of S-cone recipient receptive fields in subcortical pathways. Nearly everything we know about S-cone signals in the subcortical visual system comes from the study of visual systems in cats and primates (monkeys); in this review, we concentrate on results from macaque and marmoset monkeys. We discuss segregation of S-cone recipient (blue-on and blue-off) receptive fields in the dorsal lateral geniculate nucleus and describe their receptive field properties. We treat in some detail the question of detecting weak S-cone signals as an introduction for newcomers to the field. Finally, we briefly consider the question on how S-cone signals are distributed among nongeniculate targets.

Mapping the primate lateral geniculate nucleus: a review of experiments and methods

Mapping neuronal responses in the lateral geniculate nucleus (LGN) is key to understanding how visual information is processed in the brain. This paper focuses on our current knowledge of the dynamics the receptive field (RF) as broken down into the classical receptive field (CRF) and the extra-classical receptive field (ECRF) in primate LGN. CRFs in the LGN are known to be similar to those in the retinal ganglion cell layer in terms of both spatial and temporal characteristics, leading to the standard interpretation of the LGN as a relay center from retina to primary visual cortex. ECRFs have generally been found to be large and inhibitory, with some differences in magnitude between the magno-, parvo-, and koniocellular pathways. The specific contributions of the retina, thalamus, and visual cortex to LGN ECRF properties are presently unknown. Some reports suggest a retinal origin for extra-classical suppression based on latency arguments and other reports have suggested a thalamic origin for extra-classical suppression. This issue is complicated by the use of anesthetized animals, where cortical activity is likely to be altered. Thus further study of LGN ECRFs is warranted to reconcile these discrepancies. Producing descriptions of RF properties of LGN neurons could be enhanced by employing preferred naturalistic stimuli. Although there has been significant work in cats with natural scene stimuli and noise that statistically imitates natural scenes, we highlight a need for similar data from primates. Obtaining these data may be aided by recent advancements in experimental and analytical techniques that permit the efficient study of nonlinear RF characteristics in addition to traditional linear factors. In light of the reviewed topics, we conclude by suggesting experiments to more clearly elucidate the spatial and temporal structure of ECRFs of primate LGN neurons.

How the unstable eye sees a stable and moving world

Eye motion, even during fixation, results in constant motion of the image of the world on our retinas. Vision scientists have long sought to understand the process by which we perceive the stable parts of the world as unmoving despite this instability and perceive the moving parts with realistic motion. We used an instrument capable of delivering visual stimuli with controlled motion relative to the retina at cone-level precision while capturing the subjects' percepts of stimulus motion with a matching task. We found that the percept of stimulus motion is more complex than conventionally thought. Retinal stimuli that move in a direction that is consistent with eye motion (i.e., opposite eye motion) appear stable even if the magnitude of that motion is amplified. The apparent stabilization diminishes for stimulus motions increasingly inconsistent with eye motion direction. Remarkably, we found that this perceived direction-contingent stabilization occurs separately for each separately moving pattern on the retina rather than for the image as a whole. One consequence is that multiple patterns that move at different rates relative to each other in the visual input are perceived as immobile with respect to each other, thereby disrupting our hyperacute sensitivity to target motion against a frame of reference. This illusion of relative stability has profound implications regarding the underlying visual mechanisms. Functionally, the system compensates retinal slip induced by eye motion without requiring an extremely precise optomotor signal and, at the same time, retains an exquisite sensitivity to an object's true motion in the world.

Defining statistical perceptions with an empirical Bayesian approach

Extracting statistical structures (including textures or contrasts) from a natural stimulus is a central challenge in both biological and engineering contexts. This study interprets the process of statistical recognition in terms of hyperparameter estimations and free-energy minimization procedures with an empirical Bayesian approach. This mathematical interpretation resulted in a framework for relating physiological insights in animal sensory systems to the functional properties of recognizing stimulus statistics. We applied the present theoretical framework to two typical models of natural images that are encoded by a population of simulated retinal neurons, and demonstrated that the resulting cognitive performances could be quantified with the Fisher information measure. The current enterprise yielded predictions about the properties of human texture perception, suggesting that the perceptual resolution of image statistics depends on visual field angles, internal noise, and neuronal information processing pathways, such as the magnocellular, parvocellular, and koniocellular systems. Furthermore, the two conceptually similar natural-image models were found to yield qualitatively different predictions, striking a note of warning against confusing the two models when describing a natural image.

Adaptive Optics for Visual Simulation

A revision of the current state-of-the-art adaptive optics technology for visual sciences is provided. The human eye, as an optical system able to generate images onto the retina, exhibits optical aberrations. Those are continuously changing with time, and they are different for every subject. Adaptive optics is the technology permitting the manipulation of the aberrations, and eventually their correction. Across the different applications of adaptive optics, the current paper focuses on visual simulation. These systems are capable of manipulating the ocular aberrations and simultaneous visual testing though the modified aberrations on real eyes. Some applications of the visual simulators presented in this work are the study of the neural adaptation to the aberrations, the influence of aberrations on accommodation, and the recent development of binocular adaptive optics visual simulators allowing the study of stereopsis.

Adaptive optics scanning laser ophthalmoscope-based microperimetry

PURPOSE

To develop and test the application of an adaptive optics scanning laser ophthalmoscope (AOSLO) with eye tracking for high-resolution microperimetric testing.

METHODS

An AOSLO was used to conduct simultaneous high-resolution retinal imaging and visual function testing in six normal subjects. Visual sensitivity was measured at test locations between the fovea and 5.0° eccentricity via an increment threshold approach using a 40-trial, yes-no adaptive Bayesian staircase procedure (QUEST). A high-speed eye tracking algorithm enabled real-time video stabilization and the delivery of diffraction-limited Goldmann I-sized stimuli (diameter = 6.5 arc min = ∼32 μm; λ = 680 nm) to targeted retinal loci for 200 ms. Test locations were selected either manually by the examiner or automatically using Fourier-based image registration. Cone spacing was assessed at each test location and sensitivity was plotted against retinal eccentricity. Finally, a 4.2 arc min stimulus was used to probe the angioscotoma associated with a blood vessel located at 2.5° eccentricity.

RESULTS

Visual sensitivity decreases with eccentricity at a rate of -1.32 dB/deg (R = 0.60). The vertical and horizontal errors of the targeted stimulus delivery algorithm averaged 0.81 and 0.89 arc min (∼4 μm), respectively. Based on a predetermined exclusion criterion, the stimulus was successfully delivered to its targeted location in 90.1% of all trials. Automated recovery of test locations afforded the repeat testing of the same set of cones over a period of 3 months. Thresholds measured over a parafoveal blood vessel were 1.96 times higher (p < 0.05; one-tailed t-test) than those measured in directly adjacent retina.

CONCLUSIONS

AOSLO-based microperimetry has the potential to test visual sensitivity with fine retinotopic precision. Automated recovery of previously tested locations allows these measures to be tracked longitudinally. This approach can be implemented by researchers interested in establishing the functional correlates of photoreceptor mosaic structure in patients with retinal disease.