Receptive field center-surround interactions mediate context-dependent spatial contrast encoding in the retina

Dynamic Nonlinear Spatial Integrations on Encoding Contrasting Stimuli of Tectal Neurons

Animals detect targets using a variety of visual cues, with the visual salience of these cues determining which environmental features receive priority attention and further processing. Surround modulation plays a crucial role in generating visual saliency, which has been extensively studied in avian tectal neurons. Recent work has reported that the suppression of tectal neurons induced by motion contrasting stimulus is stronger than that by luminance contrasting stimulus. However, the underlying mechanism remains poorly understood. In this study, we built a computational model (called Generalized Linear-Dynamic Modulation) which incorporates independent nonlinear tuning mechanisms for excitatory and inhibitory inputs. This model aims to describe how tectal neurons encode contrasting stimuli. The results showed that: (1) The dynamic nonlinear integration structure substantially improved the accuracy (significant difference (p < 0.001, paired t-test) in the goodness of fit between the two models) of the predicted responses to contrasting stimuli, verifying the nonlinear processing performed by tectal neurons. (2) The modulation difference between luminance and motion contrasting stimuli emerged from the predicted response by the full model but not by that with only excitatory synaptic input (spatial luminance: 89 ± 2.8% (GL_DM) vs. 87 ± 2.1% (GL_DMexc); motion contrasting stimuli: 87 ± 1.7% (GL_DM) vs. 83 ± 2.2% (GL_DMexc)). These results validate the proposed model and further suggest the role of dynamic nonlinear spatial integrations in contextual visual information processing, especially in spatial integration, which is important for object detection performed by birds.

Heterogeneous presynaptic receptive fields contribute to directional tuning in starburst amacrine cells

The processing of visual information by retinal starburst amacrine cells (SACs) involves transforming excitatory input from bipolar cells (BCs) into directional calcium output. While previous studies have suggested that an asymmetry in the kinetic properties of BCs along the soma-dendritic axes of the postsynaptic cell could enhance directional tuning at the level of individual branches, it remains unclear whether biologically relevant presynaptic kinetics contribute to direction selectivity (DS) when visual stimulation engages the entire dendritic tree. To address this question, we built multicompartmental models of the bipolar-SAC circuit and trained them to boost directional tuning. We report that despite significant dendritic crosstalk and dissimilar directional preferences along the dendrites that occur during whole-cell stimulation, the rules that guide BC kinetics leading to optimal DS are similar to the single-dendrite condition. To correlate model predictions to empirical findings, we utilized two-photon glutamate imaging to study the dynamics of bipolar release onto ON- and OFF-starburst dendrites in the murine retina. We reveal diverse presynaptic dynamics in response to motion in both BC populations; algorithms trained on the experimental data suggested that the differences in the temporal release kinetics are likely to correspond to heterogeneous receptive field properties among the different BC types, including the spatial extent of the center and surround components. In addition, we demonstrate that circuit architecture composed of presynaptic units with experimentally recorded dynamics could enhance directional drive but not to levels that replicate empirical findings, suggesting other DS mechanisms are required to explain SAC function. Our study provides new insights into the complex mechanisms underlying DS in retinal processing and highlights the potential contribution of presynaptic kinetics to the computation of visual information by SACs.

Heterogeneous presynaptic receptive fields contribute to directional tuning in starburst amacrine cells

The processing of visual information by retinal starburst amacrine cells (SACs) involves transforming excitatory input from bipolar cells (BCs) into directional calcium output. While previous studies have suggested that an asymmetry in the kinetic properties of bipolar cells along the soma-dendritic axes of the postsynaptic cell could enhance directional tuning at the level of individual branches, it remains unclear whether biologically relevant presynaptic kinetics contribute to direction selectivity when visual stimulation engages the entire dendritic tree. To address this question, we built multicompartmental models of the bipolar-SAC circuit and trained them to boost directional tuning. We report that despite significant dendritic crosstalk and dissimilar directional preferences along the dendrites that occur during whole-cell stimulation, the rules that guide BC kinetics leading to optimal directional selectivity are similar to the single-dendrite condition. To correlate model predictions to empirical findings, we utilized two-photon glutamate imaging to study the dynamics of bipolar release onto ON- and OFF-starburst dendrites in the murine retina. We reveal diverse presynaptic dynamics in response to motion in both BC populations; algorithms trained on the experimental data suggested that the differences in the temporal release kinetics are likely to correspond to heterogeneous receptive field (RF) properties among the different BC types, including the spatial extent of the center and surround components. In addition, we demonstrate that circuit architecture composed of presynaptic units with experimentally recorded dynamics could enhance directional drive but not to levels that replicate empirical findings, suggesting other DS mechanisms are required to explain SAC function. Our study provides new insights into the complex mechanisms underlying direction selectivity in retinal processing and highlights the potential contribution of presynaptic kinetics to the computation of visual information by starburst amacrine cells.

Generalizing biological surround suppression based on center surround similarity via deep neural network models

Sensory perception is dramatically influenced by the context. Models of contextual neural surround effects in vision have mostly accounted for Primary Visual Cortex (V1) data, via nonlinear computations such as divisive normalization. However, surround effects are not well understood within a hierarchy, for neurons with more complex stimulus selectivity beyond V1. We utilized feedforward deep convolutional neural networks and developed a gradient-based technique to visualize the most suppressive and excitatory surround. We found that deep neural networks exhibited a key signature of surround effects in V1, highlighting center stimuli that visually stand out from the surround and suppressing responses when the surround stimulus is similar to the center. We found that in some neurons, especially in late layers, when the center stimulus was altered, the most suppressive surround surprisingly can follow the change. Through the visualization approach, we generalized previous understanding of surround effects to more complex stimuli, in ways that have not been revealed in visual cortices. In contrast, the suppression based on center surround similarity was not observed in an untrained network. We identified further successes and mismatches of the feedforward CNNs to the biology. Our results provide a testable hypothesis of surround effects in higher visual cortices, and the visualization approach could be adopted in future biological experimental designs.

Systematic reduction of the dimensionality of natural scenes allows accurate predictions of retinal ganglion cell spike outputs

The mammalian retina engages a broad array of linear and nonlinear circuit mechanisms to convert natural scenes into retinal ganglion cell (RGC) spike outputs. Although many individual integration mechanisms are well understood, we know less about how multiple mechanisms interact to encode the complex spatial features present in natural inputs. Here, we identified key spatial features in natural scenes that shape encoding by primate parasol RGCs. Our approach identified simplifications in the spatial structure of natural scenes that minimally altered RGC spike responses. We observed that reducing natural movies into 16 linearly integrated regions described ∼80% of the structure of parasol RGC spike responses; this performance depended on the number of regions but not their precise spatial locations. We used simplified stimuli to design high-dimensional metamers that recapitulated responses to naturalistic movies. Finally, we modeled the retinal computations that convert flashed natural images into one-dimensional spike counts.

Classical center-surround receptive fields facilitate novel object detection in retinal bipolar cells

Antagonistic interactions between center and surround receptive field (RF) components lie at the heart of the computations performed in the visual system. Circularly symmetric center-surround RFs are thought to enhance responses to spatial contrasts (i.e., edges), but how visual edges affect motion processing is unclear. Here, we addressed this question in retinal bipolar cells, the first visual neuron with classic center-surround interactions. We found that bipolar glutamate release emphasizes objects that emerge in the RF; their responses to continuous motion are smaller, slower, and cannot be predicted by signals elicited by stationary stimuli. In our hands, the alteration in signal dynamics induced by novel objects was more pronounced than edge enhancement and could be explained by priming of RF surround during continuous motion. These findings echo the salience of human visual perception and demonstrate an unappreciated capacity of the center-surround architecture to facilitate novel object detection and dynamic signal representation.

Center-surround receptive fields are typically considered to mediate edge detection. Here, by studying retinal bipolar cells responding to flashed and moving stimuli, the authors reveal an additional function: enhanced representation of newly appearing visual items.

Center-surround interactions underlie bipolar cell motion sensitivity in the mouse retina

Motion sensing is a critical aspect of vision. We studied the representation of motion in mouse retinal bipolar cells and found that some bipolar cells are radially direction selective, preferring the origin of small object motion trajectories. Using a glutamate sensor, we directly observed bipolar cells synaptic output and found that there are radial direction selective and non-selective bipolar cell types, the majority being selective, and that radial direction selectivity relies on properties of the center-surround receptive field. We used these bipolar cell receptive fields along with connectomics to design biophysical models of downstream cells. The models and additional experiments demonstrated that bipolar cells pass radial direction selective excitation to starburst amacrine cells, which contributes to their directional tuning. As bipolar cells provide excitation to most amacrine and ganglion cells, their radial direction selectivity may contribute to motion processing throughout the visual system.

Phase advancing is a common property of multiple neuron classes in the mouse retina

Behavioral interactions with moving objects are challenged by response latencies within the sensory and motor nervous systems. In vision, the combined latency from phototransduction and synaptic transmission from the retina to central visual areas amounts to 50-100 ms, depending on stimulus conditions. Time required for generating appropriate motor output adds to this latency and further compounds the behavioral delay. Neuronal adaptations that help counter sensory latency within the retina have been demonstrated in some species, but how general these specializations are, and where in the circuitry they originate, remains unclear. To address this, we studied the timing of object motion-evoked responses at multiple signaling stages within the mouse retina using two-photon fluorescence calcium and glutamate imaging, targeted whole-cell electrophysiology, and computational modeling. We found that both ON and OFF-type ganglion cells, as well as the bipolar cells that innervate them, temporally advance the position encoding of a moving object and so help counter the inherent signaling delay in the retina. Model simulations show that this predictive capability is a direct consequence of the spatial extent of the cells' linear visual receptive field, with no apparent specialized circuits that help predict beyond it.Significance StatementSignal transduction and synaptic transmission within sensory signaling pathways costs time. Not a lot of time, just tens to a few hundred milliseconds depending on the sensory system, but enough to challenge fast behavioral interactions under dynamic stimulus conditions, like catching a moving fly. To counter neuronal delays, nervous systems of many species use anticipatory mechanisms. One such mechanism in the mammalian visual system helps predict the future position of a moving target through a process called phase advancing. Here we ask for functionally diverse neuron populations in the mouse retina how common is phase advancing and demonstrate that it is common and generated at multiple signaling stages.

Stage-Dependent Changes of Visual Function and Electrical Response of the Retina in the rd10 Mouse Model

One of the critical prerequisites for the successful development of retinal prostheses is understanding the physiological features of retinal ganglion cells (RGCs) in the different stages of retinal degeneration (RD). This study used our custom-made rd10 mice, C57BL/6-Pde6bem1(R560C)Dkl/Korl mutated on the Pde6b gene in C57BL/6J mouse with the CRISPR/Cas9-based gene-editing method. We selected the postnatal day (P) 45, P70, P140, and P238 as representative ages for RD stages. The optomotor response measured the visual acuity across degeneration stages. At P45, the rd10 mice exhibited lower visual acuity than wild-type (WT) mice. At P140 and older, no optomotor response was observed. We classified RGC responses to the flashed light into ON, OFF, and ON/OFF RGCs via in vitro multichannel recording. With degeneration, the number of RGCs responding to the light stimulation decreased in all three types of RGCs. The OFF response disappeared faster than the ON response with older postnatal ages. We elicited RGC spikes with electrical stimulation and analyzed the network-mediated RGC response in the rd10 mice. Across all postnatal ages, the spikes of rd10 RGCs were less elicited by pulse amplitude modulation than in WT RGCs. The ratio of RGCs showing multiple peaks of spike burst increased in older ages. The electrically evoked RGC spikes by the pulse amplitude modulation differ across postnatal ages. Therefore, degeneration stage-dependent stimulation strategies should be considered for developing retinal prosthesis and successful vision restoration.

Retinal Encoding of Natural Scenes

An ultimate goal in retina science is to understand how the neural circuit of the retina processes natural visual scenes. Yet most studies in laboratories have long been performed with simple, artificial visual stimuli such as full-field illumination, spots of light, or gratings. The underlying assumption is that the features of the retina thus identified carry over to the more complex scenario of natural scenes. As the application of corresponding natural settings is becoming more commonplace in experimental investigations, this assumption is being put to the test and opportunities arise to discover processing features that are triggered by specific aspects of natural scenes. Here, we review how natural stimuli have been used to probe, refine, and complement knowledge accumulated under simplified stimuli, and we discuss challenges and opportunities along the way toward a comprehensive understanding of the encoding of natural scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Adaptation in cone photoreceptors contributes to an unexpected insensitivity of primate On parasol retinal ganglion cells to spatial structure in natural images

Neural circuits are constructed from nonlinear building blocks, and not surprisingly overall circuit behavior is often strongly nonlinear. But neural circuits can also behave near linearly, and some circuits shift from linear to nonlinear behavior depending on stimulus conditions. Such control of nonlinear circuit behavior is fundamental to neural computation. Here, we study a surprising stimulus dependence of the responses of macaque On (but not Off) parasol retinal ganglion cells: these cells respond nonlinearly to spatial structure in some stimuli but near linearly to spatial structure in others, including natural inputs. We show that these differences in the linearity of the integration of spatial inputs can be explained by a shift in the balance of excitatory and inhibitory synaptic inputs that originates at least partially from adaptation in the cone photoreceptors. More generally, this highlights how subtle asymmetries in signaling - here in the cone signals - can qualitatively alter circuit computation.

Rapid Analysis of Visual Receptive Fields by Iterative Tomography

Many receptive fields in the early visual system show standard (center-surround) structure and can be analyzed using simple drifting patterns and a difference-of-Gaussians (DoG) model, which treats the receptive field as a linear filter of the visual image. But many other receptive fields show nonlinear properties such as selectivity for direction of movement. Such receptive fields are typically studied using discrete stimuli (moving or flashed bars and edges) and are modelled according to the features of the visual image to which they are most sensitive. Here, we harness recent advances in tomographic image analysis to characterize rapidly and simultaneously both the linear and nonlinear components of visual receptive fields. Spiking and intracellular voltage potential responses to briefly flashed bars are analyzed using non-negative matrix factorization (NNMF) and iterative reconstruction tomography (IRT). The method yields high-resolution receptive field maps of individual neurons and neuron ensembles in primate (marmoset, both sexes) lateral geniculate and rodent (mouse, male) retina. We show that the first two IRT components correspond to DoG-equivalent center and surround of standard [magnocellular (M) and parvocellular (P)] receptive fields in primate geniculate. The first two IRT components also reveal the spatiotemporal receptive field structure of nonstandard (on/off-rectifying) receptive fields. In rodent retina we combine NNMF-IRT with patch-clamp recording and dye injection to directly map spatial receptive fields to the underlying anatomy of retinal output neurons. We conclude that NNMF-IRT provides a rapid and flexible framework for study of receptive fields in the early visual system.

Impact of Photoreceptor Loss on Retinal Circuitry

Our sense of sight relies on photoreceptors, which transduce photons into the nervous system's electrochemical interpretation of the visual world. These precious photoreceptors can be disrupted by disease, injury, and aging. Once photoreceptors start to die, but before blindness occurs, the remaining retinal circuitry can withstand, mask, or exacerbate the photoreceptor deficit and potentially be receptive to newfound therapies for vision restoration. To maximize the retina's receptivity to therapy, one must understand the conditions that influence the state of the remaining retina. In this review, we provide an overview of the retina's structure and function in health and disease. We analyze a collection of observations on photoreceptor disruption and generate a predictive model to identify parameters that influence the retina's response. Finally, we speculate on whether the retina, with its remarkable capacity to function over light levels spanning nine orders of magnitude, uses these same adaptational mechanisms to withstand and perhaps mask photoreceptor loss.

Temporal vision: measures, mechanisms and meaning

Time is largely a hidden variable in vision. It is the condition for seeing interesting things such as spatial forms and patterns, colours and movements in the external world, and yet is not meant to be noticed in itself. Temporal aspects of visual processing have received comparatively little attention in research. Temporal properties have been made explicit mainly in measurements of resolution and integration in simple tasks such as detection of spatially homogeneous flicker or light pulses of varying duration. Only through a mechanistic understanding of their basis in retinal photoreceptors and circuits can such measures guide modelling of natural vision in different species and illuminate functional and evolutionary trade-offs. Temporal vision research would benefit from bridging traditions that speak different languages. Towards that goal, I here review studies from the fields of human psychophysics, retinal physiology and neuroethology, with a focus on fundamental constraints set by early vision.

Summary: Simple measures of temporal vision such as the critical flicker frequency can be useful for modelling natural vision only if their relationship to photoreceptor responses and retinal processing is understood.

Matching visual induction effects on screens of different size

In the film industry, the same movie is expected to be watched on displays of vastly different sizes, from cinema screens to mobile phones. But visual induction, the perceptual phenomenon by which the appearance of a scene region is affected by its surroundings, will be different for the same image shown on two displays of different dimensions. This phenomenon presents a practical challenge for the preservation of the artistic intentions of filmmakers, because it can lead to shifts in image appearance between viewing destinations. In this work, we show that a neural field model based on the efficient representation principle is able to predict induction effects and how, by regularizing its associated energy functional, the model is still able to represent induction but is now invertible. From this finding, we propose a method to preprocess an image in a screen-size dependent way so that its perception, in terms of visual induction, may remain constant across displays of different size. The potential of the method is demonstrated through psychophysical experiments on synthetic images and qualitative examples on natural images.

Nonlinear Decoding of Natural Images From Large-Scale Primate Retinal Ganglion Recordings

Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.

A quantitative description of macaque ganglion cell responses to natural scenes: the interplay of time and space

KEY POINTS

Responses to natural scenes are the business of the retina. We find primate ganglion cell responses to such scenes consistent with those to simpler stimuli. A biophysical model confirmed this and predicted ganglion cell responses with close to retinal reliability. Primate ganglion cell responses to natural scenes were driven by temporal variations in colour and luminance over the receptive field centre caused by eye movements, and little influenced by interaction of centre and surround with structure in the scene. We discuss implications in the context of efficient coding of the visual environment. Much information in a higher spatiotemporal frequency band is concentrated in the magnocellular pathway.

ABSTRACT

Responses of visual neurons to natural scenes provide a link between classical descriptions of receptive field structure and visual perception of the natural environment. A natural scene video with a movement pattern resembling that of primate eye movements was used to evoke responses from macaque ganglion cells. Cell responses were well described through known properties of cell receptive fields. Different analyses converge to show that responses primarily derive from the temporal pattern of stimulation derived from eye movements, rather than spatial receptive field structure beyond centre size and position. This was confirmed using a model that predicted ganglion cell responses close to retinal reliability, with only a small contribution of the surround relative to the centre. We also found that the spatiotemporal spectrum of the stimulus is modified in ganglion cell responses, and this can reduce redundancy in the retinal signal. This is more pronounced in the magnocellular pathway, which is much better suited to transmit the detailed structure of natural scenes than the parvocellular pathway. Whitening is less important for chromatic channels. Taken together, this shows how a complex interplay across space, time and spectral content sculpts ganglion cell responses.

Intraocular Pressure Elevation Compromises Retinal Ganglion Cell Light Adaptation

Purpose

Functional adaptation to ambient light is a key characteristic of retinal ganglion cells (RGCs), but little is known about how adaptation is affected by factors that are harmful to RGC health. We explored adaptation-induced changes to RGC physiology when exposed to increased intraocular pressure (IOP), a major risk factor for glaucoma.

Methods

Wild-type mice of both sexes were subjected to 2 weeks of IOP elevation using the bead model. Retinas were assessed using a multielectrode array to record RGC responses to checkerboard white noise stimulation under both scotopic and photopic light levels. This information was used to calculate a spike-triggered average (STA) for each RGC with which to compare between lighting levels.

Results

Low but not high IOP elevation resulted in several distinct RGC functional changes: (1) diminished adaptation-dependent receptive field (RF) center-surround interactions; (2) increased likelihood of a scotopic STA; and (3) increased spontaneous firing rate. Center RF size change with lighting level varied among RGCs, and both the center and surround STA peak times were consistently increased under scotopic illumination, although none of these properties were impacted by IOP level.

Conclusions

These findings provide novel evidence that RGCs exhibit reduced light-dependent adaptation and increased excitability when IOP is elevated to low but not high levels. These results may reveal functional changes that occur early in glaucoma, which can potentially be used to identify patients with glaucoma at earlier stages when intervention is most beneficial.

Nonlinear spatial integration in retinal bipolar cells shapes the encoding of artificial and natural stimuli

The retina dissects the visual scene into parallel information channels, which extract specific visual features through nonlinear processing. The first nonlinear stage is typically considered to occur at the output of bipolar cells, resulting from nonlinear transmitter release from synaptic terminals. In contrast, we show here that bipolar cells themselves can act as nonlinear processing elements at the level of their somatic membrane potential. Intracellular recordings from bipolar cells in the salamander retina revealed frequent nonlinear integration of visual signals within bipolar cell receptive field centers, affecting the encoding of artificial and natural stimuli. These nonlinearities provide sensitivity to spatial structure below the scale of bipolar cell receptive fields in both bipolar and downstream ganglion cells and appear to arise at the excitatory input into bipolar cells. Thus, our data suggest that nonlinear signal pooling starts earlier than previously thought: that is, at the input stage of bipolar cells.

•

Some retinal bipolar cells represent visual contrast in a nonlinear fashion

•

These bipolar cells also nonlinearly integrate visual signals over space

•

The spatial nonlinearity affects the encoding of natural stimuli by bipolar cells

•

The nonlinearity results from feedforward input, not from feedback inhibition

Linear and nonlinear chromatic integration in the mouse retina

The computations performed by a neural circuit depend on how it integrates its input signals into an output of its own. In the retina, ganglion cells integrate visual information over time, space, and chromatic channels. Unlike the former two, chromatic integration is largely unexplored. Analogous to classical studies of spatial integration, we here study chromatic integration in mouse retina by identifying chromatic stimuli for which activation from the green or UV color channel is maximally balanced by deactivation through the other color channel. This reveals nonlinear chromatic integration in subsets of On, Off, and On-Off ganglion cells. Unlike the latter two, nonlinear On cells display response suppression rather than activation under balanced chromatic stimulation. Furthermore, nonlinear chromatic integration occurs independently of nonlinear spatial integration, depends on contributions from the rod pathway and on surround inhibition, and may provide information about chromatic boundaries, such as the skyline in natural scenes.

Nonlinear Spatial Integration Underlies the Diversity of Retinal Ganglion Cell Responses to Natural Images

How neurons encode natural stimuli is a fundamental question for sensory neuroscience. In the early visual system, standard encoding models assume that neurons linearly filter incoming stimuli through their receptive fields, but artificial stimuli, such as contrast-reversing gratings, often reveal nonlinear spatial processing. We investigated to what extent such nonlinear processing is relevant for the encoding of natural images in retinal ganglion cells in mice of either sex.

How neurons encode natural stimuli is a fundamental question for sensory neuroscience. In the early visual system, standard encoding models assume that neurons linearly filter incoming stimuli through their receptive fields, but artificial stimuli, such as contrast-reversing gratings, often reveal nonlinear spatial processing. We investigated to what extent such nonlinear processing is relevant for the encoding of natural images in retinal ganglion cells in mice of either sex. We found that standard linear receptive field models yielded good predictions of responses to flashed natural images for a subset of cells but failed to capture the spiking activity for many others. Cells with poor model performance displayed pronounced sensitivity to fine spatial contrast and local signal rectification as the dominant nonlinearity. By contrast, sensitivity to high-frequency contrast-reversing gratings, a classical test for nonlinear spatial integration, was not a good predictor of model performance and thus did not capture the variability of nonlinear spatial integration under natural images. In addition, we also observed a class of nonlinear ganglion cells with inverse tuning for spatial contrast, responding more strongly to spatially homogeneous than to spatially structured stimuli. These findings highlight the diversity of receptive field nonlinearities as a crucial component for understanding early sensory encoding in the context of natural stimuli.

SIGNIFICANCE STATEMENT Experiments with artificial visual stimuli have revealed that many types of retinal ganglion cells pool spatial input signals nonlinearly. However, it is still unclear how relevant this nonlinear spatial integration is when the input signals are natural images. Here we analyze retinal responses to natural scenes in large populations of mouse ganglion cells. We show that nonlinear spatial integration strongly influences responses to natural images for some ganglion cells, but not for others. Cells with nonlinear spatial integration were sensitive to spatial structure inside their receptive fields, and a small group of cells displayed a surprising sensitivity to spatially homogeneous stimuli. Traditional analyses with contrast-reversing gratings did not predict this variability of nonlinear spatial integration under natural images.

Nonlinear convergence boosts information coding in circuits with parallel outputs

Neural circuits are structured with layers of converging and diverging connectivity and selectivity-inducing nonlinearities at neurons and synapses. These components have the potential to hamper an accurate encoding of the circuit inputs. Past computational studies have optimized the nonlinearities of single neurons, or connection weights in networks, to maximize encoded information, but have not grappled with the simultaneous impact of convergent circuit structure and nonlinear response functions for efficient coding. Our approach is to compare model circuits with different combinations of convergence, divergence, and nonlinear neurons to discover how interactions between these components affect coding efficiency. We find that a convergent circuit with divergent parallel pathways can encode more information with nonlinear subunits than with linear subunits, despite the compressive loss induced by the convergence and the nonlinearities when considered separately.

A17 Amacrine Cells and Olfactory Granule Cells: Parallel Processors of Early Sensory Information

Neurons typically receive synaptic input in their dendritic arbor, integrate inputs in their soma, and send output action potentials through their axon, following Cajal's law of dynamic polarization. Two notable exceptions are retinal amacrine cells and olfactory granule cells (GCs), which flout Cajal's edict by providing synaptic output from the same dendrites that collect synaptic input. Amacrine cells, a diverse cell class comprising >60 subtypes, employ various dendritic input/output strategies, but A17 amacrine cells (A17s) in particular share further interesting functional characteristics with GCs: both receive excitatory synaptic input from neurons in the primary glutamatergic pathway and return immediate, reciprocal feedback via GABAergic inhibitory synapses to the same synaptic terminals that provided input. Both neurons thereby process signals locally within their dendrites, shaping many parallels, signaling pathways independently. The similarities between A17s and GCs cast into relief striking differences that may indicate distinct processing roles within their respective circuits: First, they employ partially dissimilar molecular mechanisms to transform excitatory input into inhibitory output; second, GCs fire action potentials, whereas A17s do not. Third, GC signals may be influenced by cortical feedback, whereas the mammalian retina receives no such retrograde input. Finally, A17s constitute just one subtype within a diverse class that is specialized in a particular task, whereas the more homogeneous GCs may play more diverse signaling roles via multiple processing modes. Here, we review these analogies and distinctions between A17 amacrine cells and granule cells, hoping to gain further insight into the operating principles of these two sensory circuits.

Evidence for the intrinsically nonlinear nature of receptive fields in vision

The responses of visual neurons, as well as visual perception phenomena in general, are highly nonlinear functions of the visual input, while most vision models are grounded on the notion of a linear receptive field (RF). The linear RF has a number of inherent problems: it changes with the input, it presupposes a set of basis functions for the visual system, and it conflicts with recent studies on dendritic computations. Here we propose to model the RF in a nonlinear manner, introducing the intrinsically nonlinear receptive field (INRF). Apart from being more physiologically plausible and embodying the efficient representation principle, the INRF has a key property of wide-ranging implications: for several vision science phenomena where a linear RF must vary with the input in order to predict responses, the INRF can remain constant under different stimuli. We also prove that Artificial Neural Networks with INRF modules instead of linear filters have a remarkably improved performance and better emulate basic human perception. Our results suggest a change of paradigm for vision science as well as for artificial intelligence.

Inference of nonlinear receptive field subunits with spike-triggered clustering

Responses of sensory neurons are often modeled using a weighted combination of rectified linear subunits. Since these subunits often cannot be measured directly, a flexible method is needed to infer their properties from the responses of downstream neurons. We present a method for maximum likelihood estimation of subunits by soft-clustering spike-triggered stimuli, and demonstrate its effectiveness in visual neurons. For parasol retinal ganglion cells in macaque retina, estimated subunits partitioned the receptive field into compact regions, likely representing aggregated bipolar cell inputs. Joint clustering revealed shared subunits between neighboring cells, producing a parsimonious population model. Closed-loop validation, using stimuli lying in the null space of the linear receptive field, revealed stronger nonlinearities in OFF cells than ON cells. Responses to natural images, jittered to emulate fixational eye movements, were accurately predicted by the subunit model. Finally, the generality of the approach was demonstrated in macaque V1 neurons.

Selectivity to approaching motion in retinal inputs to the dorsal visual pathway

To efficiently navigate through the environment and avoid potential threats, an animal must quickly detect the motion of approaching objects. Current models of primate vision place the origins of this complex computation in the visual cortex. Here, we report that detection of approaching motion begins in the retina. Several ganglion cell types, the retinal output neurons, show selectivity to approaching motion. Synaptic current recordings from these cells further reveal that this preference for approaching motion arises in the interplay between presynaptic excitatory and inhibitory circuit elements. These findings demonstrate how excitatory and inhibitory circuits interact to mediate an ethologically relevant neural function. Moreover, the elementary computations that detect approaching motion begin early in the visual stream of primates.

Inferring synaptic inputs from spikes with a conductance-based neural encoding model

Descriptive statistical models of neural responses generally aim to characterize the mapping from stimuli to spike responses while ignoring biophysical details of the encoding process. Here, we introduce an alternative approach, the conductance-based encoding model (CBEM), which describes a mapping from stimuli to excitatory and inhibitory synaptic conductances governing the dynamics of sub-threshold membrane potential. Remarkably, we show that the CBEM can be fit to extracellular spike train data and then used to predict excitatory and inhibitory synaptic currents. We validate these predictions with intracellular recordings from macaque retinal ganglion cells. Moreover, we offer a novel quasi-biophysical interpretation of the Poisson generalized linear model (GLM) as a special case of the CBEM in which excitation and inhibition are perfectly balanced. This work forges a new link between statistical and biophysical models of neural encoding and sheds new light on the biophysical variables that underlie spiking in the early visual pathway.

Understanding the retinal basis of vision across species

The vertebrate retina first evolved some 500 million years ago in ancestral marine chordates. Since then, the eyes of different species have been tuned to best support their unique visuoecological lifestyles. Visual specializations in eye designs, large-scale inhomogeneities across the retinal surface and local circuit motifs mean that all species' retinas are unique. Computational theories, such as the efficient coding hypothesis, have come a long way towards an explanation of the basic features of retinal organization and function; however, they cannot explain the full extent of retinal diversity within and across species. To build a truly general understanding of vertebrate vision and the retina's computational purpose, it is therefore important to more quantitatively relate different species' retinal functions to their specific natural environments and behavioural requirements. Ultimately, the goal of such efforts should be to build up to a more general theory of vision.

Neural sensitization improves encoding fidelity in the primate retina

An animal’s motion through the environment can induce large and frequent fluctuations in light intensity on the retina. These fluctuations pose a major challenge to neural circuits tasked with encoding visual information, as they can cause cells to adapt and lose sensitivity. Here, we report that sensitization, a short-term plasticity mechanism, solves this difficult computational problem by maintaining neuronal sensitivity in the face of these fluctuations. The numerically dominant output pathway in the macaque monkey retina, the midget (parvocellular-projecting) pathway, undergoes sensitization under specific conditions, including simulated eye movements. Sensitization is present in the excitatory synaptic inputs from midget bipolar cells and is mediated by presynaptic disinhibition from a wide-field mechanism extending >0.5 mm along the retinal surface. Direct physiological recordings and a computational model indicate that sensitization in the midget pathway supports accurate sensory encoding and prevents a loss of responsiveness during dynamic visual processing.

Light intensity on the retina can fluctuate rapidly during natural vision, posing a challenge for encoding visual information. Here, the authors report that mechanisms of sensitization/facilitation maintain the sensitivity of the numerically dominant neural pathway in the primate retina during dynamic vision.

Speed-Selectivity in Retinal Ganglion Cells is Sharpened by Broad Spatial Frequency, Naturalistic Stimuli

Motion detection represents one of the critical tasks of the visual system and has motivated a large body of research. However, it remains unclear precisely why the response of retinal ganglion cells (RGCs) to simple artificial stimuli does not predict their response to complex, naturalistic stimuli. To explore this topic, we use Motion Clouds (MC), which are synthetic textures that preserve properties of natural images and are merely parameterized, in particular by modulating the spatiotemporal spectrum complexity of the stimulus by adjusting the frequency bandwidths. By stimulating the retina of the diurnal rodent, Octodon degus with MC we show that the RGCs respond to increasingly complex stimuli by narrowing their adjustment curves in response to movement. At the level of the population, complex stimuli produce a sparser code while preserving movement information; therefore, the stimuli are encoded more efficiently. Interestingly, these properties were observed throughout different populations of RGCs. Thus, our results reveal that the response at the level of RGCs is modulated by the naturalness of the stimulus - in particular for motion - which suggests that the tuning to the statistics of natural images already emerges at the level of the retina.

Stimulus- and goal-oriented frameworks for understanding natural vision

Our knowledge of sensory processing has advanced dramatically in the last few decades, but this understanding remains far from complete, especially for stimuli with the large dynamic range and strong temporal and spatial correlations characteristic of natural visual inputs. Here we describe some of the issues that make understanding the encoding of natural images a challenge. We highlight two broad strategies for approaching this problem: a stimulus-oriented framework and a goal-oriented one. Different contexts can call for one framework or the other. Looking forward, recent advances, particularly those based in machine learning, show promise in borrowing key strengths of both frameworks and by doing so illuminating a path to a more comprehensive understanding of the encoding of natural stimuli.

How we see the forest and the trees

Signaling pathways in the retina help us see spatial detail in our visual world.