Retinal adaptation to object motion

A dynamic scale-mixture model of motion in natural scenes

Some of the most important tasks of visual and motor systems involve estimating the motion of objects and tracking them over time. Such systems evolved to meet the behavioral needs of the organism in its natural environment, and may therefore be adapted to the statistics of motion it is likely to encounter. By tracking the movement of individual points in movies of natural scenes, we begin to identify common properties of natural motion across scenes. As expected, objects in natural scenes move in a persistent fashion, with velocity correlations lasting hundreds of milliseconds. More subtly, but crucially, we find that the observed velocity distributions are heavy-tailed and can be modeled as a Gaussian scale-mixture. Extending this model to the time domain leads to a dynamic scale-mixture model, consisting of a Gaussian process multiplied by a positive scalar quantity with its own independent dynamics. Dynamic scaling of velocity arises naturally as a consequence of changes in object distance from the observer, and may approximate the effects of changes in other parameters governing the motion in a given scene. This modeling and estimation framework has implications for the neurobiology of sensory and motor systems, which need to cope with these fluctuations in scale in order to represent motion efficiently and drive fast and accurate tracking behavior.

Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction

Neuro-inspired vision systems hold great promise to address the growing demands of mass data processing for edge computing, a distributed framework that brings computation and data storage closer to the sources of data. In addition to the capability of static image sensing and processing, the hardware implementation of a neuro-inspired vision system also requires the fulfilment of detecting and recognizing moving targets. Here, we demonstrated a neuro-inspired optical sensor based on two-dimensional NbS2/MoS2 hybrid films, which featured remarkable photo-induced conductance plasticity and low electrical energy consumption. A neuro-inspired optical sensor array with 10 × 10 NbS2/MoS2 phototransistors enabled highly integrated functions of sensing, memory, and contrast enhancement capabilities for static images, which benefits convolutional neural network (CNN) with a high image recognition accuracy. More importantly, in-sensor trajectory registration of moving light spots was experimentally implemented such that the post-processing could yield a high restoration accuracy. Our neuro-inspired optical sensor array could provide a fascinating platform for the implementation of high-performance artificial vision systems.

Classical center-surround receptive fields facilitate novel object detection in retinal bipolar cells

Antagonistic interactions between center and surround receptive field (RF) components lie at the heart of the computations performed in the visual system. Circularly symmetric center-surround RFs are thought to enhance responses to spatial contrasts (i.e., edges), but how visual edges affect motion processing is unclear. Here, we addressed this question in retinal bipolar cells, the first visual neuron with classic center-surround interactions. We found that bipolar glutamate release emphasizes objects that emerge in the RF; their responses to continuous motion are smaller, slower, and cannot be predicted by signals elicited by stationary stimuli. In our hands, the alteration in signal dynamics induced by novel objects was more pronounced than edge enhancement and could be explained by priming of RF surround during continuous motion. These findings echo the salience of human visual perception and demonstrate an unappreciated capacity of the center-surround architecture to facilitate novel object detection and dynamic signal representation.

Center-surround receptive fields are typically considered to mediate edge detection. Here, by studying retinal bipolar cells responding to flashed and moving stimuli, the authors reveal an additional function: enhanced representation of newly appearing visual items.

Center-surround interactions underlie bipolar cell motion sensitivity in the mouse retina

Motion sensing is a critical aspect of vision. We studied the representation of motion in mouse retinal bipolar cells and found that some bipolar cells are radially direction selective, preferring the origin of small object motion trajectories. Using a glutamate sensor, we directly observed bipolar cells synaptic output and found that there are radial direction selective and non-selective bipolar cell types, the majority being selective, and that radial direction selectivity relies on properties of the center-surround receptive field. We used these bipolar cell receptive fields along with connectomics to design biophysical models of downstream cells. The models and additional experiments demonstrated that bipolar cells pass radial direction selective excitation to starburst amacrine cells, which contributes to their directional tuning. As bipolar cells provide excitation to most amacrine and ganglion cells, their radial direction selectivity may contribute to motion processing throughout the visual system.

Impact of Photoreceptor Loss on Retinal Circuitry

Our sense of sight relies on photoreceptors, which transduce photons into the nervous system's electrochemical interpretation of the visual world. These precious photoreceptors can be disrupted by disease, injury, and aging. Once photoreceptors start to die, but before blindness occurs, the remaining retinal circuitry can withstand, mask, or exacerbate the photoreceptor deficit and potentially be receptive to newfound therapies for vision restoration. To maximize the retina's receptivity to therapy, one must understand the conditions that influence the state of the remaining retina. In this review, we provide an overview of the retina's structure and function in health and disease. We analyze a collection of observations on photoreceptor disruption and generate a predictive model to identify parameters that influence the retina's response. Finally, we speculate on whether the retina, with its remarkable capacity to function over light levels spanning nine orders of magnitude, uses these same adaptational mechanisms to withstand and perhaps mask photoreceptor loss.

Predictive Visual Motion Extrapolation Emerges Spontaneously and without Supervision at Each Layer of a Hierarchical Neural Network with Spike-Timing-Dependent Plasticity

The fact that the transmission and processing of visual information in the brain takes time presents a problem for the accurate real-time localization of a moving object. One way this problem might be solved is extrapolation: using an object's past trajectory to predict its location in the present moment. Here, we investigate how a simulated in silico layered neural network might implement such extrapolation mechanisms, and how the necessary neural circuits might develop. We allowed an unsupervised hierarchical network of velocity-tuned neurons to learn its connectivity through spike-timing-dependent plasticity (STDP). We show that the temporal contingencies between the different neural populations that are activated by an object as it moves causes the receptive fields of higher-level neurons to shift in the direction opposite to their preferred direction of motion. The result is that neural populations spontaneously start to represent moving objects as being further along their trajectory than where they were physically detected. Because of the inherent delays of neural transmission, this effectively compensates for (part of) those delays by bringing the represented position of a moving object closer to its instantaneous position in the world. Finally, we show that this model accurately predicts the pattern of perceptual mislocalization that arises when human observers are required to localize a moving object relative to a flashed static object (the flash-lag effect; FLE).SIGNIFICANCE STATEMENT Our ability to track and respond to rapidly changing visual stimuli, such as a fast-moving tennis ball, indicates that the brain is capable of extrapolating the trajectory of a moving object to predict its current position, despite the delays that result from neural transmission. Here, we show how the neural circuits underlying this ability can be learned through spike-timing-dependent synaptic plasticity and that these circuits emerge spontaneously and without supervision. This demonstrates how the neural transmission delays can, in part, be compensated to implement the extrapolation mechanisms required to predict where a moving object is at the present moment.

Adjudicating Between Local and Global Architectures of Predictive Processing in the Subcortical Auditory Pathway

Predictive processing, a leading theoretical framework for sensory processing, suggests that the brain constantly generates predictions on the sensory world and that perception emerges from the comparison between these predictions and the actual sensory input. This requires two distinct neural elements: generative units, which encode the model of the sensory world; and prediction error units, which compare these predictions against the sensory input. Although predictive processing is generally portrayed as a theory of cerebral cortex function, animal and human studies over the last decade have robustly shown the ubiquitous presence of prediction error responses in several nuclei of the auditory, somatosensory, and visual subcortical pathways. In the auditory modality, prediction error is typically elicited using so-called oddball paradigms, where sequences of repeated pure tones with the same pitch are at unpredictable intervals substituted by a tone of deviant frequency. Repeated sounds become predictable promptly and elicit decreasing prediction error; deviant tones break these predictions and elicit large prediction errors. The simplicity of the rules inducing predictability make oddball paradigms agnostic about the origin of the predictions. Here, we introduce two possible models of the organizational topology of the predictive processing auditory network: (1) the global view, that assumes that predictions on the sensory input are generated at high-order levels of the cerebral cortex and transmitted in a cascade of generative models to the subcortical sensory pathways; and (2) the local view, that assumes that independent local models, computed using local information, are used to perform predictions at each processing stage. In the global view information encoding is optimized globally but biases sensory representations along the entire brain according to the subjective views of the observer. The local view results in a diminished coding efficiency, but guarantees in return a robust encoding of the features of sensory input at each processing stage. Although most experimental results to-date are ambiguous in this respect, recent evidence favors the global model.

High Contrast Allows the Retina to Compute More Than Just Contrast

The goal of sensory processing is to represent the environment of an animal. All sensory systems share a similar constraint: they need to encode a wide range of stimulus magnitudes within their narrow neuronal response range. The most efficient way, exploited by even the simplest nervous systems, is to encode relative changes in stimulus magnitude rather than the absolute magnitudes. For instance, the retina encodes contrast, which are the variations of light intensity occurring in time and in space. From this perspective, it is easy to understand why the bright plumage of a moving bird gains a lot of attention, while an octopus remains motionless and mimics its surroundings for concealment. Stronger contrasts simply cause stronger visual signals. However, the gains in retinal performance associated with higher contrast are far more than what can be attributed to just a trivial linear increase in signal strength. Here we discuss how this improvement in performance is reflected throughout different parts of the neural circuitry, within its neural code and how high contrast activates many non-linear mechanisms to unlock several sophisticated retinal computations that are virtually impossible in low contrast conditions.

Low rank mechanisms underlying flexible visual representations

Neuronal population responses to sensory stimuli are remarkably flexible. The responses of neurons in visual cortex have heterogeneous dependence on stimulus properties (e.g., contrast), processes that affect all stages of visual processing (e.g., adaptation), and cognitive processes (e.g., attention or task switching). Understanding whether these processes affect similar neuronal populations and whether they have similar effects on entire populations can provide insight into whether they utilize analogous mechanisms. In particular, it has recently been demonstrated that attention has low rank effects on the covariability of populations of visual neurons, which impacts perception and strongly constrains mechanistic models. We hypothesized that measuring changes in population covariability associated with other sensory and cognitive processes could clarify whether they utilize similar mechanisms or computations. Our experimental design included measurements in multiple visual areas using four distinct sensory and cognitive processes. We found that contrast, adaptation, attention, and task switching affect the variability of responses of populations of neurons in primate visual cortex in a similarly low rank way. These results suggest that a given circuit may use similar mechanisms to perform many forms of modulation and likely reflects a general principle that applies to a wide range of brain areas and sensory, cognitive, and motor processes.

What the salamander eye has been telling the vision scientist's brain

Salamanders have been habitual residents of research laboratories for more than a century, and their history in science is tightly interwoven with vision research. Nevertheless, many vision scientists - even those working with salamanders - may be unaware of how much our knowledge about vision, and particularly the retina, has been shaped by studying salamanders. In this review, we take a tour through the salamander history in vision science, highlighting the main contributions of salamanders to our understanding of the vertebrate retina. We further point out specificities of the salamander visual system and discuss the perspectives of this animal system for future vision research.

Inference of nonlinear receptive field subunits with spike-triggered clustering

Responses of sensory neurons are often modeled using a weighted combination of rectified linear subunits. Since these subunits often cannot be measured directly, a flexible method is needed to infer their properties from the responses of downstream neurons. We present a method for maximum likelihood estimation of subunits by soft-clustering spike-triggered stimuli, and demonstrate its effectiveness in visual neurons. For parasol retinal ganglion cells in macaque retina, estimated subunits partitioned the receptive field into compact regions, likely representing aggregated bipolar cell inputs. Joint clustering revealed shared subunits between neighboring cells, producing a parsimonious population model. Closed-loop validation, using stimuli lying in the null space of the linear receptive field, revealed stronger nonlinearities in OFF cells than ON cells. Responses to natural images, jittered to emulate fixational eye movements, were accurately predicted by the subunit model. Finally, the generality of the approach was demonstrated in macaque V1 neurons.

Response Latency Tuning by Retinal Circuits Modulates Signal Efficiency

In the visual system, retinal ganglion cells (RGCs) of various subtypes encode preprocessed photoreceptor signals into a spike output which is then transmitted towards the brain through parallel feature pathways. Spike timing determines how each feature signal contributes to the output of downstream neurons in visual brain centers, thereby influencing efficiency in visual perception. In this study, we demonstrate a marked population-wide variability in RGC response latency that is independent of trial-to-trial variability and recording approach. RGC response latencies to simple visual stimuli vary considerably in a heterogenous cell population but remain reliable when RGCs of a single subtype are compared. This subtype specificity, however, vanishes when the retinal circuitry is bypassed via direct RGC electrical stimulation. This suggests that latency is primarily determined by the signaling speed through retinal pathways that provide subtype specific inputs to RGCs. In addition, response latency is significantly altered when GABA inhibition or gap junction signaling is disturbed, which further supports the key role of retinal microcircuits in latency tuning. Finally, modulation of stimulus parameters affects individual RGC response delays considerably. Based on these findings, we hypothesize that retinal microcircuits fine-tune RGC response latency, which in turn determines the context-dependent weighing of each signal and its contribution to visual perception.

A Space-Variant Visual Pathway Model for Data Efficient Deep Learning

We present an investigation into adopting a model of the retino-cortical mapping, found in biological visual systems, to improve the efficiency of image analysis using Deep Convolutional Neural Nets (DCNNs) in the context of robot vision and egocentric perception systems. This work has now enabled DCNNs to process input images approaching one million pixels in size, in real time, using only consumer grade graphics processor (GPU) hardware in a single pass of the DCNN.

Speed-Selectivity in Retinal Ganglion Cells is Sharpened by Broad Spatial Frequency, Naturalistic Stimuli

Motion detection represents one of the critical tasks of the visual system and has motivated a large body of research. However, it remains unclear precisely why the response of retinal ganglion cells (RGCs) to simple artificial stimuli does not predict their response to complex, naturalistic stimuli. To explore this topic, we use Motion Clouds (MC), which are synthetic textures that preserve properties of natural images and are merely parameterized, in particular by modulating the spatiotemporal spectrum complexity of the stimulus by adjusting the frequency bandwidths. By stimulating the retina of the diurnal rodent, Octodon degus with MC we show that the RGCs respond to increasingly complex stimuli by narrowing their adjustment curves in response to movement. At the level of the population, complex stimuli produce a sparser code while preserving movement information; therefore, the stimuli are encoded more efficiently. Interestingly, these properties were observed throughout different populations of RGCs. Thus, our results reveal that the response at the level of RGCs is modulated by the naturalness of the stimulus - in particular for motion - which suggests that the tuning to the statistics of natural images already emerges at the level of the retina.

Synaptic Convergence Patterns onto Retinal Ganglion Cells Are Preserved despite Topographic Variation in Pre- and Postsynaptic Territories

Sensory processing can be tuned by a neuron's integration area, the types of inputs, and the proportion and number of connections with those inputs. Integration areas often vary topographically to sample space differentially across regions. Here, we highlight two visual circuits in which topographic changes in the postsynaptic retinal ganglion cell (RGC) dendritic territories and their presynaptic bipolar cell (BC) axonal territories are either matched or unmatched. Despite this difference, in both circuits, the proportion of inputs from each BC type, i.e., synaptic convergence between specific BCs and RGCs, remained constant across varying dendritic territory sizes. Furthermore, synapse density between BCs and RGCs was invariant across topography. Our results demonstrate a wiring design, likely engaging homotypic axonal tiling of BCs, that ensures consistency in synaptic convergence between specific BC types onto their target RGCs while enabling independent regulation of pre- and postsynaptic territory sizes and synapse number between cell pairs.

Neural Mechanisms Mediating Motion Sensitivity in Parasol Ganglion Cells of the Primate Retina

Considerable theoretical and experimental effort has been dedicated to understanding how neural circuits detect visual motion. In primates, much is known about the cortical circuits that contribute to motion processing, but the role of the retina in this fundamental neural computation is poorly understood. Here, we used a combination of extracellular and whole-cell recording to test for motion sensitivity in the two main classes of output neurons in the primate retina-midget (parvocellular-projecting) and parasol (magnocellular-projecting) ganglion cells. We report that parasol, but not midget, ganglion cells are motion sensitive. This motion sensitivity is present in synaptic excitation and disinhibition from presynaptic bipolar cells and amacrine cells, respectively. Moreover, electrical coupling between neighboring bipolar cells and the nonlinear nature of synaptic release contribute to the observed motion sensitivity. Our findings indicate that motion computations arise far earlier in the primate visual stream than previously thought.

The structured 'low temperature' phase of the retinal population code

Recent advances in experimental techniques have allowed the simultaneous recordings of populations of hundreds of neurons, fostering a debate about the nature of the collective structure of population neural activity. Much of this debate has focused on the empirical findings of a phase transition in the parameter space of maximum entropy models describing the measured neural probability distributions, interpreting this phase transition to indicate a critical tuning of the neural code. Here, we instead focus on the possibility that this is a first-order phase transition which provides evidence that the real neural population is in a 'structured', collective state. We show that this collective state is robust to changes in stimulus ensemble and adaptive state. We find that the pattern of pairwise correlations between neurons has a strength that is well within the strongly correlated regime and does not require fine tuning, suggesting that this state is generic for populations of 100+ neurons. We find a clear correspondence between the emergence of a phase transition, and the emergence of attractor-like structure in the inferred energy landscape. A collective state in the neural population, in which neural activity patterns naturally form clusters, provides a consistent interpretation for our results.

Joint Encoding of Object Motion and Motion Direction in the Salamander Retina

The processing of motion in visual scenes is important for detecting and tracking moving objects as well as for monitoring self-motion through the induced optic flow. Specialized neural circuits have been identified in the vertebrate retina for detecting motion direction or for distinguishing between object motion and self-motion, although little is known about how information about these distinct features of visual motion is combined. The salamander retina, which is a widely used model system for analyzing retinal function, contains object-motion-sensitive (OMS) ganglion cells, which strongly respond to local motion signals but are suppressed by global image motion. Yet, direction-selective (DS) ganglion cells have been conspicuously absent from characterizations of the salamander retina, despite their ubiquity in other model systems. We here show that the retina of axolotl salamanders contains at least two distinct classes of DS ganglion cells. For one of these classes, the cells display a strong preference for local over global motion in addition to their direction selectivity (OMS-DS cells) and thereby combine sensitivity to two distinct motion features. The OMS-DS cells are further distinct from standard (non-OMS) DS cells by their smaller receptive fields and different organization of preferred motion directions. Our results suggest that the two classes of DS cells specialize to encode motion direction of local and global motion stimuli, respectively, even for complex composite motion scenes. Furthermore, although the salamander DS cells are OFF-type, there is a strong analogy to the systems of ON and ON-OFF DS cells in the mammalian retina.

SIGNIFICANCE STATEMENT

The retina contains specialized cells for motion processing. Among the retinal ganglion cells, which form the output neurons of the retina, some are known to report the direction of a moving stimulus (direction-selective cells), and others distinguish the motion of an object from a moving background. But little is known about how information about local object motion and information about motion direction interact. Here, we report that direction-selective ganglion cells can be identified in the salamander retina, where their existence had been unclear. Furthermore, there are two independent systems of direction-selective cells, and one of these combines direction selectivity with sensitivity to local motion. The output of these cells could assist in tracking moving objects and estimating their future position.

Rapid and coordinated processing of global motion images by local clusters of retinal ganglion cells

Even when the body is stationary, the whole retinal image is always in motion by fixational eye movements and saccades that move the eye between fixation points. Accumulating evidence indicates that the brain is equipped with specific mechanisms for compensating for the global motion induced by these eye movements. However, it is not yet fully understood how the retina processes global motion images during eye movements. Here we show that global motion images evoke novel coordinated firing in retinal ganglion cells (GCs). We simultaneously recorded the firing of GCs in the goldfish isolated retina using a multi-electrode array, and classified each GC based on the temporal profile of its receptive field (RF). A moving target that accompanied the global motion (simulating a saccade following a period of fixational eye movements) modulated the RF properties and evoked synchronized and correlated firing among local clusters of the specific GCs. Our findings provide a novel concept for retinal information processing during eye movements.

A Computational Framework for Realistic Retina Modeling

Computational simulations of the retina have led to valuable insights about the biophysics of its neuronal activity and processing principles. A great number of retina models have been proposed to reproduce the behavioral diversity of the different visual processing pathways. While many of these models share common computational stages, previous efforts have been more focused on fitting specific retina functions rather than generalizing them beyond a particular model. Here, we define a set of computational retinal microcircuits that can be used as basic building blocks for the modeling of different retina mechanisms. To validate the hypothesis that similar processing structures may be repeatedly found in different retina functions, we implemented a series of retina models simply by combining these computational retinal microcircuits. Accuracy of the retina models for capturing neural behavior was assessed by fitting published electrophysiological recordings that characterize some of the best-known phenomena observed in the retina: adaptation to the mean light intensity and temporal contrast, and differential motion sensitivity. The retinal microcircuits are part of a new software platform for efficient computational retina modeling from single-cell to large-scale levels. It includes an interface with spiking neural networks that allows simulation of the spiking response of ganglion cells and integration with models of higher visual areas.

Fast response and high sensitivity to microsaccades in a cascading-adaptation neural network with short-term synaptic depression

Microsaccades are very small eye movements during fixation. Experimentally, they have been found to play an important role in visual information processing. However, neural responses induced by microsaccades are not yet well understood and are rarely studied theoretically. Here we propose a network model with a cascading adaptation including both retinal adaptation and short-term depression (STD) at thalamocortical synapses. In the neural network model, we compare the microsaccade-induced neural responses in the presence of STD and those without STD. It is found that the cascading with STD can give rise to faster and sharper responses to microsaccades. Moreover, STD can enhance response effectiveness and sensitivity to microsaccadic spatiotemporal changes, suggesting improved detection of small eye movements (or moving visual objects). We also explore the mechanism of the response properties in the model. Our studies strongly indicate that STD plays an important role in neural responses to microsaccades. Our model considers simultaneously retinal adaptation and STD at thalamocortical synapses in the study of microsaccade-induced neural activity, and may be useful for further investigation of the functional roles of microsaccades in visual information processing.

The Synaptic and Morphological Basis of Orientation Selectivity in a Polyaxonal Amacrine Cell of the Rabbit Retina

Much of the computational power of the retina derives from the activity of amacrine cells, a large and diverse group of GABAergic and glycinergic inhibitory interneurons. Here, we identify an ON-type orientation-selective, wide-field, polyaxonal amacrine cell (PAC) in the rabbit retina and demonstrate how its orientation selectivity arises from the structure of the dendritic arbor and the pattern of excitatory and inhibitory inputs. Excitation from ON bipolar cells and inhibition arising from the OFF pathway converge to generate a quasi-linear integration of visual signals in the receptive field center. This serves to suppress responses to high spatial frequencies, thereby improving sensitivity to larger objects and enhancing orientation selectivity. Inhibition also regulates the magnitude and time course of excitatory inputs to this PAC through serial inhibitory connections onto the presynaptic terminals of ON bipolar cells. This presynaptic inhibition is driven by graded potentials within local microcircuits, similar in extent to the size of single bipolar cell receptive fields. Additional presynaptic inhibition is generated by spiking amacrine cells on a larger spatial scale covering several hundred microns. The orientation selectivity of this PAC may be a substrate for the inhibition that mediates orientation selectivity in some types of ganglion cells. Significance statement: The retina comprises numerous excitatory and inhibitory circuits that encode specific features in the visual scene, such as orientation, contrast, or motion. Here, we identify a wide-field inhibitory neuron that responds to visual stimuli of a particular orientation, a feature selectivity that is primarily due to the elongated shape of the dendritic arbor. Integration of convergent excitatory and inhibitory inputs from the ON and OFF visual pathways suppress responses to small objects and fine textures, thus enhancing selectivity for larger objects. Feedback inhibition regulates the strength and speed of excitation on both local and wide-field spatial scales. This study demonstrates how different synaptic inputs are regulated to tune a neuron to respond to specific features in the visual scene.

Mapping nonlinear receptive field structure in primate retina at single cone resolution

The function of a neural circuit is shaped by the computations performed by its interneurons, which in many cases are not easily accessible to experimental investigation. Here, we elucidate the transformation of visual signals flowing from the input to the output of the primate retina, using a combination of large-scale multi-electrode recordings from an identified ganglion cell type, visual stimulation targeted at individual cone photoreceptors, and a hierarchical computational model. The results reveal nonlinear subunits in the circuity of OFF midget ganglion cells, which subserve high-resolution vision. The model explains light responses to a variety of stimuli more accurately than a linear model, including stimuli targeted to cones within and across subunits. The recovered model components are consistent with known anatomical organization of midget bipolar interneurons. These results reveal the spatial structure of linear and nonlinear encoding, at the resolution of single cells and at the scale of complete circuits.

DOI:http://dx.doi.org/10.7554/eLife.05241.001

Light that enters the eye begins the process of vision by activating two types of photoreceptors: rods, which support vision under low light levels, and cones, which are responsible for fine detail and color vision. Activation of either type of photoreceptor triggers responses in bipolar cells, which activate the ganglion cells that transmit visual signals to the brain.

Bipolar cells therefore belong to a class of cells called interneurons, which relay information from certain cell types to others. Interneurons play an important role in information processing throughout the brain, but directly accessing them or characterizing their role in neural computation is often difficult.

To address this problem, Freeman, Field et al. have developed a combined computational and experimental approach to describe the flow of sensory signals through the circuits within the retina of primates. Large arrays of electrodes were used to record the responses of many retinal ganglion cells in response to the activation or de-activation of pairs of cones. These experiments revealed that the responses of ganglion cells are not simply the sum of the inputs that they receive from cones; specifically, the activation of one cone is not cancelled by the deactivation of another. Instead, the data suggest that bipolar cells add cone inputs together and then pass on the total activation (but not deactivation) to ganglion cells.

By analyzing the responses of ganglion cells to numerous random patterns of cone activation, Freeman, Field et al. were able to estimate the locations and arrangements of bipolar cells that connect to them. These predicted patterns of connectivity agreed with observations from anatomical studies. This work provides detailed insights into how the primate retina works. It also suggests that similar approaches may be used to characterize how signals flow across other brain networks in which large-scale recordings are now possible.

DOI:http://dx.doi.org/10.7554/eLife.05241.002

Distinctive receptive field and physiological properties of a wide-field amacrine cell in the macaque monkey retina

At early stages of visual processing, receptive fields are typically described as subtending local regions of space and thus performing computations on a narrow spatial scale. Nevertheless, stimulation well outside of the classical receptive field can exert clear and significant effects on visual processing. Given the distances over which they occur, the retinal mechanisms responsible for these long-range effects would certainly require signal propagation via active membrane properties. Here the physiology of a wide-field amacrine cell-the wiry cell-in macaque monkey retina is explored, revealing receptive fields that represent a striking departure from the classic structure. A single wiry cell integrates signals over wide regions of retina, 5-10 times larger than the classic receptive fields of most retinal ganglion cells. Wiry cells integrate signals over space much more effectively than predicted from passive signal propagation, and spatial integration is strongly attenuated during blockade of NMDA spikes but integration is insensitive to blockade of NaV channels with TTX. Thus these cells appear well suited for contributing to the long-range interactions of visual signals that characterize many aspects of visual perception.

Visual stimulation switches the polarity of excitatory input to starburst amacrine cells

Direction-selective ganglion cells (DSGCs) are tuned to motion in one direction. Starburst amacrine cells (SACs) are thought to mediate this direction selectivity through precise anatomical wiring to DSGCs. Nevertheless, we previously found that visual adaptation can reverse DSGCs's directional tuning, overcoming the circuit anatomy. Here we explore the role of SACs in the generation and adaptation of direction selectivity. First, using pharmacogenetics and two-photon calcium imaging, we validate that SACs are necessary for direction selectivity. Next, we demonstrate that exposure to an adaptive stimulus dramatically alters SACs' synaptic inputs. Specifically, after visual adaptation, On-SACs lose their excitatory input during light onset but gain an excitatory input during light offset. Our data suggest that visual stimulation alters the interactions between rod- and cone-mediated inputs that converge on the terminals of On-cone BCs. These results demonstrate how the sensory environment can modify computations performed by anatomically defined neuronal circuits.

Wiring patterns in the mouse retina: collecting evidence across the connectome, physiology and light microscopy

The visual system has often been thought of as a parallel processor because distinct regions of the brain process different features of visual information. However, increasing evidence for convergence and divergence of circuit connections, even at the level of the retina where visual information is first processed, chips away at a model of dedicated and distinct pathways for parallel information flow. Instead, our current understanding is that parallel channels may emerge, not from exclusive microcircuits for each channel, but from unique combinations of microcircuits. This review depicts diagrammatically the current knowledge and remaining puzzles about the retinal circuit with a focus on the mouse retina. Advances in techniques for labelling cells and genetic manipulations have popularized the use of transgenic mice. We summarize evidence gained from serial electron microscopy, electrophysiology and light microscopy to illustrate the wiring patterns in mouse retina. We emphasize the need to explore proposed retinal connectivity using multiple methods to verify circuits both structurally and functionally.

Higher-order figure discrimination in fly and human vision

Visually-guided animals rely on their ability to stabilize the panorama and simultaneously track salient objects, or figures, that are distinct from the background in order to avoid predators, pursue food resources and mates, and navigate spatially. Visual figures are distinguished by luminance signals that produce coherent motion cues as well as more enigmatic 'higher-order' statistical features. Figure discrimination is thus a complex form of motion vision requiring specialized neural processing. In this minireview, we will highlight recent advances in understanding the perceptual, behavioral, and neurophysiological basis of higher-order figure detection in flies, much of which is grounded in the historical perspective and mechanistic underpinnings of human psychophysics.

Adaptation to changes in higher-order stimulus statistics in the salamander retina

Adaptation in the retina is thought to optimize the encoding of natural light signals into sequences of spikes sent to the brain. While adaptive changes in retinal processing to the variations of the mean luminance level and second-order stimulus statistics have been documented before, no such measurements have been performed when higher-order moments of the light distribution change. We therefore measured the ganglion cell responses in the tiger salamander retina to controlled changes in the second (contrast), third (skew) and fourth (kurtosis) moments of the light intensity distribution of spatially uniform temporally independent stimuli. The skew and kurtosis of the stimuli were chosen to cover the range observed in natural scenes. We quantified adaptation in ganglion cells by studying linear-nonlinear models that capture well the retinal encoding properties across all stimuli. We found that the encoding properties of retinal ganglion cells change only marginally when higher-order statistics change, compared to the changes observed in response to the variation in contrast. By analyzing optimal coding in LN-type models, we showed that neurons can maintain a high information rate without large dynamic adaptation to changes in skew or kurtosis. This is because, for uncorrelated stimuli, spatio-temporal summation within the receptive field averages away non-gaussian aspects of the light intensity distribution.

Two-photon imaging of nonlinear glutamate release dynamics at bipolar cell synapses in the mouse retina

Alpha/Y-type retinal ganglion cells encode visual information with a receptive field composed of nonlinear subunits. This nonlinear subunit structure enhances sensitivity to patterns composed of high spatial frequencies. The Y-cell's subunits are the presynaptic bipolar cells, but the mechanism for the nonlinearity remains incompletely understood. We investigated the synaptic basis of the subunit nonlinearity by combining whole-cell recording of mouse Y-type ganglion cells with two-photon fluorescence imaging of a glutamate sensor (iGluSnFR) expressed on their dendrites and throughout the inner plexiform layer. A control experiment designed to assess iGluSnFR's dynamic range showed that fluorescence responses from Y-cell dendrites increased proportionally with simultaneously recorded excitatory current. Spatial resolution was sufficient to readily resolve independent release at intermingled ON and OFF bipolar terminals. iGluSnFR responses at Y-cell dendrites showed strong surround inhibition, reflecting receptive field properties of presynaptic release sites. Responses to spatial patterns located the origin of the Y-cell nonlinearity to the bipolar cell output, after the stage of spatial integration. The underlying mechanism differed between OFF and ON pathways: OFF synapses showed transient release and strong rectification, whereas ON synapses showed relatively sustained release and weak rectification. At ON synapses, the combination of fast release onset with slower release offset explained the nonlinear response of the postsynaptic ganglion cell. Imaging throughout the inner plexiform layer, we found transient, rectified release at the central-most levels, with increasingly sustained release near the borders. By visualizing glutamate release in real time, iGluSnFR provides a powerful tool for characterizing glutamate synapses in intact neural circuits.

Spatial segregation of adaptation and predictive sensitization in retinal ganglion cells

Sensory systems change their sensitivity based on recent stimuli to adjust their response range to the range of inputs and to predict future sensory input. Here, we report the presence of retinal ganglion cells that have antagonistic plasticity, showing central adaptation and peripheral sensitization. Ganglion cell responses were captured by a spatiotemporal model with independently adapting excitatory and inhibitory subunits, and sensitization requires GABAergic inhibition. Using a simple theory of signal detection, we show that the sensitizing surround conforms to an optimal inference model that continually updates the prior signal probability. This indicates that small receptive field regions have dual functionality--to adapt to the local range of signals but sensitize based upon the probability of the presence of that signal. Within this framework, we show that sensitization predicts the location of a nearby object, revealing prediction as a functional role for adapting inhibition in the nervous system.

Restoration of retinal structure and function after selective photocoagulation

CNS neurons change their connectivity to accommodate a changing environment, form memories, or respond to injury. Plasticity in the adult mammalian retina after injury or disease was thought to be limited to restructuring resulting in abnormal retinal anatomy and function. Here we report that neurons in the mammalian retina change their connectivity and restore normal retinal anatomy and function after injury. Patches of photoreceptors in the rabbit retina were destroyed by selective laser photocoagulation, leaving retinal inner neurons (bipolar, amacrine, horizontal, ganglion cells) intact. Photoreceptors located outside of the damaged zone migrated to make new functional connections with deafferented bipolar cells located inside the lesion. The new connections restored ON and OFF responses in deafferented ganglion cells. This finding extends the previously perceived limits of restorative plasticity in the adult retina and allows for new approaches to retinal laser therapy free of current detrimental side effects such as scotomata and scarring.

Transformation of visual signals by inhibitory interneurons in retinal circuits

One of the largest mysteries of the brain lies in understanding how higher-level computations are implemented by lower-level operations in neurons and synapses. In particular, in many brain regions inhibitory interneurons represent a diverse class of cells, the individual functional roles of which are unknown. We discuss here how the operations of inhibitory interneurons influence the behavior of a circuit, focusing on recent results in the vertebrate retina. A key role in this understanding is played by a common representation of the visual stimulus that can be applied at different stages. By considering how this stimulus representation changes at each location in the circuit, we can understand how neuron-level operations such as thresholds and inhibition yield circuit-level computations such as how stimulus selectivity and gain are controlled by local and peripheral visual stimuli.

Local and global contrast adaptation in retinal ganglion cells

Retinal ganglion cells react to changes in visual contrast by adjusting their sensitivity and temporal filtering characteristics. This contrast adaptation has primarily been studied under spatially homogeneous stimulation. Yet, ganglion cell receptive fields are often characterized by spatial subfields, providing a substrate for nonlinear spatial processing. This raises the question whether contrast adaptation follows a similar subfield structure or whether it occurs globally over the receptive field even for local stimulation. We therefore recorded ganglion cell activity in isolated salamander retinas while locally changing visual contrast. Ganglion cells showed primarily global adaptation characteristics, with notable exceptions in certain aspects of temporal filtering. Surprisingly, some changes in filtering were most pronounced for locations where contrast did not change. This seemingly paradoxical effect can be explained by a simple computational model, which emphasizes the importance of local nonlinearities in the retina and suggests a reevaluation of previously reported local contrast adaptation.

Learning quadratic receptive fields from neural responses to natural stimuli

Models of neural responses to stimuli with complex spatiotemporal correlation structure often assume that neurons are selective for only a small number of linear projections of a potentially high-dimensional input. In this review, we explore recent modeling approaches where the neural response depends on the quadratic form of the input rather than on its linear projection, that is, the neuron is sensitive to the local covariance structure of the signal preceding the spike. To infer this quadratic dependence in the presence of arbitrary (e.g., naturalistic) stimulus distribution, we review several inference methods, focusing in particular on two information theory-based approaches (maximization of stimulus energy and of noise entropy) and two likelihood-based approaches (Bayesian spike-triggered covariance and extensions of generalized linear models). We analyze the formal relationship between the likelihood-based and information-based approaches to demonstrate how they lead to consistent inference. We demonstrate the practical feasibility of these procedures by using model neurons responding to a flickering variance stimulus.

Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.

Evolving concepts of sensory adaptation

Sensory systems constantly adapt their responses to match the current environment. These adjustments occur at many levels of the system and increasingly appear to calibrate even for highly abstract perceptual representations of the stimulus. The similar effects of adaptation across very different stimulus domains point to common design principles but also continue to raise questions about the purpose of adaptation.

Divergence of visual channels in the inner retina

Bipolar cells form parallel channels that carry visual signals from the outer to the inner retina. Each type of bipolar cell is thought to carry a distinct visual message to select types of amacrine cells and ganglion cells. However, the number of ganglion cell types exceeds that of the bipolar cells providing their input, suggesting that bipolar cell signals diversify on transmission to ganglion cells. We explored in the salamander retina how signals from individual bipolar cells feed into multiple ganglion cells and found that each bipolar cell was able to evoke distinct responses among ganglion cells, differing in kinetics, adaptation and rectification properties. This signal divergence resulted primarily from interactions with amacrine cells that allowed each bipolar cell to send distinct signals to its target ganglion cells. Our findings indicate that individual bipolar cell-ganglion cell connections have distinct transfer functions. This expands the number of visual channels in the inner retina and enhances the computational power and feature selectivity of early visual processing.

Linking the computational structure of variance adaptation to biophysical mechanisms

In multiple sensory systems, adaptation to the variance of a sensory input changes the sensitivity, kinetics, and average response over timescales ranging from < 100 ms to tens of seconds. Here, we present a simple, biophysically relevant model of retinal contrast adaptation that accurately captures both the membrane potential response and all adaptive properties. The adaptive component of this model is a first-order kinetic process of the type used to describe ion channel gating and synaptic transmission. From the model, we conclude that all adaptive dynamics can be accounted for by depletion of a signaling mechanism, and that variance adaptation can be explained as adaptation to the mean of a rectified signal. The model parameters show strong similarity to known properties of bipolar cell synaptic vesicle pools. Diverse types of adaptive properties that implement theoretical principles of efficient coding can be generated by a single type of molecule or synapse with just a few microscopic states.

The retinal hypercircuit: a repeating synaptic interactive motif underlying visual function

Abstract

The vertebrate retina generates a stack of about a dozen different movies that represent the visual world as dynamic neural images or movies. The stack is embodied as separate strata that span the inner plexiform layer (IPL). At each stratum, ganglion cell dendrites reach up to read out inhibitory interactions between three different amacrine cell classes that shape bipolar-to-ganglion cell transmission. The nexus of these five cell classes represents a functional module, a retinal ‘hypercircuit’, that is repeated across the surface of each of the dozen strata that span the depth of the IPL. Individual differences in the characteristics of each cell class at each stratum lead to the unique processing characteristics of each neural image throughout the stack. This review shows how the interactions between the morphological and physiological characteristics of each cell class generate many of the known retinal visual functions including motion detection, directional selectivity, local edge detection, looming detection, object motion and looming detection.

Adaptation and visual coding

Visual coding is a highly dynamic process and continuously adapting to the current viewing context. The perceptual changes that result from adaptation to recently viewed stimuli remain a powerful and popular tool for analyzing sensory mechanisms and plasticity. Over the last decade, the footprints of this adaptation have been tracked to both higher and lower levels of the visual pathway and over a wider range of timescales, revealing that visual processing is much more adaptable than previously thought. This work has also revealed that the pattern of aftereffects is similar across many stimulus dimensions, pointing to common coding principles in which adaptation plays a central role. However, why visual coding adapts has yet to be fully answered.

Illuminating synapses and circuitry in the retina

In the central nervous system, space is at a premium. This is especially true in the retina, where synapses, cells, and circuitry have evolved to maximize signal-processing capacity within a thin, optically transparent tissue. For example, at some retinal synapses, single presynaptic active zones contact multiple postsynaptic targets; some individual neurons perform completely different tasks depending on visual conditions, while others execute hundreds of circuit computations in parallel; and the retinal network adapts, at various levels, to the ever-changing visual world. Each of these features reflects efficient use of limited cellular resources to optimally encode visual information.

Separability of stimulus parameter encoding by on-off directionally selective rabbit retinal ganglion cells

The ganglion cell output of the retina constitutes a bottleneck in sensory processing in that ganglion cells must encode multiple stimulus parameters in their responses. Here we investigate encoding strategies of On-Off directionally selective retinal ganglion cells (On-Off DS RGCs) in rabbits, a class of cells dedicated to representing motion. The exquisite axial discrimination of these cells to preferred vs. null direction motion is well documented: it is invariant with respect to speed, contrast, spatial configuration, spatial frequency, and motion extent. However, these cells have broad direction tuning curves and their responses also vary as a function of other parameters such as speed and contrast. In this study, we examined whether the variation in responses across multiple stimulus parameters is systematic, that is the same for all cells, and separable, such that the response to a stimulus is a product of the effects of each stimulus parameter alone. We extracellularly recorded single On-Off DS RGCs in a superfused eyecup preparation while stimulating them with moving bars. We found that spike count responses of these cells scaled as independent functions of direction, speed, and luminance. Moreover, the speed and luminance functions were common across the whole sample of cells. Based on these findings, we developed a model that accurately predicted responses of On-Off DS RGCs as products of separable functions of direction, speed, and luminance (r = 0.98; P < 0.0001). Such a multiplicatively separable encoding strategy may simplify the decoding of these cells' outputs by the higher visual centers.

Applicability of White-Noise Techniques to Analyzing Motion Responses

Motion processing in visual neurons is often understood in terms of how they integrate light stimuli in space and time. These integrative properties, known as the spatiotemporal receptive fields (STRFs), are sometimes obtained using white-noise techniques where a continuous random contrast sequence is delivered to each spatial location within the cell's field of view. In contrast, motion stimuli such as moving bars are usually presented intermittently. Here we compare the STRF prediction of a neuron's response to a moving bar with the measured response in second-order interneurons (L-neurons) of dragonfly ocelli (simple eyes). These low-latency neurons transmit sudden changes in intensity and motion information to mediate flight and gaze stabilization reflexes. A white-noise analysis is made of the responses of L-neurons to random bar stimuli delivered either every frame (densely) or intermittently (sparsely) with a temporal sequence matched to the bar motion stimulus. Linear STRFs estimated using the sparse stimulus were significantly better at predicting the responses to moving bars than the STRFs estimated using a traditional dense white-noise stimulus, even when second-order nonlinear terms were added. Our results strongly suggest that visual adaptation significantly modifies the linear STRF properties of L-neurons in dragonfly ocelli during dense white-noise stimulation. We discuss the ability to predict the responses of visual neurons to arbitrary stimuli based on white-noise analysis. We also discuss the likely functional advantages that adaptive receptive field structures provide for stabilizing attitude during hover and forward flight in dragonflies.

Eye smarter than scientists believed: neural computations in circuits of the retina

We rely on our visual system to cope with the vast barrage of incoming light patterns and to extract features from the scene that are relevant to our well-being. The necessary reduction of visual information already begins in the eye. In this review, we summarize recent progress in understanding the computations performed in the vertebrate retina and how they are implemented by the neural circuitry. A new picture emerges from these findings that helps resolve a vexing paradox between the retina's structure and function. Whereas the conventional wisdom treats the eye as a simple prefilter for visual images, it now appears that the retina solves a diverse set of specific tasks and provides the results explicitly to downstream brain areas.

A retinal circuit that computes object motion

Certain ganglion cells in the retina respond sensitively to differential motion between the receptive field center and surround, as produced by an object moving over the background, but are strongly suppressed by global image motion, as produced by the observer's head or eye movements. We investigated the circuit basis for this object motion sensitive (OMS) response by recording intracellularly from all classes of retinal interneurons while simultaneously recording the spiking output of many ganglion cells. Fast, transient bipolar cells respond linearly to motion in the receptive field center. The synaptic output from their terminals is rectified and then pooled by the OMS ganglion cell. A type of polyaxonal amacrine cell is driven by motion in the surround, again via pooling of rectified inputs, but from a different set of bipolar cell terminals. By direct intracellular current injection, we found that these polyaxonal amacrine cells selectively suppress the synaptic input of OMS ganglion cells. A quantitative model of these circuit elements and their interactions explains how an important visual computation is accomplished by retinal neurons and synapses.

Functional circuitry of visual adaptation in the retina

The visual system continually adjusts its sensitivity, or 'adapts', to the conditions of the immediate environment. Adaptation increases responses when input signals are weak, to improve the signal-to-noise ratio, and decreases responses when input signals are strong, to prevent response saturation. Retinal ganglion cells adapt primarily to two properties of light input: the mean intensity and the variance of intensity over time (contrast). This review focuses on cellular mechanisms for contrast adaptation in mammalian retina. High contrast over the ganglion cell's receptive field centre reduces the gain of spiking responses. The mechanism for gain control arises partly in presynaptic bipolar cell inputs and partly in the process of spike generation. Following strong contrast stimulation, ganglion cells exhibit a prolonged after-hyperpolarization, driven primarily by suppression of glutamate release from presynaptic bipolar cells. Ganglion cells also adapt to high contrast over their peripheral receptive field. Long-range adaptive signals are carried by amacrine cells that inhibit the ganglion cell directly, causing hyperpolarization, and inhibit presynaptic bipolar terminals, reducing gain of their synaptic output. Thus, contrast adaptation in ganglion cells involves multiple synaptic and intrinsic mechanisms for gain control and hyperpolarization. Several forms of adaptation in ganglion cells originate in presynaptic bipolar cells.