Closed-loop measurements of iso-response stimuli reveal dynamic nonlinear stimulus integration in the retina

Retinal Encoding of Natural Scenes

An ultimate goal in retina science is to understand how the neural circuit of the retina processes natural visual scenes. Yet most studies in laboratories have long been performed with simple, artificial visual stimuli such as full-field illumination, spots of light, or gratings. The underlying assumption is that the features of the retina thus identified carry over to the more complex scenario of natural scenes. As the application of corresponding natural settings is becoming more commonplace in experimental investigations, this assumption is being put to the test and opportunities arise to discover processing features that are triggered by specific aspects of natural scenes. Here, we review how natural stimuli have been used to probe, refine, and complement knowledge accumulated under simplified stimuli, and we discuss challenges and opportunities along the way toward a comprehensive understanding of the encoding of natural scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Neural circuits of fear and defensive behavior

Innate fear-related behavioral responses have evolved as strategies for survival. The neural circuits responsible for defensive responses, studied mainly in rodents, have been substantially preserved across evolution. Amygdala collects sensory information (visual, auditory and olfactory) in the cortical division and conveys it to the striatal output division. Distinct amygdala nuclei/subnuclei are activated by different fearful stimuli, such as exposure to a predator or to an aggressive conspecific. The same stimuli segregation is observed in downstream structures, i.e., hypothalamus and PAG. In guinea pigs, the circuits underlying Tonic Immobility (TI) and freezing in response to a natural predator, have been mapped in different subnuclei of the same amygdala area. In the PAG circuits, defensive responses are differentially represented along the dorso-ventral and rostro-caudal axis. The coordination of behavioral, anti-nociceptive and autonomic responses is due to the overlapping of the involved neurons in longitudinal columns.

Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.

For understanding how sensory systems operate in the natural environment, an important goal is to develop models that capture neuronal responses to natural stimuli. For retinal ganglion cells, which connect the eye to the brain, current standard models often fail to capture responses to natural visual scenes. This shortcoming is at least partly rooted in the fact that ganglion cells may combine visual signals over space in a nonlinear fashion. We here show that a simple model, which not only considers the average light intensity inside a cell’s receptive field but also the variance of light intensity over space, can partly account for these nonlinearities and thereby improve current standard models. This provides an easy-to-obtain benchmark for modeling ganglion cell responses to natural images.

Coding of chromatic spatial contrast by macaque V1 neurons

Color perception relies on comparisons between adjacent lights, but how the brain performs these comparisons is poorly understood. To elucidate the underlying neural mechanisms, we recorded spiking responses of individual V1 neurons in macaque monkeys to pairs of stimuli within the classical receptive field (RF). We estimated the spatial-chromatic RF of each neuron and then presented customized colored edges using a novel closed-loop technique. We found that many double-opponent (DO) cells, which have spatially and chromatically opponent RFs, responded to chromatic contrast as a weighted sum, akin to how other V1 cells responded to luminance contrast. Yet other neurons integrated chromatic signals non-linearly, confirming that linear signal integration is not an obligate property of V1 neurons. The functional similarity of cone-opponent DO cells and cone non-opponent simple cells suggests that these two groups may share a common underlying neural circuitry, promotes the construction of image-computable models for full-color image representation, and sheds new light on V1 complex cells.

Linear and nonlinear chromatic integration in the mouse retina

The computations performed by a neural circuit depend on how it integrates its input signals into an output of its own. In the retina, ganglion cells integrate visual information over time, space, and chromatic channels. Unlike the former two, chromatic integration is largely unexplored. Analogous to classical studies of spatial integration, we here study chromatic integration in mouse retina by identifying chromatic stimuli for which activation from the green or UV color channel is maximally balanced by deactivation through the other color channel. This reveals nonlinear chromatic integration in subsets of On, Off, and On-Off ganglion cells. Unlike the latter two, nonlinear On cells display response suppression rather than activation under balanced chromatic stimulation. Furthermore, nonlinear chromatic integration occurs independently of nonlinear spatial integration, depends on contributions from the rod pathway and on surround inhibition, and may provide information about chromatic boundaries, such as the skyline in natural scenes.

Nonlinear Spatial Integration Underlies the Diversity of Retinal Ganglion Cell Responses to Natural Images

How neurons encode natural stimuli is a fundamental question for sensory neuroscience. In the early visual system, standard encoding models assume that neurons linearly filter incoming stimuli through their receptive fields, but artificial stimuli, such as contrast-reversing gratings, often reveal nonlinear spatial processing. We investigated to what extent such nonlinear processing is relevant for the encoding of natural images in retinal ganglion cells in mice of either sex.

How neurons encode natural stimuli is a fundamental question for sensory neuroscience. In the early visual system, standard encoding models assume that neurons linearly filter incoming stimuli through their receptive fields, but artificial stimuli, such as contrast-reversing gratings, often reveal nonlinear spatial processing. We investigated to what extent such nonlinear processing is relevant for the encoding of natural images in retinal ganglion cells in mice of either sex. We found that standard linear receptive field models yielded good predictions of responses to flashed natural images for a subset of cells but failed to capture the spiking activity for many others. Cells with poor model performance displayed pronounced sensitivity to fine spatial contrast and local signal rectification as the dominant nonlinearity. By contrast, sensitivity to high-frequency contrast-reversing gratings, a classical test for nonlinear spatial integration, was not a good predictor of model performance and thus did not capture the variability of nonlinear spatial integration under natural images. In addition, we also observed a class of nonlinear ganglion cells with inverse tuning for spatial contrast, responding more strongly to spatially homogeneous than to spatially structured stimuli. These findings highlight the diversity of receptive field nonlinearities as a crucial component for understanding early sensory encoding in the context of natural stimuli.

SIGNIFICANCE STATEMENT Experiments with artificial visual stimuli have revealed that many types of retinal ganglion cells pool spatial input signals nonlinearly. However, it is still unclear how relevant this nonlinear spatial integration is when the input signals are natural images. Here we analyze retinal responses to natural scenes in large populations of mouse ganglion cells. We show that nonlinear spatial integration strongly influences responses to natural images for some ganglion cells, but not for others. Cells with nonlinear spatial integration were sensitive to spatial structure inside their receptive fields, and a small group of cells displayed a surprising sensitivity to spatially homogeneous stimuli. Traditional analyses with contrast-reversing gratings did not predict this variability of nonlinear spatial integration under natural images.

Selectivity and robustness of sparse coding networks

We investigate how the population nonlinearities resulting from lateral inhibition and thresholding in sparse coding networks influence neural response selectivity and robustness. We show that when compared to pointwise nonlinear models, such population nonlinearities improve the selectivity to a preferred stimulus and protect against adversarial perturbations of the input. These findings are predicted from the geometry of the single-neuron iso-response surface, which provides new insight into the relationship between selectivity and adversarial robustness. Inhibitory lateral connections curve the iso-response surface outward in the direction of selectivity. Since adversarial perturbations are orthogonal to the iso-response surface, adversarial attacks tend to be aligned with directions of selectivity. Consequently, the network is less easily fooled by perceptually irrelevant perturbations to the input. Together, these findings point to benefits of integrating computational principles found in biological vision systems into artificial neural networks.

What the salamander eye has been telling the vision scientist's brain

Salamanders have been habitual residents of research laboratories for more than a century, and their history in science is tightly interwoven with vision research. Nevertheless, many vision scientists - even those working with salamanders - may be unaware of how much our knowledge about vision, and particularly the retina, has been shaped by studying salamanders. In this review, we take a tour through the salamander history in vision science, highlighting the main contributions of salamanders to our understanding of the vertebrate retina. We further point out specificities of the salamander visual system and discuss the perspectives of this animal system for future vision research.

Inference of nonlinear receptive field subunits with spike-triggered clustering

Responses of sensory neurons are often modeled using a weighted combination of rectified linear subunits. Since these subunits often cannot be measured directly, a flexible method is needed to infer their properties from the responses of downstream neurons. We present a method for maximum likelihood estimation of subunits by soft-clustering spike-triggered stimuli, and demonstrate its effectiveness in visual neurons. For parasol retinal ganglion cells in macaque retina, estimated subunits partitioned the receptive field into compact regions, likely representing aggregated bipolar cell inputs. Joint clustering revealed shared subunits between neighboring cells, producing a parsimonious population model. Closed-loop validation, using stimuli lying in the null space of the linear receptive field, revealed stronger nonlinearities in OFF cells than ON cells. Responses to natural images, jittered to emulate fixational eye movements, were accurately predicted by the subunit model. Finally, the generality of the approach was demonstrated in macaque V1 neurons.

Introducing chaotic codes for the modulation of code modulated visual evoked potentials (c-VEP) in normal adults for visual fatigue reduction

Code modulated Visual Evoked Potentials (c-VEP) based BCI studies usually employ m-sequences as a modulating codes for their broadband spectrum and correlation property. However, subjective fatigue of the presented codes has been a problem. In this study, we introduce chaotic codes containing broadband spectrum and similar correlation property. We examined whether the introduced chaotic codes could be decoded from EEG signals and also compared the subjective fatigue level with m-sequence codes in normal subjects. We generated chaotic code from one-dimensional logistic map and used it with conventional 31-bit m-sequence code. In a c-VEP based study in normal subjects (n = 44, 21 females) we presented these codes visually and recorded EEG signals from the corresponding codes for their four lagged versions. Canonical correlation analysis (CCA) and spatiotemporal beamforming (STB) methods were used for target identification and comparison of responses. Additionally, we compared the subjective self-declared fatigue using VAS caused by presented m-sequence and chaotic codes. The introduced chaotic code was decoded from EEG responses with CCA and STB methods. The maximum total accuracy values of 93.6 ± 11.9% and 94 ± 14.4% were achieved with STB method for chaotic and m-sequence codes for all subjects respectively. The achieved accuracies in all subjects were not significantly different in m-sequence and chaotic codes. There was significant reduction in subjective fatigue caused by chaotic codes compared to the m-sequence codes. Both m-sequence and chaotic codes were similar in their accuracies as evaluated by CCA and STB methods. The chaotic codes significantly reduced subjective fatigue compared to the m-sequence codes.

Adaptive Stimulus Design for Dynamic Recurrent Neural Network Models

We present an adaptive stimulus design method for efficiently estimating the parameters of a dynamic recurrent network model with interacting excitatory and inhibitory neuronal populations. Although stimuli that are optimized for model parameter estimation should, in theory, have advantages over nonadaptive random stimuli, in practice it remains unclear in what way and to what extent an optimal design of time-varying stimuli may actually improve parameter estimation for this common type of recurrent network models. Here we specified the time course of each stimulus by a Fourier series whose amplitudes and phases were determined by maximizing a utility function based on the Fisher information matrix. To facilitate the optimization process, we have derived differential equations that govern the time evolution of the gradients of the utility function with respect to the stimulus parameters. The network parameters were estimated by maximum likelihood from the spike train data generated by an inhomogeneous Poisson process from the continuous network state. The adaptive design process was repeated in a closed loop, alternating between optimal stimulus design and parameter estimation from the updated stimulus-response data. Our results confirmed that, compared with random stimuli, optimally designed stimuli elicited responses with significantly better likelihood values for parameter estimation. Furthermore, all individual parameters, including the time constants and the connection weights, were recovered more accurately by the optimal design method. We also examined how the errors of different parameter estimates were correlated, and proposed heuristic formulas to account for the correlation patterns by an approximate parameter-confounding theory. Our results suggest that although adaptive optimal stimulus design incurs considerable computational cost even for the simplest excitatory-inhibitory recurrent network model, it may potentially help save time in experiments by reducing the number of stimuli needed for network parameter estimation.

The dynamic receptive fields of retinal ganglion cells

Retinal ganglion cells (RGCs) were one of the first classes of sensory neurons to be described in terms of a receptive field (RF). Over the last six decades, our understanding of the diversity of RGC types and the nuances of their response properties has grown exponentially. We will review the current understanding of RGC RFs mostly from studies in mammals, but including work from other vertebrates as well. We will argue for a new paradigm that embraces the fluidity of RGC RFs with an eye toward the neuroethology of vision. Specifically, we will focus on (1) different methods for measuring RGC RFs, (2) RF models, (3) feature selectivity and the distinction between fluid and stable RF properties, and (4) ideas about the future of understanding RGC RFs.

Receptive field center-surround interactions mediate context-dependent spatial contrast encoding in the retina

Antagonistic receptive field surrounds are a near-universal property of early sensory processing. A key assumption in many models for retinal ganglion cell encoding is that receptive field surrounds are added only to the fully formed center signal. But anatomical and functional observations indicate that surrounds are added before the summation of signals across receptive field subunits that creates the center. Here, we show that this receptive field architecture has an important consequence for spatial contrast encoding in the macaque monkey retina: the surround can control sensitivity to fine spatial structure by changing the way the center integrates visual information over space. The impact of the surround is particularly prominent when center and surround signals are correlated, as they are in natural stimuli. This effect of the surround differs substantially from classic center-surround models and raises the possibility that the surround plays unappreciated roles in shaping ganglion cell sensitivity to natural inputs.

Inferring hidden structure in multilayered neural circuits

A central challenge in sensory neuroscience involves understanding how neural circuits shape computations across cascaded cell layers. Here we attempt to reconstruct the response properties of experimentally unobserved neurons in the interior of a multilayered neural circuit, using cascaded linear-nonlinear (LN-LN) models. We combine non-smooth regularization with proximal consensus algorithms to overcome difficulties in fitting such models that arise from the high dimensionality of their parameter space. We apply this framework to retinal ganglion cell processing, learning LN-LN models of retinal circuitry consisting of thousands of parameters, using 40 minutes of responses to white noise. Our models demonstrate a 53% improvement in predicting ganglion cell spikes over classical linear-nonlinear (LN) models. Internal nonlinear subunits of the model match properties of retinal bipolar cells in both receptive field structure and number. Subunits have consistently high thresholds, supressing all but a small fraction of inputs, leading to sparse activity patterns in which only one subunit drives ganglion cell spiking at any time. From the model’s parameters, we predict that the removal of visual redundancies through stimulus decorrelation across space, a central tenet of efficient coding theory, originates primarily from bipolar cell synapses. Furthermore, the composite nonlinear computation performed by retinal circuitry corresponds to a boolean OR function applied to bipolar cell feature detectors. Our methods are statistically and computationally efficient, enabling us to rapidly learn hierarchical non-linear models as well as efficiently compute widely used descriptive statistics such as the spike triggered average (STA) and covariance (STC) for high dimensional stimuli. This general computational framework may aid in extracting principles of nonlinear hierarchical sensory processing across diverse modalities from limited data.

Computation in neural circuits arises from the cascaded processing of inputs through multiple cell layers. Each of these cell layers performs operations such as filtering and thresholding in order to shape a circuit’s output. It remains a challenge to describe both the computations and the mechanisms that mediate them given limited data recorded from a neural circuit. A standard approach to describing circuit computation involves building quantitative encoding models that predict the circuit response given its input, but these often fail to map in an interpretable way onto mechanisms within the circuit. In this work, we build two layer linear-nonlinear cascade models (LN-LN) in order to describe how the retinal output is shaped by nonlinear mechanisms in the inner retina. We find that these LN-LN models, fit to ganglion cell recordings alone, identify filters and nonlinearities that are readily mapped onto individual circuit components inside the retina, namely bipolar cells and the bipolar-to-ganglion cell synaptic threshold. This work demonstrates how combining simple prior knowledge of circuit properties with partial experimental recordings of a neural circuit’s output can yield interpretable models of the entire circuit computation, including parts of the circuit that are hidden or not directly observed in neural recordings.

Active learning of cortical connectivity from two-photon imaging data

Understanding how groups of neurons interact within a network is a fundamental question in system neuroscience. Instead of passively observing the ongoing activity of a network, we can typically perturb its activity, either by external sensory stimulation or directly via techniques such as two-photon optogenetics. A natural question is how to use such perturbations to identify the connectivity of the network efficiently. Here we introduce a method to infer sparse connectivity graphs from in-vivo, two-photon imaging of population activity in response to external stimuli. A novel aspect of the work is the introduction of a recommended distribution, incrementally learned from the data, to optimally refine the inferred network. Unlike existing system identification techniques, this “active learning” method automatically focuses its attention on key undiscovered areas of the network, instead of targeting global uncertainty indicators like parameter variance. We show how active learning leads to faster inference while, at the same time, provides confidence intervals for the network parameters. We present simulations on artificial small-world networks to validate the methods and apply the method to real data. Analysis of frequency of motifs recovered show that cortical networks are consistent with a small-world topology model.

Dethroning the Fano Factor: A Flexible, Model-Based Approach to Partitioning Neural Variability

Neurons in many brain areas exhibit high trial-to-trial variability, with spike counts that are overdispersed relative to a Poisson distribution. Recent work (Goris, Movshon, & Simoncelli, 2014 ) has proposed to explain this variability in terms of a multiplicative interaction between a stochastic gain variable and a stimulus-dependent Poisson firing rate, which produces quadratic relationships between spike count mean and variance. Here we examine this quadratic assumption and propose a more flexible family of models that can account for a more diverse set of mean-variance relationships. Our model contains additive gaussian noise that is transformed nonlinearly to produce a Poisson spike rate. Different choices of the nonlinear function can give rise to qualitatively different mean-variance relationships, ranging from sublinear to linear to quadratic. Intriguingly, a rectified squaring nonlinearity produces a linear mean-variance function, corresponding to responses with a constant Fano factor. We describe a computationally efficient method for fitting this model to data and demonstrate that a majority of neurons in a V1 population are better described by a model with a nonquadratic relationship between mean and variance. Finally, we demonstrate a practical use of our model via an application to Bayesian adaptive stimulus selection in closed-loop neurophysiology experiments, which shows that accounting for overdispersion can lead to dramatic improvements in adaptive tuning curve estimation.

Space-time codependence of retinal ganglion cells can be explained by novel and separable components of their receptive fields

Reverse correlation methods such as spike‐triggered averaging consistently identify the spatial center in the linear receptive fields (RFs) of retinal ganglion cells (GCs). However, the spatial antagonistic surround observed in classical experiments has proven more elusive. Tests for the antagonistic surround have heretofore relied on models that make questionable simplifying assumptions such as space–time separability and radial homogeneity/symmetry. We circumvented these, along with other common assumptions, and observed a linear antagonistic surround in 754 of 805 mouse GCs. By characterizing the RF's space–time structure, we found the overall linear RF's inseparability could be accounted for both by tuning differences between the center and surround and differences within the surround. Finally, we applied this approach to characterize spatial asymmetry in the RF surround. These results shed new light on the spatiotemporal organization of GC linear RFs and highlight a major contributor to its inseparability.

Diversity in spatial scope of contrast adaptation among mouse retinal ganglion cells

Retinal ganglion cells adapt to changes in visual contrast by adjusting their response kinetics and sensitivity. While much work has focused on the time scales of these adaptation processes, less is known about the spatial scale of contrast adaptation. For example, do small, localized contrast changes affect a cell's signal processing across its entire receptive field? Previous investigations have provided conflicting evidence, suggesting that contrast adaptation occurs either locally within subregions of a ganglion cell's receptive field or globally over the receptive field in its entirety. Here, we investigated the spatial extent of contrast adaptation in ganglion cells of the isolated mouse retina through multielectrode-array recordings. We applied visual stimuli so that ganglion cell receptive fields contained regions where the average contrast level changed periodically as well as regions with constant average contrast level. This allowed us to analyze temporal stimulus integration and sensitivity separately for stimulus regions with and without contrast changes. We found that the spatial scope of contrast adaptation depends strongly on cell identity, with some ganglion cells displaying clear local adaptation, whereas others, in particular large transient ganglion cells, adapted globally to contrast changes. Thus, the spatial scope of contrast adaptation in mouse retinal ganglion cells appears to be cell-type specific. This could reflect differences in mechanisms of contrast adaptation and may contribute to the functional diversity of different ganglion cell types.NEW & NOTEWORTHY Understanding whether adaptation of a neuron in a sensory system can occur locally inside the receptive field or whether it always globally affects the entire receptive field is important for understanding how the neuron processes complex sensory stimuli. For mouse retinal ganglion cells, we here show that both local and global contrast adaptation exist and that this diversity in spatial scope can contribute to the functional diversity of retinal ganglion cell types.

Joint Encoding of Object Motion and Motion Direction in the Salamander Retina

The processing of motion in visual scenes is important for detecting and tracking moving objects as well as for monitoring self-motion through the induced optic flow. Specialized neural circuits have been identified in the vertebrate retina for detecting motion direction or for distinguishing between object motion and self-motion, although little is known about how information about these distinct features of visual motion is combined. The salamander retina, which is a widely used model system for analyzing retinal function, contains object-motion-sensitive (OMS) ganglion cells, which strongly respond to local motion signals but are suppressed by global image motion. Yet, direction-selective (DS) ganglion cells have been conspicuously absent from characterizations of the salamander retina, despite their ubiquity in other model systems. We here show that the retina of axolotl salamanders contains at least two distinct classes of DS ganglion cells. For one of these classes, the cells display a strong preference for local over global motion in addition to their direction selectivity (OMS-DS cells) and thereby combine sensitivity to two distinct motion features. The OMS-DS cells are further distinct from standard (non-OMS) DS cells by their smaller receptive fields and different organization of preferred motion directions. Our results suggest that the two classes of DS cells specialize to encode motion direction of local and global motion stimuli, respectively, even for complex composite motion scenes. Furthermore, although the salamander DS cells are OFF-type, there is a strong analogy to the systems of ON and ON-OFF DS cells in the mammalian retina.

SIGNIFICANCE STATEMENT

The retina contains specialized cells for motion processing. Among the retinal ganglion cells, which form the output neurons of the retina, some are known to report the direction of a moving stimulus (direction-selective cells), and others distinguish the motion of an object from a moving background. But little is known about how information about local object motion and information about motion direction interact. Here, we report that direction-selective ganglion cells can be identified in the salamander retina, where their existence had been unclear. Furthermore, there are two independent systems of direction-selective cells, and one of these combines direction selectivity with sensitivity to local motion. The output of these cells could assist in tracking moving objects and estimating their future position.

Inference of neuronal functional circuitry with spike-triggered non-negative matrix factorization

Neurons in sensory systems often pool inputs over arrays of presynaptic cells, giving rise to functional subunits inside a neuron’s receptive field. The organization of these subunits provides a signature of the neuron’s presynaptic functional connectivity and determines how the neuron integrates sensory stimuli. Here we introduce the method of spike-triggered non-negative matrix factorization for detecting the layout of subunits within a neuron’s receptive field. The method only requires the neuron’s spiking responses under finely structured sensory stimulation and is therefore applicable to large populations of simultaneously recorded neurons. Applied to recordings from ganglion cells in the salamander retina, the method retrieves the receptive fields of presynaptic bipolar cells, as verified by simultaneous bipolar and ganglion cell recordings. The identified subunit layouts allow improved predictions of ganglion cell responses to natural stimuli and reveal shared bipolar cell input into distinct types of ganglion cells.

How a neuron integrates sensory information requires knowledge about its functional presynaptic connections. Here the authors report a new method using non-negative matrix factorization to identify the layout of presynaptic bipolar cell inputs onto retinal ganglion cells and predict their responses to natural stimuli.

Sensitivity to image recurrence across eye-movement-like image transitions through local serial inhibition in the retina

Standard models of stimulus encoding in the retina postulate that image presentations activate neurons according to the increase of preferred contrast inside the receptive field. During natural vision, however, images do not arrive in isolation, but follow each other rapidly, separated by sudden gaze shifts. We here report that, contrary to standard models, specific ganglion cells in mouse retina are suppressed after a rapid image transition by changes in visual patterns across the transition, but respond with a distinct spike burst when the same pattern reappears. This sensitivity to image recurrence depends on opposing effects of glycinergic and GABAergic inhibition and can be explained by a circuit of local serial inhibition. Rapid image transitions thus trigger a mode of operation that differs from the processing of simpler stimuli and allows the retina to tag particular image parts or to detect transition types that lead to recurring stimulus patterns.

DOI:http://dx.doi.org/10.7554/eLife.22431.001

Circuit Mechanisms of a Retinal Ganglion Cell with Stimulus-Dependent Response Latency and Activation Beyond Its Dendrites

Center-surround antagonism has been used as the canonical model to describe receptive fields of retinal ganglion cells (RGCs) for decades. We describe a newly identified RGC type in the mouse, called the ON delayed (OND) RGC, with receptive field properties that deviate from center-surround organization. Responding with an unusually long latency to light stimulation, OND RGCs respond earlier as the visual stimulus increases in size. Furthermore, OND RGCs are excited by light falling far beyond their dendrites. We unravel details of the circuit mechanisms behind these phenomena, revealing new roles for inhibition in controlling both temporal and spatial receptive field properties. The non-canonical receptive field properties of the OND RGC-integration of long temporal and large spatial scales-suggest that unlike typical RGCs, it may encode a slowly varying, global property of the visual scene.

Neural Circuit Inference from Function to Structure

Advances in technology are opening new windows on the structural connectivity and functional dynamics of brain circuits. Quantitative frameworks are needed that integrate these data from anatomy and physiology. Here, we present a modeling approach that creates such a link. The goal is to infer the structure of a neural circuit from sparse neural recordings, using partial knowledge of its anatomy as a regularizing constraint. We recorded visual responses from the output neurons of the retina, the ganglion cells. We then generated a systematic sequence of circuit models that represents retinal neurons and connections and fitted them to the experimental data. The optimal models faithfully recapitulated the ganglion cell outputs. More importantly, they made predictions about dynamics and connectivity among unobserved neurons internal to the circuit, and these were subsequently confirmed by experiment. This circuit inference framework promises to facilitate the integration and understanding of big data in neuroscience.

Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains

Advances in neuronal recording techniques are leading to ever larger numbers of simultaneously monitored neurons. This poses the important analytical challenge of how to capture compactly all sensory information that neural population codes carry in their spatial dimension (differences in stimulus tuning across neurons at different locations), in their temporal dimension (temporal neural response variations), or in their combination (temporally coordinated neural population firing). Here we investigate the utility of tensor factorizations of population spike trains along space and time. These factorizations decompose a dataset of single-trial population spike trains into spatial firing patterns (combinations of neurons firing together), temporal firing patterns (temporal activation of these groups of neurons) and trial-dependent activation coefficients (strength of recruitment of such neural patterns on each trial). We validated various factorization methods on simulated data and on populations of ganglion cells simultaneously recorded in the salamander retina. We found that single-trial tensor space-by-time decompositions provided low-dimensional data-robust representations of spike trains that capture efficiently both their spatial and temporal information about sensory stimuli. Tensor decompositions with orthogonality constraints were the most efficient in extracting sensory information, whereas non-negative tensor decompositions worked well even on non-independent and overlapping spike patterns, and retrieved informative firing patterns expressed by the same population in response to novel stimuli. Our method showed that populations of retinal ganglion cells carried information in their spike timing on the ten-milliseconds-scale about spatial details of natural images. This information could not be recovered from the spike counts of these cells. First-spike latencies carried the majority of information provided by the whole spike train about fine-scale image features, and supplied almost as much information about coarse natural image features as firing rates. Together, these results highlight the importance of spike timing, and particularly of first-spike latencies, in retinal coding.

Synaptic Rectification Controls Nonlinear Spatial Integration of Natural Visual Inputs

A central goal in the study of any sensory system is to predict neural responses to complex inputs, especially those encountered during natural stimulation. Nowhere is the transformation from stimulus to response better understood than the vertebrate retina. Nevertheless, descriptions of retinal computation are largely based on stimulation using artificial visual stimuli, and it is unclear how these descriptions map onto the encoding of natural stimuli. We demonstrate that nonlinear spatial integration, a common feature of retinal ganglion cell (RGC) processing, shapes neural responses to natural visual stimuli in primate Off parasol RGCs, whereas On parasol RGCs exhibit surprisingly linear spatial integration. Despite this asymmetry, both cell types show strong nonlinear integration when presented with artificial stimuli. We show that nonlinear integration of natural stimuli is a consequence of rectified excitatory synaptic input and that accounting for nonlinear spatial integration substantially improves models that predict RGC responses to natural images.

A Retinal Mechanism Inspired Color Constancy Model

In this paper, we propose a novel model for the computational color constancy, inspired by the amazing ability of the human vision system (HVS) to perceive the color of objects largely constant as the light source color changes. The proposed model imitates the color processing mechanisms in the specific level of the retina, the first stage of the HVS, from the adaptation emerging in the layers of cone photoreceptors and horizontal cells (HCs) to the color-opponent mechanism and disinhibition effect of the non-classical receptive field in the layer of retinal ganglion cells (RGCs). In particular, HC modulation provides a global color correction with cone-specific lateral gain control, and the following RGCs refine the processing with iterative adaptation until all the three opponent channels reach their stable states (i.e., obtain stable outputs). Instead of explicitly estimating the scene illuminant(s), such as most existing algorithms, our model directly removes the effect of scene illuminant. Evaluations on four commonly used color constancy data sets show that the proposed model produces competitive results in comparison with the state-of-the-art methods for the scenes under either single or multiple illuminants. The results indicate that single opponency, especially the disinhibitory effect emerging in the receptive field's subunit-structured surround of RGCs, plays an important role in removing scene illuminant(s) by inherently distinguishing the spatial structures of surfaces from extensive illuminant(s).

Functional Circuitry of the Retina

The mammalian retina is an important model system for studying neural circuitry: Its role in sensation is clear, its cell types are relatively well defined, and its responses to natural stimuli-light patterns-can be studied in vitro. To solve the retina, we need to understand how the circuits presynaptic to its output neurons, ganglion cells, divide the visual scene into parallel representations to be assembled and interpreted by the brain. This requires identifying the component interneurons and understanding how their intrinsic properties and synapses generate circuit behaviors. Because the cellular composition and fundamental properties of the retina are shared across species, basic mechanisms studied in the genetically modifiable mouse retina apply to primate vision. We propose that the apparent complexity of retinal computation derives from a straightforward mechanism-a dynamic balance of synaptic excitation and inhibition regulated by use-dependent synaptic depression-applied differentially to the parallel pathways that feed ganglion cells.

Mapping nonlinear receptive field structure in primate retina at single cone resolution

The function of a neural circuit is shaped by the computations performed by its interneurons, which in many cases are not easily accessible to experimental investigation. Here, we elucidate the transformation of visual signals flowing from the input to the output of the primate retina, using a combination of large-scale multi-electrode recordings from an identified ganglion cell type, visual stimulation targeted at individual cone photoreceptors, and a hierarchical computational model. The results reveal nonlinear subunits in the circuity of OFF midget ganglion cells, which subserve high-resolution vision. The model explains light responses to a variety of stimuli more accurately than a linear model, including stimuli targeted to cones within and across subunits. The recovered model components are consistent with known anatomical organization of midget bipolar interneurons. These results reveal the spatial structure of linear and nonlinear encoding, at the resolution of single cells and at the scale of complete circuits.

DOI:http://dx.doi.org/10.7554/eLife.05241.001

Light that enters the eye begins the process of vision by activating two types of photoreceptors: rods, which support vision under low light levels, and cones, which are responsible for fine detail and color vision. Activation of either type of photoreceptor triggers responses in bipolar cells, which activate the ganglion cells that transmit visual signals to the brain.

Bipolar cells therefore belong to a class of cells called interneurons, which relay information from certain cell types to others. Interneurons play an important role in information processing throughout the brain, but directly accessing them or characterizing their role in neural computation is often difficult.

To address this problem, Freeman, Field et al. have developed a combined computational and experimental approach to describe the flow of sensory signals through the circuits within the retina of primates. Large arrays of electrodes were used to record the responses of many retinal ganglion cells in response to the activation or de-activation of pairs of cones. These experiments revealed that the responses of ganglion cells are not simply the sum of the inputs that they receive from cones; specifically, the activation of one cone is not cancelled by the deactivation of another. Instead, the data suggest that bipolar cells add cone inputs together and then pass on the total activation (but not deactivation) to ganglion cells.

By analyzing the responses of ganglion cells to numerous random patterns of cone activation, Freeman, Field et al. were able to estimate the locations and arrangements of bipolar cells that connect to them. These predicted patterns of connectivity agreed with observations from anatomical studies. This work provides detailed insights into how the primate retina works. It also suggests that similar approaches may be used to characterize how signals flow across other brain networks in which large-scale recordings are now possible.

DOI:http://dx.doi.org/10.7554/eLife.05241.002

Spike-Triggered Covariance Analysis Reveals Phenomenological Diversity of Contrast Adaptation in the Retina

When visual contrast changes, retinal ganglion cells adapt by adjusting their sensitivity as well as their temporal filtering characteristics. The latter has classically been described by contrast-induced gain changes that depend on temporal frequency. Here, we explored a new perspective on contrast-induced changes in temporal filtering by using spike-triggered covariance analysis to extract multiple parallel temporal filters for individual ganglion cells. Based on multielectrode-array recordings from ganglion cells in the isolated salamander retina, we found that contrast adaptation of temporal filtering can largely be captured by contrast-invariant sets of filters with contrast-dependent weights. Moreover, differences among the ganglion cells in the filter sets and their contrast-dependent contributions allowed us to phenomenologically distinguish three types of filter changes. The first type is characterized by newly emerging features at higher contrast, which can be reproduced by computational models that contain response-triggered gain-control mechanisms. The second type follows from stronger adaptation in the Off pathway as compared to the On pathway in On-Off-type ganglion cells. Finally, we found that, in a subset of neurons, contrast-induced filter changes are governed by particularly strong spike-timing dynamics, in particular by pronounced stimulus-dependent latency shifts that can be observed in these cells. Together, our results show that the contrast dependence of temporal filtering in retinal ganglion cells has a multifaceted phenomenology and that a multi-filter analysis can provide a useful basis for capturing the underlying signal-processing dynamics.

Our sensory systems have to process stimuli under a wide range of environmental conditions. To cope with this challenge, the involved neurons adapt by adjusting their signal processing to the recently encountered intensity range. In the visual system, one finds, for example, that higher visual contrast leads to changes in how visual signals are temporally filtered, making signal processing faster and more band-pass-like at higher contrast. By analyzing signals from neurons in the retina of salamanders, we here found that these adaptation effects can be described by a fixed set of filters, independent of contrast, whose relative contributions change with contrast. Also, we found that different phenomena contribute to this adaptation. In particular, some cells change their relative sensitivity to light increments and light decrements, whereas other cells are influenced by a strong contrast-dependence of the exact timing of their responses. Our results show that contrast adaptation in the retina is not an entirely homogeneous phenomenon, and that models with multiple filters can help in characterizing sensory adaptation.

General features of the retinal connectome determine the computation of motion anticipation

Motion anticipation allows the visual system to compensate for the slow speed of phototransduction so that a moving object can be accurately located. This correction is already present in the signal that ganglion cells send from the retina but the biophysical mechanisms underlying this computation are not known. Here we demonstrate that motion anticipation is computed autonomously within the dendritic tree of each ganglion cell and relies on feedforward inhibition. The passive and non-linear interaction of excitatory and inhibitory synapses enables the somatic voltage to encode the actual position of a moving object instead of its delayed representation. General rather than specific features of the retinal connectome govern this computation: an excess of inhibitory inputs over excitatory, with both being randomly distributed, allows tracking of all directions of motion, while the average distance between inputs determines the object velocities that can be compensated for.

DOI:http://dx.doi.org/10.7554/eLife.06250.001

The retina is a structure at the back of the eye that converts light into nerve impulses, which are then processed in the brain to produce the images that we see. It normally takes about one-tenth of a second for the retina to send a signal to the brain after an object first moves into view. This is about the same time it takes a tennis ball to travel several meters during a tennis match, yet we are still able to see where the moving tennis ball is in real time. This is because a process called ‘motion anticipation’ is able to compensate for the delay in processing the position of a moving object. However, it was not known precisely how motion anticipation occurs.

Inside the retina, cells called photoreceptors detect light and ultimately send signals (via some intermediate cell types) to nerve cells known as retinal ganglion cells. These signals can either excite a retinal ganglion cell to cause it to send an electrical signal to the brain, or inhibit it, which temporarily prevents electrical activity. Each cell receives signals from several photoreceptors, which each connect to a different site along branch-like structures called dendrites that project out of the retinal ganglion cells.

Johnston and Lagnado have now investigated how motion anticipation occurs in the retina by using electrical recordings of the activity in the retinas of goldfish combined with computer simulations of this activity. This revealed inhibitory signals, sent from photoreceptors to retinal ganglion cells via a type of intermediate cell (called amacrine cells), play a key role in motion anticipation. The ability to track motion effectively in all directions requires more inhibitory signals to be sent to the dendrites of a retinal ganglion cell than excitatory signals. These two types of input must also be randomly distributed across the cell. Furthermore, it is the density of these input sites on a dendrite that determines how well the retina can compensate for the motion of a fast-moving object. The building blocks required for motion anticipation in the retina are also found in visual areas higher in the brain. Therefore, further work may reveal that higher visual areas also use this mechanism to predict the future location of moving objects.

DOI:http://dx.doi.org/10.7554/eLife.06250.002

A synaptic mechanism for temporal filtering of visual signals

The visual system transmits information about fast and slow changes in light intensity through separate neural pathways. We used in vivo imaging to investigate how bipolar cells transmit these signals to the inner retina. We found that the volume of the synaptic terminal is an intrinsic property that contributes to different temporal filters. Individual cells transmit through multiple terminals varying in size, but smaller terminals generate faster and larger calcium transients to trigger vesicle release with higher initial gain, followed by more profound adaptation. Smaller terminals transmitted higher stimulus frequencies more effectively. Modeling global calcium dynamics triggering vesicle release indicated that variations in the volume of presynaptic compartments contribute directly to all these differences in response dynamics. These results indicate how one neuron can transmit different temporal components in the visual signal through synaptic terminals of varying geometries with different adaptational properties.

Online stimulus optimization rapidly reveals multidimensional selectivity in auditory cortical neurons

Neurons in sensory brain regions shape our perception of the surrounding environment through two parallel operations: decomposition and integration. For example, auditory neurons decompose sounds by separately encoding their frequency, temporal modulation, intensity, and spatial location. Neurons also integrate across these various features to support a unified perceptual gestalt of an auditory object. At higher levels of a sensory pathway, neurons may select for a restricted region of feature space defined by the intersection of multiple, independent stimulus dimensions. To further characterize how auditory cortical neurons decompose and integrate multiple facets of an isolated sound, we developed an automated procedure that manipulated five fundamental acoustic properties in real time based on single-unit feedback in awake mice. Within several minutes, the online approach converged on regions of the multidimensional stimulus manifold that reliably drove neurons at significantly higher rates than predefined stimuli. Optimized stimuli were cross-validated against pure tone receptive fields and spectrotemporal receptive field estimates in the inferior colliculus and primary auditory cortex. We observed, from midbrain to cortex, increases in both level invariance and frequency selectivity, which may underlie equivalent sparseness of responses in the two areas. We found that onset and steady-state spike rates increased proportionately as the stimulus was tailored to the multidimensional receptive field. By separately evaluating the amount of leverage each sound feature exerted on the overall firing rate, these findings reveal interdependencies between stimulus features as well as hierarchical shifts in selectivity and invariance that may go unnoticed with traditional approaches.

Nonlinear spatial integration in the receptive field surround of retinal ganglion cells

Throughout different sensory systems, individual neurons integrate incoming signals over their receptive fields. The characteristics of this signal integration are crucial determinants for the neurons' functions. For ganglion cells in the vertebrate retina, receptive fields are characterized by the well-known center-surround structure and, although several studies have addressed spatial integration in the receptive field center, little is known about how visual signals are integrated in the surround. Therefore, we set out here to characterize signal integration and to identify relevant nonlinearities in the receptive field surround of ganglion cells in the isolated salamander retina by recording spiking activity with extracellular electrodes under visual stimulation of the center and surround. To quantify nonlinearities of spatial integration independently of subsequent nonlinearities of spike generation, we applied the technique of iso-response measurements as follows: using closed-loop experiments, we searched for different stimulus patterns in the surround that all reduced the center-evoked spiking activity by the same amount. The identified iso-response stimuli revealed strongly nonlinear spatial integration in the receptive field surrounds of all recorded cells. Furthermore, cell types that had been shown previously to have different nonlinearities in receptive field centers showed similar surround nonlinearities but differed systematically in the adaptive characteristics of the surround. Finally, we found that there is an optimal spatial scale of surround suppression; suppression was most effective when surround stimulation was organized into subregions of several hundred micrometers in diameter, indicating that the surround is composed of subunits that have strong center-surround organization themselves.

Bayesian active learning of neural firing rate maps with transformed gaussian process priors

A firing rate map, also known as a tuning curve, describes the nonlinear relationship between a neuron's spike rate and a low-dimensional stimulus (e.g., orientation, head direction, contrast, color). Here we investigate Bayesian active learning methods for estimating firing rate maps in closed-loop neurophysiology experiments. These methods can accelerate the characterization of such maps through the intelligent, adaptive selection of stimuli. Specifically, we explore the manner in which the prior and utility function used in Bayesian active learning affect stimulus selection and performance. Our approach relies on a flexible model that involves a nonlinearly transformed gaussian process (GP) prior over maps and conditionally Poisson spiking. We show that infomax learning, which selects stimuli to maximize the information gain about the firing rate map, exhibits strong dependence on the seemingly innocuous choice of nonlinear transformation function. We derive an alternate utility function that selects stimuli to minimize the average posterior variance of the firing rate map and analyze the surprising relationship between prior parameterization, stimulus selection, and active learning performance in GP-Poisson models. We apply these methods to color tuning measurements of neurons in macaque primary visual cortex.

Retinal representation of the elementary visual signal

The propagation of visual signals from individual cone photoreceptors through parallel neural circuits was examined in the primate retina. Targeted stimulation of individual cones was combined with simultaneous recording from multiple retinal ganglion cells of identified types. The visual signal initiated by an individual cone produced strong responses with different kinetics in three of the four numerically dominant ganglion cell types. The magnitude and kinetics of light responses in each ganglion cell varied nonlinearly with stimulus strength but in a manner that was independent of the cone of origin after accounting for the overall input strength of each cone. Based on this property of independence, the receptive field profile of an individual ganglion cell could be well estimated from responses to stimulation of each cone individually. Together, these findings provide a quantitative account of how elementary visual inputs form the ganglion cell receptive field.

Light adaptation increases response latency of alpha ganglion cells via a threshold-like nonlinearity

Adaptation is an important process of sensory systems to adjust sensitivity to ensure the appropriate information encoding. Sensitivity and kinetics of retinal ganglion cell (RGC) responses have been studied extensively using a brief flash superimposed on different but steady backgrounds. However, it is still unclear if light adaptation exerts any effect on more complex response properties, such as response nonlinearity. In this study, we found that the latency of spike responses to a repeated flashing spot stimulation increased by 30 ms in the mouse ON α RGCs (An ON-type RGC is excited when a spot is turned on in the center of its receptive field). A single dimming event preceding the test flash on a steady adapting background could also produce similar effect in increasing latency of light responses. A simple computational model with a linear transformation of the light stimulus and a threshold-like nonlinearity could account for the experimental data. Moreover, the strength of the measured nonlinearity and the response latency were affected by the duration of light adaptation. The possible biological processes underlying this nonlinearity were explored. Voltage clamp recording revealed the presence of the increase in latency and threshold-like nonlinearity in the excitatory input of RGCs. However, no comparable nonlinearity was observed in the light responses of the ON cone bipolar cells. We further excluded GABAergic and glycinergic inhibition, N-methyl-D-aspartate receptor rectification and voltage-gated Na(+) channels as potential sources of this nonlinearity by pharmacological experiments. Our results indicate the bipolar cell terminals as the potential site of nonlinearity. Computational modeling constrained by experimental data supports that conclusion and suggests the voltage-sensitive Ca(++) channels and Ca(++)-dependent vesicle release in the bipolar cell terminals as mechanistic basis.

Spatial segregation of adaptation and predictive sensitization in retinal ganglion cells

Sensory systems change their sensitivity based on recent stimuli to adjust their response range to the range of inputs and to predict future sensory input. Here, we report the presence of retinal ganglion cells that have antagonistic plasticity, showing central adaptation and peripheral sensitization. Ganglion cell responses were captured by a spatiotemporal model with independently adapting excitatory and inhibitory subunits, and sensitization requires GABAergic inhibition. Using a simple theory of signal detection, we show that the sensitizing surround conforms to an optimal inference model that continually updates the prior signal probability. This indicates that small receptive field regions have dual functionality--to adapt to the local range of signals but sensitize based upon the probability of the presence of that signal. Within this framework, we show that sensitization predicts the location of a nearby object, revealing prediction as a functional role for adapting inhibition in the nervous system.

Adaptive stimulus optimization for sensory systems neuroscience

In this paper, we review several lines of recent work aimed at developing practical methods for adaptive on-line stimulus generation for sensory neurophysiology. We consider various experimental paradigms where on-line stimulus optimization is utilized, including the classical optimal stimulus paradigm where the goal of experiments is to identify a stimulus which maximizes neural responses, the iso-response paradigm which finds sets of stimuli giving rise to constant responses, and the system identification paradigm where the experimental goal is to estimate and possibly compare sensory processing models. We discuss various theoretical and practical aspects of adaptive firing rate optimization, including optimization with stimulus space constraints, firing rate adaptation, and possible network constraints on the optimal stimulus. We consider the problem of system identification, and show how accurate estimation of non-linear models can be highly dependent on the stimulus set used to probe the network. We suggest that optimizing stimuli for accurate model estimation may make it possible to successfully identify non-linear models which are otherwise intractable, and summarize several recent studies of this type. Finally, we present a two-stage stimulus design procedure which combines the dual goals of model estimation and model comparison and may be especially useful for system identification experiments where the appropriate model is unknown beforehand. We propose that fast, on-line stimulus optimization enabled by increasing computer power can make it practical to move sensory neuroscience away from a descriptive paradigm and toward a new paradigm of real-time model estimation and comparison.

Local and global contrast adaptation in retinal ganglion cells

Retinal ganglion cells react to changes in visual contrast by adjusting their sensitivity and temporal filtering characteristics. This contrast adaptation has primarily been studied under spatially homogeneous stimulation. Yet, ganglion cell receptive fields are often characterized by spatial subfields, providing a substrate for nonlinear spatial processing. This raises the question whether contrast adaptation follows a similar subfield structure or whether it occurs globally over the receptive field even for local stimulation. We therefore recorded ganglion cell activity in isolated salamander retinas while locally changing visual contrast. Ganglion cells showed primarily global adaptation characteristics, with notable exceptions in certain aspects of temporal filtering. Surprisingly, some changes in filtering were most pronounced for locations where contrast did not change. This seemingly paradoxical effect can be explained by a simple computational model, which emphasizes the importance of local nonlinearities in the retina and suggests a reevaluation of previously reported local contrast adaptation.

Learning quadratic receptive fields from neural responses to natural stimuli

Models of neural responses to stimuli with complex spatiotemporal correlation structure often assume that neurons are selective for only a small number of linear projections of a potentially high-dimensional input. In this review, we explore recent modeling approaches where the neural response depends on the quadratic form of the input rather than on its linear projection, that is, the neuron is sensitive to the local covariance structure of the signal preceding the spike. To infer this quadratic dependence in the presence of arbitrary (e.g., naturalistic) stimulus distribution, we review several inference methods, focusing in particular on two information theory-based approaches (maximization of stimulus energy and of noise entropy) and two likelihood-based approaches (Bayesian spike-triggered covariance and extensions of generalized linear models). We analyze the formal relationship between the likelihood-based and information-based approaches to demonstrate how they lead to consistent inference. We demonstrate the practical feasibility of these procedures by using model neurons responding to a flickering variance stimulus.

Alert response to motion onset in the retina

Previous studies have shown that motion onset is very effective at capturing attention and is more salient than smooth motion. Here, we find that this salience ranking is present already in the firing rate of retinal ganglion cells. By stimulating the retina with a bar that appears, stays still, and then starts moving, we demonstrate that a subset of salamander retinal ganglion cells, fast OFF cells, responds significantly more strongly to motion onset than to smooth motion. We refer to this phenomenon as an alert response to motion onset. We develop a computational model that predicts the time-varying firing rate of ganglion cells responding to the appearance, onset, and smooth motion of a bar. This model, termed the adaptive cascade model, consists of a ganglion cell that receives input from a layer of bipolar cells, represented by individual rectified subunits. Additionally, both the bipolar and ganglion cells have separate contrast gain control mechanisms. This model captured the responses to our different motion stimuli over a wide range of contrasts, speeds, and locations. The alert response to motion onset, together with its computational model, introduces a new mechanism of sophisticated motion processing that occurs early in the visual system.

A method for closed-loop presentation of sensory stimuli conditional on the internal brain-state of awake animals

Brain activity often consists of interactions between internal-or on-going-and external-or sensory-activity streams, resulting in complex, distributed patterns of neural activity. Investigation of such interactions could benefit from closed-loop experimental protocols in which one stream can be controlled depending on the state of the other. We describe here methods to present rapid and precisely timed visual stimuli to awake animals, conditional on features of the animal's on-going brain state; those features are the presence, power and phase of oscillations in local field potentials (LFP). The system can process up to 64 channels in real time. We quantified its performance using simulations, synthetic data and animal experiments (chronic recordings in the dorsal cortex of awake turtles). The delay from detection of an oscillation to the onset of a visual stimulus on an LCD screen was 47.5ms and visual-stimulus onset could be locked to the phase of ongoing oscillations at any frequency ≤40Hz. Our software's architecture is flexible, allowing on-the-fly modifications by experimenters and the addition of new closed-loop control and analysis components through plugins. The source code of our system "StimOMatic" is available freely as open-source.

Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.

The iso-response method: measuring neuronal stimulus integration with closed-loop experiments

Throughout the nervous system, neurons integrate high-dimensional input streams and transform them into an output of their own. This integration of incoming signals involves filtering processes and complex non-linear operations. The shapes of these filters and non-linearities determine the computational features of single neurons and their functional roles within larger networks. A detailed characterization of signal integration is thus a central ingredient to understanding information processing in neural circuits. Conventional methods for measuring single-neuron response properties, such as reverse correlation, however, are often limited by the implicit assumption that stimulus integration occurs in a linear fashion. Here, we review a conceptual and experimental alternative that is based on exploring the space of those sensory stimuli that result in the same neural output. As demonstrated by recent results in the auditory and visual system, such iso-response stimuli can be used to identify the non-linearities relevant for stimulus integration, disentangle consecutive neural processing steps, and determine their characteristics with unprecedented precision. Automated closed-loop experiments are crucial for this advance, allowing rapid search strategies for identifying iso-response stimuli during experiments. Prime targets for the method are feed-forward neural signaling chains in sensory systems, but the method has also been successfully applied to feedback systems. Depending on the specific question, “iso-response” may refer to a predefined firing rate, single-spike probability, first-spike latency, or other output measures. Examples from different studies show that substantial progress in understanding neural dynamics and coding can be achieved once rapid online data analysis and stimulus generation, adaptive sampling, and computational modeling are tightly integrated into experiments.