Adaptation of the simple or complex nature of V1 receptive fields to visual statistics

Perception-Guided Quality Metric of 3D Point Clouds Using Hybrid Strategy

Full-reference point cloud quality assessment (FR-PCQA) aims to infer the quality of distorted point clouds with available references. Most of the existing FR-PCQA metrics ignore the fact that the human visual system (HVS) dynamically tackles visual information according to different distortion levels (i.e., distortion detection for high-quality samples and appearance perception for low-quality samples) and measure point cloud quality using unified features. To bridge the gap, in this paper, we propose a perception-guided hybrid metric (PHM) that adaptively leverages two visual strategies with respect to distortion degree to predict point cloud quality: to measure visible difference in high-quality samples, PHM takes into account the masking effect and employs texture complexity as an effective compensatory factor for absolute difference; on the other hand, PHM leverages spectral graph theory to evaluate appearance degradation in low-quality samples. Variations in geometric signals on graphs and changes in the spectral graph wavelet coefficients are utilized to characterize geometry and texture appearance degradation, respectively. Finally, the results obtained from the two components are combined in a non-linear method to produce an overall quality score of the tested point cloud. The results of the experiment on five independent databases show that PHM achieves state-of-the-art (SOTA) performance and offers significant performance improvement in multiple distortion environments. The code is publicly available at https://github.com/zhangyujie-1998/PHM.

Neural synchrony in cortical networks: mechanisms and implications for neural information processing and coding

Synchronization of neuronal discharges on the millisecond scale has long been recognized as a prevalent and functionally important attribute of neural activity. In this article, I review classical concepts and corresponding evidence of the mechanisms that govern the synchronization of distributed discharges in cortical networks and relate those mechanisms to their possible roles in coding and cognitive functions. To accommodate the need for a selective, directed synchronization of cells, I propose that synchronous firing of distributed neurons is a natural consequence of spike-timing-dependent plasticity (STDP) that associates cells repetitively receiving temporally coherent input: the “synchrony through synaptic plasticity” hypothesis. Neurons that are excited by a repeated sequence of synaptic inputs may learn to selectively respond to the onset of this sequence through synaptic plasticity. Multiple neurons receiving coherent input could thus actively synchronize their firing by learning to selectively respond at corresponding temporal positions. The hypothesis makes several predictions: first, the position of the cells in the network, as well as the source of their input signals, would be irrelevant as long as their input signals arrive simultaneously; second, repeating discharge patterns should get compressed until all or some part of the signals are synchronized; and third, this compression should be accompanied by a sparsening of signals. In this way, selective groups of cells could emerge that would respond to some recurring event with synchronous firing. Such a learned response pattern could further be modulated by synchronous network oscillations that provide a dynamic, flexible context for the synaptic integration of distributed signals. I conclude by suggesting experimental approaches to further test this new hypothesis.

Horizontal connectivity in V1: Prediction of coherence in contour and motion integration

This study demonstrates the functional importance of the Surround context relayed laterally in V1 by the horizontal connectivity, in controlling the latency and the gain of the cortical response to the feedforward visual drive. We report here four main findings: 1) a centripetal apparent motion sequence results in a shortening of the spiking latency of V1 cells, when the orientation of the local inducer and the global motion axis are both co-aligned with the RF orientation preference; 2) this contextual effects grows with visual flow speed, peaking at 150–250°/s when it matches the propagation speed of horizontal connectivity (0.15–0.25 mm/ms); 3) For this speed range, the axial sensitivity of V1 cells is tilted by 90° to become co-aligned with the orientation preference axis; 4) the strength of modulation by the surround context correlates with the spatiotemporal coherence of the apparent motion flow. Our results suggest an internally-generated binding process, linking local (orientation /position) and global (motion/direction) features as early as V1. This long-range diffusion process constitutes a plausible substrate in V1 of the human psychophysical bias in speed estimation for collinear motion. Since it is demonstrated in the anesthetized cat, this novel form of contextual control of the cortical gain and phase is a built-in property in V1, whose expression does not require behavioral attention and top-down control from higher cortical areas. We propose that horizontal connectivity participates in the propagation of an internal “prediction” wave, shaped by visual experience, which links contour co-alignment and global axial motion at an apparent speed in the range of saccade-like eye movements.

Deep networks may capture biological behavior for shallow, but not deep, empirical characterizations

We assess whether deep convolutional networks (DCN) can account for a most fundamental property of human vision: detection/discrimination of elementary image elements (bars) at different contrast levels. The human visual process can be characterized to varying degrees of "depth," ranging from percentage of correct detection to detailed tuning and operating characteristics of the underlying perceptual mechanism. We challenge deep networks with the same stimuli/tasks used with human observers and apply equivalent characterization of the stimulus-response coupling. In general, we find that popular DCN architectures do not account for signature properties of the human process. For shallow depth of characterization, some variants of network-architecture/training-protocol produce human-like trends; however, more articulate empirical descriptors expose glaring discrepancies. Networks can be coaxed into learning those richer descriptors by shadowing a human surrogate in the form of a tailored circuit perturbed by unstructured input, thus ruling out the possibility that human-model misalignment in standard protocols may be attributable to insufficient representational power. These results urge caution in assessing whether neural networks do or do not capture human behavior: ultimately, our ability to assess "success" in this area can only be as good as afforded by the depth of behavioral characterization against which the network is evaluated. We propose a novel set of metrics/protocols that impose stringent constraints on the evaluation of DCN behavior as an adequate approximation to biological processes.

Periodic clustering of simple and complex cells in visual cortex

Neurons in the primary visual cortex (V1) are often classified as simple or complex cells, but it is debated whether they are discrete hierarchical classes of neurons or if they represent a continuum of variation within a single class of cells. Herein, we show that simple and complex cells may arise commonly from the feedforward projections from the retina. From analysis of the cortical receptive fields in cats, we show evidence that simple and complex cells originate from the periodic variation of ON-OFF segregation in the feedforward projection of retinal mosaics, by which they organize into periodic clusters in V1. From data in cats, we observed that clusters of simple and complex receptive fields correlate topographically with orientation maps, which supports our model prediction. Our results suggest that simple and complex cells are not two distinct neural populations but arise from common retinal afferents, simultaneous with orientation tuning.

Efficient and adaptive sensory codes

The ability to adapt to changes in stimulus statistics is a hallmark of sensory systems. Here, we developed a theoretical framework that can account for the dynamics of adaptation from an information processing perspective. We use this framework to optimize and analyze adaptive sensory codes, and we show that codes optimized for stationary environments can suffer from prolonged periods of poor performance when the environment changes. To mitigate the adversarial effects of these environmental changes, sensory systems must navigate tradeoffs between the ability to accurately encode incoming stimuli and the ability to rapidly detect and adapt to changes in the distribution of these stimuli. We derive families of codes that balance these objectives, and we demonstrate their close match to experimentally observed neural dynamics during mean and variance adaptation. Our results provide a unifying perspective on adaptation across a range of sensory systems, environments, and sensory tasks.

Functional reallocation of sensory processing resources caused by long-term neural adaptation to altered optics

The eye’s optics are a major determinant of visual perception. Elucidating how long-term exposure to optical defects affects visual processing is key to understanding the capacity for, and limits of, sensory plasticity. Here, we show evidence of functional reallocation of sensory processing resources following long-term exposure to poor optical quality. Using adaptive optics to bypass all optical defects, we assessed visual processing in neurotypically-developed adults with healthy eyes and with keratoconus – a corneal disease causing severe optical aberrations. Under fully-corrected optical conditions, keratoconus patients showed altered contrast sensitivity, with impaired sensitivity for fine spatial details and better-than-typical sensitivity for coarse spatial details. Both gains and losses in sensitivity were more pronounced in patients experiencing poorer optical quality in their daily life and mediated by changes in signal enhancement mechanisms. These findings show that adult neural processing adapts to better match the changes in sensory inputs caused by long-term exposure to altered optics.

A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex

To understand how the brain processes sensory information to guide behavior, we must know how stimulus representations are transformed throughout the visual cortex. Here we report an open, large-scale physiological survey of activity in the awake mouse visual cortex: the Allen Brain Observatory Visual Coding dataset. This publicly available dataset includes the cortical activity of nearly 60,000 neurons from six visual areas, four layers, and 12 transgenic mouse lines in a total of 243 adult mice, in response to a systematic set of visual stimuli. We classify neurons on the basis of joint reliabilities to multiple stimuli and validate this functional classification with models of visual responses. While most classes are characterized by responses to specific subsets of the stimuli, the largest class is not reliably responsive to any of the stimuli and becomes progressively larger in higher visual areas. These classes reveal a functional organization wherein putative dorsal areas show specialization for visual motion signals.

Data-Driven Approaches to Understanding Visual Neuron Activity

With modern neurophysiological methods able to record neural activity throughout the visual pathway in the context of arbitrarily complex visual stimulation, our understanding of visual system function is becoming limited by the available models of visual neurons that can be directly related to such data. Different forms of statistical models are now being used to probe the cellular and circuit mechanisms shaping neural activity, understand how neural selectivity to complex visual features is computed, and derive the ways in which neurons contribute to systems-level visual processing. However, models that are able to more accurately reproduce observed neural activity often defy simple interpretations. As a result, rather than being used solely to connect with existing theories of visual processing, statistical modeling will increasingly drive the evolution of more sophisticated theories.

Multiscale modeling of mechanotransduction in the utricle

We review recent progress in using numerical models to relate utricular hair bundle and otoconial membrane (OM) structure to the functional requirements imposed by natural behavior in turtles. The head movements section reviews the evolution of experimental attempts to understand vestibular system function with emphasis on turtles, including data showing that accelerations occurring during natural head movements achieve higher magnitudes and frequencies than previously assumed. The structure section reviews quantitative anatomical data documenting topographical variation in the structures underlying macromechanical and micromechanical responses of the turtle utricle to head movement: hair bundles, OM, and bundle-OM coupling. The macromechanics section reviews macromechanical models that incorporate realistic anatomical and mechanical parameters and reveal that the system is significantly underdamped, contrary to previous assumptions. The micromechanics: hair bundle motion and met currents section reviews work based on micromechanical models, which demonstrates that topographical variation in the structure of hair bundles and OM, and their mode of coupling, result in regional specializations for signaling of low frequency (or static) head position and high frequency head accelerations. We conclude that computational models based on empirical data are especially promising for investigating mechanotransduction in this challenging sensorimotor system.

Nonlinear Processing of Shape Information in Rat Lateral Extrastriate Cortex

In rodents, the progression of extrastriate areas located laterally to primary visual cortex (V1) has been assigned to a putative object-processing pathway (homologous to the primate ventral stream), based on anatomical considerations. Recently, we found functional support for such attribution (Tafazoli et al., 2017), by showing that this cortical progression is specialized for coding object identity despite view changes, the hallmark property of a ventral-like pathway. Here, we sought to clarify what computations are at the base of such specialization. To this aim, we performed multielectrode recordings from V1 and laterolateral area LL (at the apex of the putative ventral-like hierarchy) of male adult rats, during the presentation of drifting gratings and noise movies. We found that the extent to which neuronal responses were entrained to the phase of the gratings sharply dropped from V1 to LL, along with the quality of the receptive fields inferred through reverse correlation. Concomitantly, the tendency of neurons to respond to different oriented gratings increased, whereas the sharpness of orientation tuning declined. Critically, these trends are consistent with the nonlinear summation of visual inputs that is expected to take place along the ventral stream, according to the predictions of hierarchical models of ventral computations and a meta-analysis of the monkey literature. This suggests an intriguing homology between the mechanisms responsible for building up shape selectivity and transformation tolerance in the visual cortex of primates and rodents, reasserting the potential of the latter as models to investigate ventral stream functions at the circuitry level.SIGNIFICANCE STATEMENT Despite the growing popularity of rodents as models of visual functions, it remains unclear whether their visual cortex contains specialized modules for processing shape information. To addresses this question, we compared how neuronal tuning evolves from rat primary visual cortex (V1) to a downstream visual cortical region (area LL) that previous work has implicated in shape processing. In our experiments, LL neurons displayed a stronger tendency to respond to drifting gratings with different orientations while maintaining a sustained response across the whole duration of the drift cycle. These trends match the increased complexity of pattern selectivity and the augmented tolerance to stimulus translation found in monkey visual temporal cortex, thus revealing a homology between shape processing in rodents and primates.

The empirical characteristics of human pattern vision defy theoretically-driven expectations

Contrast is the most fundamental property of images. Consequently, any comprehensive model of biological vision must incorporate this attribute and provide a veritable description of its impact on visual perception. Current theoretical and computational models predict that vision should modify its characteristics at low contrast: for example, it should become broader (more lowpass) to protect from noise, as often demonstrated by individual neurons. We find that the opposite is true for human discrimination of elementary image elements: vision becomes sharper, not broader, as contrast approaches threshold levels. Furthermore, it suffers from increased internal variability at low contrast and it transitions from a surprisingly linear regime at high contrast to a markedly nonlinear processing mode in the low-contrast range. These characteristics are hard-wired in that they happen on a single trial without memory or expectation. Overall, the empirical results urge caution when attempting to interpret human vision from the standpoint of optimality and related theoretical constructs. Direct measurements of this phenomenon indicate that the actual constraints derive from intrinsic architectural features, such as the co-existence of complex-cell-like and simple-cell-like components. Small circuits built around these elements can indeed account for the empirical results, but do not appear to operate in a manner that conforms to optimality even approximately. More generally, our results provide a compelling demonstration of how far we still are from securing an adequate computational account of the most basic operations carried out by human vision.

Microsaccade-rhythmic modulation of neural synchronization and coding within and across cortical areas V1 and V2

Primates sample their visual environment actively through saccades and microsaccades (MSs). Saccadic eye movements not only modulate neural spike rates but might also affect temporal correlations (synchrony) among neurons. Neural synchrony plays a role in neural coding and modulates information transfer between cortical areas. The question arises of how eye movements shape neural synchrony within and across cortical areas and how it affects visual processing. Through local field recordings in macaque early visual cortex while monitoring eye position and through neural network simulations, we find 2 distinct synchrony regimes in early visual cortex that are embedded in a 3- to 4-Hz MS-related rhythm during visual fixation. In the period shortly after an MS (“transient period”), synchrony was high within and between cortical areas. In the subsequent period (“sustained period”), overall synchrony dropped and became selective to stimulus properties. Only mutually connected neurons with similar stimulus responses exhibited sustained narrow-band gamma synchrony (25–80 Hz), both within and across cortical areas. Recordings in macaque V1 and V2 matched the model predictions. Furthermore, our modeling provides predictions on how (micro)saccade-modulated gamma synchrony in V1 shapes V2 receptive fields (RFs). We suggest that the rhythmic alternation between synchronization regimes represents a basic repeating sampling strategy of the visual system.

During visual exploration, we continuously move our eyes in a quick, coordinated manner several times a second to scan our environment. These movements are called saccades. Even while we fixate on a visual object, we unconsciously execute small saccades that are termed microsaccades (MSs). Despite MSs being relatively small, they are suggested to be critical to maintain and support accurate perception during visual fixation. Here, we studied in macaques the influence of MSs on the synchronization of neural rhythms—which are important to regulate information flow in the brain—in areas of the cerebral cortex that are important for early processing of visual information, and we complemented the analysis with computational modeling. We found that synchronization properties shortly after an MS were distinct from synchronization in the later phase. Specifically, we found an early and spectrally broadband synchronization within and between visual cortices that was broadly tuned over the cortical space and stimulus properties. This was followed by narrow-band synchronization in the gamma range (25–80 Hz) that was spatially and stimulus specific. This suggests that the manner in which information is transmitted and integrated between early visual cortices depends on the timing relative to MSs. We illustrate this in a computational model showing that the receptive field (RF) of neurons in the secondary visual cortex are expected to be different depending on MS timing. Our results highlight the significance of MS timing for understanding cortical dynamics and suggest that the regulation of synchronization might be one mechanism by which MSs support visual perception.

Big data and the industrialization of neuroscience: A safe roadmap for understanding the brain?

New technologies in neuroscience generate reams of data at an exponentially increasing rate, spurring the design of very-large-scale data-mining initiatives. Several supranational ventures are contemplating the possibility of achieving, within the next decade(s), full simulation of the human brain.

Representation of tactile scenes in the rodent barrel cortex

After half a century of research, the sensory features coded by neurons of the rodent barrel cortex remain poorly understood. Still, views of the sensory representation of whisker information are increasingly shifting from a labeled line representation of single-whisker deflections to a selectivity for specific elements of the complex statistics of the multi-whisker deflection patterns that take place during spontaneous rodent behavior - so called natural tactile scenes. Here we review the current knowledge regarding the coding of patterns of whisker stimuli by barrel cortex neurons, from responses to single-whisker deflections to the representation of complex tactile scenes. A number of multi-whisker tunings have already been identified, including center-surround feature extraction, angular tuning during edge-like multi-whisker deflections, and even tuning to specific statistical properties of the tactile scene such as the level of correlation across whiskers. However, a more general model of the representation of multi-whisker information in the barrel cortex is still missing. This is in part because of the lack of a human intuition regarding the perception emerging from a whisker system, but also because in contrast to other primary sensory cortices such as the visual cortex, the spatial feature selectivity of barrel cortex neurons rests on highly nonlinear interactions that remained hidden to classical receptive field approaches.

Multidimensional receptive field processing by cat primary auditory cortical neurons

The receptive fields of many auditory cortical neurons are multidimensional and are best represented by more than one stimulus feature. The number of these dimensions, their characteristics, and how they differ with stimulus context have been relatively unexplored. Standard methods that are often used to characterize multidimensional stimulus selectivity, such as spike-triggered covariance (STC) or maximally informative dimensions (MIDs), are either limited to Gaussian stimuli or are only able to recover a small number of stimulus features due to data limitations. An information theoretic extension of STC, the maximum noise entropy (MNE) model, can be used with non-Gaussian stimulus distributions to find an arbitrary number of stimulus dimensions. When we applied the MNE model to auditory cortical neurons, we often found more than two stimulus features that influenced neuronal firing. Excitatory and suppressive features coded different acoustic contexts: excitatory features encoded higher temporal and spectral modulations, while suppressive features had lower modulation frequency preferences. We found that the excitatory and suppressive features themselves were sensitive to stimulus context when we employed two stimuli that differed only in their short-term correlation structure: while the linear features were similar, the secondary features were strongly affected by stimulus statistics. These results show that multidimensional receptive field processing is influenced by feature type and stimulus context.

Sensory noise predicts divisive reshaping of receptive fields

In order to respond reliably to specific features of their environment, sensory neurons need to integrate multiple incoming noisy signals. Crucially, they also need to compete for the interpretation of those signals with other neurons representing similar features. The form that this competition should take depends critically on the noise corrupting these signals. In this study we show that for the type of noise commonly observed in sensory systems, whose variance scales with the mean signal, sensory neurons should selectively divide their input signals by their predictions, suppressing ambiguous cues while amplifying others. Any change in the stimulus context alters which inputs are suppressed, leading to a deep dynamic reshaping of neural receptive fields going far beyond simple surround suppression. Paradoxically, these highly variable receptive fields go alongside and are in fact required for an invariant representation of external sensory features. In addition to offering a normative account of context-dependent changes in sensory responses, perceptual inference in the presence of signal-dependent noise accounts for ubiquitous features of sensory neurons such as divisive normalization, gain control and contrast dependent temporal dynamics.

The feature-specific propagation of orientation and direction adaptation from areas 17 to 21a in cats

Adaptation plays a key role in visual information processing, and investigations on the adaptation across different visual regions will be helpful to understand how information is processed dynamically along the visual streams. Recent studies have found the enhanced adaptation effects in the early visual system (from LGN to V1) and the dorsal stream (from V1 to MT). However, it remains unclear how adaptation effect propagates along the form/orientation stream in the visual system. In this study, we compared the orientation and direction adaptation evoked by drifting gratings and stationary flashing gratings, as well as moving random dots, in areas 17 and 21a simultaneously of cats. Recorded by single-unit and intrinsic signal optical imaging, induced by both top-up and biased adaptation protocols, the orientation adaptation effect was greater in response decline and preferred orientation shifts in area 21a compared to area 17. However, for the direction adaptation, no difference was observed between these two areas. These results suggest the feature-specific propagation of the adaptation effect along the visual stream.

Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1

Neurons in the primary visual cortex are known for responding vigorously but with high variability to classical stimuli such as drifting bars or gratings. By contrast, natural scenes are encoded more efficiently by sparse and temporal precise spiking responses. We used a conductance-based model of the visual system in higher mammals to investigate how two specific features of the thalamo-cortical pathway, namely push-pull receptive field organization and fast synaptic depression, can contribute to this contextual reshaping of V1 responses. By comparing cortical dynamics evoked respectively by natural vs. artificial stimuli in a comprehensive parametric space analysis, we demonstrate that the reliability and sparseness of the spiking responses during natural vision is not a mere consequence of the increased bandwidth in the sensory input spectrum. Rather, it results from the combined impacts of fast synaptic depression and push-pull inhibition, the later acting for natural scenes as a form of “effective” feed-forward inhibition as demonstrated in other sensory systems. Thus, the combination of feedforward-like inhibition with fast thalamo-cortical synaptic depression by simple cells receiving a direct structured input from thalamus composes a generic computational mechanism for generating a sparse and reliable encoding of natural sensory events.

Synaptic Correlates of Low-Level Perception in V1

The computational role of primary visual cortex (V1) in low-level perception remains largely debated. A dominant view assumes the prevalence of higher cortical areas and top-down processes in binding information across the visual field. Here, we investigated the role of long-distance intracortical connections in form and motion processing by measuring, with intracellular recordings, their synaptic impact on neurons in area 17 (V1) of the anesthetized cat. By systematically mapping synaptic responses to stimuli presented in the nonspiking surround of V1 receptive fields, we provide the first quantitative characterization of the lateral functional connectivity kernel of V1 neurons. Our results revealed at the population level two structural-functional biases in the synaptic integration and dynamic association properties of V1 neurons. First, subthreshold responses to oriented stimuli flashed in isolation in the nonspiking surround exhibited a geometric organization around the preferred orientation axis mirroring the psychophysical "association field" for collinear contour perception. Second, apparent motion stimuli, for which horizontal and feedforward synaptic inputs summed in-phase, evoked dominantly facilitatory nonlinear interactions, specifically during centripetal collinear activation along the preferred orientation axis, at saccadic-like speeds. This spatiotemporal integration property, which could constitute the neural correlate of a human perceptual bias in speed detection, suggests that local (orientation) and global (motion) information is already linked within V1. We propose the existence of a "dynamic association field" in V1 neurons, whose spatial extent and anisotropy are transiently updated and reshaped as a function of changes in the retinal flow statistics imposed during natural oculomotor exploration.

SIGNIFICANCE STATEMENT

The computational role of primary visual cortex in low-level perception remains debated. The expression of this "pop-out" perception is often assumed to require attention-related processes, such as top-down feedback from higher cortical areas. Using intracellular techniques in the anesthetized cat and novel analysis methods, we reveal unexpected structural-functional biases in the synaptic integration and dynamic association properties of V1 neurons. These structural-functional biases provide a substrate, within V1, for contour detection and, more unexpectedly, global motion flow sensitivity at saccadic speed, even in the absence of attentional processes. We argue for the concept of a "dynamic association field" in V1 neurons, whose spatial extent and anisotropy changes with retinal flow statistics, and more generally for a renewed focus on intracortical computation.

Categorically distinct types of receptive fields in early visual cortex

In the visual cortex, distinct types of neurons have been identified based on cellular morphology, response to injected current, or expression of specific markers, but neurophysiological studies have revealed visual receptive field (RF) properties that appear to be on a continuum, with only two generally recognized classes: simple and complex. Most previous studies have characterized visual responses of neurons using stereotyped stimuli such as bars, gratings, or white noise and simple system identification approaches (e.g., reverse correlation). Here we estimate visual RF models of cortical neurons using visually rich natural image stimuli and regularized regression system identification methods and characterize their spatial tuning, temporal dynamics, spatiotemporal behavior, and spiking properties. We quantitatively demonstrate the existence of three functionally distinct categories of simple cells, distinguished by their degree of orientation selectivity (isotropic or oriented) and the nature of their output nonlinearity (expansive or compressive). In addition, these three types have differing average values of several other properties. Cells with nonoriented RFs tend to have smaller RFs, shorter response durations, no direction selectivity, and high reliability. Orientation-selective neurons with an expansive output nonlinearity have Gabor-like RFs, lower spontaneous activity and responsivity, and spiking responses with higher sparseness. Oriented RFs with a compressive nonlinearity are spatially nondescript and tend to show longer response latency. Our findings indicate multiple physiologically defined types of RFs beyond the simple/complex dichotomy, suggesting that cortical neurons may have more specialized functional roles rather than lying on a multidimensional continuum.

Spatial Correlations in Natural Scenes Modulate Response Reliability in Mouse Visual Cortex

Intrinsic neuronal variability significantly limits information encoding in the primary visual cortex (V1). Certain stimuli can suppress this intertrial variability to increase the reliability of neuronal responses. In particular, responses to natural scenes, which have broadband spatiotemporal statistics, are more reliable than responses to stimuli such as gratings. However, very little is known about which stimulus statistics modulate reliable coding and how this occurs at the neural ensemble level. Here, we sought to elucidate the role that spatial correlations in natural scenes play in reliable coding. We developed a novel noise-masking method to systematically alter spatial correlations in natural movies, without altering their edge structure. Using high-speed two-photon calcium imaging in vivo, we found that responses in mouse V1 were much less reliable at both the single neuron and population level when spatial correlations were removed from the image. This change in reliability was due to a reorganization of between-neuron correlations. Strongly correlated neurons formed ensembles that reliably and accurately encoded visual stimuli, whereas reducing spatial correlations reduced the activation of these ensembles, leading to an unreliable code. Together with an ensemble-specific normalization model, these results suggest that the coordinated activation of specific subsets of neurons underlies the reliable coding of natural scenes.

Cortical Correlates of Low-Level Perception: From Neural Circuits to Percepts

Low-level perception results from neural-based computations, which build a multimodal skeleton of unconscious or self-generated inferences on our environment. This review identifies bottleneck issues concerning the role of early primary sensory cortical areas, mostly in rodent and higher mammals (cats and non-human primates), where perception substrates can be searched at multiple scales of neural integration. We discuss the limitation of purely bottom-up approaches for providing realistic models of early sensory processing and the need for identification of fast adaptive processes, operating within the time of a percept. Future progresses will depend on the careful use of comparative neuroscience (guiding the choices of experimental models and species adapted to the questions under study), on the definition of agreed-upon benchmarks for sensory stimulation, on the simultaneous acquisition of neural data at multiple spatio-temporal scales, and on the in vivo identification of key generic integration and plasticity algorithms validated experimentally and in simulations.

Hierarchical Novelty-Familiarity Representation in the Visual System by Modular Predictive Coding

Predictive coding has been previously introduced as a hierarchical coding framework for the visual system. At each level, activity predicted by the higher level is dynamically subtracted from the input, while the difference in activity continuously propagates further. Here we introduce modular predictive coding as a feedforward hierarchy of prediction modules without back-projections from higher to lower levels. Within each level, recurrent dynamics optimally segregates the input into novelty and familiarity components. Although the anatomical feedforward connectivity passes through the novelty-representing neurons, it is nevertheless the familiarity information which is propagated to higher levels. This modularity results in a twofold advantage compared to the original predictive coding scheme: the familiarity-novelty representation forms quickly, and at each level the full representational power is exploited for an optimized readout. As we show, natural images are successfully compressed and can be reconstructed by the familiarity neurons at each level. Missing information on different spatial scales is identified by novelty neurons and complements the familiarity representation. Furthermore, by virtue of the recurrent connectivity within each level, non-classical receptive field properties still emerge. Hence, modular predictive coding is a biologically realistic metaphor for the visual system that dynamically extracts novelty at various scales while propagating the familiarity information.

Primary visual cortex represents the difference between past and present

The visual system is confronted with rapidly changing stimuli in everyday life. It is not well understood how information in such a stream of input is updated within the brain. We performed voltage-sensitive dye imaging across the primary visual cortex (V1) to capture responses to sequences of natural scene contours. We presented vertically and horizontally filtered natural images, and their superpositions, at 10 or 33 Hz. At low frequency, the encoding was found to represent not the currently presented images, but differences in orientation between consecutive images. This was in sharp contrast to more rapid sequences for which we found an ongoing representation of current input, consistent with earlier studies. Our finding that for slower image sequences, V1 does no longer report actual features but represents their relative difference in time counteracts the view that the first cortical processing stage must always transfer complete information. Instead, we show its capacities for change detection with a new emphasis on the role of automatic computation evolving in the 100-ms range, inevitably affecting information transmission further downstream.

Imaging light responses of foveal ganglion cells in the living macaque eye

The fovea dominates primate vision, and its anatomy and perceptual abilities are well studied, but its physiology has been little explored because of limitations of current physiological methods. In this study, we adapted a novel in vivo imaging method, originally developed in mouse retina, to explore foveal physiology in the macaque, which permits the repeated imaging of the functional response of many retinal ganglion cells (RGCs) simultaneously. A genetically encoded calcium indicator, G-CaMP5, was inserted into foveal RGCs, followed by calcium imaging of the displacement of foveal RGCs from their receptive fields, and their intensity-response functions. The spatial offset of foveal RGCs from their cone inputs makes this method especially appropriate for fovea by permitting imaging of RGC responses without excessive light adaptation of cones. This new method will permit the tracking of visual development, progression of retinal disease, or therapeutic interventions, such as insertion of visual prostheses.

Hidden complexity of synaptic receptive fields in cat V1

In the primary visual cortex (V1), Simple and Complex receptive fields (RFs) are usually characterized on the basis of the linearity of the cell spiking response to stimuli of opposite contrast. Whether or not this classification reflects a functional dichotomy in the synaptic inputs to Simple and Complex cells is still an open issue. Here we combined intracellular membrane potential recordings in cat V1 with 2D dense noise stimulation to decompose the Simple-like and Complex-like components of the subthreshold RF into a parallel set of functionally distinct subunits. Results show that both Simple and Complex RFs exhibit a remarkable diversity of excitatory and inhibitory Complex-like contributions, which differ in orientation and spatial frequency selectivity from the linear RF, even in layer 4 and layer 6 Simple cells. We further show that the diversity of Complex-like contributions recovered at the subthreshold level is expressed in the cell spiking output. These results demonstrate that the Simple or Complex nature of V1 RFs does not rely on the diversity of Complex-like components received by the cell from its synaptic afferents but on the imbalance between the weights of the Simple-like and Complex-like synaptic contributions.

Spatiotemporal receptive fields of barrel cortex revealed by reverse correlation of synaptic input

Of all sensory areas, barrel cortex is among the best understood in terms of circuitry, yet least understood in terms of sensory function. We combined intracellular recording in rats with a novel multi-directional multi-whisker stimulator system to estimate receptive fields by reverse correlation of stimuli to synaptic inputs. Spatiotemporal receptive fields were identified orders of magnitude faster than by conventional spike-based approaches, even for neurons with little spiking activity. Given a suitable stimulus representation, a linear model captured the stimulus-response relationship for all neurons with surprisingly high accuracy. In contrast to conventional single-whisker stimuli, complex stimuli revealed dramatically sharpened receptive fields, largely due to adaptation. This phenomenon allowed the surround to facilitate rather than suppress responses to the principal whisker. Optimized stimuli enhanced firing in layers 4-6 but not 2/3, which remained sparsely active. Surround facilitation through adaptation may be required for discriminating complex shapes and textures during natural sensing.

The mechanism for processing random-dot motion at various speeds in early visual cortices

All moving objects generate sequential retinotopic activations representing a series of discrete locations in space and time (motion trajectory). How direction-selective neurons in mammalian early visual cortices process motion trajectory remains to be clarified. Using single-cell recording and optical imaging of intrinsic signals along with mathematical simulation, we studied response properties of cat visual areas 17 and 18 to random dots moving at various speeds. We found that, the motion trajectory at low speed was encoded primarily as a direction signal by groups of neurons preferring that motion direction. Above certain transition speeds, the motion trajectory is perceived as a spatial orientation representing the motion axis of the moving dots. In both areas studied, above these speeds, other groups of direction-selective neurons with perpendicular direction preferences were activated to encode the motion trajectory as motion-axis information. This applied to both simple and complex neurons. The average transition speed for switching between encoding motion direction and axis was about 31°/s in area 18 and 15°/s in area 17. A spatio-temporal energy model predicted the transition speeds accurately in both areas, but not the direction-selective indexes to random-dot stimuli in area 18. In addition, above transition speeds, the change of direction preferences of population responses recorded by optical imaging can be revealed using vector maximum but not vector summation method. Together, this combined processing of motion direction and axis by neurons with orthogonal direction preferences associated with speed may serve as a common principle of early visual motion processing.

Animation of natural scene by virtual eye-movements evokes high precision and low noise in V1 neurons

Synaptic noise is thought to be a limiting factor for computational efficiency in the brain. In visual cortex (V1), ongoing activity is present in vivo, and spiking responses to simple stimuli are highly unreliable across trials. Stimulus statistics used to plot receptive fields, however, are quite different from those experienced during natural visuomotor exploration. We recorded V1 neurons intracellularly in the anaesthetized and paralyzed cat and compared their spiking and synaptic responses to full field natural images animated by simulated eye-movements to those evoked by simpler (grating) or higher dimensionality statistics (dense noise). In most cells, natural scene animation was the only condition where high temporal precision (in the 10–20 ms range) was maintained during sparse and reliable activity. At the subthreshold level, irregular but highly reproducible membrane potential dynamics were observed, even during long (several 100 ms) “spike-less” periods. We showed that both the spatial structure of natural scenes and the temporal dynamics of eye-movements increase the signal-to-noise ratio by a non-linear amplification of the signal combined with a reduction of the subthreshold contextual noise. These data support the view that the sparsening and the time precision of the neural code in V1 may depend primarily on three factors: (1) broadband input spectrum: the bandwidth must be rich enough for recruiting optimally the diversity of spatial and time constants during recurrent processing; (2) tight temporal interplay of excitation and inhibition: conductance measurements demonstrate that natural scene statistics narrow selectively the duration of the spiking opportunity window during which the balance between excitation and inhibition changes transiently and reversibly; (3) signal energy in the lower frequency band: a minimal level of power is needed below 10 Hz to reach consistently the spiking threshold, a situation rarely reached with visual dense noise.

On the relation between receptive field structure and stimulus selectivity in the tree shrew primary visual cortex

There are notable differences in functional properties of primary visual cortex (V1) neurons among mammalian species, particularly those concerning the occurrence of simple and complex cells and the generation of orientation selectivity. Here, we present quantitative data on receptive field (RF) structure, response modulation, and orientation tuning for single neurons in V1 of the tree shrew, a close relative of primates. We find that spatial RF subfield segregation, a criterion for identifying simple cells, was exceedingly small in the tree shrew V1. In contrast, many neurons exhibited elevated F1/F0 modulation that is often used as a simple cell marker. This apparent discrepancy can be explained by the robust stimulus polarity preference in tree shrew V1, which inflates F1/F0 ratio values. RF structure mapped with sparse-noise-which is spatially restricted and emphasizes thalamo-cortical feed-forward inputs-appeared unrelated to orientation selectivity. However, RF structure mapped using the Hartley subspace stimulus-which covers a large area of the visual field and recruits considerable intracortical processing-did predict orientation preference. Our findings reveal a number of striking similarities in V1 functional organization between tree shrews and primates, emphasizing the important role of intracortical recurrent processing in shaping V1 response properties in these species.

Effects of contrast and contrast adaptation on static receptive field features in macaque area V1

The spatiotemporal features of the “static” receptive field (RF), as revealed with flashing bars or spots, determine other RF properties. We examined how some of these static RF features vary with contrast and contrast adaptation in area V1 of the anesthetized macaque monkey. RFs were mapped with light and dark flashing bars presented at three different contrasts, with the low and medium contrasts eliciting approximately 1/3 and 2/3 of the high-contrast response amplitude. The main results are as follows: 1) RF widths decreased when contrast decreased; however, the amount of decrease was less than that expected from an iceberg model and closer to the expectation of a contrast invariance of the RF width. 2) Area tuning experiments with drifting gratings showed an opposite effect of contrast: an increase in preferred stimulus diameter when contrast decreased. This implies that the effect of contrast on preferred stimulus size is not predictable from the static RF. 3) Contrast adaptation attenuated the effect of contrast on RF amplitude but did not significantly modify RF width. 4) RF subregion overlap was only marginally affected by changes in contrast and contrast adaptation; the classification of cells as simple and complex, when established from subregion overlap, appears to be robust with respect to changes in contrast and adaptation state. Previous studies have shown that the spatiotemporal features of the RF depend largely on the stimuli used to map the RF. This study shows that contrast is one elemental feature that contributes to the dynamics of the RF.

The accuracy of membrane potential reconstruction based on spiking receptive fields

A common technique used to study the response selectivity of neurons is to measure the relationship between sensory stimulation and action potential responses. Action potentials, however, are only indirectly related to the synaptic inputs that determine the underlying, subthreshold, response selectivity. We present a method to predict membrane potential, the measurable result of the convergence of synaptic inputs, based on spike rate alone and then test its utility by comparing predictions to actual membrane potential recordings from simple cells in primary visual cortex. Using a noise stimulus, we found that spike rate receptive fields were in precise correspondence with membrane potential receptive fields (R(2) = 0.74). On average, spike rate alone could predict 44% of membrane potential fluctuations to dynamic noise stimuli, demonstrating the utility of this method to extract estimates of subthreshold responses. We also found that the nonlinear relationship between membrane potential and spike rate could also be extracted from spike rate data alone by comparing predictions from the noise stimulus with the actual spike rate. Our analysis reveals that linear receptive field models extracted from noise stimuli accurately reflect the underlying membrane potential selectivity and thus represent a method to generate estimates of the underlying average membrane potential from spike rate data alone.