Predictive coding accounts for V1 response properties recorded using reverse correlation

Model-based characterization of the selectivity of neurons in primary visual cortex

Statistical models are increasingly being used to understand the complexity of stimulus selectivity in primary visual cortex (V1) in the context of complex time-varying stimuli, replacing averaging responses to simple parametric stimuli. Although such models often can more accurately reflect the computations performed by V1 neurons in more natural visual environments, they do not by themselves provide insight into V1 neural selectivity to basic stimulus features such as receptive field size, spatial frequency tuning, and phase invariance. Here, we present a battery of analyses that can be directly applied to encoding models to link complex encoding models to more interpretable aspects of stimulus selectivity. We apply this battery to nonlinear models of V1 neurons recorded in awake macaque during random bar stimuli. In linking model properties to more classical measurements, we demonstrate several novel aspects of V1 selectivity not available to simpler experimental measurements. For example, this approach reveals that individual spatiotemporal elements of the V1 models often have a smaller spatial scale than the neuron as a whole, resulting in nontrivial tuning to spatial frequencies. In addition, we propose measures of nonlinear integration that suggest that classical classifications of V1 neurons into simple versus complex cells will be spatial-frequency dependent. In total, rather than obfuscate classical characterizations of V1 neurons, model-based characterizations offer a means to more fully understand their selectivity, and link their classical tuning properties to their roles in more complex, natural, visual processing.NEW & NOTEWORTHY Visual neurons are increasingly being studied with more complex, natural visual stimuli, and increasingly complex models are necessary to characterize their response properties. Here, we describe a battery of analyses that relate these more complex models to classical characterizations. Using such model-based characterizations of V1 neurons furthermore yields several new insights into V1 processing not possible to capture in more classical means to measure their visual selectivity.

The cognitive penetrability of perception: A blocked debate and a tentative solution

Despite the extensive body of psychological findings suggesting that cognition influences perception, the debate between defenders and detractors of the cognitive penetrability of perception persists. While detractors demand more strictness in psychological experiments, proponents consider that empirical studies show that cognitive penetrability occurs. These considerations have led some theorists to propose that the debate has reached a dead end. The issue about where perception ends and cognition begins is, I argue, one of the reasons why the debate is cornered. Another reason is the inability of psychological studies to present uncontroversial interpretations of the results obtained. To dive into other kinds of empirical sources is, therefore, required to clarify the debate. In this paper, I explain where the debate is blocked, and suggest that neuroscientific evidence together with the predictive coding account, might decant the discussion on the side of the penetrability thesis.

Predictive Coding with Neural Transmission Delays: A Real-Time Temporal Alignment Hypothesis

Hierarchical predictive coding is an influential model of cortical organization, in which sequential hierarchical levels are connected by backward connections carrying predictions, as well as forward connections carrying prediction errors. To date, however, predictive coding models have largely neglected to take into account that neural transmission itself takes time. For a time-varying stimulus, such as a moving object, this means that backward predictions become misaligned with new sensory input. We present an extended model implementing both forward and backward extrapolation mechanisms that realigns backward predictions to minimize prediction error. This realignment has the consequence that neural representations across all hierarchical levels become aligned in real time. Using visual motion as an example, we show that the model is neurally plausible, that it is consistent with evidence of extrapolation mechanisms throughout the visual hierarchy, that it predicts several known motion-position illusions in human observers, and that it provides a solution to the temporal binding problem.

Predictive Coding: A Possible Explanation of Filling-In at the Blind Spot

Filling-in at the blind spot is a perceptual phenomenon in which the visual system fills the informational void, which arises due to the absence of retinal input corresponding to the optic disc, with surrounding visual attributes. It is known that during filling-in, nonlinear neural responses are observed in the early visual area that correlates with the perception, but the knowledge of underlying neural mechanism for filling-in at the blind spot is far from complete. In this work, we attempted to present a fresh perspective on the computational mechanism of filling-in process in the framework of hierarchical predictive coding, which provides a functional explanation for a range of neural responses in the cortex. We simulated a three-level hierarchical network and observe its response while stimulating the network with different bar stimulus across the blind spot. We find that the predictive-estimator neurons that represent blind spot in primary visual cortex exhibit elevated non-linear response when the bar stimulated both sides of the blind spot. Using generative model, we also show that these responses represent the filling-in completion. All these results are consistent with the finding of psychophysical and physiological studies. In this study, we also demonstrate that the tolerance in filling-in qualitatively matches with the experimental findings related to non-aligned bars. We discuss this phenomenon in the predictive coding paradigm and show that all our results could be explained by taking into account the efficient coding of natural images along with feedback and feed-forward connections that allow priors and predictions to co-evolve to arrive at the best prediction. These results suggest that the filling-in process could be a manifestation of the general computational principle of hierarchical predictive coding of natural images.

Investigating bottom-up auditory attention

Bottom-up attention is a sensory-driven selection mechanism that directs perception toward a subset of the stimulus that is considered salient, or attention-grabbing. Most studies of bottom-up auditory attention have adapted frameworks similar to visual attention models whereby local or global “contrast” is a central concept in defining salient elements in a scene. In the current study, we take a more fundamental approach to modeling auditory attention; providing the first examination of the space of auditory saliency spanning pitch, intensity and timbre; and shedding light on complex interactions among these features. Informed by psychoacoustic results, we develop a computational model of auditory saliency implementing a novel attentional framework, guided by processes hypothesized to take place in the auditory pathway. In particular, the model tests the hypothesis that perception tracks the evolution of sound events in a multidimensional feature space, and flags any deviation from background statistics as salient. Predictions from the model corroborate the relationship between bottom-up auditory attention and statistical inference, and argues for a potential role of predictive coding as mechanism for saliency detection in acoustic scenes.

Classification using sparse representations: a biologically plausible approach

Representing signals as linear combinations of basis vectors sparsely selected from an overcomplete dictionary has proven to be advantageous for many applications in pattern recognition, machine learning, signal processing, and computer vision. While this approach was originally inspired by insights into cortical information processing, biologically plausible approaches have been limited to exploring the functionality of early sensory processing in the brain, while more practical applications have employed non-biologically plausible sparse coding algorithms. Here, a biologically plausible algorithm is proposed that can be applied to practical problems. This algorithm is evaluated using standard benchmark tasks in the domain of pattern classification, and its performance is compared to a wide range of alternative algorithms that are widely used in signal and image processing. The results show that for the classification tasks performed here, the proposed method is competitive with the best of the alternative algorithms that have been evaluated. This demonstrates that classification using sparse representations can be performed in a neurally plausible manner, and hence, that this mechanism of classification might be exploited by the brain.

Visual nonclassical receptive field effects emerge from sparse coding in a dynamical system

Extensive electrophysiology studies have shown that many V1 simple cells have nonlinear response properties to stimuli within their classical receptive field (CRF) and receive contextual influence from stimuli outside the CRF modulating the cell's response. Models seeking to explain these non-classical receptive field (nCRF) effects in terms of circuit mechanisms, input-output descriptions, or individual visual tasks provide limited insight into the functional significance of these response properties, because they do not connect the full range of nCRF effects to optimal sensory coding strategies. The (population) sparse coding hypothesis conjectures an optimal sensory coding approach where a neural population uses as few active units as possible to represent a stimulus. We demonstrate that a wide variety of nCRF effects are emergent properties of a single sparse coding model implemented in a neurally plausible network structure (requiring no parameter tuning to produce different effects). Specifically, we replicate a wide variety of nCRF electrophysiology experiments (e.g., end-stopping, surround suppression, contrast invariance of orientation tuning, cross-orientation suppression, etc.) on a dynamical system implementing sparse coding, showing that this model produces individual units that reproduce the canonical nCRF effects. Furthermore, when the population diversity of an nCRF effect has also been reported in the literature, we show that this model produces many of the same population characteristics. These results show that the sparse coding hypothesis, when coupled with a biophysically plausible implementation, can provide a unified high-level functional interpretation to many response properties that have generally been viewed through distinct mechanistic or phenomenological models.

Simple cells in the primary visual cortex (V1) demonstrate many response properties that are either nonlinear or involve response modulations (i.e., stimuli that do not cause a response in isolation alter the cell's response to other stimuli). These non-classical receptive field (nCRF) effects are generally modeled individually and their collective role in biological vision is not well understood. Previous work has shown that classical receptive field (CRF) properties of V1 cells (i.e., the spatial structure of the visual field responsive to stimuli) could be explained by the sparse coding hypothesis, which is an optimal coding model that conjectures a neural population should use the fewest number of cells simultaneously to represent each stimulus. In this paper, we have performed extensive simulated physiology experiments to show that many nCRF response properties are simply emergent effects of a dynamical system implementing this same sparse coding model. These results suggest that rather than representing disparate information processing operations themselves, these nCRF effects could be consequences of an optimal sensory coding strategy that attempts to represent each stimulus most efficiently. This interpretation provides a potentially unifying high-level functional interpretation to many response properties that have generally been viewed through distinct models.

Distinguishing theory from implementation in predictive coding accounts of brain function

It is often helpful to distinguish between a theory (Marr's computational level) and a specific implementation of that theory (Marr's physical level). However, in the target article, a single implementation of predictive coding is presented as if this were the theory of predictive coding itself. Other implementations of predictive coding have been formulated which can explain additional neurobiological phenomena.

Image segmentation using a sparse coding model of cortical area V1

Algorithms that encode images using a sparse set of basis functions have previously been shown to explain aspects of the physiology of a primary visual cortex (V1), and have been used for applications, such as image compression, restoration, and classification. Here, a sparse coding algorithm, that has previously been used to account for the response properties of orientation tuned cells in primary visual cortex, is applied to the task of perceptually salient boundary detection. The proposed algorithm is currently limited to using only intensity information at a single scale. However, it is shown to out-perform the current state-of-the-art image segmentation method (Pb) when this method is also restricted to using the same information.

How prediction errors shape perception, attention, and motivation

Prediction errors (PE) are a central notion in theoretical models of reinforcement learning, perceptual inference, decision-making and cognition, and prediction error signals have been reported across a wide range of brain regions and experimental paradigms. Here, we will make an attempt to see the forest for the trees and consider the commonalities and differences of reported PE signals in light of recent suggestions that the computation of PE forms a fundamental mode of brain function. We discuss where different types of PE are encoded, how they are generated, and the different functional roles they fulfill. We suggest that while encoding of PE is a common computation across brain regions, the content and function of these error signals can be very different and are determined by the afferent and efferent connections within the neural circuitry in which they arise.