Humans make efficient use of natural image statistics when performing spatial interpolation

Contiguity in perception: origins in cellular associative computations

Our brains are good at detecting and learning associative structures; according to some linguistic theories, this capacity even constitutes a prerequisite for the development of syntax and compositionality in language and verbalized thought. I will argue that the search for associative motifs in input patterns is an evolutionary old brain function that enables contiguity in sensory perception and orientation in time and space. It has its origins in an elementary material property of cells that is particularly evident at chemical synapses: input-assigned calcium influx that activates calcium sensor proteins involved in memory storage. This machinery for the detection and learning of associative motifs generates knowledge about input relationships and integrates this knowledge into existing networks through updates in connectivity patterns.

Towards a Closed Form Second-Order Natural Scene Statistics Model

Previous work on natural scene statistics (NSS)-based image models has focused primarily on characterizing the univariate bandpass statistics of single pixels. These models have proven to be powerful tools driving a variety of computer vision and image/video processing applications, including depth estimation, image quality assessment, and image denoising, among others. Multivariate NSS models descriptive of the joint distributions of spatially separated bandpass image samples have, however, received relatively little attention. Here, we develop a closed form bivariate spatial correlation model of bandpass and normalized image samples that completes an existing 2D joint generalized Gaussian distribution model of adjacent bandpass pixels. Our model is built using a set of diverse, high-quality naturalistic photographs, and as a control, we study the model properties on white noise. We also study the way the model fits are affected when the images are modified by common distortions.

Adaptation in the visual cortex: a case for probing neuronal populations with natural stimuli

The perception of, and neural responses to, sensory stimuli in the present are influenced by what has been observed in the past—a phenomenon known as adaptation. We focus on adaptation in visual cortical neurons as a paradigmatic example. We review recent work that represents two shifts in the way we study adaptation, namely (i) going beyond single neurons to study adaptation in populations of neurons and (ii) going beyond simple stimuli to study adaptation to natural stimuli. We suggest that efforts in these two directions, through a closer integration of experimental and modeling approaches, will enable a more complete understanding of cortical processing in natural environments.

Statistics of colors in paintings and natural scenes

Painters reproduce some spatial statistical regularities of natural scenes. To what extent they replicate their color statistics is an open question. We investigated this question by analyzing the colors of 50 natural scenes of rural and urban environments and 44 paintings with abstract and figurative compositions. The analysis was carried out using hyperspectral imaging data from both sets and focused on the gamut and distribution of colors in the CIELAB space. The results showed that paintings, like natural scenes, have gamuts with elongated shapes in the yellow-blue direction but more tilted to the red direction. It was also found that the fraction of discernible colors, expressed as a function of the number of occurrences in the scene or painting, is well described by power laws. These have similar distribution of slopes in a log-log scale for paintings and natural scenes. These features are observed in both abstract and figurative compositions. These results suggest that the underlying chromatic structure of artistic compositions generally follows the main statistical features of the natural environment.

Duration estimates within a modality are integrated sub-optimally

Perceived duration can be influenced by various properties of sensory stimuli. For example, visual stimuli of higher temporal frequency are perceived to last longer than those of lower temporal frequency. How does the brain form a representation of duration when each of two simultaneously presented stimuli influences perceived duration in different way? To answer this question, we investigated the perceived duration of a pair of dynamic visual stimuli of different temporal frequencies in comparison to that of a single visual stimulus of either low or high temporal frequency. We found that the duration representation of simultaneously occurring visual stimuli is best described by weighting the estimates of duration based on each individual stimulus. However, the weighting performance deviates from the prediction of statistically optimal integration. In addition, we provided a Bayesian account to explain a difference in the apparent sensitivity of the psychometric curves introduced by the order in which the two stimuli are displayed in a two-alternative forced-choice task.

Neural coding of uncertainty and probability

Organisms must act in the face of sensory, motor, and reward uncertainty stemming from a pandemonium of stochasticity and missing information. In many tasks, organisms can make better decisions if they have at their disposal a representation of the uncertainty associated with task-relevant variables. We formalize this problem using Bayesian decision theory and review recent behavioral and neural evidence that the brain may use knowledge of uncertainty, confidence, and probability.

A conceptual framework of computations in mid-level vision

If a picture is worth a thousand words, as an English idiom goes, what should those words-or, rather, descriptors-capture? What format of image representation would be sufficiently rich if we were to reconstruct the essence of images from their descriptors? In this paper, we set out to develop a conceptual framework that would be: (i) biologically plausible in order to provide a better mechanistic understanding of our visual system; (ii) sufficiently robust to apply in practice on realistic images; and (iii) able to tap into underlying structure of our visual world. We bring forward three key ideas. First, we argue that surface-based representations are constructed based on feature inference from the input in the intermediate processing layers of the visual system. Such representations are computed in a largely pre-semantic (prior to categorization) and pre-attentive manner using multiple cues (orientation, color, polarity, variation in orientation, and so on), and explicitly retain configural relations between features. The constructed surfaces may be partially overlapping to compensate for occlusions and are ordered in depth (figure-ground organization). Second, we propose that such intermediate representations could be formed by a hierarchical computation of similarity between features in local image patches and pooling of highly-similar units, and reestimated via recurrent loops according to the task demands. Finally, we suggest to use datasets composed of realistically rendered artificial objects and surfaces in order to better understand a model's behavior and its limitations.

Task-dependent neural representations of salient events in dynamic auditory scenes

Selecting pertinent events in the cacophony of sounds that impinge on our ears every day is regulated by the acoustic salience of sounds in the scene as well as their behavioral relevance as dictated by top-down task-dependent demands. The current study aims to explore the neural signature of both facets of attention, as well as their possible interactions in the context of auditory scenes. Using a paradigm with dynamic auditory streams with occasional salient events, we recorded neurophysiological responses of human listeners using EEG while manipulating the subjects' attentional state as well as the presence or absence of a competing auditory stream. Our results showed that salient events caused an increase in the auditory steady-state response (ASSR) irrespective of attentional state or complexity of the scene. Such increase supplemented ASSR increases due to task-driven attention. Salient events also evoked a strong N1 peak in the ERP response when listeners were attending to the target sound stream, accompanied by an MMN-like component in some cases and changes in the P1 and P300 components under all listening conditions. Overall, bottom-up attention induced by a salient change in the auditory stream appears to mostly modulate the amplitude of the steady-state response and certain event-related potentials to salient sound events; though this modulation is affected by top-down attentional processes and the prominence of these events in the auditory scene as well.