Serial Dependence in Perception

Much evidence has shown that perception is biased towards previously presented similar stimuli, an effect recently termed serial dependence. Serial dependence affects nearly every aspect of perception, often causing gross perceptual distortions, especially for weak and ambiguous stimuli. Despite unwanted side-effects, empirical evidence and Bayesian modeling show that serial dependence acts to improve efficiency and is generally beneficial to the system. Consistent with models of predictive coding, the Bayesian priors of serial dependence are generated at high levels of cortical analysis, incorporating much perceptual experience, but feed back to lower sensory areas. These feedback loops may drive oscillations in the alpha range, linked strongly with serial dependence. The discovery of top-down predictive perceptual processes is not new, but the new, more quantitative approach characterizing serial dependence promises to lead to a deeper understanding of predictive perceptual processes and their underlying neural mechanisms.

From intermediate shape-centered representations to the perception of oriented shapes: response to commentaries

In this response paper, we start by addressing the main points made by the commentators on the target article's main theoretical conclusions: the existence and characteristics of the intermediate shape-centered representations (ISCRs) in the visual system, their emergence from edge detection mechanisms operating on different types of visual properties, and how they are eventually reunited in higher order frames of reference underlying conscious visual perception. We also address the much-commented issue of the possible neural mechanisms of the ISCRs. In the final section, we address more specific and general comments, questions, and suggestions which, albeit very interesting, were less directly focused on the main conclusions of the target paper.

Shape-centered representations of bounded regions of space mediate the perception of objects

We report the study of a woman who perceives 2D bounded regions of space ("shapes") defined by sharp edges of medium to high contrast as if they were rotated by 90, 180 degrees around their centre, mirrored across their own axes, or both. In contrast, her perception of 3D, strongly blurred or very low contrast shapes, and of stimuli emerging from a collection of shapes, is intact. This suggests that a stage in the process of constructing the conscious visual representation of a scene consists of representing mutually exclusive bounded regions extracted from the initial retinotopic space in "shape-centered" frames of reference. The selectivity of the disorder to shapes originally biased toward the parvocellular subcortical pathway, and the absence of any other type of error, additionally invite new hypotheses about the operations involved in computing these "intermediate shape-centered representations" and in mapping them onto higher frames for perception and action.

Deep neural networks capture texture sensitivity in V2

Deep convolutional neural networks (CNNs) trained on visual objects have shown intriguing ability to predict some response properties of visual cortical neurons. However, the factors (e.g., if the model is trained or not, receptive field size) and computations (e.g., convolution, rectification, pooling, normalization) that give rise to such ability, at what level, and the role of intermediate processing stages in explaining changes that develop across areas of the cortical hierarchy are poorly understood. We focused on the sensitivity to textures as a paradigmatic example, since recent neurophysiology experiments provide rich data pointing to texture sensitivity in secondary (but not primary) visual cortex (V2). We initially explored the CNN without any fitting to the neural data and found that the first two layers of the CNN showed qualitative correspondence to the first two cortical areas in terms of texture sensitivity. We therefore developed a quantitative approach to select a population of CNN model neurons that best fits the brain neural recordings. We found that the CNN could develop compatibility to secondary cortex in the second layer following rectification and that this was improved following pooling but only mildly influenced by the local normalization operation. Higher layers of the CNN could further, though modestly, improve the compatibility with the V2 data. The compatibility was reduced when incorporating random rather than learned weights. Our results show that the CNN class of model is effective for capturing changes that develop across early areas of cortex, and has the potential to help identify the computations that give rise to hierarchical processing in the brain (code is available in GitHub).

Material Perception

Under typical viewing conditions, human observers effortlessly recognize materials and infer their physical, functional, and multisensory properties at a glance. Without touching materials, we can usually tell whether they would feel hard or soft, rough or smooth, wet or dry. We have vivid visual intuitions about how deformable materials like liquids or textiles respond to external forces and how surfaces like chrome, wax, or leather change appearance when formed into different shapes or viewed under different lighting. These achievements are impressive because the retinal image results from complex optical interactions between lighting, shape, and material, which cannot easily be disentangled. Here I argue that because of the diversity, mutability, and complexity of materials, they pose enormous challenges to vision science: What is material appearance, and how do we measure it? How are material properties estimated and represented? Resolving these questions causes us to scrutinize the basic assumptions of mid-level vision.

Representation of Maximally Regular Textures in Human Visual Cortex

Naturalistic textures with an intermediate degree of statistical regularity can capture key structural features of natural images (Freeman and Simoncelli, 2011). V2 and later visual areas are sensitive to these features, while primary visual cortex is not (Freeman et al., 2013). Here we expand on this work by investigating a class of textures that have maximal formal regularity, the 17 crystallographic wallpaper groups (Fedorov, 1891). We used texture stimuli from four of the groups that differ in the maximum order of rotation symmetry they contain, and measured neural responses in human participants using functional MRI and high-density EEG. We found that cortical area V3 has a parametric representation of the rotation symmetries in the textures that is not present in either V1 or V2, the first discovery of a stimulus property that differentiates processing in V3 from that of lower-level areas. Parametric responses were also seen in higher-order ventral stream areas V4, VO1, and lateral occipital complex (LOC), but not in dorsal stream areas. The parametric response pattern was replicated in the EEG data, and source localization indicated that responses in V3 and V4 lead responses in LOC, which is consistent with a feedforward mechanism. Finally, we presented our stimuli to four well developed feedforward models and found that none of them were able to account for our results. Our results highlight structural regularity as an important stimulus dimension for distinguishing the early stages of visual processing, and suggest a previously unrecognized role for V3 in the visual form-processing hierarchy. Significance statement: Hierarchical processing is a fundamental organizing principle in visual neuroscience, with each successive processing stage being sensitive to increasingly complex stimulus properties. Here, we probe the encoding hierarchy in human visual cortex using a class of visual textures--wallpaper patterns--that are maximally regular. Through a combination of fMRI and EEG source imaging, we find specific responses to texture regularity that depend parametrically on the maximum order of rotation symmetry in the textures. These parametric responses are seen in several areas of the ventral visual processing stream, as well as in area V3, but not in V1 or V2. This is the first demonstration of a stimulus property that differentiates processing in V3 from that of lower-level visual areas.

Context affects lightness at the level of surfaces

Visual perception of object attributes such as surface lightness is crucial for successful interaction with the environment. How the visual system assigns lightness to image regions is not yet understood. It has been shown that the context in which a surface is embedded influences its perceived lightness, but whether that influence involves predominantly low-, mid-, or high-level visual mechanisms has not been resolved. To answer this question, we measured whether perceptual attributes of target image regions affected their perceived lightness when they were placed in different contexts. We varied the sharpness of the edge while keeping total target flux fixed. Targets with a sharp edge were consistent with the perceptual interpretation of a surface, and in that case, observers perceived significant brightening or darkening of the target. Targets with blurred edges rather appeared to be spotlights instead of surfaces; for targets with blurred edges, there was much less of a contextual effect on target lightness. The results indicate that the effect of context on the lightness of an image region is not fixed but is strongly affected by image manipulations that modify the perceptual attributes of the target, implying that a mid-level scene interpretation affects lightness perception.

Serial versus parallel processing in mid-level vision: filling-in the details of spatial interpolation

The relationship between boundary completion and surface filling-in, two core mechanisms of mid-level vision, remains unclear. Here, we integrate recent empirical findings to shine new light onto the neural mechanisms of boundary completion and surface filling-in as well as their relation to each other. Specifically, we discuss several psychophysical and neurophysiological studies that, when taken together, support a model where object boundaries and visual surfaces are interpolated in parallel, with one process impacting the other. We suggest that visual boundary completion and surface filling-in are two interacting processes that are supported by neural processes that are distributed throughout several areas of the early visual system.

A conceptual framework of computations in mid-level vision

If a picture is worth a thousand words, as an English idiom goes, what should those words-or, rather, descriptors-capture? What format of image representation would be sufficiently rich if we were to reconstruct the essence of images from their descriptors? In this paper, we set out to develop a conceptual framework that would be: (i) biologically plausible in order to provide a better mechanistic understanding of our visual system; (ii) sufficiently robust to apply in practice on realistic images; and (iii) able to tap into underlying structure of our visual world. We bring forward three key ideas. First, we argue that surface-based representations are constructed based on feature inference from the input in the intermediate processing layers of the visual system. Such representations are computed in a largely pre-semantic (prior to categorization) and pre-attentive manner using multiple cues (orientation, color, polarity, variation in orientation, and so on), and explicitly retain configural relations between features. The constructed surfaces may be partially overlapping to compensate for occlusions and are ordered in depth (figure-ground organization). Second, we propose that such intermediate representations could be formed by a hierarchical computation of similarity between features in local image patches and pooling of highly-similar units, and reestimated via recurrent loops according to the task demands. Finally, we suggest to use datasets composed of realistically rendered artificial objects and surfaces in order to better understand a model's behavior and its limitations.

Reliability and validity of the Leuven Perceptual Organization Screening Test (L-POST)

Neuropsychological tests of visual perception mostly assess high-level processes like object recognition. Object recognition, however, relies on distinct mid-level processes of perceptual organization that are only implicitly tested in classical tests. Furthermore, the psychometric properties of the existing instruments are limited. To fill this gap, the Leuven perceptual organization screening test (L-POST) was developed, in which a wide range of mid-level phenomena are measured in 15 subtests. In this study, we evaluated reliability and validity of the L-POST. Performance on the test is evaluated relative to a norm sample of more than 1,500 healthy control participants. Cronbach's alpha of the norm sample and test-retest correlations for 20 patients provide evidence for adequate reliability of L-POST performance. The convergent and discriminant validity of the test was assessed in 40 brain-damaged patients, whose performance on the L-POST was compared with standard clinical tests of visual perception and other measures of cognitive function. The L-POST showed high sensitivity to visual dysfunction and decreased performance was specific to visual problems. In conclusion, the L-POST is a reliable and valid screening test for perceptual organization. It offers a useful online tool for researchers and clinicians to get a broader overview of the mid-level processes that are preserved or disrupted in a given patient.

Towards a new kind of experimental psycho-aesthetics? Reflections on the Parallellepipeda project

Experimental psycho-aesthetics-the science aimed at understanding the factors that determine aesthetic experience-is reviewed briefly as background to describe the Parallellepipeda project, a cross-over project between artists and scientists in Leuven. In particular, I sketch how it started and developed further, with close interactions between the participating artists and scientists. A few examples of specific research projects are mentioned to illustrate the kind of research questions we address and the methodological approach we have taken. We often found an effect of providing participants with additional information, a difference between novice and expert participants, and a shift with increasing experience with an artwork, in the direction of tolerating more complexity and acquiring more order from it. By establishing more connections between parts of an artwork and more associations to the artwork, it becomes a stronger Gestalt, which is more easily mastered by the viewer and leads to increased appreciation. In the final part of the paper, I extract some general lessons from the project regarding a possible new way of doing psycho-aesthetics research, which is able to solve some of the problems of traditional experimental psycho-aesthetics (eg, trade-off between experimental control and ecological validity).