The "side" matters: how configurality is reflected in completion

Capturing the objects of vision with neural networks

Human visual perception carves a scene at its physical joints, decomposing the world into objects, which are selectively attended, tracked and predicted as we engage our surroundings. Object representations emancipate perception from the sensory input, enabling us to keep in mind that which is out of sight and to use perceptual content as a basis for action and symbolic cognition. Human behavioural studies have documented how object representations emerge through grouping, amodal completion, proto-objects and object files. By contrast, deep neural network models of visual object recognition remain largely tethered to sensory input, despite achieving human-level performance at labelling objects. Here, we review related work in both fields and examine how these fields can help each other. The cognitive literature provides a starting point for the development of new experimental tasks that reveal mechanisms of human object perception and serve as benchmarks driving the development of deep neural network models that will put the object into object recognition.

Distinct spatiotemporal mechanisms underlie extra-classical receptive field modulation in macaque V1 microcircuits

Complex scene perception depends upon the interaction between signals from the classical receptive field (CRF) and the extra-classical receptive field (eCRF) in primary visual cortex (V1) neurons. Although much is known about V1 eCRF properties, we do not yet know how the underlying mechanisms map onto the cortical microcircuit. We probed the spatio-temporal dynamics of eCRF modulation using a reverse correlation paradigm, and found three principal eCRF mechanisms: tuned-facilitation, untuned-suppression, and tuned-suppression. Each mechanism had a distinct timing and spatial profile. Laminar analysis showed that the timing, orientation-tuning, and strength of eCRF mechanisms had distinct signatures within magnocellular and parvocellular processing streams in the V1 microcircuit. The existence of multiple eCRF mechanisms provides new insights into how V1 responds to spatial context. Modeling revealed that the differences in timing and scale of these mechanisms predicted distinct patterns of net modulation, reconciling many previous disparate physiological and psychophysical findings.

Is information theory, or the assumptions that surround it, holding back neuroscience?

The challenges raised in this article are not with information theory per se, but the assumptions surrounding it. Neuroscience isn't sufficiently critical about the appropriate ‘receiver’ or ‘channel’, focuses on decoding ‘parts’, and often relies on a flawed ‘veridicality’ assumption. If these problematic assumptions were questioned, information theory could be better directed to help us understand how the brain works.

Bouba and Kiki inside objects: Sound-shape correspondence for objects with a hole

Visual holes (cutouts in a surface) have recently intrigued vision scientists as interesting and useful stimuli in the studies of shape perception and as a perceptual conundrum regarding figure/ground organization. Adopting the Bouba/Kiki paradigm, this study addressed a controversial issue of whether the perceived shape of a closed region alters when the region changes from a solid object to an empty hole, in a more direct manner than previous studies did. Observers were presented with two doughnut-like cardboard cutouts, one with a flower-shaped hole and the other with a star-shaped hole, and then matched them with two nonsense words. The curvature profile of the hole boundary was manipulated so that the shape of the interior region (i.e., a hole) and that of the exterior region (i.e., material edges) give rise to opposite shape impressions (i.e., one rounded and the other spiky). The results of Experiment 1 revealed that shape-name matching for holed objects is based on the interior shapes of holes, but not those of materially defined inner edges. The following three experiments replicated the same pattern of results even when holes appeared like oral apertures in animal character faces (Experiments 2-3) and when they were irregular, non-symmetric, and low in semantic association with familiar real-world objects (Experiment 4). Lastly, Experiment 5 showed that shape-name matching for "C"-shaped, negative-part stimuli is also interior-shape-based if the opening of the interior region is relatively small. These findings suggest that the interior shapes of holes are automatically accessible. I conclude with a discussion of my hypothesis that the only global-level, unitary shape representation of a bounded region of a single connected surface is that of the interior region for both objects and holes, imposing an important constraint in visual shape processing.

From Michotte Until Today: Why the Dichotomous Classification of Modal and Amodal Completions Is Inadequate

The distinction between modal and amodal completion is ubiquitous in the perception literature. It goes back to the seminal publication "Les compléments amodaux des structures perceptives" by A. Michotte, G. Thinès, and G. Crabbé (Publications Universitaires de Louvain: Louvain) in 1964. We review and discuss this work in this article and show commonalities and differences to today's view. We then argue that the dichotomous distinction between modal and amodal completions is problematic in phenomenological, empirical, logical, and theoretical terms. Finally, we propose alternative criteria allowing for a more differentiated classification scheme for completion phenomena. This scheme seems to be consistent with all known empirical findings and can also be generalized to nonvisual domains of perception.

The Segmentation of Proto-Objects in the Monkey Primary Visual Cortex

During visual perception, the brain enhances the representations of image regions that belong to figures and suppresses those that belong to the background. Natural images contain many regions that initially appear to be part of a figure when analyzed locally (proto-objects) but are actually part of the background if the whole image is considered. These proto-grounds must be correctly assigned to the background to allow correct shape identification and guide behavior. To understand how the brain resolves this conflict between local and global processing, we recorded neuronal activity from the primary visual cortex (V1) of macaque monkeys while they discriminated between n/u shapes that have a central proto-ground region. We studied the fine-grained spatiotemporal profile of neural activity evoked by the n/u shape and found that neural representation of the object proceeded from a coarse-to-fine resolution. Approximately 100 ms after the stimulus onset, the representation of the proto-ground region was enhanced together with the rest of the n/u surface, but after ∼115 ms, the proto-ground was suppressed back to the level of the background. Suppression of the proto-ground was only present in animals that had been trained to perform the shape-discrimination task, and it predicted the choice of the animal on a trial-by-trial basis. Attention enhanced figure-ground modulation, but it had no effect on the strength of proto-ground suppression. The results indicate that the accuracy of scene segmentation is sharpened by a suppressive process that resolves local ambiguities by assigning proto-grounds to the background.

Dynamic Volume Completion and Deformation

A new class of dynamic volume completion is introduced, where image elements (e.g., occluding semi-ellipses placed at the edge of an object) can link across a gap between two or more objects, leading to the perception of illusory volumes that deform as those image elements are set into relative motion. These new demonstrations provide further evidence that volume completion is not dictated solely by contour relatability constraints, but is instead a dynamic process of 3D shape construction that also takes into account dynamic cues to object shape, even in the absence of any contour relatability whatsoever.

Consistency of Border-Ownership Cells across Artificial Stimuli, Natural Stimuli, and Stimuli with Ambiguous Contours

Segmentation and recognition of objects in a visual scene are two problems that are hard to solve separately from each other. When segmenting an ambiguous scene, it is helpful to already know the present objects and their shapes. However, for recognizing an object in clutter, one would like to consider its isolated segment alone to avoid confounds from features of other objects. Border-ownership cells (Zhou et al., 2000) appear to play an important role in segmentation, as they signal the side-of-figure of artificial stimuli. The present work explores the role of border-ownership cells in dorsal macaque visual areas V2 and V3 in the segmentation of natural object stimuli and locally ambiguous stimuli. We report two major results. First, compared with previous estimates, we found a smaller percentage of cells that were consistent across artificial stimuli used previously. Second, we found that the average response of those neurons that did respond consistently to the side-of-figure of artificial stimuli also consistently signaled, as a population, the side-of-figure for borders of single faces, occluding faces and, with higher latencies, even stimuli with illusory contours, such as Mooney faces and natural faces completely missing local edge information. In contrast, the local edge or the outlines of the face alone could not always evoke a significant border-ownership signal. Our results underscore that border ownership is coded by a population of cells, and indicate that these cells integrate a variety of cues, including low-level features and global object context, to compute the segmentation of the scene.

SIGNIFICANCE STATEMENT

To distinguish different objects in a natural scene, the brain must segment the image into regions corresponding to objects. The so-called "border-ownership" cells appear to be dedicated to this task, as they signal for a given edge on which side the object is that owns it. Here, we report that individual border-ownership cells are unreliable when tested across a battery of artificial stimuli used previously but can signal border-ownership consistently as a population. We show that these border-ownership population signals are also suited for signaling border-ownership for natural objects and at longer latency, even for stimuli without local edge information. Our results suggest that border-ownership cells integrate both local, low-level and global, high-level cues to segment the scene.

No evidence for surface organization in Kanizsa configurations during continuous flash suppression

Does one need to be aware of a visual stimulus for it to be perceptually organized into a coherent whole? The answer to this question regarding the interplay between Gestalts and visual awareness remains unclear. Using interocular suppression as the paradigm for rendering stimuli invisible, conflicting evidence has been obtained as to whether the traditional Kanizsa surface is constructed during interocular suppression. While Sobel and Blake (2003) and Harris, Schwarzkopf, Song, Bahrami, and Rees (2011) failed to find evidence for this, Wang, Weng, and He (2012) showed that standard configurations of Kanizsa pacmen would break interocular suppression faster than their rotated counterparts. In the current study, we replicated the findings by Wang et al. (2012) but show that neither an account based on the construction of a surface nor one based on the long-range collinearities in the standard Kanizsa configuration stimulus could fully explain the difference in breakthrough times. We discuss these findings in the context of differences in the amplitudes of the Fourier orientation spectra for all stimulus types. Thus, we find no evidence that the integration of separate elements takes place during interocular suppression of Kanizsa stimuli, suggesting that this Gestalt involving figure-ground assignment is not constructed when rendered nonconscious using interocular suppression.

Visual perception of shape altered by inferred causal history

One of the main functions of vision is to represent object shape. Most theories of shape perception focus exclusively on geometrical computations (e.g., curvatures, symmetries, axis structure). Here, however, we find that shape representations are also profoundly influenced by an object’s causal origins: the processes in its past that formed it. Observers placed dots on objects to report their perceived symmetry axes. When objects appeared ‘complete’—created entirely by a single generative process—responses closely approximated the object’s geometrical axes. However, when objects appeared ‘bitten’—as if parts had been removed by a distinct causal process—the responses deviated significantly from the geometrical axes, as if the bitten regions were suppressed from the computation of symmetry. This suppression of bitten regions was also found when observers were not asked about symmetry axes but about the perceived front and back of objects. The findings suggest that visual shape representations are more sophisticated than previously appreciated. Objects are not only parsed according to what features they have, but also to how or why they have those features.

The Time-Course of Ultrarapid Categorization: The Influence of Scene Congruency and Top-Down Processing

Although categorization can take place at different levels of abstraction, classic studies on semantic labeling identified the basic level, for example, dog, as entry point for categorization. Ultrarapid categorization tasks have contradicted these findings, indicating that participants are faster at detecting superordinate-level information, for example, animal, in a complex visual image. We argue that both seemingly contradictive findings can be reconciled within the framework of parallel distributed processing and its successor Leabra (Local, Error-driven and Associative, Biologically Realistic Algorithm). The current study aimed at verifying this prediction in an ultrarapid categorization task with a dynamically changing presentation time (PT) for each briefly presented object, followed by a perceptual mask. Furthermore, we manipulated two defining task variables: level of categorization (basic vs. superordinate categorization) and object presentation mode (object-in-isolation vs. object-in-context). In contradiction with previous ultrarapid categorization research, focusing on reaction time, we used accuracy as our main dependent variable. Results indicated a consistent superordinate processing advantage, coinciding with an overall improvement in performance with longer PT and a significantly more accurate detection of objects in isolation, compared with objects in context, at lower stimulus PT. This contextual disadvantage disappeared when PT increased, indicating that figure-ground separation with recurrent processing is vital for meaningful contextual processing to occur.

Suppressive and enhancing effects in early visual cortex during illusory shape perception: A comment on

In a recent functional magnetic resonance imaging study, Kok and de Lange (2014) observed that BOLD activity for a Kanizsa illusory shape stimulus, in which pacmen-like inducers elicit an illusory shape percept, was either enhanced or suppressed relative to a nonillusory control configuration depending on whether the spatial profile of BOLD activity in early visual cortex was related to the illusory shape or the inducers, respectively. The authors argued that these findings fit well with the predictive coding framework, because top-down predictions related to the illusory shape are not met with bottom-up sensory input and hence the feedforward error signal is enhanced. Conversely, for the inducing elements, there is a match between top-down predictions and input, leading to a decrease in error. Rather than invoking predictive coding as the explanatory framework, the suppressive effect related to the inducers might be caused by neural adaptation to perceptually stable input due to the trial sequence used in the experiment.

Temporal dynamics of different cases of bi-stable figure-ground perception

Segmentation of a visual scene in "figure" and "ground" is essential for perception of the three-dimensional layout of a scene. In cases of bi-stable perception, two distinct figure-ground interpretations alternate over time. We were interested in the temporal dynamics of these alternations, in particular when the same image is presented repeatedly, with short blank periods in-between. Surprisingly, we found that the intermittent presentation of Rubin's classical "face-or-vase" figure, which is frequently taken as a standard case of bi-stable figure-ground perception, often evoked perceptual switches during the short presentations and stabilization was not prominent. Interestingly, bi-stable perception of Kanizsa's anomalous transparency figure did strongly stabilize across blanks. We also found stabilization for the Necker cube, which we used for comparison. The degree of stabilization (and the lack of it) varied across stimuli and across individuals. Our results indicate, against common expectation, that the stabilization phenomenon cannot be generally evoked by intermittent presentation. We argue that top-down feedback factors such as familiarity, semantics, expectation, and perceptual bias contribute to the complex processes underlying the temporal dynamics of bi-stable figure-ground perception.

Depth perception of illusory surfaces

The perception of an illusory surface, a subjectively perceived surface that is not given in the image, is one of the most intriguing phenomena in vision. It strongly influences the perception of some fundamental properties, namely, depth, lightness and contours. Recently, we suggested (1) that the context-sensitive mechanism of depth computation plays a key role in creating the illusion, (2) that the illusory lightness perception can be explained by an influence of depth perception on the lightness computation, and (3) that the perception of variations of the Kanizsa figure can be well-reproduced by implementing these principles in a model (Kogo, Strecha, et al., 2010). However, depth perception, lightness perception, contour perception, and their interactions can be influenced by various factors. It is essential to measure the differences between the variation figures in these aspects separately to further understand the mechanisms. As a first step, we report here the results of a new experimental paradigm to compare the depth perception of the Kanizsa figure and its variations. One of the illusory figures was presented side-by-side with a non-illusory variation whose stereo disparities were varied. Participants had to decide in which of these two figures the central region appeared closer. The results indicate that the depth perception of the illusory surface was indeed different in the variation figures. Furthermore, there was a non-linear interaction between the occlusion cues and stereo disparity cues. Implications of the results for the neuro-computational mechanisms are discussed.