Bayesian contour extrapolation: geometric determinants of good continuation

Ambiguity is a linking feature for interocular grouping

Ambiguity is implicit in neural representations of the physical world. Previous work has examined how the visual system resolves ambiguous neural signals that represent various features, such as the percept resulting from rivalrous chromaticities or forms. Relatively little is known, however, about the contribution of unambiguous neural representations to perceptual resolution of ambiguous ones. This is addressed here by measuring perceptual resolution of ambiguity by grouping, which is operationalized as the tendency for multiple similar ambiguous representations to be seen as identical to each other. Multiple chromatically ambiguous representations were created using interocular switch rivalry and presented together with a nearby but separate unambiguous (non-rivalrous) chromaticity. The magnitude of grouping the chromatic regions was compared when ambiguous regions were seen alone versus with unambiguous regions seen simultaneously. Contrary to prevailing theory that the resolution of the ambiguous percepts would follow the unambiguous ones, the ambiguous chromatic regions consistently appeared identical to each other, but their appearance was not found to be attracted to the unambiguous color percept. This supports the proposition that the ambiguity itself in a neural representation is a linking feature contributing to perceptual disambiguation.

Speed behaviour upon approaching freeway curves

The actual speed behaviour when drivers approach a curve is very relevant to assess the road design and safety but is mostly overlooked in the scientific literature. Most research into curve driving behaviour is focussed at the behaviour inside the curve, although the speed selection is done before curve entry. The main objective of this research is to identify which freeway characteristics play a role in driving speed selection. High Frequency Floating Car Data, detailed reconstruction of the curves and their surroundings, as well as three dimensional sight distance analysis, were used to analyse individual speed profiles on 153 Dutch freeway curves. By defining the positions where the acceleration approaches 0 m/s² before and after a curve starts, the positions when the driver started and stopped decelerating upon curve entry were defined. Further correlation and regression analysis of those positions revealed that the radius of the curve is indeed a main explaining variable, as well as the speed driven before deceleration starts. Sight distances and cross section characteristics play a further role in determining the position where deceleration starts. Deceleration ends at approximately 135 m after curve start, and the speed in a curve is also correlated with the deflection angle and length of a curve. Sight distances do not play a role in selecting the speed in a curve based on this research. Overall, the findings indicate a non-constant nature and variability of speed behaviour upon curve entry. This can be used for safer freeway curve design and to assess traffic safety based on actual speed behaviour.

Physically Implied Surfaces

In addition to seeing objects that are directly in view, we also represent objects that are merely implied (e.g., by occlusion, motion, and other cues). What can imply the presence of an object? Here, we explored (in three preregistered experiments; N = 360 adults) the role of physical interaction in creating impressions of objects that are not actually present. After seeing an actor collide with an invisible wall or step onto an invisible box, participants gave facilitated responses to actual, visible surfaces that appeared where the implied wall or box had been-a Stroop-like pattern of facilitation and interference that suggested automatic inferences about the relevant implied surfaces. Follow-up experiments ruled out confounding geometric cues and anticipatory responses. We suggest that physical interactions can trigger representations of the participating surfaces such that we automatically infer the presence of objects implied only by their physical consequences.

Visual perception of shape altered by inferred causal history

One of the main functions of vision is to represent object shape. Most theories of shape perception focus exclusively on geometrical computations (e.g., curvatures, symmetries, axis structure). Here, however, we find that shape representations are also profoundly influenced by an object’s causal origins: the processes in its past that formed it. Observers placed dots on objects to report their perceived symmetry axes. When objects appeared ‘complete’—created entirely by a single generative process—responses closely approximated the object’s geometrical axes. However, when objects appeared ‘bitten’—as if parts had been removed by a distinct causal process—the responses deviated significantly from the geometrical axes, as if the bitten regions were suppressed from the computation of symmetry. This suppression of bitten regions was also found when observers were not asked about symmetry axes but about the perceived front and back of objects. The findings suggest that visual shape representations are more sophisticated than previously appreciated. Objects are not only parsed according to what features they have, but also to how or why they have those features.

Decoding information about dynamically occluded objects in visual cortex

During dynamic occlusion, an object passes behind an occluding surface and then later reappears. Even when completely occluded from view, such objects are experienced as continuing to exist or persist behind the occluder even though they are no longer visible. The contents and neural basis of this persistent representation remain poorly understood. Questions remain as to whether there is information maintained about the object itself (i.e. its shape or identity) or non-object-specific information such as its position or velocity as it is tracked behind an occluder, as well as which areas of visual cortex represent such information. Recent studies have found that early visual cortex is activated by "invisible" objects during visual imagery and by unstimulated regions along the path of apparent motion, suggesting that some properties of dynamically occluded objects may also be neurally represented in early visual cortex. We applied functional magnetic resonance imaging in human subjects to examine representations within visual cortex during dynamic occlusion. For gradually occluded, but not for instantly disappearing objects, there was an increase in activity in early visual cortex (V1, V2, and V3). This activity was spatially-specific, corresponding to the occluded location in the visual field. However, the activity did not encode enough information about object identity to discriminate between different kinds of occluded objects (circles vs. stars) using MVPA. In contrast, object identity could be decoded in spatially-specific subregions of higher-order, topographically organized areas such as ventral, lateral, and temporal occipital areas (VO, LO, and TO) as well as the functionally defined LOC and hMT+. These results suggest that early visual cortex may only represent the dynamically occluded object's position or motion path, while later visual areas represent object-specific information.

An overview of quantitative approaches in Gestalt perception

Gestalt psychology is often criticized as lacking quantitative measurements and precise mathematical models. While this is true of the early Gestalt school, today there are many quantitative approaches in Gestalt perception and the special issue of Vision Research "Quantitative Approaches in Gestalt Perception" showcases the current state-of-the-art. In this article we give an overview of these current approaches. For example, ideal observer models are one of the standard quantitative tools in vision research and there is a clear trend to try and apply this tool to Gestalt perception and thereby integrate Gestalt perception into mainstream vision research. More generally, Bayesian models, long popular in other areas of vision research, are increasingly being employed to model perceptual grouping as well. Thus, although experimental and theoretical approaches to Gestalt perception remain quite diverse, we are hopeful that these quantitative trends will pave the way for a unified theory.

The role of shape complexity in the detection of closed contours

The detection of contours in noise has been extensively studied, but the detection of closed contours, such as the boundaries of whole objects, has received relatively little attention. Closed contours pose substantial challenges not present in the simple (open) case, because they form the outlines of whole shapes and thus take on a range of potentially important configural properties. In this paper we consider the detection of closed contours in noise as a probabilistic decision problem. Previous work on open contours suggests that contour complexity, quantified as the negative log probability (Description Length, DL) of the contour under a suitably chosen statistical model, impairs contour detectability; more complex (statistically surprising) contours are harder to detect. In this study we extended this result to closed contours, developing a suitable probabilistic model of whole shapes that gives rise to several distinct though interrelated measures of shape complexity. We asked subjects to detect either natural shapes (Exp. 1) or experimentally manipulated shapes (Exp. 2) embedded in noise fields. We found systematic effects of global shape complexity on detection performance, demonstrating how aspects of global shape and form influence the basic process of object detection.

Bayesian hierarchical grouping: Perceptual grouping as mixture estimation

We propose a novel framework for perceptual grouping based on the idea of mixture models, called Bayesian hierarchical grouping (BHG). In BHG, we assume that the configuration of image elements is generated by a mixture of distinct objects, each of which generates image elements according to some generative assumptions. Grouping, in this framework, means estimating the number and the parameters of the mixture components that generated the image, including estimating which image elements are "owned" by which objects. We present a tractable implementation of the framework, based on the hierarchical clustering approach of Heller and Ghahramani (2005). We illustrate it with examples drawn from a number of classical perceptual grouping problems, including dot clustering, contour integration, and part decomposition. Our approach yields an intuitive hierarchical representation of image elements, giving an explicit decomposition of the image into mixture components, along with estimates of the probability of various candidate decompositions. We show that BHG accounts well for a diverse range of empirical data drawn from the literature. Because BHG provides a principled quantification of the plausibility of grouping interpretations over a wide range of grouping problems, we argue that it provides an appealing unifying account of the elusive Gestalt notion of Prägnanz.

Contour complexity and contour detection

Itis well-known that "smooth" chains of oriented elements-contours-are more easily detected amid background noise than more undulating (i.e., "less smooth") chains. Here, we develop a Bayesian framework for contour detection and show that it predicts that contour detection performance should decrease with the contour's complexity, quantified as the description length (DL; i.e., the negative logarithm of probability integrated along the contour). We tested this prediction in two experiments in which subjects were asked to detect simple open contours amid pixel noise. In Experiment 1, we demonstrate a consistent decline in performance with increasingly complex contours, as predicted by the Bayesian model. In Experiment 2, we confirmed that this effect is due to integrated complexity along the contour, and does not seem to depend on local stretches of linear structure. The results corroborate the probabilistic model of contours, and show how contour detection can be understood as a special case of a more general process-the identification of organized patterns in the environment.

Revealing individual differences in strategy selection through visual motion extrapolation

Humans are constantly challenged to make use of internal models to fill in missing sensory information. We measured human performance in a simple motion extrapolation task where no feedback was provided in order to elucidate the models of object motion incorporated into observers' extrapolation strategies. There was no "right" model for extrapolation in this task. Observers consistently adopted one of two models, linear or quadratic, but different observers chose different models. We further demonstrate that differences in motion sensitivity impact the choice of internal models for many observers. These results demonstrate that internal models and individual differences in those models can be elicited by unconstrained, predictive-based psychophysical tasks.

Tangent Bundle Elastica and Computer Vision

Visual curve completion, an early visual process that completes the occluded parts between observed boundary fragments (a.k.a. inducers), is a major problem in perceptual organization and a critical step toward higher level visual tasks in both biological and machine vision. Most computational contributions to solving this problem suggest desired perceptual properties that the completed contour should satisfy in the image plane, and then seek the mathematical curves that provide them. Alternatively, few studies (including by the authors) have suggested to frame the problem not in the image plane but rather in the unit tangent bundleR (2) × S(1), the space that abstracts the primary visual cortex, where curve completion allegedly occurs. Combining both schools, here we propose and develop a biologically plausible theory of elastica in the tangent bundle that provides not only perceptually superior completion results but also a rigorous computational prediction that inducer curvatures greatly affects the shape of the completed curve, as indeed indicated by human perception.

Task-specific response strategy selection on the basis of recent training experience

The goal of training is to produce learning for a range of activities that are typically more general than the training task itself. Despite a century of research, predicting the scope of learning from the content of training has proven extremely difficult, with the same task producing narrowly focused learning strategies in some cases and broadly scoped learning strategies in others. Here we test the hypothesis that human subjects will prefer a decision strategy that maximizes performance and reduces uncertainty given the demands of the training task and that the strategy chosen will then predict the extent to which learning is transferable. To test this hypothesis, we trained subjects on a moving dot extrapolation task that makes distinct predictions for two types of learning strategy: a narrow model-free strategy that learns an input-output mapping for training stimuli, and a general model-based strategy that utilizes humans' default predictive model for a class of trajectories. When the number of distinct training trajectories is low, we predict better performance for the mapping strategy, but as the number increases, a predictive model is increasingly favored. Consonant with predictions, subject extrapolations for test trajectories were consistent with using a mapping strategy when trained on a small number of training trajectories and a predictive model when trained on a larger number. The general framework developed here can thus be useful both in interpreting previous patterns of task-specific versus task-general learning, as well as in building future training paradigms with certain desired outcomes.

Predicting what humans will learn from a training task, in particular, whether learning will generalize beyond the specifics of the given experience, is of both significant practical and theoretical interest. However, a principled understanding of the relationship between training conditions and learning generalization remains elusive. In this paper, we develop a computational framework for predicting which of two basic decision-making strategies will be utilized by human subjects - 1) simple stimulus-response mappings or 2) predictive models. Through simulation, we show that the nature of the training experience determines which of these categories leads to better in-task performance; repetitive training on a small set of examples favors simple stimulus-response mappings, whereas training on a large set of examples favors predictive strategies. We then show that humans trained under these various conditions do indeed utilize the predicted strategy. Finally, we show that the strategies that are utilized during training predict generalization of learning. Those who learn simple mappings fail to generalize their new skills, in contrast to those who use default predictive strategies. The framework developed here is useful both in interpreting previous patterns of learning, as well as in building training paradigms with given desired outcomes.

Structure propagation for image registration

Mosaicing is a commonly used technique in many medical imaging applications where subimages are stitched together in order to obtain a larger field of view. However, stitching, which involves alignment or registration in overlapping regions, is often challenging when the information shared by subimages is absent or small. While it is not possible to perform an alignment without overlap using existing techniques, imaging artifacts such as distortions towards image boundaries present further complications during registration by decreasing the reliability of available information. Without taking these into consideration, a registration approach might violate the continuity and the smoothness of structures across subimages. In this paper, we propose a novel registration approach for the stitching of subimages in such challenging scenarios. By using a perceptual grouping approach, we extend subimages beyond their boundaries by propagating available structures in order to obtain structural maps in the extended regions. These maps are then used to establish correspondences between subimages when the shared information is absent, small or unreliable. Using our approach ensures the continuity and the smoothness of structures across subimage boundaries. Furthermore, since only structures are used, the proposed method can also be used for the stitching of multi-modal images. Our approach is unique in that it also enables contactless stitching. We demonstrate the effectiveness of the proposed method by performing several experiments on synthetic and medical images. Moreover, we show how stitching is possible in the presence of a physical gap between subimages.

A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations

Our first review article (Wagemans et al., 2012) on the occasion of the centennial anniversary of Gestalt psychology focused on perceptual grouping and figure-ground organization. It concluded that further progress requires a reconsideration of the conceptual and theoretical foundations of the Gestalt approach, which is provided here. In particular, we review contemporary formulations of holism within an information-processing framework, allowing for operational definitions (e.g., integral dimensions, emergent features, configural superiority, global precedence, primacy of holistic/configural properties) and a refined understanding of its psychological implications (e.g., at the level of attention, perception, and decision). We also review 4 lines of theoretical progress regarding the law of Prägnanz-the brain's tendency of being attracted towards states corresponding to the simplest possible organization, given the available stimulation. The first considers the brain as a complex adaptive system and explains how self-organization solves the conundrum of trading between robustness and flexibility of perceptual states. The second specifies the economy principle in terms of optimization of neural resources, showing that elementary sensors working independently to minimize uncertainty can respond optimally at the system level. The third considers how Gestalt percepts (e.g., groups, objects) are optimal given the available stimulation, with optimality specified in Bayesian terms. Fourth, structural information theory explains how a Gestaltist visual system that focuses on internal coding efficiency yields external veridicality as a side effect. To answer the fundamental question of why things look as they do, a further synthesis of these complementary perspectives is required.

Principles of contour information: Reply to Lim and Leek (2012)

Lim and Leek (2012) presented a formalization of information along object contours, which they argued was an alternative to the approach taken in our article (Feldman & Singh, 2005). Here, we summarize the 2 approaches, showing that--notwithstanding Lim and Leek's (2012) critical rhetoric--their approach is substantially identical to ours, except for the technical details of the formalism. Following the logic of our article point by point, Lim and Leek (a) defined probabilistic expectations about the geometry of smooth contours (which they based on differential contour geometry, while we used a discrete approximation--the only essential difference in their approach), (b) assumed that information along the contour was proportional to the negative logarithm of probability, following standard information theory, and then (c) extended this formulation to closed contours. We analyze what they described as errors in our approach, all of which rest on mathematical misunderstandings or bizarre misreadings of our article. We also show that their extension to 3-dimensional surfaces and their "modified minima rule" contain fatal deficiencies.

A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization

In 1912, Max Wertheimer published his paper on phi motion, widely recognized as the start of Gestalt psychology. Because of its continued relevance in modern psychology, this centennial anniversary is an excellent opportunity to take stock of what Gestalt psychology has offered and how it has changed since its inception. We first introduce the key findings and ideas in the Berlin school of Gestalt psychology, and then briefly sketch its development, rise, and fall. Next, we discuss its empirical and conceptual problems, and indicate how they are addressed in contemporary research on perceptual grouping and figure-ground organization. In particular, we review the principles of grouping, both classical (e.g., proximity, similarity, common fate, good continuation, closure, symmetry, parallelism) and new (e.g., synchrony, common region, element and uniform connectedness), and their role in contour integration and completion. We then review classic and new image-based principles of figure-ground organization, how it is influenced by past experience and attention, and how it relates to shape and depth perception. After an integrated review of the neural mechanisms involved in contour grouping, border ownership, and figure-ground perception, we conclude by evaluating what modern vision science has offered compared to traditional Gestalt psychology, whether we can speak of a Gestalt revival, and where the remaining limitations and challenges lie. A better integration of this research tradition with the rest of vision science requires further progress regarding the conceptual and theoretical foundations of the Gestalt approach, which is the focus of a second review article.

The neural correlates of visuospatial perceptual and oculomotor extrapolation

The human visual system must perform complex visuospatial extrapolations (VSE) across space and time in order to extract shape and form from the retinal projection of a cluttered visual environment characterized by occluded surfaces and moving objects. Even if we exclude the temporal dimension, for instance when judging whether an extended finger is pointing towards one object or another, the mechanisms of VSE remain opaque. Here we investigated the neural correlates of VSE using functional magnetic resonance imaging in sixteen human observers while they judged the relative position of, or saccaded to, a (virtual) target defined by the extrapolated path of a pointer. Using whole brain and region of interest (ROI) analyses, we compared the brain activity evoked by these VSE tasks to similar control judgements or eye movements made to explicit (dot) targets that did not require extrapolation. The data show that activity in an occipitotemporal region that included the lateral occipital cortex (LOC) was significantly greater during VSE than during control tasks. A similar, though less pronounced, pattern was also evident in regions of the fronto-parietal cortex that included the frontal eye fields. However, none of the ROIs examined exhibited a significant interaction between target type (extrapolated/explicit) and response type (oculomotor/perceptual). These findings are consistent with a close association between visuoperceptual and oculomotor responses, and highlight a critical role for the LOC in the process of VSE.

Studies of the perception of incomplete outline images of different sizes

The aim of the present work was to assess the range of angular sizes of fragmented images of objects at which perception of the images was scale-independent. Measurements were made of human subjects' recognition thresholds for the shapes of the objects over a wide range of angular sizes (0.19-50 degrees). The experiments used the Gollin test--a method for studying the recognition of fragmented outline images of objects with which the observer is familiar. The results obtained demonstrated that there is a wide range of angular sizes, from 1.0 degrees to 50 degrees, over which the perception thresholds of incomplete outline images do not change with changes in size, along with a narrow range of stimulus sizes, 0.19-1.0 degrees, over which there is a significant size dependence. We suggest that the increase in thresholds and the failure to recognize images of small size occur as a result of an increased contribution of sampling noise at the level of the human retina.

Contour statistics in natural images: grouping across occlusions

Correctly interpreting a natural image requires dealing properly with the effects of occlusion, and hence, contour grouping across occlusions is a major component of many natural visual tasks. To better understand the mechanisms of contour grouping across occlusions, we (a) measured the pair-wise statistics of edge elements from contours in natural images, as a function of edge element geometry and contrast polarity, (b) derived the ideal Bayesian observer for a contour occlusion task where the stimuli were extracted directly from natural images, and then (c) measured human performance in the same contour occlusion task. In addition to discovering new statistical properties of natural contours, we found that naïve human observers closely parallel ideal performance in our contour occlusion task. In fact, there was no region of the four-dimensional stimulus space (three geometry dimensions and one contrast dimension) where humans did not closely parallel the performance of the ideal observer (i.e., efficiency was approximately constant over the entire space). These results reject many other contour grouping hypotheses and strongly suggest that the neural mechanisms of contour grouping are tightly related to the statistical properties of contours in natural images.

Precision and consistency of contour interpolation

We investigated conditions under which observers can interpolate occluded contours by a single, stable, smooth contour. Observers viewed partly-occluded contours defined by linear segments and estimated the position and tangent orientation of the contour at multiple locations within the occluded region. We measured the precision and consistency of observers' settings as indices of successful interpolation. We found that although increasing the relative offset between inducers led to a decrease in both precision and consistency, increasing turning angle affected primarily precision. We discuss conditions under which interpolation settings are consistent with a single, stable smooth contour.