Bayesian hierarchical grouping: Perceptual grouping as mixture estimation

Hierarchical Constraints on the Distribution of Attention in Dynamic Displays

Human vision is remarkably good at recovering the latent hierarchical structure of dynamic scenes. Here, we explore how visual attention operates with this hierarchical motion representation. The way in which attention responds to surface physical features has been extensively explored. However, we know little about how the distribution of attention can be distorted by the latent hierarchical structure. To explore this topic, we conducted two experiments to investigate the relationship between minimal graph distance (MGD), one key factor in hierarchical representation, and attentional distribution. In Experiment 1, we constructed three hierarchical structures consisting of two moving objects with different MGDs. In Experiment 2, we generated three moving objects from one hierarchy to eliminate the influence of different structures. Attention was probed by the classic congruent-incongruent cueing paradigm. Our results show that the cueing effect is significantly smaller when the MGD between two objects is shorter, which suggests that attention is not evenly distributed across multiple moving objects but distorted by their latent hierarchical structure. As neither the latent structure nor the graph distance was part of the explicit task, our results also imply that both the construction of hierarchical representation and the attention to that representation are spontaneous and automatic.

Prägnanz in visual perception

How do we perceptually and cognitively organize incoming stimulation? A century ago, Gestalt psychologists posited the law of Prägnanz: psychological organization will always be as 'good' as possible given the prevailing conditions. To make the Prägnanz law a useful statement, it needs to be specified further (a) what a 'good' psychological organization entails, (b) how the Prägnanz tendency can be realized, and (c) which conditions need to be taken into account. Although the Gestalt school did provide answers to these questions, modern-day mentions of Prägnanz or good Gestalt often lack these clarifications. The concept of Prägnanz has been (mis)understood in many different ways, and by looking back on the rich history of the concept, we will attempt to present a more fine-grained view and promote a renewed understanding of the central role of Prägnanz in visual perception and beyond. We review Gestalt psychology's answers to the questions listed above, and also discuss the four main uses of the Prägnanz concept in more detail: (a) a Prägnanz tendency in each organizational process, (b) Prägnanz as a property of a Gestalt, (c) Prägnanz steps as internal reference points, and (d) Prägnanz in relation to aesthetic appreciation. As a key takeaway, Prägnanz is a multifaceted Gestalt psychological concept indicating the "goodness" of an experienced organization. Both the removal of unnecessary details and the emphasis on characteristic features of the overall organization compared to a reference organization can contribute to the emergence of a 'better' Gestalt. The stimulus constellation is not the only factor in determining the goodness of an organization, also the stimulus' interaction with an individual in a specific spatial and temporal context plays a role. Taking the ideas on Prägnanz as a generative framework and keeping the original Gestalt psychological context in mind, future research on perceptual organization can improve our understanding of the principles underlying psychological organization by further specifying how different organizational principles interact in concrete situations. Public significance statement: This paper reviews what a 'good' psychological organization entails, and how the incoming stimulation is clarified in human perception to achieve the best possible psychological organization. The review debunks common misconceptions on the meaning of "goodness" and synthesizes the most important perspectives and developments on "goodness" from its conception until now.

Hierarchical motion perception as causal inference

Since motion can only be defined relative to a reference frame, which reference frame guides perception? A century of psychophysical studies has produced conflicting evidence: retinotopic, egocentric, world-centric, or even object-centric. We introduce a hierarchical Bayesian model mapping retinal velocities to perceived velocities. Our model mirrors the structure in the world, in which visual elements move within causally connected reference frames. Friction renders velocities in these reference frames mostly stationary, formalized by an additional delta component (at zero) in the prior. Inverting this model automatically segments visual inputs into groups, groups into supergroups, etc. and "perceives" motion in the appropriate reference frame. Critical model predictions are supported by two new experiments, and fitting our model to the data allows us to infer the subjective set of reference frames used by individual observers. Our model provides a quantitative normative justification for key Gestalt principles providing inspiration for building better models of visual processing in general.

Bringing Bayes and Shannon to the Study of Behavioural and Neurobiological Timing and Associative Learning

Bayesian parameter estimation and Shannon’s theory of information provide tools for analysing and understanding data from behavioural and neurobiological experiments on interval timing—and from experiments on Pavlovian and operant conditioning, because timing plays a fundamental role in associative learning. In this tutorial, we explain basic concepts behind these tools and show how to apply them to estimating, on a trial-by-trial, reinforcement-by-reinforcement and response-by-response basis, important parameters of timing behaviour and of the neurobiological manifestations of timing in the brain. These tools enable quantification of relevant variables in the trade-off between acting as an ideal observer should act and acting as an ideal agent should act, which is also known as the trade-off between exploration (information gathering) and exploitation (information utilization) in reinforcement learning. They enable comparing the strength of the evidence for a measurable association to the strength of the behavioural evidence that the association has been perceived. A GitHub site and an OSF site give public access to well-documented Matlab and Python code and to raw data to which these tools have been applied.

A language of thought for the mental representation of geometric shapes

In various cultures and at all spatial scales, humans produce a rich complexity of geometric shapes such as lines, circles or spirals. Here, we propose that humans possess a language of thought for geometric shapes that can produce line drawings as recursive combinations of a minimal set of geometric primitives. We present a programming language, similar to Logo, that combines discrete numbers and continuous integration to form higher-level structures based on repetition, concatenation and embedding, and we show that the simplest programs in this language generate the fundamental geometric shapes observed in human cultures. On the perceptual side, we propose that shape perception in humans involves searching for the shortest program that correctly draws the image (program induction). A consequence of this framework is that the mental difficulty of remembering a shape should depend on its minimum description length (MDL) in the proposed language. In two experiments, we show that encoding and processing of geometric shapes is well predicted by MDL. Furthermore, our hypotheses predict additive laws for the psychological complexity of repeated, concatenated or embedded shapes, which we confirm experimentally.

Simple Assumptions to Improve Markov Illuminance and Reflectance

Murray recently introduced a novel computational lightness model, Markov illuminance and reflectance (MIR). MIR is a promising new approach that simulates human lightness processing using a conditional random field (CRF) where natural-scene statistics of reflectance and illumination are implemented. Although MIR can account for various lightness illusions and phenomena, it has limitations, such as the inability to predict reverse-contrast phenomena. In this study, we improved MIR performance by modifying its inference process, the prior on X-junctions, and that on general illumination changes. Our modified model improved predictions for Checkerboard assimilation, the simplified Checkershadow and its control figure, the influence of luminance noise, and White's effect and its several variants. In particular, White's effect is a partial reverse contrast that is challenging for computational models, so this improvement is a significant advance for the MIR framework. This study showed the high extensibility and potential of MIR, which shows the promise for further sophistication.

Perceptual Grouping Explains Similarities in Constellations Across Cultures

Cultures around the world organize stars into constellations, or asterisms, and these groupings are often considered to be arbitrary and culture specific. Yet there are striking similarities in asterisms across cultures, and groupings such as Orion, the Big Dipper, the Pleiades, and the Southern Cross are widely recognized across many different cultures. Psychologists have informally suggested that these shared patterns are explained by Gestalt laws of grouping, but there have been no systematic attempts to catalog asterisms that recur across cultures or to explain the perceptual basis of these groupings. Here, we compiled data from 27 cultures around the world and found that a simple computational model of perceptual grouping accounts for many of the recurring cross-cultural asterisms. Our results suggest that basic perceptual principles account for more of the structure of asterisms across cultures than previously acknowledged and highlight ways in which specific cultures depart from this shared baseline.

Point-estimating observer models for latent cause detection

The spatial distribution of visual items allows us to infer the presence of latent causes in the world. For instance, a spatial cluster of ants allows us to infer the presence of a common food source. However, optimal inference requires the integration of a computationally intractable number of world states in real world situations. For example, optimal inference about whether a common cause exists based on N spatially distributed visual items requires marginalizing over both the location of the latent cause and 2^N possible affiliation patterns (where each item may be affiliated or non-affiliated with the latent cause). How might the brain approximate this inference? We show that subject behaviour deviates qualitatively from Bayes-optimal, in particular showing an unexpected positive effect of N (the number of visual items) on the false-alarm rate. We propose several “point-estimating” observer models that fit subject behaviour better than the Bayesian model. They each avoid a costly computational marginalization over at least one of the variables of the generative model by “committing” to a point estimate of at least one of the two generative model variables. These findings suggest that the brain may implement partially committal variants of Bayesian models when detecting latent causes based on complex real world data.

Perceptual systems are designed to make sense of fragmented sensory data by inferring common, latent causes. Seeing a cluster of insects might allow us to infer the presence of a common food source, whereas the same number of insects scattered over a larger area of land might not evoke the same suspicions. The ability to reliably make this inference based on statistical information about the environment is surprisingly non-trivial: making the best possible inference requires making full use of the probabilistic information provided by the sensory data, which would require considering a combinatorially explosive number of hypothetical world states. In this paper, we test human subjects on their ability to perform a causal detection task: subjects are asked to judge whether an underlying cause of clustering is present or absent, based on the spatial distribution of those items. We show that subjects do not reason optimally on this task, and that particular computational short cuts (“committing” to certain world states over others, rather than representing them all) might underlie perceptual decision-making in these causal detection schemes.

A computational model for gestalt proximity principle on dot patterns and beyond

The human visual system has the ability to group parts of stimuli into larger, inherently structured units. In this article, a computational model inspired by tolerance space theory simulating the human perceptual grouping of dot patterns is proposed. Tolerance space theory introduces a tolerance relation to a discrete set to formulate the continuity of the discrete patterns. The model proposed herein includes one- and two-reach methods based on the assumption that dot patterns can be represented in the proposed extended tolerance space (ETS). Both methods are used to construct a ratio neighborhood graph (RANG), calculate tolerance from the diagram, compute the new RANG, and then rebuild continuous structures from the new RANG with a combinatorial procedure. Experiments are conducted to show the high consistency of the proposed model with human perception for various shapes of dot patterns, its ability to simulate Gestalt proximity and similarity principles, and its potential application in computer vision. In addition, the close relationship of the proposed model with the Pure Distance Law is comprehensively revealed, and the hierarchical representation of perceptual grouping is simulated with an adaptation of the proposed model based on the ETS.

Statistically defined visual chunks engage object-based attention

Although objects are the fundamental units of our representation interpreting the environment around us, it is still not clear how we handle and organize the incoming sensory information to form object representations. By utilizing previously well-documented advantages of within-object over across-object information processing, here we test whether learning involuntarily consistent visual statistical properties of stimuli that are free of any traditional segmentation cues might be sufficient to create object-like behavioral effects. Using a visual statistical learning paradigm and measuring efficiency of 3-AFC search and object-based attention, we find that statistically defined and implicitly learned visual chunks bias observers' behavior in subsequent search tasks the same way as objects defined by visual boundaries do. These results suggest that learning consistent statistical contingencies based on the sensory input contributes to the emergence of object representations.

Parts beget parts: Bootstrapping hierarchical object representations through visual statistical learning

Previous research has shown that humans are able to acquire statistical regularities among shape parts that form various spatial configurations, via exposure to these configurations without any task or feedback. The present study extends this approach of visual statistical learning to examine whether prior knowledge of parts, acquired in a separate learning context, facilitates acquisition of multi-layer hierarchical representations of objects. After participants had learned to encode a shape-pair as a chunk into memory, they viewed cluttered scenes containing multiple shape chunks. One of the larger configurations was constructed by combining the learned shape-pair with an unfamiliar, complementary shape-pair. Although the complementary shape-pair had never been presented separately during learning, it was remembered better than other shape pairs that were parts of larger configurations. The greater perceived familiarity of the complementary shape-pair depended on the encoding strength of the previously learned shape-pair. This "parts-beget-parts" effect suggests that statistical learning, in combination with prior knowledge, can represent objects as a coherent whole and also as a spatial configuration of parts by bootstrapping multi-layer hierarchical structures.

Recognizing Linear Building Patterns in Topographic Data by Using Two New Indices based on Delaunay Triangulation

Building pattern recognition is fundamental to a wide range of downstream applications, such as urban landscape evaluation, social analyses, and map generalization. Although many studies have been conducted, there is still a lack of satisfactory results, due to the imprecision of the relative direction model of any two adjacent buildings and the ineffective extraction methods. This study aims to provide an alternative for quantifying the direction and the spatial continuity of any two buildings on the basis of the Delaunay triangulation for the recognition of linear building patterns. First, constrained Delaunay triangulations (CDTs) are created for all buildings within each block and every two adjacent buildings. Then, the spatial continuity index (SCI), the direction index (DI), and other spatial relations (e.g., distance) of every two adjacent buildings are derived using the CDT. Finally, the building block is modelled as a graph based on derived matrices, and a graph segmentation approach is proposed to extract linear building patterns. In the segmentation process, the edges of the graph are removed first, according to the global thresholds of the SCI and distance, and are subsequently subdivided into subgraphs on direction rules. The proposed method is tested using three datasets. The experimental results suggest that the proposed method can recognize both collinear and curvilinear building patterns, given that the correctness values are all above 92% for the three study areas. The results also demonstrate that the novel SCI can effectively filter many insignificant neighbor relationships in the graph segmentation process. It is noteworthy that the proposed DI is capable of measuring building relative directions accurately and works efficiently in linear building pattern extraction.

Dubious Claims about Simplicity and Likelihood: Comment on Pinna and Conti (2019)

Pinna and Conti (Brain Sci., 2019, 9, 149, doi:10.3390/brainsci9060149) presented phenomena concerning the salience and role of contrast polarity in human visual perception, particularly in amodal completion. These phenomena are indeed illustrative thereof, but here, the focus is on their claims (1) that neither simplicity nor likelihood approaches can account for these phenomena; and (2) that simplicity and likelihood are equivalent. I argue that their first claim is based on incorrect assumptions, whereas their second claim is simply untrue.

Perception and conception in understanding evolutionary trees

Relationships depicted in evolutionary trees depend solely on levels of most recent common ancestry. Integrating discipline-based education research in biology with perceptual/cognitive psychology, the authors predicted, however, that the Gestalt principles of perceptual grouping would affect how students interpret these relationships. Experiment 1 (N = 93) found that students segment 6-9 branch trees in accordance with the Gestalt principle of connectedness. Experiment 2 (N = 310) found that students in introductory through advanced biology classes predominantly believed, incorrectly, that the evolutionary relationships among a set of target taxa differed in two trees because the grouping of those taxa differed. Experiment 3 (N = 99) found that students from these same classes were more likely to make inferences consistent with the depicted evolutionary relationships when Gestalt grouping supported those inferences. The authors discuss implications for improving students' understanding of cladograms.

Statistical learning and Gestalt-like principles predict melodic expectations

Expectation, or prediction, has become a major theme in cognitive science. Music offers a powerful system for studying how expectations are formed and deployed in the processing of richly structured sequences that unfold rapidly in time. We ask to what extent expectations about an upcoming note in a melody are driven by two distinct factors: Gestalt-like principles grounded in the auditory system (e.g.a preference for subsequent notes to move in small intervals), and statistical learning of melodic structure. We use multinomial regression modeling to evaluate the predictions of computationally implemented models of melodic expectation against behavioral data from a musical cloze task, in which participants hear a novel melodic opening and are asked to sing the note they expect to come next. We demonstrate that both Gestalt-like principles and statistical learning contribute to listeners' online expectations. In conjunction with results in the domain of language, our results point to a larger-than-previously-assumed role for statistical learning in predictive processing across cognitive domains, even in cases that seem potentially governed by a smaller set of theoretically motivated rules. However, we also find that both of the models tested here leave much variance in the human data unexplained, pointing to a need for models of melodic expectation that incorporate underlying hierarchical and/or harmonic structure. We propose that our combined behavioral (melodic cloze) and modeling (multinomial regression) approach provides a powerful method for further testing and development of models of melodic expectation.

On Bayesian Simplicity in Human Visual Perceptual Organization

A returning idea among some Bayesians in research on human visual perceptual organization is that the surprisal of something (i.e., the negative logarithm of its probability) expresses its complexity (i.e., the length of its shortest description). Bayes' rule is a powerful modeling tool and descriptive simplicity is a rich concept, but this idea is wishful thinking at best: If true, it would unify the simplicity and likelihood principles, which reflect two traditionally opposed schools of thought on perceptual organization. Some rapprochement between the two principles can certainly be discerned, but the aforementioned idea lacks formal underpinning and confounds otherwise perfectly good ideas. Here, this idea is revisited and its latest version is debunked step by step. In addition, I argue that its likely origin lies, inadvertently, in a standard Bayesian textbook: The author made (a) a pivotal mistake and (b) a compelling argument that was overinterpreted by others.

Inflections of the Bayesian Paradigm in Perceptual Psychology

Bayesian modeling has gained a conspicuous position in contemporary perceptual psychology. It can be examined from two viewpoints: a formal one, concerning the logical attributes of and the algebraic operations on the components of the models, and a substantive one, concerning the empirical meaning of those components. We maintain that, while there is homogeneity between Bayesian models of visual perception in their formal setup, remarkable differences can be found in their substantive aspect, that is, how the question "Where do probabilities come from?" is answered when designing the models. In particular, we focus on an inflection that we call "congenial" because it consistently embodies the inversion idea of the Bayes' rule in terms of optical inversion and highlight delicate issues that face this inflection for a consistent realization of the scientific program it represents. We also suggest ideas concerning the organization of the Bayesian area within perceptual psychology, which appears variegated, with the congenial inflection in a central position, and a fringe of disputable classification along the border.

Visual perception of complex shape-transforming processes

Morphogenesis-or the origin of complex natural form-has long fascinated researchers from practically every branch of science. However, we know practically nothing about how we perceive and understand such processes. Here, we measured how observers visually infer shape-transforming processes. Participants viewed pairs of objects ('before' and 'after' a transformation) and identified points that corresponded across the transformation. This allowed us to map out in spatial detail how perceived shape and space were affected by the transformations. Participants' responses were strikingly accurate and mutually consistent for a wide range of non-rigid transformations including complex growth-like processes. A zero-free-parameter model based on matching and interpolating/extrapolating the positions of high-salience contour features predicts the data surprisingly well, suggesting observers infer spatial correspondences relative to key landmarks. Together, our findings reveal the operation of specific perceptual organization processes that make us remarkably adept at identifying correspondences across complex shape-transforming processes by using salient object features. We suggest that these abilities, which allow us to parse and interpret the causally significant features of shapes, are invaluable for many tasks that involve 'making sense' of shape.

An overview of quantitative approaches in Gestalt perception

Gestalt psychology is often criticized as lacking quantitative measurements and precise mathematical models. While this is true of the early Gestalt school, today there are many quantitative approaches in Gestalt perception and the special issue of Vision Research "Quantitative Approaches in Gestalt Perception" showcases the current state-of-the-art. In this article we give an overview of these current approaches. For example, ideal observer models are one of the standard quantitative tools in vision research and there is a clear trend to try and apply this tool to Gestalt perception and thereby integrate Gestalt perception into mainstream vision research. More generally, Bayesian models, long popular in other areas of vision research, are increasingly being employed to model perceptual grouping as well. Thus, although experimental and theoretical approaches to Gestalt perception remain quite diverse, we are hopeful that these quantitative trends will pave the way for a unified theory.