The emergence of visual objects in space-time

Additivity of grouping by proximity and luminance similarity is dependent on relative grouping strength: An analysis of individual differences in grouping sensitivity

It has previously been shown that grouping by proximity is well described by a linear function relating the perceived orientation of a dot lattice to the ratio of the distances between the dots in the different orientations. Similarly, luminance influences how observers perceptually group stimuli. Using the dot lattice paradigm, it has been shown that proximity and luminance similarity interact additively, which means that their effects can be summed to predict an observers' percept. In this study, we revisit the additive interplay between proximity and luminance similarity and we ask whether this pattern might be the result of inappropriately averaging different types of observers or the imbalance between the strength of proximity grouping and luminance similarity grouping. To address these questions, we first ran a replication of the original study reporting the additive interplay between proximity and luminance similarity. Our results showed a convincing replication at the aggregate and individual level. However, at the individual level, all observers showed grouping by proximity whereas some observers did not show grouping by luminance similarity. In response, we ran a second experiment with enlarged luminance differences to reinforce the strength of grouping by luminance similarity and balance the strength of the two grouping cues. Interestingly, in this second experiment, additivity was not observed but instead a significant interaction was obtained. This disparity suggests that the additivity or interaction between two grouping cues in a visual stimulus is not a general rule of perceptual grouping but a consequence of relative grouping strength.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

Unifying Physics and Psychophysics on the Basis of Symmetry, Least-Action ≈ Simplicity Principle, and Conservation Laws ≈ Veridicality

Psychophysics is the branch of experimental psychology that deals with the study of sensation and perception. A consensus has grown up among experts in psychophysics in the last hundred years that the human being’s percepts are inferences, which are based on a minimum, or simplicity, principle that is applied to the currently available sensory data. These educated guesses play the critical role in establishing veridical perceptual representations of the three-dimensional environment, where by “veridical” we mean that the percept agrees with what is “out there.” These veridical representations cannot be achieved without making use of symmetries, much like those known in physics, where they are essential for characterizing our physical world and deriving the conservation laws. But, unlike in physics, the important role that symmetry plays in psychophysics has been demonstrated and explained only within the last 10 years. Symmetries represent regularities in our physical world. These symmetries also serve as the source of the redundancies that are inherent in 3D objects and make vision possible. The main goal of this article is to show that the similarity between the mathematical formalisms used in physics and in psychophysics is not coincidental and that exploring this similarity can benefit the sciences of perception and cognition. This article includes a brief tutorial about symmetry groups and their relationship to transformation groups as well as to their invariants. It was included to make this material available to readers who are not familiar with these topics.

Perceptual inference employs intrinsic alpha frequency to resolve perceptual ambiguity

The brain uses its intrinsic dynamics to actively predict observed sensory inputs, especially under perceptual ambiguity. However, it remains unclear how this inference process is neurally implemented in biasing perception of ambiguous inputs towards the predicted percepts. The process of perceptual inference can be well illustrated by the phenomenon of bistable apparent motion in the Ternus display, in which subjective perception spontaneously alternates between element motion (EM) and group motion (GM) percepts depending on whether two consecutively presented frames are grouped over time or not. The frequency of alpha-band oscillations has long been hypothesized to gate the temporal window of perceptual grouping over time. Under this hypothesis, variation in the intrinsic alpha frequency should predict perceptual outcome of the bistable Ternus display. Moreover, we hypothesize that the perception system employs this prior knowledge on intrinsic alpha frequency to resolve perceptual ambiguity, by shifting perceptual inference towards the predicted percepts. Using electroencephalography and intracranial recordings, we showed that both between and within subjects, lower prestimulus alpha frequencies (PAFs) predicted the EM percepts since the two frames fell in the same alpha cycle and got temporally integrated, while higher PAFs predicted the GM percepts since the two frames fell in different alpha cycles. Multivariate decoding analysis between the EM percepts with lower PAFs and the GM percepts with higher PAFs further revealed a representation of the subsequently reported bistable percept in the neural signals shortly before the actual appearance of the second frame. Therefore, perceptual inference, based on variation in intrinsic PAFs, biases poststimulus neural representations by inducing preactivation of the predicted percepts. In addition, enhanced prestimulus blood-oxygen-level-dependent (BOLD) signals and network dynamics in the frontoparietal network, together with reduced prestimulus alpha power, upon perceiving the EM percepts suggest that temporal grouping is an attention-demanding process.

Paradoxical perception of object identity in visual motion

In the course of perceptual organization, incomplete optical stimulation can evoke the experience of complete objects with distinct perceptual identities. According to a well-known principle of perceptual organization, stimulus parts separated by shorter spatial distances are more likely to appear as parts of the same perceptual identity. Whereas this principle of proximity has been confirmed in many studies of perceptual grouping in static displays, we show that it does not generalize to perception of object identity in dynamic displays, where the parts are separated by spatial and temporal distances. We use ambiguous displays which contain multiple moving parts and which can be perceived two ways: as two large objects that gradually change their size or as multiple smaller objects that rotate independent of one another. Grouping over long and short distances corresponds to the perception of the respectively large and small objects. We find that grouping over long distances is often preferred to grouping over short distances, against predictions of the proximity principle. Even though these effects are observed at high luminance contrast, we show that they are consistent with results obtained at the threshold of luminance contrast, in agreement with predictions of a theory of efficient motion measurement. This is evidence that the perception of object identity can be explained by a computational principle of neural economy rather than by the empirical principle of proximity.

Recent advances in exploring the neural underpinnings of auditory scene perception

Studies of auditory scene analysis have traditionally relied on paradigms using artificial sounds-and conventional behavioral techniques-to elucidate how we perceptually segregate auditory objects or streams from each other. In the past few decades, however, there has been growing interest in uncovering the neural underpinnings of auditory segregation using human and animal neuroscience techniques, as well as computational modeling. This largely reflects the growth in the fields of cognitive neuroscience and computational neuroscience and has led to new theories of how the auditory system segregates sounds in complex arrays. The current review focuses on neural and computational studies of auditory scene perception published in the last few years. Following the progress that has been made in these studies, we describe (1) theoretical advances in our understanding of the most well-studied aspects of auditory scene perception, namely segregation of sequential patterns of sounds and concurrently presented sounds; (2) the diversification of topics and paradigms that have been investigated; and (3) how new neuroscience techniques (including invasive neurophysiology in awake humans, genotyping, and brain stimulation) have been used in this field.

Synchrony, Binding, and Consciousness

LaRock provides a rather perplexing critique of the binding problem. He argues that neural synchrony, the proposed solution to the binding problem, is not sufficient to explain the unity of objects in visual consciousness. Did anyone claim it was? Neural synchrony can at best be necessary for consciousness, not sufficient. The binding problem originally has nothing to do with the conscious content of our perception. Misidentification of binding and consciousness may lead to tunnel vision with respect to other roles that neural synchrony could play in conscious experience. A different perspective is offered on how neural synchrony relates to visual consciousness.

What Happens in a Moment

There has been evidence for the very brief, temporal quantization of perceptual experience at regular intervals below 100 ms for several decades. We briefly describe how earlier studies led to the concept of "psychological moment" of between 50 and 60 ms duration. According to historical theories, within the psychological moment all events would be processed as co-temporal. More recently, a link with physiological mechanisms has been proposed, according to which the 50-60 ms psychological moment would be defined by the upper limit required by neural mechanisms to synchronize and thereby represent a snapshot of current perceptual event structure. However, our own experimental developments also identify a more fine-scaled, serialized process structure within the psychological moment. Our data suggests that not all events are processed as co-temporal within the psychological moment and instead, some are processed successively. This evidence questions the analog relationship between synchronized process and simultaneous experience and opens debate on the ontology and function of "moments" in psychological experience.

Perceiving the initial note: quantitative models of how listeners parse cyclical auditory patterns

In this paper, we explore the rules followed by the auditory system in grouping temporal patterns. Imagine the following cyclical pattern (which we call an "auditory necklace"-AN for short-because those patterns are best visualized as beads arranged on a circle) consisting of notes (1s) and rests (0s): … 1110011011100110 …. It is perceived either as repeating 11100110 or as repeating 11011100. We devised a method to explore the temporal segmentation of ANs. In two experiments, while an AN was played, a circular array of icons appeared on the screen. At the time of each event (i.e., note or rest), one icon was highlighted; the highlight moved cyclically around the circular array. The participants were asked to click on the icon that corresponded to the note they perceived as the starting point, or clasp, of the AN. The best account of the segmentation of our ANs is based on Garner's (1974) run and gap principles. An important feature of our probabilistic model is the way in which it combines the effects of run length and gap length: additively. This result is an auditory analogue of Kubovy and van den Berg's (2008) discovery of the additivity of the effects of two visual grouping principles (proximity and similarity) conjointly applied to the same stimulus.

Non-rigid illusory contours and global shape transformations defined by spatiotemporal boundary formation

Spatiotemporal boundary formation (SBF) is the perception of form, global motion, and continuous boundaries from relations of discrete changes in local texture elements (Shipley and Kellman, 1994). In two experiments, small, circular elements underwent small displacements whenever an edge of an invisible (virtual) object passed over them. Unlike previous studies that examined only rigidly translating objects, we tested virtual objects whose properties changed continuously. Experiment 1 tested rigid objects that changed in orientation, scale, and velocity. Experiment 2 tested objects that transformed non-rigidly taking on a series of shapes. Robust SBF occurred for all of the rigid transformations tested, as well as for non-rigid virtual objects, producing the perception of continuously bounded, smoothly deforming shapes. These novel illusions involve perhaps the most extreme cases of visual perception of continuous boundaries and shape from minimal information. They show that SBF encompasses a wider range of illusory phenomena than previously understood, and they present substantial challenges for existing models of SBF.

A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization

In 1912, Max Wertheimer published his paper on phi motion, widely recognized as the start of Gestalt psychology. Because of its continued relevance in modern psychology, this centennial anniversary is an excellent opportunity to take stock of what Gestalt psychology has offered and how it has changed since its inception. We first introduce the key findings and ideas in the Berlin school of Gestalt psychology, and then briefly sketch its development, rise, and fall. Next, we discuss its empirical and conceptual problems, and indicate how they are addressed in contemporary research on perceptual grouping and figure-ground organization. In particular, we review the principles of grouping, both classical (e.g., proximity, similarity, common fate, good continuation, closure, symmetry, parallelism) and new (e.g., synchrony, common region, element and uniform connectedness), and their role in contour integration and completion. We then review classic and new image-based principles of figure-ground organization, how it is influenced by past experience and attention, and how it relates to shape and depth perception. After an integrated review of the neural mechanisms involved in contour grouping, border ownership, and figure-ground perception, we conclude by evaluating what modern vision science has offered compared to traditional Gestalt psychology, whether we can speak of a Gestalt revival, and where the remaining limitations and challenges lie. A better integration of this research tradition with the rest of vision science requires further progress regarding the conceptual and theoretical foundations of the Gestalt approach, which is the focus of a second review article.

Attention modulates spatio-temporal grouping

Dynamic stimuli are ubiquitous in natural viewing conditions implying that grouping operations need to operate, not only in space, but also jointly in space and time. Moreover, in natural viewing, attention plays an important role in controlling how resources are allocated. We investigated how attention interacts with spatio-temporal perceptual grouping by using a bistable stimulus, called the Ternus-Pikler display. Ternus-Pikler displays can give rise to two different motion percepts, called Element Motion (EM) and Group Motion (GM), the former dominating at short Inter-Stimulus Intervals (ISIs) and the latter at long ISIs. Our results indicate that GM grouping requires more attentional resources than EM grouping. Different theoretical accounts of perceptual grouping and attention are discussed and evaluated in the light of the current results.

Selecting salient frames for spatiotemporal video modeling and segmentation

We propose a new statistical generative model for spatiotemporal video segmentation. The objective is to partition a video sequence into homogeneous segments that can be used as "building blocks" for semantic video segmentation. The baseline framework is a Gaussian mixture model (GMM)-based video modeling approach that involves a six-dimensional spatiotemporal feature space. Specifically, we introduce the concept of frame saliency to quantify the relevancy of a video frame to the GMM-based spatiotemporal video modeling. This helps us use a small set of salient frames to facilitate the model training by reducing data redundancy and irrelevance. A modified expectation maximization algorithm is developed for simultaneous GMM training and frame saliency estimation, and the frames with the highest saliency values are extracted to refine the GMM estimation for video segmentation. Moreover, it is interesting to find that frame saliency can imply some object behaviors. This makes the proposed method also applicable to other frame-related video analysis tasks, such as key-frame extraction, video skimming, etc. Experiments on real videos demonstrate the effectiveness and efficiency of the proposed method.

Contour discontinuities subserve two types of form analysis that underlie motion processing

Form analysis subserves motion processing in at least two ways: first, in terms of figural segmentation dedicated to solving the problem of figure-to-figure matching over time, and second, in terms of defining trackable features whose unambiguous motion signals can be generalized to ambiguously moving portions of an object. The former is a primarily ventral process involving the lateral occipital complex and also retinotopic areas such as V2 and V4, and the latter is a dorsal process involving V3A. Contour discontinuities, such as corners, deep concavities, maxima of positive curvature, junctions, and terminators, play a central role in both types of form analysis. Transformational apparent motion will be discussed in the context of figural segmentation and matching, and rotational motion in the context of trackable features. In both cases the analysis of form must proceed in parallel with the analysis of motion, in order to constrain the ongoing analysis of motion.

Effects of the orientation of moving objects on the perception of streaming/bouncing motion displays

In this study, we examined the contribution of the orientation of moving objects to perception of a streaming/bouncing motion display. In three experiments, participants reported which of the two types of motion, streaming or bouncing, they perceived. The following independent variables were used: orientation differences between Gabor micropatterns (Gabors) and their path of motion (all the experiments) and the presence/absence of a transient tone (Experiment 1), transient visual flash (Experiment 2), or concurrent secondary task (Experiment 3) at the coincidence of Gabors. The results showed that the events at coincidence generally biased responses toward the perception of bouncing. On the other hand, alignment of Gabors with their motion axes significantly reduced the frequency of bounce perception. The results also indicated that an object whose orientation was parallel to its motion path strengthened the spatiotemporal integration of local motion signals along a straight motion path, resulting in the perception of streaming. We suggest that the effect of collinearity between Gabors and their motion path is relatively free from the effect of attention distraction.

Spectral grouping using the Nyström method

Spectral graph theoretic methods have recently shown great promise for the problem of image segmentation. However, due to the computational demands of these approaches, applications to large problems such as spatiotemporal data and high resolution imagery have been slow to appear. The contribution of this paper is a method that substantially reduces the computational requirements of grouping algorithms based on spectral partitioning making it feasible to apply them to very large grouping problems. Our approach is based on a technique for the numerical solution of eigenfunction problems known as the Nyström method. This method allows one to extrapolate the complete grouping solution using only a small number of samples. In doing so, we leverage the fact that there are far fewer coherent groups in a scene than pixels.

Acts of perceptual inquiry: problems for any stimulus-based simplicity theory

Stimulus-specified simplicity is simply insufficient to predict the appearance of objects' 3D forms and figural shapes, etc., as originally intended. Which information a viewer chooses to attend can determine both what 3D structure is perceived and its attendant perceptual consequences; moreover, a shape's meaningfulness or denotivity [M.A. Peterson, Current Directions in Psychological Science 3 (1994) 105] can overcome simplicity-based figure-ground segregation. In both cases, perceptual consequences, such as subsequent perceived movements, are constrained or primed in ways that can help us corroborate and define what the viewer has perceived; that may help us in studying the underlying events in brain processing; and that should help in designing such perceptual applications as still and animated displays. Demonstrations, theoretical framework, and potential research tools are offered.

Perceptual switching, eye movements, and the bus paradox

According to a widely cited finding by Ellis and Stark (1978 Perception 7 575-581), the duration of eye fixations is longer at the instant of perceptual reversal of an ambiguous figure than before or after the reversal. However, long fixations are more likely to include samples of an independent random event than are short fixations. This sampling bias would produce the pattern of results also when no correlation exists between fixation duration and perceptual reversals. When an appropriate correction is applied to the measurement of fixation durations, the effect disappears. In fact, there are fewer actual button-presses during the long intervals than would be expected by chance. Moving-window analyses performed on eye-fixation data reveal that no unique eye event is associated with switching behaviour. However, several indicators, such as blink frequency, saccade frequency, and the direction of the saccade, are each differentially sensitive to perceptual and response-related aspects of the switching process. The time course of these indicators depicts switching behaviour as a process of cascaded stages.

Interpolating sampled contours in 3-D: analyses of variability and bias

In two experiments, we examined how observers interpolated the missing parts of sampled, planar contours in 3-D space. We varied (1) contour type (linear or parabolic), (2) orientation of the plane containing the contour and (3) the number of points on a sampled contour.Interpolation performance was very accurate, comparable to results from Vernier tasks. Setting variability was highest along the line of sight and for the parabolic contour. Setting variability did not decrease with increasing number of points on either contour, suggesting that observers do not use all available, relevant information in this task.

The duration of 3-d form analysis in transformational apparent motion

Transformational apparent motion (TAM) occurs when a figure changes discretely from one configuration to another overlapping configuration. Rather than an abrupt shape change, the initial shape is perceived to transform smoothly into the final shape as if animated by a series of intermediate shapes. We find that TAM follows an analysis of form that takes 80-140 msec. Form analysis can function both at and away from equiluminance and can occur over contours defined by uniform regions as well as outlines. Moreover, the forms analyzed can be 3-D, resulting in motion paths that appear to smoothly project out from or into the stimulus plane. The perceived transformation is generally the one that involves the least change in the shape or location of the initial figure in a 3-D sense. We conclude that perception of TAM follows an analysis of 3-D form that takes approximately 100 msec. This stage of form analysis may be common to both TAM and second-order motion.

Neural synergy in visual grouping: when good continuation meets common fate

A modified version of the 'path finder' display consisting of many small oriented Gabor patches was used to study the joint contributions of spatial and temporal structures to shape perception. A two-interval forced-choice procedure measured detectability of curved 'paths' defined by orientation ('good continuation') and/or by temporal synchrony of change in motion direction ('common fate'). When orientation was completely random (no spatial 'path' cue) temporal synchrony still supported reliable performance, but only when correlation of change among 'path' elements was high. When combined, these two weak spatial and temporal structures yielded performance in excess of probability summation: 'paths' weakly defined by orientation were highly conspicuous when the constituent Gabors underwent synchronized changes in direction of motion, even though the individual directions of path elements were uncorrelated. Spatial grouping from temporal structure may arise from correlated transients associated with synchronized changes in motion direction. Evidently these two mechanisms for promotion of spatial grouping interact synergistically.