A computational model for reference-frame synthesis with applications to motion perception

A neural model for vector decomposition and relative-motion perception

The perception of motion not only depends on the detection of motion signals but also on choosing and applying reference-frames according to which motion is interpreted. Here we propose a neural model that implements the common-fate principle for reference-frame selection. The model starts with a retinotopic layer of directionally-tuned motion detectors. The Gestalt common-fate principle is applied to the activities of these detectors to implement in two neural populations the direction and the magnitude (speed) of the reference-frame. The output activities of retinotopic motion-detectors are decomposed using the direction of the reference-frame. The direction and magnitude of the reference-frame are then applied to these decomposed motion-vectors to generate activities that reflect relative-motion perception, i.e., the perception of motion with respect to the prevailing reference-frame. We simulated this model for classical relative motion stimuli, viz., the three-dot, rotating-wheel, and point-walker (biological motion) paradigms and found the model performance to be close to theoretical vector decomposition values. In the three-dot paradigm, the model made the prediction of perceived curved-trajectories for the target dot when its horizontal velocity was slower or faster than the flanking dots. We tested this prediction in two psychophysical experiments and found a good qualitative and quantitative agreement between the model and the data. Our results show that a simple neural network using solely motion information can account for the perception of group and relative motion.

Features integrate along a motion trajectory when object integrity is preserved

Information about a moving object is usually poor at each retinotopic location because photoreceptor activation is short, noisy, and affected by shadows, reflections of other objects, and so on. Integration across the motion trajectory may yield a much better estimate about the objects’ features. Using the sequential metacontrast paradigm, we have shown previously that features, indeed, integrate along a motion trajectory in a long-lasting window of unconscious processing. In the sequential metacontrast paradigm, a percept of two diverging streams is elicited by the presentation of a central line followed by a sequence of flanking pairs of lines. When several lines are spatially offset, the offsets integrate mandatorily for several hundreds of milliseconds along the motion trajectory of the streams. We propose that, within these long-lasting windows, stimuli are first grouped based on Gestalt principles of grouping. These processes establish reference frames that are used to attribute features. Features are then integrated following their respective reference frame. Here using occlusion and bouncing effects, we show that indeed such grouping operations are in place. We found that features integrate only when the spatiotemporal integrity of the object is preserved. Moreover, when several moving objects are present, only features belonging to the same object integrate. Overall, our results show that feature integration is a deliberate strategy of the brain and long-lasting windows of processing can be seen as periods of sense making.

Spinning objects and partial occlusion: Smart neural responses to symmetry

In humans, extrastriate visual areas are strongly activated by symmetry. However, perfect symmetry is rare in natural visual images. Recent findings showed that when parts of a symmetric shape are presented at different points in time the process relies on a perceptual memory buffer. Does this temporal integration need a retinotopic reference frame? For the first time we tested integration of parts both in the temporal and spatial domain, using a non-retinotopic frame of reference. In Experiment 1, an irregular polygonal shape (either symmetric or asymmetric) was partly occluded by a rectangle for 500 ms (T1). The rectangle moved to the opposite side to reveal the other half of the shape, whilst occluding the previously visible half (T2). The reference frame for the object was static: the two parts stimulated retinotopically corresponding receptive fields (revealed over time). A symmetry-specific ERP response from ~300 ms after T2 was observed. In Experiment 2 dynamic occlusion was combined with an additional step at T2: the new half-shape and occluder were rotated by 90°. Therefore, there was a moving frame of reference and the retinal correspondence between the two parts was disrupted. A weaker but significant symmetry-specific response was recorded. This result extends previous findings: global symmetry representation can be achieved in extrastriate areas non-retinotopically, through integration in both temporal and spatial domain.

Human visual motion perception shows hallmarks of Bayesian structural inference

Motion relations in visual scenes carry an abundance of behaviorally relevant information, but little is known about how humans identify the structure underlying a scene's motion in the first place. We studied the computations governing human motion structure identification in two psychophysics experiments and found that perception of motion relations showed hallmarks of Bayesian structural inference. At the heart of our research lies a tractable task design that enabled us to reveal the signatures of probabilistic reasoning about latent structure. We found that a choice model based on the task's Bayesian ideal observer accurately matched many facets of human structural inference, including task performance, perceptual error patterns, single-trial responses, participant-specific differences, and subjective decision confidence-especially, when motion scenes were ambiguous and when object motion was hierarchically nested within other moving reference frames. Our work can guide future neuroscience experiments to reveal the neural mechanisms underlying higher-level visual motion perception.

Unconscious retinotopic motion processing affects non-retinotopic motion perception

Unconscious visual stimuli can affect conscious perception: For example, an invisible prime can affect responses to a subsequent target. The invisible interpretation of an ambiguous figure can have similar effects. Invisibility in these situations is typically explained by stimulus-suppression in early, retinotopic brain areas. We have previously argued that invisibility is closely linked to Gestalt ("object") organization principles. For example, motion is typically perceived in non-retinotopic, object-centered, and not in retinotopic coordinates. Such is the case for a bicycle-reflector that is perceived as circling, although its retinotopic trajectory is cycloidal. Here, we used a modified Ternus-Pikler display in which, just as in everyday vision, the retinotopic motion is invisible and the non-retinotopic motion is perceived. Nevertheless, the invisible retinotopic motion, can strongly degrade the conscious non-retinotopic motion percept. This effect cannot be explained by inhibition at a retinotopic processing stage.

Confirming the appearance of excess success: Reply to van Boxtel and Koch (2016)

van Boxtel and Koch (Psychonomic Bulletin & Review. doi: 10.3758/s13423-016-1010-0 , 2016) reported finding problems in the Test for Excess Success (TES) analysis in Francis (Psychonomic Bulletin & Review, 21, 1180-1187, 2014). They argued that their findings undermined the general analysis and the conclusions of the specific TES analysis for their article (van Boxtel & Koch in Psychological Science, 23(4), 410-418, 2012). As shown in this paper, their reported problems reflect misunderstandings about both the general properties of a TES analysis and how it was applied to their specific set of findings. Another look at the findings and theoretical claims in van Boxtel and Koch (Psychological Science, 23(4), 410-418, 2012) confirms the appearance of excess success.

An overview of quantitative approaches in Gestalt perception

Gestalt psychology is often criticized as lacking quantitative measurements and precise mathematical models. While this is true of the early Gestalt school, today there are many quantitative approaches in Gestalt perception and the special issue of Vision Research "Quantitative Approaches in Gestalt Perception" showcases the current state-of-the-art. In this article we give an overview of these current approaches. For example, ideal observer models are one of the standard quantitative tools in vision research and there is a clear trend to try and apply this tool to Gestalt perception and thereby integrate Gestalt perception into mainstream vision research. More generally, Bayesian models, long popular in other areas of vision research, are increasingly being employed to model perceptual grouping as well. Thus, although experimental and theoretical approaches to Gestalt perception remain quite diverse, we are hopeful that these quantitative trends will pave the way for a unified theory.

Retinotopic encoding of the Ternus-Pikler display reflected in the early visual areas

The visual representation of the world is often assumed to be retinotopic, and many visual brain areas are indeed organized retinotopically. Visual perception, however, is not based on a reference frame anchored in retinotopic coordinates. For example, when an object moves, motion of its constituent parts is perceived relative to the object rather than in retinotopic coordinates. The moving object thus serves as a nonretinotopic reference system for computing the properties of its parts. It is largely unknown how the brain accomplishes this feat. Here, we used the Ternus-Pikler display to pit retinotopic processing in a stationary reference system against nonretinotopic processing in a moving one. Using 7T fMRI, we found that the average blood-oxygen-level dependent activations in V1, V2, and V3 reflected the retinotopic properties, but not the nonretinotopic percepts, of the Ternus-Pikler display. In the human motion processing complex (hMT+), activations were compatible with both retinotopic and nonretinotopic encoding. Thus, hMT+ may be the first visual area encoding the nonretinotopic percepts of the Ternus-Pikler display.

Putting low-level vision into global context: Why vision cannot be reduced to basic circuits

To cope with the complexity of vision, most models in neuroscience and computer vision are of hierarchical and feedforward nature. Low-level vision, such as edge and motion detection, is explained by basic low-level neural circuits, whose outputs serve as building blocks for more complex circuits computing higher level features such as shape and entire objects. There is an isomorphism between states of the outer world, neural circuits, and perception, inspired by the positivistic philosophy of the mind. Here, we show that although such an approach is conceptually and mathematically appealing, it fails to explain many phenomena including crowding, visual masking, and non-retinotopic processing.