A neural model of visual figure-ground segregation from kinetic occlusion

Interaction of contour geometry and optic flow in determining relative depth of surfaces

Dynamic occlusion, such as the accretion and deletion of texture near a boundary, is a major factor in determining relative depth of surfaces. However, the shape of the contour bounding the dynamic texture can significantly influence what kind of 3D shape, and what relative depth, are conveyed by the optic flow. This can lead to percepts that are inconsistent with traditional accounts of shape and depth from motion, where accreting/deleting texture can indicate the figural region, and/or 3D rotation can be perceived despite the constant speed of the optic flow. This suggests that the speed profile of the dynamic texture and the shape of its bounding contours combine to determine relative depth in a way that is not explained by existing models. Here, we investigated how traditional structure-from-motion principles and contour geometry interact to determine the relative-depth interpretation of dynamic textures. We manipulated the consistency of the dynamic texture with rotational or translational motion by varying the speed profile of the texture. In Experiment 1, we used a multi-region figure-ground display consisting of regions with dots moving horizontally in opposite directions in adjacent regions. In Experiment 2, we used stimuli including two regions separated by a common border, with dot textures moving horizontally in opposite directions. Both contour geometry (convexity) and the speed profile of the dynamic dot texture influenced relative-depth judgments, but contour geometry was the stronger factor. The results underscore the importance of contour geometry, which most current models disregard, in determining depth from motion.

The interpretation of dynamic occlusion: Combining contour geometry and accretion/deletion of texture

Conventional accounts of motion perception mostly treat accretion/deletion-the appearance or disappearance of texture at a boundary between regions-as an essentially decisive cue to relative depth: the accreting/deleting surface is interpreted as being behind adjacent surfaces. Under certain circumstances, however, accretion/deletion can be perceived in a radically different way: the accreting or deleting surface is seen as rotating in depth in front of adjacent surfaces. This alternative interpretation suggests a problem in conventional accounts of motion interpretation that cannot account for this phenomenon, in part because they ignore the role of contour geometry. In two experiments, we examined the combined role of contour convexity and accretion/deletion in determining the perception of relative depth by parametrically manipulating the strength of each cue. Our results show that convexity plays a more substantial role, often dominating the 3D percept, even in cases when the saliency of the convexity cue is substantially weakened on a contour where the texture was accreting/deleting at high rates. These results highlight the need for a rethinking of theories of perceptual organization in the critical case of moving stimuli.

When Is Accreting/Deleting Texture Seen as In Front? Interpretation of Depth From Texture Motion

Standard accounts of accretion/deletion of texture treat it as a definite cue to depth ordering: The accreting/deleting surface is interpreted as being behind the adjoining surface. Froyen, Feldman, and Singh showed that accretion/deletion can also, under certain circumstances, be perceived as a 3D column rotating in front, with the accretion/deletion explained by self-occlusion. These displays differ from traditional accretion/deletion displays in a number of factors, including the presence of figure/ground cues, accretion/deletion on both sides of boundaries, and in the number of distinct regions. In a series of experiments, we systematically manipulated each of these factors in order to determine what factors are actually instrumental in creating the rotating column (accretion/deletion in front) interpretation. In Experiment 1, the width of each region was kept fixed while manipulating the number of regions, and in Experiment 2, the width of the overall display was kept fixed. Observers indicated which set of regions they perceived to be in front. In both experiments, accreting/deleting regions were most likely to be seen in front when geometric figural cues favored a figural interpretation and when textural motion was introduced in all regions (rather than on just one side of each boundary). The number of regions had a relatively small effect (although this effect was larger in Experiment 2). These findings indicate that the geometry of the occluding contour is a critical factor in the interpretation of accretion/deleting, and future models of 3D interpretation involving accretion/deletion must include contour geometry as a key component.

Feature-Linking Model for Image Enhancement

Inspired by gamma-band oscillations and other neurobiological discoveries, neural networks research shifts the emphasis toward temporal coding, which uses explicit times at which spikes occur as an essential dimension in neural representations. We present a feature-linking model (FLM) that uses the timing of spikes to encode information. The first spiking time of FLM is applied to image enhancement, and the processing mechanisms are consistent with the human visual system. The enhancement algorithm achieves boosting the details while preserving the information of the input image. Experiments are conducted to demonstrate the effectiveness of the proposed method. Results show that the proposed method is effective.

Geometric figure-ground cues override standard depth from accretion-deletion

Accretion-deletion is widely considered a decisive cue to surface depth ordering, with the accreting or deleting surface interpreted as behind an adjoining surface. However, Froyen, Feldman, and Singh (2013) have shown that when accretion-deletion occurs on both sides of a contour, accreting-deleting regions can also be perceived as in front and as self-occluding due to rotation in three dimensions. In this study we ask whether geometric figure–ground cues can override the traditional “depth from accretion-deletion” interpretation even when accretion-deletion takes place only on one side of a contour. We used two tasks: a relative-depth task (front/back), and a motion-classification task (translation/rotation). We conducted two experiments, in which texture in only one set of alternating regions was moving; the other set was static. Contrary to the traditional interpretation of accretion-deletion, the moving convex and symmetric regions were perceived as figural and rotating in three dimensions in roughly half of the trials. In the second experiment, giving different motion directions to the moving regions (thereby weakening motion-based grouping) further weakened the traditional accretion-deletion interpretation. Our results show that the standard “depth from accretion-deletion” interpretation is overridden by static geometric cues to figure–ground. Overall, the results demonstrate a rich interaction between accretion-deletion, figure–ground, and structure from motion that is not captured by existing models of depth from motion.

Figure-ground modulation in awake primate thalamus

Figure-ground discrimination refers to the perception of an object, the figure, against a nondescript background. Neural mechanisms of figure-ground detection have been associated with feedback interactions between higher centers and primary visual cortex and have been held to index the effect of global analysis on local feature encoding. Here, in recordings from visual thalamus of alert primates, we demonstrate a robust enhancement of neuronal firing when the figure, as opposed to the ground, component of a motion-defined figure-ground stimulus is located over the receptive field. In this paradigm, visual stimulation of the receptive field and its near environs is identical across both conditions, suggesting the response enhancement reflects higher integrative mechanisms. It thus appears that cortical activity generating the higher-order percept of the figure is simultaneously reentered into the lowest level that is anatomically possible (the thalamus), so that the signature of the evolving representation of the figure is imprinted on the input driving it in an iterative process.

A neural model of border-ownership from kinetic occlusion

Camouflaged animals that have very similar textures to their surroundings are difficult to detect when stationary. However, when an animal moves, humans readily see a figure at a different depth than the background. How do humans perceive a figure breaking camouflage, even though the texture of the figure and its background may be statistically identical in luminance? We present a model that demonstrates how the primate visual system performs figure-ground segregation in extreme cases of breaking camouflage based on motion alone. Border-ownership signals develop as an emergent property in model V2 units whose receptive fields are nearby kinetically defined borders that separate the figure and background. Model simulations support border-ownership as a general mechanism by which the visual system performs figure-ground segregation, despite whether figure-ground boundaries are defined by luminance or motion contrast. The gradient of motion- and luminance-related border-ownership signals explains the perceived depth ordering of the foreground and background surfaces. Our model predicts that V2 neurons, which are sensitive to kinetic edges, are selective to border-ownership (magnocellular B cells). A distinct population of model V2 neurons is selective to border-ownership in figures defined by luminance contrast (parvocellular B cells). B cells in model V2 receive feedback from neurons in V4 and MT with larger receptive fields to bias border-ownership signals toward the figure. We predict that neurons in V4 and MT sensitive to kinetically defined figures play a crucial role in determining whether the foreground surface accretes, deletes, or produces a shearing motion with respect to the background.

Computing with a canonical neural circuits model with pool normalization and modulating feedback

Evidence suggests that the brain uses an operational set of canonical computations like normalization, input filtering, and response gain enhancement via reentrant feedback. Here, we propose a three-stage columnar architecture of cascaded model neurons to describe a core circuit combining signal pathways of feedforward and feedback processing and the inhibitory pooling of neurons to normalize the activity. We present an analytical investigation of such a circuit by first reducing its detail through the lumping of initial feedforward response filtering and reentrant modulating signal amplification. The resulting excitatory-inhibitory pair of neurons is analyzed in a 2D phase-space. The inhibitory pool activation is treated as a separate mechanism exhibiting different effects. We analyze subtractive as well as divisive (shunting) interaction to implement center-surround mechanisms that include normalization effects in the characteristics of real neurons. Different variants of a core model architecture are derived and analyzed--in particular, individual excitatory neurons (without pool inhibition), the interaction with an inhibitory subtractive or divisive (i.e., shunting) pool, and the dynamics of recurrent self-excitation combined with divisive inhibition. The stability and existence properties of these model instances are characterized, which serve as guidelines to adjust these properties through proper model parameterization. The significance of the derived results is demonstrated by theoretical predictions of response behaviors in the case of multiple interacting hypercolumns in a single and in multiple feature dimensions. In numerical simulations, we confirm these predictions and provide some explanations for different neural computational properties. Among those, we consider orientation contrast-dependent response behavior, different forms of attentional modulation, contrast element grouping, and the dynamic adaptation of the silent surround in extraclassical receptive field configurations, using only slight variations of the same core reference model.

Rotating columns: relating structure-from-motion, accretion/deletion, and figure/ground

We present a novel phenomenon involving an interaction between accretion deletion, figure-ground interpretation, and structure-from-motion. Our displays contain alternating light and dark vertical regions in which random-dot textures moved horizontally at constant speed but in opposite directions in alternating regions. This motion is consistent with all the light regions in front, with the dark regions completing amodally into a single large surface moving in the background, or vice versa. Surprisingly, the regions that are perceived as figural are also perceived as 3-D volumes rotating in depth (like rotating columns)-despite the fact that dot motion is not consistent with 3-D rotation. In a series of experiments, we found we could manipulate which set of regions is perceived as rotating volumes simply by varying known geometric cues to figure ground, including convexity, parallelism, symmetry, and relative area. Subjects indicated which colored regions they perceived as rotating. For our displays we found convexity to be a stronger cue than either symmetry or parallelism. We furthermore found a smooth monotonic decay of the proportion by which subjects perceive symmetric regions as figural, as a function of their relative area. Our results reveal an intriguing new interaction between accretion-deletion, figure-ground, and 3-D motion that is not captured by existing models. They also provide an effective tool for measuring figure-ground perception.