A neural network architecture for figure-ground separation of connected scenic figures

Neural dynamics of spreading attentional labels in mental contour tracing

Behavioral and neural data suggest that visual attention spreads along contour segments to bind them into a unified object representation. Such attentional labeling segregates the target contour from distractors in a process known as mental contour tracing. A recurrent competitive map is developed to simulate the dynamics of mental contour tracing. In the model, local excitation opposes global inhibition and enables enhanced activity to propagate on the path offered by the contour. The extent of local excitatory interactions is modulated by the output of the multi-scale contour detection network, which constrains the speed of activity spreading in a scale-dependent manner. Furthermore, an L-junction detection network enables tracing to switch direction at the L-junctions, but not at the X- or T-junctions, thereby preventing spillover to a distractor contour. Computer simulations reveal that the model exhibits a monotonic increase in tracing time as a function of the distance to be traced. Also, the speed of tracing increases with decreasing proximity to the distractor contour and with the reduced curvature of the contours. The proposed model demonstrated how an elaborated version of the winner-takes-all network can implement a complex cognitive operation such as contour tracing.

The time dimension for scene analysis

A fundamental issue in neural computation is the binding problem, which refers to how sensory elements in a scene organize into perceived objects, or percepts. The issue of binding is hotly debated in recent years in neuroscience and related communities. Much of the debate, however, gives little attention to computational considerations. This review intends to elucidate the computational issues that bear directly on the binding issue. The review starts with two problems considered by Rosenblatt to be the most challenging to the development of perceptron theory more than 40 years ago, and argues that the main challenge is the figure-ground separation problem, which is intrinsically related to the binding problem. The theme of the review is that the time dimension is essential for systematically attacking Rosenblatt's challenge. The temporal correlation theory as well as its special form--oscillatory correlation theory-is discussed as an adequate representation theory to address the binding problem. Recent advances in understanding oscillatory dynamics are reviewed, and these advances have overcome key computational obstacles for the development of the oscillatory correlation theory. We survey a variety of studies that address the scene analysis problem. The results of these studies have substantially advanced the capability of neural networks for figure-ground separation. A number of issues regarding oscillatory correlation are considered and clarified. Finally, the time dimension is argued to be necessary for versatile computing.

Recurrent network with large representational capacity

A recurrent network is proposed with the ability to bind image features into a unified surface representation within a single layer and without capacity limitations or border effects. A group of cells belonging to the same object or surface is labeled with the same activity amplitude, while cells in different groups are kept segregated due to lateral inhibition. Labeling is achieved by activity spreading through local excitatory connections. In order to prevent uncontrolled spreading, a separate network computes the intensity difference between neighboring locations and signals the presence of the surface boundary, which constrains local excitation. The quality of surface representation is not compromised due to the self-excitation. The model is also applied on gray-level images. In order to remove small, noisy regions, a feedforward network is proposed that computes the size of surfaces. Size estimation is based on the difference of dendritic inhibition in lateral excitatory and inhibitory pathways, which allows the network to selectively integrate signals only from cells with the same activity amplitude. When the output of the size estimation network is combined with the recurrent network, good segmentation results are obtained. Both networks are based on biophysically realistic mechanisms such as dendritic inhibition and multiplicative integration among different dendritic branches.

Using afterimages to test neural mechanisms for perceptual filling-in

Many theories of visual perception propose that brightness information spreads from edges to define the perceived intensity of the interior of visual surfaces. Several theories of visual perception have hypothesized that this filling-in process is similar to a diffusion of information where the signals coding brightness spread to nearest neighbors. This paper shows that diffusive mechanisms fail to account for the characteristics of certain afterimage percepts that seem to be dependent on the filling-in process. A psychophysical experiment tests a key property of diffusion-based filling-in mechanisms and finds data that rejects this class of models. A non-diffusive based filling-in mechanism is proposed and is shown to act much like the diffusive based mechanism in many instances, but also produces afterimage percepts that match the experimental data.

A dynamically coupled neural oscillator network for image segmentation

We propose a dynamically coupled neural oscillator network for image segmentation. Instead of pair-wise coupling, an ensemble of oscillators coupled in a local region is used for grouping. We introduce a set of neighborhoods to generate dynamical coupling structures associated with a specific oscillator. Based on the proximity and similarity principles, two grouping rules are proposed to explicitly consider the distinct cases of whether an oscillator is inside a homogeneous image region or near a boundary between different regions. The use of dynamical coupling makes our segmentation network robust to noise on an image, and unlike image processing algorithms no iterative operation is needed for noise removal. For fast computation, a segmentation algorithm is abstracted from the underlying oscillatory dynamics, and has been applied to synthetic and real images. Simulation results demonstrate the effectiveness of our oscillator network in image segmentation.

Temporal dynamics of binocular disparity processing with corticogeniculate interactions

A neural model is developed to probe how corticogeniculate feedback may contribute to the dynamics of binocular vision. Feedforward and feedback interactions among retinal, lateral geniculate, and cortical simple and complex cells are used to simulate psychophysical and neurobiological data concerning the dynamics of binocular disparity processing, including correct registration of disparity in response to dynamically changing stimuli, binocular summation of weak stimuli, and fusion of anticorrelated stimuli when they are delayed, but not when they are simultaneous. The model exploits dynamic rebounds between opponent ON and OFF cells that are due to imbalances in habituative transmitter gates. It shows how corticogeniculate feedback can carry out a top-down matching process that inhibits incorrect disparity responses and reduces persistence of previously correct responses to dynamically changing displays.

Neural dynamics of binocular brightness perception

How does the visual cortex combine information from both eyes to generate perceptual representations of object surfaces? Important clues about this process may be derived from data about the perceived brightness of surface regions under binocular viewing conditions, including data about binocular brightness summation in response to Ganzfelds, the U-shaped data of Fechner's paradox that violates binocular brightness summation, and the effects of different combinations of monocular and binocular contours and surface luminance differences on threshold sensitivity to monocular flashes of light. How to reconcile these apparently contradictory data properties has been a severe challenge to previous models, and none has explained them all. The present article quantitatively simulates them all by further developing the FACADE vision model. Key model processes discount the illuminant and compute image contrasts in each monocular channel using shunting on-center off-surround networks; binocularly fuse these discounted monocular signals using shunting on-center off-surround networks with nonlinear excitatory and inhibitory signals; and use these binocularly fused activities to trigger filling-in of a binocular surface representation that represents perceived surface brightness. Previous models that have suggested explanations of subsets of these data are discussed.

Object selection based on oscillatory correlation

One of the classical topics in neural networks is winner-take-all (WTA), which has been widely used in unsupervised (competitive) learning, cortical processing, and attentional control. Owing to global connectivity, WTA networks, however, do not encode spatial relations in the input, and thus cannot support sensory and perceptual processing where spatial relations are important. We propose a new architecture that maintains spatial relations between input features. This selection network builds on Locally Excitatory Globally Inhibitory Oscillator Networks (LEGION) dynamics and slow inhibition. In an input scene with many objects (patterns), the network selects the largest object. This system can be easily adjusted to select several largest objects, which then alternate in time. We analyze the speed of selection, and further show that a two-stage selection network gains efficiency by combining selection with parallel removal of noisy regions. The network is applied to select the most salient object in gray-level images. As a special case, the selection network without local excitation gives rise to a new form of oscillatory WTA.

A neural network for enhancing boundaries and surfaces in synthetic aperture radar images

A neural network system for boundary segmentation and surface representation, inspired by a new local-circuit model of visual processing in the cerebral cortex, is used to enhance images of range data gathered by a synthetic aperture radar (SAR) sensor. Boundary segmentation is accomplished by an improved Boundary Contour System (BCS) model which completes coherent boundaries that retain their sensitivity to image contrasts and locations. A Feature Contour System (FCS) model compensates for local contrast variations and uses the compensated signals to diffusively fill-in surface regions within the BCS boundaries. Image noise pixels that are not supported by BCS boundaries are hereby eliminated. More generally, BCS/FCS processing normalizes input dynamic range, reduces noise, and enhances contrasts between surface regions. BCS/FCS processing hereby makes structures such as motor vehicles, roads, and buildings more salient to human observers than in original imagery. The new BCS model improves image enhancement with significant reductions in processing time and complexity over previous BCS applications. The new system also outperforms several established techniques for image enhancement.

Image segmentation based on oscillatory correlation

We study the image segmentation on the basis of locally excitatory, globally inhibitory oscillator networks (LEGION), whereby the phases of oscillators encode the binding of pixels. We introduce a lateral potential for each oscillators so that only oscillators with strong connections from their neighborhood can develop high potentials. Based on the concept of the lateral potential, a solution to remove noisy regions in an image is proposed for LEGION, so that it suppresses the oscillators corresponding to noisy regions but without affecting those corresponding to major regions. We show that the resulting oscillator network separates an image into several major regions, plus a background consisting of all noisy regions, and we illustrate network properties by computer stimulation. The network exhibits a natural capacity in segmenting images. The oscillatory dynamics leads to a computer algorithm, which is applied successfully to segmenting real gray-level images. A number of issues regarding biological plausibility and perceptual organization are discussed. We argue that LEGION provides a novel and effective framework for image segmentation and figure-ground segregation.

Mechanisms of Neural Architecture for Visual Contrast and Brightness Perception

A neural architecture is proposed that serves as a framework for further empirical as well as theoretical investigations for a unified theory for contrast and brightness perception. The work further extends the brightness perception model developed by Grossberg and Todorovic. The proposed new computational architecture utilizes a (retinal) preprocessing stage with center-surround antagonisms of both polarities. The preprocessed data are shown to multiplex contrast as well as luminance information that can be de-multiplexed subsequently using a scheme of cross-channel interaction. Based on a hypothesized luminance-related channel, a three-stage process is suggested for brightness reconstruction. The separate channel for the representation of luminance-related information provides a key mechanism to assign the reconstructed brightness to an absolute reference level. The architecture provides a framework for the analysis of processes in brightness perception. Copyright 1996 Elsevier Science Ltd.

Cortical dynamics of boundary segmentation and reset: persistence, afterimages, and residual traces

In previous work with a neural-network model of boundary segmentation and reset, the percept of persistence was linked to the duration of a boundary segmentation after stimulus offset. In particular, the model simulated the decrease of persistence duration with an increase in stimulus duration and luminance. Further evidence is revealed for the neural mechanisms involved in the theory. Simulations show that the model reset signals generate orientational afterimages, such as the MacKay effect, when the reset signals can be grouped by a subsequent boundary segmentation that generates illusory contours through them. Simulations also show that the same mechanisms explain properties of residual traces, which increase in duration with stimulus duration and luminance. The model hereby discloses previously unsuspected mechanistic links between data about persistence and afterimages, and helps to clarify the sometimes controversial issues surrounding distinctions between persistence, residual traces, and afterimages.

Cortical dynamics of form and motion integration: persistence, apparent motion, and illusory contours

How does the visual system generate percepts of moving forms? How does this happen when the forms are emergent percepts, such as illusory contours or segregated textures, and the motion percept is apparent motion between the emergent forms? We develop a neural model of form-motion interactions to explain and simulate parametric properties of psychophysical motion data and to make predictions about how the parallel cortical processing streams V1-->MT and V1-->V2-->MT control form-motion interactions. The model explains how an illusory contour can move in apparent motion to another illusory contour or to a luminance-derived contour; how illusory contour persistence relates to the upper interstimulus interval (ISI) threshold for apparent motion; and how upper and lower ISI thresholds for seeing apparent motion between two flashes decrease with stimulus duration and narrow with spatial separation (Korte's laws). The model accounts for these data by suggesting how the persistence of a boundary segmentation in the V1-->V2 processing stream influences the quality of apparent motion in the V1-->MT stream through V2-->MT interactions. These data may all be explained by an analysis of how orientationally tuned form perception mechanisms and directionally tuned motion perception mechanisms interact.

Brightness perception, illusory contours, and corticogeniculate feedback

A neural network model is developed to explain how visual thalamocortical interactions give rise to boundary percepts such as illusory contours and surface percepts such as filled-in brightnesses. Top-down feedback interactions are needed in addition to bottom-up feed-forward interactions to simulate these data. One feedback loop is modeled between lateral geniculate nucleus (LGN) and cortical area V1, and another within cortical areas V1 and V2. The first feedback loop realizes a matching process which enhances LGN cell activities that are consistent with those of active cortical cells, and suppresses LGN activities that are not. This corticogeniculate feedback, being endstopped and oriented, also enhances LGN ON cell activations at the ends of thin dark lines, thereby leading to enhanced cortical brightness percepts when the lines group into closed illusory contours. The second feedback loop generates boundary representations, including illusory contours, that coherently bind distributed cortical features together. Brightness percepts form within the surface representations through a diffusive filling-in process that is contained by resistive gating signals from the boundary representations. The model is used to simulate illusory contours and surface brightness induced by Ehrenstein disks, Kanizsa squares, Glass patterns, and café wall patterns in single contrast, reverse contrast, and mixed contrast configurations. These examples illustrate how boundary and surface mechanisms can generate percepts that are highly context-sensitive, including how illusory contours can be amodally recognized without being seen, how model simple cells in V1 respond preferentially to luminance discontinuities using inputs from both LGN ON and OFF cells, how model bipole cells in V2 with two colinear receptive fields can help to complete curved illusory contours, how short-range simple cell groupings and long-range bipole cell groupings can sometimes generate different outcomes, and how model double-opponent, filling-in and boundary segmentation mechanisms in V4 interact to generate surface brightness percepts in which filling-in of enhanced brightness and darkness can occur before the net brightness distribution is computed by double-opponent interactions.

3-D vision and figure-ground separation by visual cortex

A neural network theory of three-dimensional (3-D) vision, called FACADE theory, is described. The theory proposes a solution of the classical figure-ground problem for biological vision. It does so by suggesting how boundary representations and surface representations are formed within a boundary contour system (BCS) and a feature contour system (FCS). The BCS and FCS interact reciprocally to form 3-D boundary and surface representations that are mutually consistent. Their interactions generate 3-D percepts wherein occluding and occluded object parts are separated, completed, and grouped. The theory clarifies how preattentive processes of 3-D perception and figure-ground separation interact reciprocally with attentive processes of spatial localization, object recognition, and visual search. A new theory of stereopsis is proposed that predicts how cells sensitive to multiple spatial frequencies, disparities, and orientations are combined by context-sensitive filtering, competition, and cooperation to form coherent BCS boundary segmentations. Several factors contribute to figure-ground pop-out, including: boundary contrast between spatially contiguous boundaries, whether due to scenic differences in luminance, color, spatial frequency, or disparity; partially ordered interactions from larger spatial scales and disparities to smaller scales and disparities; and surface filling-in restricted to regions surrounded by a connected boundary. Phenomena such as 3-D pop-out from a 2-D picture, Da Vinci stereopsis, 3-D neon color spreading, completion of partially occluded objects, and figure-ground reversals are analyzed. The BCS and FCS subsystems model aspects of how the two parvocellular cortical processing streams that join the lateral geniculate nucleus to prestriate cortical area V4 interact to generate a multiplexed representation of Form-And-Color-And-DEpth, or FACADE, within area V4. Area V4 is suggested to support figure-ground separation and to interact with cortical mechanisms of spatial attention, attentive object learning, and visual search. Adaptive resonance theory (ART) mechanisms model aspects of how prestriate visual cortex interacts reciprocally with a visual object recognition system in inferotemporal (IT) cortex for purposes of attentive object learning and categorization. Object attention mechanisms of the What cortical processing stream through IT cortex are distinguished from spatial attention mechanisms of the Where cortical processing stream through parietal cortex. Parvocellular BCS and FCS signals interact with the model What stream. Parvocellular FCS and magnocellular motion BCS signals interact with the model Where stream.(ABSTRACT TRUNCATED AT 400 WORDS)