Figure-ground segregation at contours: a neural mechanism in the visual cortex of the alert monkey

Figure-ground segmentation based on motion in the archerfish

Figure-ground segmentation is a fundamental process in visual perception that involves separating visual stimuli into distinct meaningful objects and their surrounding context, thus allowing the brain to interpret and understand complex visual scenes. Mammals exhibit varying figure-ground segmentation capabilities, ranging from primates that can perform well on figure-ground segmentation tasks to rodents that perform poorly. To explore figure-ground segmentation capabilities in teleost fish, we studied how the archerfish, an expert visual hunter, performs figure-ground segmentation. We trained archerfish to discriminate foreground objects from the background, where the figures were defined by motion as well as by discontinuities in intensity and texture. Specifically, the figures were defined by grating, naturalistic texture, and random noise moving in counterphase with the background. The archerfish performed the task well and could distinguish between all three types of figures and grounds. Their performance was comparable to that of primates and outperformed rodents. These findings suggest the existence of a complex visual process in the archerfish visual system that enables the delineation of figures as distinct from backgrounds, and provide insights into object recognition in this animal.

Nonsymbolic numerosity in sets with illusory-contours exploits a context-sensitive, but contrast-insensitive, visual boundary formation process

The visual mechanisms underlying approximate numerical representation are still intensely debated because numerosity information is often confounded with continuous sensory cues (e.g., texture density, area, convex hull). However, numerosity is underestimated when a few items are connected by illusory contours (ICs) lines without changing other physical cues, suggesting in turn that numerosity processing may rely on discrete visual input. Yet, in these previous works, ICs were generated by black-on-gray inducers producing an illusory brightness enhancement, which could represent a further continuous sensory confound. To rule out this possibility, we tested participants in a numerical discrimination task in which we manipulated the alignment of 0, 2, or 4 pairs of open/closed inducers and their contrast polarity. In Experiment 1, aligned open inducers had only one polarity (all black or all white) generating ICs lines brighter or darker than the gray background. In Experiment 2, open inducers had always opposite contrast polarity (one black and one white inducer) generating ICs without strong brightness enhancement. In Experiment 3, reverse-contrast inducers were aligned but closed with a line preventing ICs completion. Results showed that underestimation triggered by ICs lines was independent of inducer contrast polarity in both Experiment 1 and Experiment 2, whereas no underestimation was found in Experiment 3. Taken together, these results suggest that mere brightness enhancement is not the primary cause of the numerosity underestimation induced by ICs lines. Rather, a boundary formation mechanism insensitive to contrast polarity may drive the effect, providing further support to the idea that numerosity processing exploits discrete inputs.

Motion trajectories and object properties influence perceived direction of motion

Judging the motion of objects is a fundamental task that the visual system executes in everyday life in order for us to navigate and interact safely with our surroundings. A number of strategies have been suggested to explain how the visual system uses motion information from different points of an object to compute veridical directions of motion. These include combining ambiguous signals from object contours via a vector summation (VS) or intersection of constraints (IOC) calculation, pooling information using a maximum likelihood or tracking object features. We measured the perceived direction of motion for a range of cross-shaped stimuli (composed of two superimposed lines) to test how accurately humans perceive their motion and compared data to predictions from these strategies. Crosses of different shapes (defined by the angle between the component lines) translated along 16 directions of motion with constant speed. The crosses either moved along one of their symmetry axes (balanced conditions with line components equidistant to the direction of motion) or had their symmetry axis tilted relative to the motion (unbalanced conditions) Data show reproducible differences between observers, including occasional bimodal behaviour, and exhibit the following common patterns. There is a general dependence on direction of motion: For all conditions, when motion is along cardinal axes (horizontal and vertical), perception is largely veridical. For non-cardinal directions, biases are typically small (<10 deg) when crosses are balanced but large biases occur (≥30 deg) when crosses are tilted relative to their direction of motion. Factors influencing the pattern of biases are the shape and tilt of the cross as well as the proximity of its direction of motion to cardinal axes. The dependence of the biases on the direction of motion is inconsistent with any isotropic mechanisms including VS, IOC, maximum likelihood or feature tracking. Instead, perception is biased by a number of intrinsic properties of the cross and external references. The strength of these cues depends on the type, with elongation producing the strongest weight, and their proximity to the direction of motion. This suggests that the visual system may rely on a number of static cues to improve the known low precision for non-cardinal directions of motion, a process which can, however, result in large perceptual biases in certain circumstances.

The "side" matters: how configurality is reflected in completion

The perception of figure-ground organization is a highly context-sensitive phenomenon. Accumulating evidence suggests that the so-called completion phenomenon is tightly linked to this figure-ground organization. While many computational models have applied borderline completion algorithms based on the detection of boundary alignments, we point out the problems of this approach. We hypothesize that completion is a result of computing the figure-ground organization. Specifically, the global interactions in the neural network activate the "border-ownership" sensitive neurons at the location where no luminance contrast is given and this activation corresponds to the perception of illusory contours. The implications of this result to the general property of emerging Gestalt percepts are discussed.

Contributions of indirect pathways to visual response properties in macaque middle temporal area MT

The primate visual cortex exhibits a remarkable degree of interconnectivity. Each visual area receives an average of 10 to 15 inputs, many of them from cortical areas with overlapping, but not identical, functional properties. In this study, we assessed the functional significance of this anatomical parallelism to the middle temporal area (MT) of the macaque visual cortex. MT receives major feedforward inputs from areas V1, V2, and V3, but little is known about the properties of each of these pathways. We previously demonstrated that reversible inactivation of V2 and V3 causes a disproportionate degradation of tuning for binocular disparity of MT neurons, relative to direction tuning (Ponce et al., 2008). Here we show that MT neurons continued to encode speed and size information during V2/3 inactivation; however, many became significantly less responsive to fast speeds and others showed a modest decrease in surround suppression. These changes resemble previously reported effects of reducing stimulus contrast (Pack et al., 2005; Krekelberg et al., 2006), but we show here that they differ in their temporal dynamics. We find no evidence that the indirect pathways selectively target different functional regions within MT. Overall, our findings suggest that the indirect pathways to MT primarily convey modality-specific information on binocular disparity, but that they also contribute to the processing of stimuli moving at fast speeds.

Neural computations underlying depth perception

Neural mechanisms underlying depth perception are reviewed with respect to three computational goals: determining surface depth order, gauging depth intervals, and representing 3D surface geometry and object shape. Accumulating evidence suggests that these three computational steps correspond to different stages of cortical processing. Early visual areas appear to be involved in depth ordering, while depth intervals, expressed in terms of relative disparities, are likely represented at intermediate stages. Finally, 3D surfaces appear to be processed in higher cortical areas, including an area in which individual neurons encode 3D surface geometry, and a population of these neurons may therefore represent 3D object shape. How these processes are integrated to form a coherent 3D percept of the world remains to be understood.

Global visual processing in macaques studied using Kanizsa illusory shapes

The ability to extract form information from a visual scene, for object recognition or figure-ground segregation, is a fundamental visual system function. Many studies of nonhuman primates have addressed the neural mechanisms involved in global form processing, but few have sought to demonstrate this ability behaviorally. In this study, we probed global visual processing in macaque monkeys (Macaca nemestrina) using classical Kanizsa illusory shapes as an assay of global form perception. We trained three monkeys on a "similarity match-to-sample" form discrimination task, first with complete forms embedded in fields of noncontour-inducing "pacman" elements. We then tested them with classic Kanizsa illusory shapes embedded in fields of randomly oriented elements. Two of the three subjects reached our criterion performance level of 80% correct or better on four of five illusory test conditions, demonstrating clear evidence of Kanizsa illusory form perception; the third subject mastered three of five conditions. Performance limits for illusory form discrimination were obtained by manipulating support ratio and by measuring threshold for discriminating "fat" and "thin" illusory squares. Our results indicate that macaque monkeys are capable of global form processing similarly to humans and that the perceptual mechanisms for "filling-in" contour gaps exist in macaques as they do in humans.

Early stages of figure-ground segregation during perception of the face-vase

The temporal sequence of neural processes supporting figure-ground perception was investigated by recording ERPs associated with subjects' perceptions of the face-vase figure. In Experiment 1, subjects continuously reported whether they perceived the face or the vase as the foreground figure by pressing one of two buttons. Each button press triggered a probe flash to the face region, the vase region, or the borders between the two. The N170/vertex positive potential (VPP) component of the ERP elicited by probes to the face region was larger when subjects perceived the faces as figure. Preceding the N170/VPP, two additional components were identified. First, when the borders were probed, ERPs differed in amplitude as early as 110 msec after probe onset depending on subjects' figure-ground perceptions. Second, when the face or vase regions were probed, ERPs were more positive (at ∼ 150-200 msec) when that region was perceived as figure versus background. These components likely reflect an early "border ownership" stage, and a subsequent "figure-ground segregation" stage of processing. To explore the influence of attention on these stages of processing, two additional experiments were conducted. In Experiment 2, subjects selectively attended to the face or vase region, and the same early ERP components were again produced. In Experiment 3, subjects performed an identical selective attention task, but on a display lacking distinctive figure-ground borders, and neither of the early components were produced. Results from these experiments suggest sequential stages of processing underlying figure-ground perception, each which are subject to modifications by selective attention.

A neural model of the temporal dynamics of figure-ground segregation in motion perception

How does the visual system manage to segment a visual scene into surfaces and objects and manage to attend to a target object? Based on psychological and physiological investigations, it has been proposed that the perceptual organization and segmentation of a scene is achieved by the processing at different levels of the visual cortical hierarchy. According to this, motion onset detection, motion-defined shape segregation, and target selection are accomplished by processes which bind together simple features into fragments of increasingly complex configurations at different levels in the processing hierarchy. As an alternative to this hierarchical processing hypothesis, it has been proposed that the processing stages for feature detection and segregation are reflected in different temporal episodes in the response patterns of individual neurons. Such temporal epochs have been observed in the activation pattern of neurons as low as in area V1. Here, we present a neural network model of motion detection, figure-ground segregation and attentive selection which explains these response patterns in an unifying framework. Based on known principles of functional architecture of the visual cortex, we propose that initial motion and motion boundaries are detected at different and hierarchically organized stages in the dorsal pathway. Visual shapes that are defined by boundaries, which were generated from juxtaposed opponent motions, are represented at different stages in the ventral pathway. Model areas in the different pathways interact through feedforward and modulating feedback, while mutual interactions enable the communication between motion and form representations. Selective attention is devoted to shape representations by sending modulating feedback signals from higher levels (working memory) to intermediate levels to enhance their responses. Areas in the motion and form pathway are coupled through top-down feedback with V1 cells at the bottom end of the hierarchy. We propose that the different temporal episodes in the response pattern of V1 cells, as recorded in recent experiments, reflect the strength of modulating feedback signals. This feedback results from the consolidated shape representations from coherent motion patterns and the attentive modulation of responses along the cortical hierarchy. The model makes testable predictions concerning the duration and delay of the temporal episodes of V1 cell responses as well as their response variations that were caused by modulating feedback signals.

Phenomenology and neurophysiological correlations: two approaches to perception research

This article argues that phenomenological description and neurophysiological correlation complement each other in perception research. Whilst phenomena constitute the material, neuronal mechanisms are indispensable for their explanation. Numerous examples of neurophysiological correlates show that the correlation of phenomenology and neurophysiology is fruitful. Phenomena for which neuronal mechanism have been found include: (in area V1) filling-in of real and artificial scotomata, contour integration, figure-ground segregation by orientation contrast, amodal completion, and motion transparency; (in V2) modal completion, border ownership, surface transparency, and cyclopean perception; (in V3) alignment in dotted contours, and filling-in with dynamic texture; (in V4) colour constancy; (in MT) shape by accretion/deletion, grouping by coherent motion, apparent motion in motion quartets, motion in apertures, and biological motion. Results suggest that in monkey visual cortex, occlusion cues, including stereo depth, are predominantly processed in lower areas, whereas mechanisms for grouping and motion are primarily represented in higher areas. More correlations are likely to emerge as neuroscientists strive for a better understanding of visual perception. The paper concludes with a review of major achievements in visual neuroscience pertinent to the study of the phenomena under consideration.

Globally consistent depth sorting of overlapping 2D surfaces in a model using local recurrent interactions

The human visual system utilizes depth information as a major cue to group together visual items constituting an object and to segregate them from items belonging to other objects in the visual scene. Depth information can be inferred from a variety of different visual cues, such as disparity, occlusions and perspective. Many of these cues provide only local and relative information about the depth of objects. For example, at occlusions, T-junctions indicate the local relative depth precedence of surface patches. However, in order to obtain a globally consistent interpretation of the depth relations between the surfaces and objects in a visual scene, a mechanism is necessary that globally propagates such local and relative information. We present a computational framework in which depth information derived from T-junctions is propagated along surface contours using local recurrent interactions between neighboring neurons. We demonstrate that within this framework a globally consistent depth sorting of overlapping surfaces can be obtained on the basis of local interactions. Unlike previous approaches in which locally restricted cell interactions could merely distinguish between two depths (figure and ground), our model can also represent several intermediate depth positions. Our approach is an extension of a previous model of recurrent V1-V2 interaction for contour processing and illusory contour formation. Based on the contour representation created by this model, a recursive scheme of local interactions subsequently achieves a globally consistent depth sorting of several overlapping surfaces. Within this framework, the induction of illusory contours by the model of recurrent V1-V2 interaction gives rise to the figure-ground segmentation of illusory figures such as a Kanizsa square.

Higher order visual processing in macaque extrastriate cortex

The extrastriate cortex of primates encompasses a substantial portion of the cerebral cortex and is devoted to the higher order processing of visual signals and their dispatch to other parts of the brain. A first step towards the understanding of the function of this cortical tissue is a description of the selectivities of the various neuronal populations for higher order aspects of the image. These selectivities present in the various extrastriate areas support many diverse representations of the scene before the subject. The list of the known selectivities includes that for pattern direction and speed gradients in middle temporal/V5 area; for heading in medial superior temporal visual area, dorsal part; for orientation of nonluminance contours in V2 and V4; for curved boundary fragments in V4 and shape parts in infero-temporal area (IT); and for curvature and orientation in depth from disparity in IT and CIP. The most common putative mechanism for generating such emergent selectivity is the pattern of excitatory and inhibitory linear inputs from the afferent area combined with nonlinear mechanisms in the afferent and receiving area.

Neuronal response to texture- and contrast-defined boundaries in early visual cortex

Natural scenes contain a variety of visual cues that facilitate boundary perception (e.g., luminance, contrast, and texture). Here we explore whether single neurons in early visual cortex can process both contrast and texture cues. We recorded neural responses in cat A18 to both illusory contours formed by abutting gratings (ICs, texture-defined) and contrast-modulated gratings (CMs, contrast-defined). We found that if a neuron responded to one of the two stimuli, it also responded to the other. These neurons signaled similar contour orientation, spatial frequency, and movement direction of the two stimuli. A given neuron also exhibited similar selectivity for spatial frequency of the fine, stationary grating components (carriers) of the stimuli. These results suggest that the cue-invariance of early cortical neurons extends to different kinds of texture or contrast cues, and might arise from a common nonlinear mechanism.

Glass pattern responses in macaque V2 neurons

Area V2 of macaque visual cortex is known to respond well to conventional oriented bar and grating stimuli, but some recent physiological data have shown that it may play an important role in coding more complicated patterns. Most of these data come from testing done with stimuli presented within the classical receptive field (CRF), whereas relatively little attention has been paid to the role played by the extra-classical surround. We have previously shown that neurons in primary visual cortex (V1) respond to translational Glass patterns in a manner that is predictable from their responses to grating stimuli. In this article, we first extend our experiments and modeling of Glass pattern responses in V1 to include V2. We explored the sensitivity of V2 cells to global form cues in Glass patterns confined to the CRF. Our results indicate that V2 neurons respond to the local signals in Glass patterns in a manner similar to V1 and that those responses are not influenced by global form present in the surround. It appears that the coding of the more complicated global structure in Glass patterns takes place further downstream in the visual system.

Functional neuroimaging findings on the human perception of illusory contours

Illusory contours (IC) have attracted a considerable interest in recent years to derive models of how sensory information is processed and integrated within the visual system. In addition to various findings from neuropsychology, neurophysiology, and psychophysics, several recent studies have used functional neuroimaging to identify the cerebral substrates underlying human perception of IC (in particular Kanizsa figures). In this paper, we review the results from more than 20 neuroimaging studies on IC perception and highlight the great diversity of findings across these studies. We then provide a detailed discussion about the localization ('where' debate) and the timing ('when' debate) of IC processing as suggested by functional neuroimaging. Cortical responses involving visual areas as early as V1/V2 and latencies as rapid as 100 ms have been reported in several studies. Particular issues concerning the role of the right hemisphere and the retinotopic encoding of IC are also discussed. These different findings are tentatively brought together to propose different hypothetical cortical mechanisms that might be responsible for the visual formation of IC. Several remaining questions on IC processing that could potentially be explored with functional neuroimaging techniques are finally emphasized.

Figure and ground in the visual cortex: v2 combines stereoscopic cues with gestalt rules

Figure-ground organization is a process by which the visual system identifies some image regions as foreground and others as background, inferring 3D layout from 2D displays. A recent study reported that edge responses of neurons in area V2 are selective for side-of-figure, suggesting that figure-ground organization is encoded in the contour signals (border ownership coding). Here, we show that area V2 combines two strategies of computation, one that exploits binocular stereoscopic information for the definition of local depth order, and another that exploits the global configuration of contours (Gestalt factors). These are combined in single neurons so that the "near" side of the preferred 3D edge generally coincides with the preferred side-of-figure in 2D displays. Thus, area V2 represents the borders of 2D figures as edges of surfaces, as if the figures were objects in 3D space. Even in 3D displays, Gestalt factors influence the responses and can enhance or null the stereoscopic depth information.

A common neuronal code for perceptual processes in visual cortex? Comparing choice and attentional correlates in V5/MT

In the past two decades, sensory neuroscience has moved from describing response properties to external stimuli in cerebral cortex to establishing connections between neuronal activity and sensory perception. The seminal studies by Newsome, Movshon and colleagues in the awake behaving macaque firmly link single cells in extrastriate area V5/MT and perception of motion. A decade later, extrastriate visual cortex appears awash with neuronal correlates for many different perceptual tasks. Examples are attentional signals, choice signals for ambiguous images, correlates for binocular rivalry, stereo and shape perception, and so on. These diverse paradigms are aimed at elucidating the neuronal code for perceptual processes, but it has been little studied how they directly compare or even interact. In this paper, I explore to what degree the measured neuronal signals in V5/MT for choice and attentional paradigms might reflect a common neuronal mechanism for visual perception.

Spatiotemporal analysis of event-related potentials to upright, inverted, and contrast-reversed faces: Effects on encoding and recognition

In an n-back face recognition task where subjects responded to repeated stimuli, ERPs were recorded to upright, inverted, and contrast-reversed faces. The effects of inversion and contrast reversal on face encoding and recognition were investigated using the multivariate spatiotemporal partial least squares (PLS) analysis. The configural manipulations affected early processing (100-200 ms) at posterior sites: Inversion effects were parietal and lateral, whereas contrast-reversal effects were more occipital and medial, suggesting different underlying generators. A later reactivation of face processing areas was unique to inverted faces, likely due to processing difficulties. PLS also indicated that the "old-new" repetition effect was maximal for upright faces and likely involved frontotemporal areas. Marked processing differences between inverted and contrast-reversed faces were seen, but these effects were similar at encoding and recognition.

Cortical activation to illusory shapes as measured with magnetoencephalography

Spatiotemporal patterns of cortical activation during the perceptual grouping of elements to form illusory shapes were estimated using anatomically constrained magnetoencephalography. Subjects were shown an array of Kanizsa-style figures which were either aligned to form illusory squares or misaligned so that no illusory contour or shape was perceived. Differential activity is more pronounced in the right hemisphere. After a weakly significant modulation at approximately 110 ms in the occipital pole, a prominent peak appears at approximately 155 ms in the lateral occipital cortex. Modulation then appears to spread back from this location toward the occipital pole, as well as ventrally to involve ventral occipital and temporal cortices for the next 180 ms, eventually involving ventral orbitofrontal cortex at 325 ms. The prominent lateral occipital response is consistent with fMRI studies with similar stimuli which found activation in that region as well as in V3A, V4v, V7, and V8. Furthermore, the timing of this activation, after the occipital pole but before ventral temporal, is consistent with a putative role for this region in midlevel vision. The late ventral temporal response (235 ms) is centered in the lingual and fusiform areas implicated in object identification. The V1/V2 modulation at this time may reflect top-down modulation by lateral occipitotemporal and ventral temporal areas.

Contour detection based on nonclassical receptive field inhibition

We propose a biologically motivated method, called nonclassical receptive field (non-CRF) inhibition (more generally, surround inhibition or suppression), to improve contour detection in machine vision. Non-CRF inhibition is exhibited by 80% of the orientation-selective neurons in the primary visual cortex of monkeys and has been shown to influence human visual perception as well. Essentially, the response of an edge detector at a certain point is suppressed by the responses of the operator in the region outside the supported area. We combine classical edge detection with isotropic and anisotropic inhibition, both of which have counterparts in biology. We also use a biologically motivated method (the Gabor energy operator) for edge detection. The resulting operator responds strongly to isolated lines, edges, and contours, but exhibits weak or no response to edges that are part of texture. We use natural images with associated ground truth contour maps to assess the performance of the proposed operator for detecting contours while suppressing texture edges. Our method enhances contour detection in cluttered visual scenes more effectively than classical edge detectors used in machine vision (Canny edge detector). Therefore, the proposed operator is more useful for contour-based object recognition tasks, such as shape comparison, than traditional edge detectors, which do not distinguish between contour and texture edges. Traditional edge detection algorithms can, however, also be extended with surround suppression. This study contributes also to the understanding of inhibitory mechanisms in biology.

Contextual influences on visual processing

The visual image formed on the retina represents an amalgam of visual scene properties, including the reflectances of surfaces, their relative positions, and the type of illumination. The challenge facing the visual system is to extract the "meaning" of the image by decomposing it into its environmental causes. For each local region of the image, that extraction of meaning is only possible if information from other regions is taken into account. Of particular importance is a set of image cues revealing surface occlusion and/or lighting conditions. These information-rich cues direct the perceptual interpretation of other more ambiguous image regions. This context-dependent transformation from image to perception has profound-but frequently under-appreciated-implications for neurophysiological studies of visual processing: To demonstrate that neuronal responses are correlated with perception of visual scene properties, rather than visual image features, neuronal sensitivity must be assessed in varied contexts that differentially influence perceptual interpretation. We review a number of recent studies that have used this context-based approach to explore the neuronal bases of visual scene perception.

Stereoscopic illusory contours--cortical neuron responses and human perception

In human perception, figure-ground segregation suggests that stereoscopic cues are grouped over wide areas of the visual field. For example, two abutting rectangles of equal luminance and size are seen as a uniform surface when presented at the same depth, but appear as two surfaces separated by an illusory contour and a step in depth when presented with different retinal disparities. Here, we describe neurons in the monkey visual cortex that signal such illusory contours and can be selective for certain figure-ground directions that human observers perceive at these contours. The results suggest that these neurons group stereoscopic cues over distances up to 8 degrees. In addition, we compare these results with human perception and show that the mean stimulus parameters required by these neurons also induce optimal percepts of illusory contours in human observers.

The spatiotemporal dynamics of illusory contour processing: combined high-density electrical mapping, source analysis, and functional magnetic resonance imaging

Because environmental information is often suboptimal, visual perception must frequently rely on the brain's reconstruction of contours absent from retinal images. Illusory contour (IC) stimuli have been used to investigate these "filling-in" processes. Intracranial recordings and neuroimaging studies show IC sensitivity in lower-tier area V2, and to a lesser extent V1. Some interpret these data as evidence for feedforward processing of IC stimuli, beginning at lower-tier visual areas. On the basis of lesion, visual evoked potentials (VEP), and neuroimaging evidence, others contend that IC sensitivity is a later, higher-order process. Whether IC sensitivity seen in lower-tier areas indexes feedforward or feedback processing remains unresolved. In a series of experiments, we addressed the spatiotemporal dynamics of IC processing. Centrally presented IC stimuli resulted in early VEP modulation (88-100 msec) over lateral-occipital (LOC) scalp--the IC effect. The IC effect followed visual response onset by 40 msec. Scalp current density topographic mapping, source analysis, and functional magnetic resonance imaging results all localized the IC effect to bilateral LOC areas. We propose that IC sensitivity described in V2 and V1 may reflect predominantly feedback modulation from higher-tier LOC areas, where IC sensitivity first occurs. Two additional observations further support this proposal. The latency of the IC effect shifted dramatically later (approximately 120 msec) when stimuli were laterally presented, indicating that retinotopic position alters IC processing. Immediately preceding the IC effect, the VEP modulated with inducer eccentricity--the configuration effect. We interpret this to represent contributions from global stimulus parameters to scene analysis. In contrast to the IC effect, the topography of the configuration effect was restricted to central parieto-occipital scalp.

The Role of the Immune System in Psychology and Neuroscience

The immune system and the nervous system are connected in a dynamic network that has important implications for psychology. First, if analyzed in functional terms, the immune system and the nervous system are not distinctly separated: The immune system can take part in neuronal signaling through the production of an array of transmitters and hormones, and brain cells can process and present antigen and produce immune proteins. Moreover, synapses seem to be formed between immune and neural cells in lymphoid tissue. The basis of this phenomenon may be a common evolutionary background of physiological systems that by tradition have been viewed as discrete rather than overlapping. Second, the immune system is actively involved in homeostatic regulation. Signals from immune cells can profoundly change the physiological state of the organism, with changes observed in metabolism, stress axes activity, behavior, motivation and cognition. Many of these changes have probably evolved to ease recuperation. Third, the activity in the immune system is dependent of homeostatic regulation by the neuroendocrine system in a biologically important network that is also capable of mediating psychological impact on immunity. In this review, it is argued that immunology should be ecological in nature and thereby related to psychological and neural science. Hypothetically, an ecological immunology will show cross-fertilizing properties, increasing the explanatory power of the seemingly disparate scientific disciplines involved.

Simulation of neuronal responses defining depth order and contrast polarity at illusory contours in monkey area V2

Neurophysiological, brain imaging, and perceptual studies in animals and humans suggest that illusory (occluding) contours are represented at an early level of visual cortical processing. Comparatively little is known about the mechanisms defining the depth order and the brightness illusion associated with such contours. Baumann et al. (1997) found neurons in area V2 of the alert monkey that signaled not only illusory contours but also the figure-ground direction that human observers perceive at such contours. The majority of these neurons showed this property independent stimulus contrast; a small minority preferred a certain combination of figure-ground direction and contrast polarity at these contours. In this article, we simulate the responses of these neurons by means of a grouping mechanism that uses occlusion cues (line-ends, corners) to define figure-ground direction and contrast polarity at such contours.

Real and illusory contour processing in area V1 of the primate: a cortical balancing act

It is known that neurons in area V2 (the second visual area) can signal the orientation of illusory contours in the primate. Whether area V1 (primary visual cortex) can signal illusory contour orientation is more controversial. While some electrophysiology studies have ruled out illusory signaling in V1, other reports suggest that V1 shows some illusory-specific response. Here, using optical imaging and single unit electrophysiology, we report that primate V1 does show an orientation-specific response to the 'abutting line grating' illusory contour. However, this response does not signal an illusory contour in the conventional sense. Rather, we find that illusory contour stimulation leads to an activation map that, after appropriate subtraction of real line signal, is inversely related to the real orientation map. The illusory contour orientation is thus negatively signaled or de-emphasized in V1. This 'activation reversal' is robust, is not due merely to presence of line ends, is not dependent on inducer orientation, and is not due to precise position of line end stimulation of V1 cells. These data suggest a resolution for previous apparently contradictory experimental findings. We propose that the de-emphasis of illusory contour orientation in V1 may be an important signal of contour identity and may, together with illusory signal from V2, provide a unique signature for illusory contour representation.

Object-based vision and attention in primates

In forming a representation of a visible object, the brain must analyze the visual scene pre-attentively, select an object through active attention, and form representations of the multiple attributes of the selected object. During the past two years, progress has been made in understanding the neural underpinnings of these processes by means of single-neuron recording in monkeys.

Anatomy and physiology of a neural mechanism defining depth order and contrast polarity at illusory contours

We studied the anatomy and physiology of neurons in monkey visual cortex, which contribute to mechanisms segregating figure and ground at contours based on information provided by occlusion cues. First, we defined the location of neurons sensitive to occluding (illusory) contours. These neurons were found most frequently in the pale cytochrome oxidase stripes of area V2 but rarely in V1. In area V2, they were found in all laminae and with similar frequencies. The few neurons recorded in area V1 concentrated in the upper laminae. Second, we studied the properties and anatomical location of neurons sensitive to occlusion cues (dark and light line-ends, corners). These neurons had end-stopped receptive fields and were found with similar frequencies in both areas. In area V1, they concentrated in the upper laminae. In area V2, they were found in all laminae and cytochrome oxidase stripes. These neurons responded to short stimuli of optimal length (bars, edges) and to stimuli terminating in their receptive field (line-ends, corners). Overall, about half of these neurons detected the direction of such terminations and about 60% were selective for certain types of termination. In summary, our results suggest that in monkey visual cortex, occlusion cues are represented in areas V1 and V2, whereas grouping mechanisms detecting occluding contours concentrate in area V2.

Coding of border ownership in monkey visual cortex

Areas V1 and V2 of the visual cortex have traditionally been conceived as stages of local feature representations. We investigated whether neural responses carry information about how local features belong to objects. Single-cell activity was recorded in areas V1, V2, and V4 of awake behaving monkeys. Displays were used in which the same local feature (contrast edge or line) could be presented as part of different figures. For example, the same light-dark edge could be the left side of a dark square or the right side of a light square. Each display was also presented with reversed contrast. We found significant modulation of responses as a function of the side of the figure in >50% of neurons of V2 and V4 and in 18% of neurons of the top layers of V1. Thus, besides the local contrast border information, neurons were found to encode the side to which the border belongs ("border ownership coding"). A majority of these neurons coded border ownership and the local polarity of luminance-chromaticity contrast. The others were insensitive to contrast polarity. Another 20% of the neurons of V2 and V4, and 48% of top layer V1, coded local contrast polarity, but not border ownership. The border ownership-related response differences emerged soon (<25 msec) after the response onset. In V2 and V4, the differences were found to be nearly independent of figure size up to the limit set by the size of our display (21 degrees ). Displays that differed only far outside the conventional receptive field could produce markedly different responses. When tested with more complex displays in which figure-ground cues were varied, some neurons produced invariant border ownership signals, others failed to signal border ownership for some of the displays, but neurons that reversed signals were rare. The influence of visual stimulation far from the receptive field center indicates mechanisms of global context integration. The short latencies and incomplete cue invariance suggest that the border-ownership effect is generated within the visual cortex rather than projected down from higher levels.

Representation of stereoscopic edges in monkey visual cortex

Form perception in random-dot stereograms is based on information that resides in the correlation between the two images, but is not present in either image alone. We have studied the coding of stereoscopic figures in the neural activity of areas V1 and V2 of alert behaving monkeys. While cells in V1 generally responded according to the disparity of the surface at the receptive field, we found cells in area V2 that responded selectively to the figure edges. These cells signaled the location and orientation of contrast borders as well as stereoscopic edges, and were often selective for the direction of the step in depth. We concluded that stereoscopic edges are explicitly represented in area V2.

Illusory contours and spatial judgment

We investigated whether, in the human visual system, the mechanisms responsible for relative location judgments are the same when those judgments are made in the context of illusory contours and in the context of mentally joining two points. We asked subjects to align a dot with the oblique contour of an illusory surface or to align a dot with two markers at an oblique orientation. The systematic errors differed in direction for these two conditions. All the systematic errors were orientation dependent. The errors in aligning a dot with an illusory contour seem to be related to the asymmetrical shape of the single objects, which are able to induce an illusory contour, as well as figure-ground segregation.

Moving illusory contours activate primary visual cortex: an fMRI study

Identifying the cortical areas activated by illusory contours provides valuable information on the mechanisms of object perception. We applied functional magnetic resonance imaging to identify the visual areas of the human brain involved in the perception of a moving Kanizsa-type illusory contour. Our results indicate that, in addition to other cortical regions, areas V5 and V1 are activated. Activity in area V1 was particularly prominent.

From elements to perception: local and global processing in visual neurons

Gestalt psychologists in the early part of the century challenged psychophysical notions that perceptual phenomena can be understood from a punctate (atomistic) analysis of the elements present in the stimulus. Their ideas slowed later attempts to explain vision in terms of single-cell recordings from individual neurons. A rapprochement between Gestalt phenomenology and neurophysiology seemed unlikely when the first ECVP was held in Marburg, Germany, in 1978. Since that time, response properties of neurons have been discovered that invite an interpretation of visual phenomena (including illusions) in terms of neuronal processing by long-range interactions, as first proposed by Mach and Hering in the last century. This article traces a personal journey into the early days of neurophysiological vision research to illustrate the progress that has taken place from the first attempts to correlate single-cell responses with visual perceptions. Whereas initially the receptive-field properties of individual classes of cells--e.g., contrast, wavelength, orientation, motion, disparity, and spatial-frequency detectors--were used to account for relatively simple visual phenomena, nowadays complex perceptions are interpreted in terms of long-range interactions, involving many neurons. This change in paradigm from local to global processing was made possible by recent findings, in the cortex, on horizontal interactions and backward propagation (feedback loops) in addition to classical feedforward processing. These mechanisms are exemplified by studies of the tilt effect and tilt aftereffect, direction-specific motion adaptation, illusory contours, filling-in and fading, figure--ground segregation by orientation and motion contrast, and pop-out in dynamic visual-noise patterns. Major questions for future research and a discussion of their epistemological implications conclude the article.

Orientation topography of layer 4 lateral networks revealed by optical imaging in cat visual cortex (area 18)

The functional specificity of corticocortical connections with respect to the topography of orientation selectivity was studied by optical imaging of intrinsic signals and bulk injections of fluorescent latex beads (green and red) and biocytin into layer 4. The distributions of retrogradely labelled cells and anterogradely labelled axon terminals were histologically reconstructed from all cortical laminae, and the resulting anatomical maps compared with the optically imaged functional maps. Layer 4 injections produced extensive horizontal labelling up to 2-3 mm from the injection centres albeit without the clear patchy pattern described after layer 2/3 injections (Gilbert & Wiesel 1989, J. Neurosci., 9, 2432-2442; Kisvárday et al. 1997, Cerebral Cortex, 7, 605-618). The functional (orientation) distribution of the labelled projections was analysed according to laminar location and lateral spread. With regard to the former, no major difference in the orientation topography between supragranular- (upper tier), granular- (middle tier) and infragranular (lower tier) layers was seen. Laterally, proximal and distal projections were distinguished and further dissected into three orientation categories, iso- (+/- 30 degrees ), oblique- (+/- 30-60 degrees ) and cross-orientations (+/- 60-90 degrees ) with respect to the orientation preference at the injection sites. The majority of distal connections (retrograde and anterograde) was equally distributed across orientations (35.4% iso-, 33.7% oblique-, and 30.9% cross-orientations) that are equivalent with a preponderance to dissimilar orientations (oblique- and cross-orientations, 64.6%). In one case, distal excitatory and inhibitory connections could be morphologically distinguished. For both categories, a marked bias to dissimilar orientations was found (excitatory, 63.7%; inhibitory, 86.6%). Taken together, these results suggest that the long-range layer 4 circuitry has a different functional role from that of the iso-orientation biased (52.9%, Kisvárday et al. 1997, Cerebral Cortex, 7, 605-618) layer 2/3 circuitry, and is perhaps involved in feature difference-based mechanisms, e.g. figure ground segregation.

The role of the primary visual cortex in higher level vision

In the classical feed-forward, modular view of visual processing, the primary visual cortex (area V1) is a module that serves to extract local features such as edges and bars. Representation and recognition of objects are thought to be functions of higher extrastriate cortical areas. This paper presents neurophysiological data that show the later part of V1 neurons' responses reflecting higher order perceptual computations related to Ullman's (Cognition 1984; 18:97-159) visual routines and Marr's (Vision NJ: Freeman 1982) full primal sketch, 2 1/2D sketch and 3D model. Based on theoretical reasoning and the experimental evidence, we propose a possible reinterpretation of the functional role of V1. In this framework, because of V1 neurons' precise encoding of orientation and spatial information, higher level perceptual computations and representations that involve high resolution details, fine geometry and spatial precision would necessarily involve V1 and be reflected in the later part of its neurons' activities.

Colour in a larger perspective: the rebirth of Gestalt psychology

This overview takes the reader from the classical contrast and assimilation studies of the past to today's colour research, in a broad sense, with its renewed emphasis on the phenomenological qualities of visual perception. It shows how the shift in paradigm from local to global effects in single-unit recordings prompted a reappraisal of appearance in visual experiments, not just in colour, but in the perception of motion, texture, and depth as well. Gestalt ideas placed in the context of modern concepts are shown to inspire psychophysicists, neurophysiologists, and computational vision scientists alike. Feedforward, horizontal interactions, and feedback are discussed as potential neuronal mechanisms to account for phenomena such as uniform surfaces, filling-in, and grouping arising from processes beyond the classical receptive field. A look forward towards future developments in the field of figure-ground segregation (Gestalt formation) concludes the article.