Approaches to visual cortical function

Contextual modulation emerges by integrating feedforward and feedback processing in mouse visual cortex

Sensory systems use context to infer meaning. Accordingly, context profoundly influences neural responses to sensory stimuli. However, a cohesive understanding of the circuit mechanisms governing contextual effects across different stimulus conditions is still lacking. Here we present a unified circuit model of mouse visual cortex that accounts for the main standard forms of contextual modulation. This data-driven and biologically realistic circuit, including three primary inhibitory cell types, sheds light on how bottom-up, top-down, and recurrent inputs are integrated across retinotopic space to generate contextual effects in layer 2/3. We establish causal relationships between neural responses, geometrical features of the inputs, and the connectivity patterns. The model not only reveals how a single canonical cortical circuit differently modulates sensory response depending on context but also generates multiple testable predictions, offering insights that apply to broader neural circuitry.

Towards an integrative theory of consciousness: part 2 (an anthology of various other models)

The study of consciousness has today moved beyond neurobiology and cognitive models. In the past few years, there has been a surge of research into various newer areas. The present article looks at the non-neurobiological and non-cognitive theories regarding this complex phenomenon, especially ones that self-psychology, self-theory, artificial intelligence, quantum physics, visual cognitive science and philosophy have to offer. Self-psychology has proposed the need to understand the self and its development, and the ramifications of the self for morality and empathy, which will help us understand consciousness better. There have been inroads made from the fields of computer science, machine technology and artificial intelligence, including robotics, into understanding the consciousness of these machines and their implications for human consciousness. These areas are explored. Visual cortex and emotional theories along with their implications are discussed. The phylogeny and evolution of the phenomenon of consciousness is also highlighted, with theories on the emergence of consciousness in fetal and neonatal life. Quantum physics and its insights into the mind, along with the implications of consciousness and physics and their interface are debated. The role of neurophilosophy to understand human consciousness, the functions of such a concept, embodiment, the dark side of consciousness, future research needs and limitations of a scientific theory of consciousness complete the review. The importance and salient features of each theory are discussed along with certain pitfalls, if present. A need for the integration of various theories to understand consciousness from a holistic perspective is stressed.

A CORF computational model of a simple cell that relies on LGN input outperforms the Gabor function model

Simple cells in primary visual cortex are believed to extract local contour information from a visual scene. The 2D Gabor function (GF) model has gained particular popularity as a computational model of a simple cell. However, it short-cuts the LGN, it cannot reproduce a number of properties of real simple cells, and its effectiveness in contour detection tasks has never been compared with the effectiveness of alternative models. We propose a computational model that uses as afferent inputs the responses of model LGN cells with center-surround receptive fields (RFs) and we refer to it as a Combination of Receptive Fields (CORF) model. We use shifted gratings as test stimuli and simulated reverse correlation to explore the nature of the proposed model. We study its behavior regarding the effect of contrast on its response and orientation bandwidth as well as the effect of an orthogonal mask on the response to an optimally oriented stimulus. We also evaluate and compare the performances of the CORF and GF models regarding contour detection, using two public data sets of images of natural scenes with associated contour ground truths. The RF map of the proposed CORF model, determined with simulated reverse correlation, can be divided in elongated excitatory and inhibitory regions typical of simple cells. The modulated response to shifted gratings that this model shows is also characteristic of a simple cell. Furthermore, the CORF model exhibits cross orientation suppression, contrast invariant orientation tuning and response saturation. These properties are observed in real simple cells, but are not possessed by the GF model. The proposed CORF model outperforms the GF model in contour detection with high statistical confidence (RuG data set: p<10(-4), and Berkeley data set: p<10(-4)). The proposed CORF model is more realistic than the GF model and is more effective in contour detection, which is assumed to be the primary biological role of simple cells.

From perceptive fields to Gestalt

Studies on visual psychophysics and perception conducted in the Freiburg psychophysics laboratory during the last 35 years are reviewed. Many of these were inspired by single-cell neurophysiology in cat and monkey. The aim was to correlate perceptual phenomena and their effects to possible neuronal mechanisms from retina to visual cortex and beyond. Topics discussed include perceptive field organization, figure-ground segregation and grouping, fading and filling-in, and long-range color interaction. While some of these studies succeeded in linking perception to neuronal response patterns, others require further investigation. The task of probing the human brain with perceptual phenomena continues to be a challenge for the future.

A computational model of monkey cortical grating cells

Grating cells were discovered in the V1 and V2 areas of the monkey visual cortex by von der Heydt et al. (1992). These cells responded vigorously to grating patterns of appropriate orientation and periodicity. Computational models inspired by these findings were used as texture operator (Kruzinga and Petkov 1995, 1999; Petkov and Kruzinga 1997) and for the emergence and self-organization of grating cells (Brunner et al. 1998; Bauer et al. 1999). The aim of this paper is to create a grating cell operator that demonstrates similar responses to monkey grating cells by applying operator to the same stimuli as in the experiments carried out by von der Heydt et al. (1992). Operator will be tested on images that contain periodic patterns as suggested by De Valois (1988). In order to learn more about the role of grating cells in natural vision, operator is applied to 338 real-world images of textures obtained from three different databases. The results suggest that grating cells respond strongly to regular alternating periodic patterns of a certain orientation. Such patterns are common in images of human-made structures, like buildings, fabrics, and tiles, and to regular natural periodic patterns, which are relatively rare in nature.

Contour detection based on nonclassical receptive field inhibition

We propose a biologically motivated method, called nonclassical receptive field (non-CRF) inhibition (more generally, surround inhibition or suppression), to improve contour detection in machine vision. Non-CRF inhibition is exhibited by 80% of the orientation-selective neurons in the primary visual cortex of monkeys and has been shown to influence human visual perception as well. Essentially, the response of an edge detector at a certain point is suppressed by the responses of the operator in the region outside the supported area. We combine classical edge detection with isotropic and anisotropic inhibition, both of which have counterparts in biology. We also use a biologically motivated method (the Gabor energy operator) for edge detection. The resulting operator responds strongly to isolated lines, edges, and contours, but exhibits weak or no response to edges that are part of texture. We use natural images with associated ground truth contour maps to assess the performance of the proposed operator for detecting contours while suppressing texture edges. Our method enhances contour detection in cluttered visual scenes more effectively than classical edge detectors used in machine vision (Canny edge detector). Therefore, the proposed operator is more useful for contour-based object recognition tasks, such as shape comparison, than traditional edge detectors, which do not distinguish between contour and texture edges. Traditional edge detection algorithms can, however, also be extended with surround suppression. This study contributes also to the understanding of inhibitory mechanisms in biology.

From elements to perception: local and global processing in visual neurons

Gestalt psychologists in the early part of the century challenged psychophysical notions that perceptual phenomena can be understood from a punctate (atomistic) analysis of the elements present in the stimulus. Their ideas slowed later attempts to explain vision in terms of single-cell recordings from individual neurons. A rapprochement between Gestalt phenomenology and neurophysiology seemed unlikely when the first ECVP was held in Marburg, Germany, in 1978. Since that time, response properties of neurons have been discovered that invite an interpretation of visual phenomena (including illusions) in terms of neuronal processing by long-range interactions, as first proposed by Mach and Hering in the last century. This article traces a personal journey into the early days of neurophysiological vision research to illustrate the progress that has taken place from the first attempts to correlate single-cell responses with visual perceptions. Whereas initially the receptive-field properties of individual classes of cells--e.g., contrast, wavelength, orientation, motion, disparity, and spatial-frequency detectors--were used to account for relatively simple visual phenomena, nowadays complex perceptions are interpreted in terms of long-range interactions, involving many neurons. This change in paradigm from local to global processing was made possible by recent findings, in the cortex, on horizontal interactions and backward propagation (feedback loops) in addition to classical feedforward processing. These mechanisms are exemplified by studies of the tilt effect and tilt aftereffect, direction-specific motion adaptation, illusory contours, filling-in and fading, figure--ground segregation by orientation and motion contrast, and pop-out in dynamic visual-noise patterns. Major questions for future research and a discussion of their epistemological implications conclude the article.

Toward a theory of visual consciousness

The visual brain consists of several parallel, functionally specialized processing systems, each having several stages (nodes) which terminate their tasks at different times; consequently, simultaneously presented attributes are perceived at the same time if processed at the same node and at different times if processed by different nodes. Clinical evidence shows that these processing systems can act fairly autonomously. Damage restricted to one system compromises specifically the perception of the attribute that that system is specialized for; damage to a given node of a processing system that leaves earlier nodes intact results in a degraded perceptual capacity for the relevant attribute, which is directly related to the physiological capacities of the cells left intact by the damage. By contrast, a system that is spared when all others are damaged can function more or less normally. Moreover, internally created visual percepts-illusions, afterimages, imagery, and hallucinations-activate specifically the nodes specialized for the attribute perceived. Finally, anatomical evidence shows that there is no final integrator station in the brain, one which receives input from all visual areas; instead, each node has multiple outputs and no node is recipient only. Taken together, the above evidence leads us to propose that each node of a processing-perceptual system creates its own microconsciousness. We propose that, if any binding occurs to give us our integrated image of the visual world, it must be a binding between microconsciousnesses generated at different nodes. Since any two microconsciousnesses generated at any two nodes can be bound together, perceptual integration is not hierarchical, but parallel and postconscious. By contrast, the neural machinery conferring properties on those cells whose activity has a conscious correlate is hierarchical, and we refer to it as generative binding, to distinguish it from the binding that might occur between the microconsciousnesses.

Occlusion cues contribute to orientation judgments of occlusion-defined contours

Occlusion cues defining a contour in a 2-D stimulus pattern were shown to contribute to the accuracy of orientation judgments of that contour. The stimulus pattern was altered so that the occlusion cues became ambiguous, by introducing a textured background suggesting transparency of the stimulus pattern. Orientation judgments then became significantly less accurate. This finding shows that occlusion cues in 2-D patterns can be behaviorally relevant, in addition to generating the subjective percept commonly known as an illusory contour. The disruptive effect of the textured background on orientation judgments remained when no texture elements were present in the vicinity of the contour. This suggests that the generation of occlusion-defined contours relies as much on an evaluation of the surfaces at either side of the contour as being opaque as it does on local encoding of occlusion cues close to the contour. Finally, orientation sensitivity measured with contours defined by other than occlusion cues was not altered after the introduction of a textured background.

Effects of visual cortex lesions on orientation discrimination of illusory contours in the cat

We have trained five cats in orientation discrimination using different contours, and compared the deficits caused by lesions of cortical areas 17 and 18 (tier I) to the deficits induced by removal of those areas receiving afferents originating in areas 17 and 18 (tier II). As contour stimuli we used two types of illusory contours and a luminance bar. The two illusory contours were defined by opposed line-ends. One of them coincided with a luminance gradient whereas the other did not. Tier I lesions destroyed the capacity to discriminate the orientation of both illusory contours, and also caused an important, though less severe, deficit in bar orientation discrimination. The deficits induced by tier I lesions were permanent. Tier II lesions also caused significant deficits in orientation discrimination of illusory contours, but only a negligible deficit in bar orientation discrimination, and this result was not a mere consequence of a difference in difficulty between the tasks involving bars and illusory contours. In addition, tier II lesions differentiated between illusory contour types, the deficit being more pronounced for the illusory contour without luminance gradient than for the one with luminance gradient. In contrast to tier I lesions, tier II lesions allowed significant recovery, leading to small final deficits for all contour types tested.

The role of features in preattentive vision: comparison of orientation, motion and color cues

Arrays of lines or blobs were used to investigate the role of features vs feature contrast in preattentive vision. This study continues earlier work on orientation cues and extends it into the dimensions of motion and color. Tests were performed on patterns displaying continuous feature gradients, i.e. continuous variation, from element to element, in either line orientation, direction of motion, or color. In different series of experiments, the following four aspects of visual perception were investigated: (i) detection of a salient target ("odd man out" paradigm), (ii) segmentation of texture fields, (iii) search strategies for given targets, and (iv) figure-ground discrimination by grouping. Features were, in general, not found to play an important role in these tasks and performance was instead related to feature contrast. Only in the case of color did performance also depend upon hue, i.e. feature properties themselves. Whereas in all dimensions tested pop-out ("saliency") and segmentation were obtained from target or border elements whose local feature contrast was well above the level of variation elsewhere in the pattern, performance in the two latter tasks differed between orientation, motion, and color. In search, orientation targets were detected quickly when sufficiently distinct from their neighbors, but were apparently searched for serially if feature contrast was similar to that of other elements nearby. Color targets, however, were always detected fast in these patterns, independently of local feature contrast. Also, the perceived grouping of orientation or motion defined targets depended only on local feature contrast and not on the similarity of target elements. In fact, figures of dissimilar elements were seen as easily as figures of similar elements, indicating that, in these dimensions, stimulus coherence is not essential for the discrimination of figure and ground. For color, however, figures made up of the same targets were always seen slightly better than figures composed of different colors.

Subjective contours--bridging the gap between psychophysics and physiology

Much is known about the initial stages of visual processing up to the striate cortex, but how is visual information represented and handled at subsequent stages? Phenomena of contour, color and movement perception have been used to identify functions of neurons and to reveal functional differences between cortical areas that application of classical receptive-field concepts has not suggested. These differences can be related to theoretical stages of visual processing that provide stability of perception under changing conditions of stimulation.

Texton segregation by associated differences in global and local luminance distribution

Perceptual segregation of visual textures has been attributed to certain features ('textons') such as (elongated) blobs of given size and orientation, line crossings, and line ends. Differences in the spatial distribution of these features were assumed to be detected pre-attentively and to provide the instantaneous impression of segregating texture areas and of borders between them. This paper questions the validity of this general view and, in particular, the role of some of these features in texture discrimination. It is demonstrated that for some textons, perceptual segregation is independent of detection and discrimination of the texton itself. In addition, segregation can be strongly affected by positional or luminance jitter of texture elements or by other modifications that change the luminance distribution in the pattern but do not affect the supposed texton differences. From the textons reported in the literature, only differences in orientation were found to be fairly robust against such modifications.

Simple cells in the visual cortex of the cat can be narrowly tuned for spatial frequency

The existence in cat cortex of linearly operating simple cells with relative spatial-frequency bandwidths of between 0.5 and 1 octave is confirmed.

Interactions among spatial frequency and orientation channels adapted concurrently

Interactions between size and orientation-specific mechanisms in the human visual system were investigated using a sequential adaptation technique. Subjects adapted to a vertical, 4 c/deg high-contrast (0.7) sinewave grating that was interleaved at a rate of 0.5 Hz with another adapting grating differing either in (1) spatial frequency or (2) orientation. Before and after adaptation contrast thresholds were measured for a vertical 4 c/deg sinewave test grating. The resultant elevation in contrast threshold was plotted as a function of the (1) spatial frequency or (2) orientation differences between the first and second adapting gratings. Maximum threshold elevation was found when both adapting gratings shared the same spatial frequency and orientation. Minimum elevations were found when the second grating's spatial frequency or orientation differed by approx. 1.5 octaves or 45 deg, respectively. Beyond these values threshold elevations reapproached the baseline value measured in a control condition, where the 4.0 c/deg adapting grating was interleaved with a blank. The minimum threshold elevations were 0.2-0.3 log units below the baseline level. The results suggest the existence of inhibitory interactions between neural mechanisms tuned to the size and orientation of retinal images.