Boundary cue invariance in cortical orientation maps

Orientation selectivity mapping in the visual cortex

The orientation map is one of the most well-studied functional maps of the visual cortex. However, results from the literature are of different qualities. Clear boundaries among different orientation domains and blurred uncertain distinctions were shown in different studies. These unclear imaging results will lead to an inaccuracy in depicting cortical structures, and the lack of consideration in experimental design will also lead to biased depictions of the cortical features. How we accurately define orientation domains will impact the entire field of research. In this study, we test how spatial frequency (SF), stimulus size, location, chromatic, and data processing methods affect the orientation functional maps (including a large area of dorsal V4, and parts of dorsal V1) acquired by intrinsic signal optical imaging. Our results indicate that, for large imaging fields, large grating stimuli with mixed SF components should be considered to acquire the orientation map. A diffusion model image enhancement based on the difference map could further improve the map quality. In addition, the similar outcomes of achromatic and chromatic gratings indicate two alternative types of afferents from LGN, pooling in V1 to generate cue-invariant orientation selectivity.

Bipartite invariance in mouse primary visual cortex

A defining characteristic of intelligent systems, whether natural or artificial, is the ability to generalize and infer behaviorally relevant latent causes from high-dimensional sensory input, despite significant variations in the environment. To understand how brains achieve generalization, it is crucial to identify the features to which neurons respond selectively and invariantly. However, the high-dimensional nature of visual inputs, the non-linearity of information processing in the brain, and limited experimental time make it challenging to systematically characterize neuronal tuning and invariances, especially for natural stimuli. Here, we extended "inception loops" - a paradigm that iterates between large-scale recordings, neural predictive models, and in silico experiments followed by in vivo verification - to systematically characterize single neuron invariances in the mouse primary visual cortex. Using the predictive model we synthesized Diverse Exciting Inputs (DEIs), a set of inputs that differ substantially from each other while each driving a target neuron strongly, and verified these DEIs' efficacy in vivo. We discovered a novel bipartite invariance: one portion of the receptive field encoded phase-invariant texture-like patterns, while the other portion encoded a fixed spatial pattern. Our analysis revealed that the division between the fixed and invariant portions of the receptive fields aligns with object boundaries defined by spatial frequency differences present in highly activating natural images. These findings suggest that bipartite invariance might play a role in segmentation by detecting texture-defined object boundaries, independent of the phase of the texture. We also replicated these bipartite DEIs in the functional connectomics MICrONs data set, which opens the way towards a circuit-level mechanistic understanding of this novel type of invariance. Our study demonstrates the power of using a data-driven deep learning approach to systematically characterize neuronal invariances. By applying this method across the visual hierarchy, cell types, and sensory modalities, we can decipher how latent variables are robustly extracted from natural scenes, leading to a deeper understanding of generalization.

Revealing Detail along the Visual Hierarchy: Neural Clustering Preserves Acuity from V1 to V4

How primates perceive objects along with their detailed features remains a mystery. This ability to make fine visual discriminations depends upon a high-acuity analysis of spatial frequency (SF) along the visual hierarchy from V1 to inferotemporal cortex. By studying the transformation of SF across macaque parafoveal V1, V2, and V4, we discovered SF-selective functional domains in V4 encoding higher SFs up to 12 cycles/°. These intermittent higher-SF-selective domains, surrounded by domains encoding lower SFs, violate the inverse relationship between SF preference and retinal eccentricity. The neural activities of higher- and lower-SF domains correspond to local and global features, respectively, of the same stimuli. Neural response latencies in high-SF domains are around 10 ms later than in low-SF domains, consistent with the coarse-to-fine nature of perception. Thus, our finding of preserved resolution from V1 into V4, separated both spatially and temporally, may serve as a connecting link for detailed object representation.

Mice Can Use Second-Order, Contrast-Modulated Stimuli to Guide Visual Perception

Visual processing along the primate ventral stream takes place in a hierarchy of areas, characterized by an increase in both complexity of neuronal preferences and invariance to changes of low-level stimulus attributes. A basic type of invariance is form-cue invariance, where neurons have similar preferences in response to first-order stimuli, defined by changes in luminance, and global features of second-order stimuli, defined by changes in texture or contrast. Whether in mice, a now popular model system for early visual processing, visual perception can be guided by second-order stimuli is currently unknown. Here, we probed mouse visual perception and neural responses in areas V1 and LM using various types of second-order, contrast-modulated gratings with static noise carriers. These gratings differ in their spatial frequency composition and thus in their ability to invoke first-order mechanisms exploiting local luminance features. We show that mice can transfer learning of a coarse orientation discrimination task involving first-order, luminance-modulated gratings to the contrast-modulated gratings, albeit with markedly reduced discrimination performance. Consistent with these behavioral results, we demonstrate that neurons in area V1 and LM are less responsive and less selective to contrast-modulated than to luminance-modulated gratings, but respond with broadly similar preferred orientations. We conclude that mice can, at least in a rudimentary form, use second-order stimuli to guide visual perception.

SIGNIFICANCE STATEMENT

To extract object boundaries in natural scenes, the primate visual system does not only rely on differences in local luminance but can also take into account differences in texture or contrast. Whether the mouse, which has a much simpler visual system, can use such second-order information to guide visual perception is unknown. Here we tested mouse perception of second-order, contrast-defined stimuli and measured their neural representations in two areas of visual cortex. We find that mice can use contrast-defined stimuli to guide visual perception, although behavioral performance and neural representations were less robust than for luminance-defined stimuli. These findings shed light on basic steps of feature extraction along the mouse visual cortical hierarchy, which may ultimately lead to object recognition.

Breaking cover: neural responses to slow and fast camouflage-breaking motion

Primates need to detect and recognize camouflaged animals in natural environments. Camouflage-breaking movements are often the only visual cue available to accomplish this. Specifically, sudden movements are often detected before full recognition of the camouflaged animal is made, suggesting that initial processing of motion precedes the recognition of motion-defined contours or shapes. What are the neuronal mechanisms underlying this initial processing of camouflaged motion in the primate visual brain? We investigated this question using intrinsic-signal optical imaging of macaque V1, V2 and V4, along with computer simulations of the neural population responses. We found that camouflaged motion at low speed was processed as a direction signal by both direction- and orientation-selective neurons, whereas at high-speed camouflaged motion was encoded as a motion-streak signal primarily by orientation-selective neurons. No population responses were found to be invariant to the camouflage contours. These results suggest that the initial processing of camouflaged motion at low and high speeds is encoded as direction and motion-streak signals in primate early visual cortices. These processes are consistent with a spatio-temporal filter mechanism that provides for fast processing of motion signals, prior to full recognition of camouflage-breaking animals.

Possible functions of contextual modulations and receptive field nonlinearities: pop-out and texture segmentation

When analyzing a visual image, the brain has to achieve several goals quickly. One crucial goal is to rapidly detect parts of the visual scene that might be behaviorally relevant, while another one is to segment the image into objects, to enable an internal representation of the world. Both of these processes can be driven by local variations in any of several image attributes such as luminance, color, and texture. Here, focusing on texture defined by local orientation, we propose that the two processes are mediated by separate mechanisms that function in parallel. More specifically, differences in orientation can cause an object to "pop out" and attract visual attention, if its orientation differs from that of the surrounding objects. Differences in orientation can also signal a boundary between objects and therefore provide useful information for image segmentation. We propose that contextual response modulations in primary visual cortex (V1) are responsible for orientation pop-out, while a different kind of receptive field nonlinearity in secondary visual cortex (V2) is responsible for orientation-based texture segmentation. We review a recent experiment that led us to put forward this hypothesis along with other research literature relevant to this notion.

Orientation-cue invariant population responses to contrast-modulated and phase-reversed contour stimuli in macaque V1 and V2

Visual scenes can be readily decomposed into a variety of oriented components, the processing of which is vital for object segregation and recognition. In primate V1 and V2, most neurons have small spatio-temporal receptive fields responding selectively to oriented luminance contours (first order), while only a subgroup of neurons signal non-luminance defined contours (second order). So how is the orientation of second-order contours represented at the population level in macaque V1 and V2? Here we compared the population responses in macaque V1 and V2 to two types of second-order contour stimuli generated either by modulation of contrast or phase reversal with those to first-order contour stimuli. Using intrinsic signal optical imaging, we found that the orientation of second-order contour stimuli was represented invariantly in the orientation columns of both macaque V1 and V2. A physiologically constrained spatio-temporal energy model of V1 and V2 neuronal populations could reproduce all the recorded population responses. These findings suggest that, at the population level, the primate early visual system processes the orientation of second-order contours initially through a linear spatio-temporal filter mechanism. Our results of population responses to different second-order contour stimuli support the idea that the orientation maps in primate V1 and V2 can be described as a spatial-temporal energy map.

The mechanism for processing random-dot motion at various speeds in early visual cortices

All moving objects generate sequential retinotopic activations representing a series of discrete locations in space and time (motion trajectory). How direction-selective neurons in mammalian early visual cortices process motion trajectory remains to be clarified. Using single-cell recording and optical imaging of intrinsic signals along with mathematical simulation, we studied response properties of cat visual areas 17 and 18 to random dots moving at various speeds. We found that, the motion trajectory at low speed was encoded primarily as a direction signal by groups of neurons preferring that motion direction. Above certain transition speeds, the motion trajectory is perceived as a spatial orientation representing the motion axis of the moving dots. In both areas studied, above these speeds, other groups of direction-selective neurons with perpendicular direction preferences were activated to encode the motion trajectory as motion-axis information. This applied to both simple and complex neurons. The average transition speed for switching between encoding motion direction and axis was about 31°/s in area 18 and 15°/s in area 17. A spatio-temporal energy model predicted the transition speeds accurately in both areas, but not the direction-selective indexes to random-dot stimuli in area 18. In addition, above transition speeds, the change of direction preferences of population responses recorded by optical imaging can be revealed using vector maximum but not vector summation method. Together, this combined processing of motion direction and axis by neurons with orthogonal direction preferences associated with speed may serve as a common principle of early visual motion processing.

Responses to orientation discontinuities in V1 and V2: physiological dissociations and functional implications

Segmenting the visual image into objects is a crucial stage of visual processing. Object boundaries are typically associated with differences in luminance, but discontinuities in texture also play an important role. We showed previously that a subpopulation of neurons in V2 in anesthetized macaques responds to orientation discontinuities parallel to their receptive field orientation. Such single-cell responses could be a neurophysiological correlate of texture boundary detection. Neurons in V1, on the other hand, are known to have contextual response modulations such as iso-orientation surround suppression, which also produce responses to orientation discontinuities. Here, we use pseudorandom multiregion grating stimuli of two frame durations (20 and 40 ms) to probe and compare texture boundary responses in V1 and V2 in anesthetized macaque monkeys. In V1, responses to texture boundaries were observed for only the 40 ms frame duration and were independent of the orientation of the texture boundary. However, in transient V2 neurons, responses to such texture boundaries were robust for both frame durations and were stronger for boundaries parallel to the neuron's preferred orientation. The dependence of these processes on stimulus duration and orientation indicates that responses to texture boundaries in V2 arise independently of contextual modulations in V1. In addition, because the responses in transient V2 neurons are sensitive to the orientation of the texture boundary but those of V1 neurons are not, we suggest that V2 responses are the correlate of texture boundary detection, whereas contextual modulation in V1 serves other purposes, possibly related to orientation "pop-out."

Linking brain to behavior for the visual perception of figures and objects

The dissociation of a figure from its background is an essential feat of visual perception, as it allows us to detect, recognize, and interact with shapes and objects in our environment. In order to understand how the human brain gives rise to the perception of figures, we here review experiments that explore the links between activity in visual cortex and performance of perceptual tasks related to figure perception. We organize our review according to a proposed model that attempts to contextualize figure processing within the more general framework of object processing in the brain. Overall, the current literature provides us with individual linking hypotheses as to cortical regions that are necessary for particular tasks related to figure perception. Attempts to reach a more complete understanding of how the brain instantiates figure and object perception, however, will have to consider the temporal interaction between the many regions involved, the details of which may vary widely across different tasks.

Distinct functional organizations for processing different motion signals in V1, V2, and V4 of macaque

Motion perception is qualitatively invariant across different objects and forms, namely, the same motion information can be conveyed by many different physical carriers, and it requires the processing of motion signals consisting of direction, speed, and axis or trajectory of motion defined by a moving object. Compared with the representation of orientation, the cortical processing of these different motion signals within the early ventral visual pathway of the primate remains poorly understood. Using drifting full-field noise stimuli and intrinsic optical imaging, along with cytochrome-oxidase staining, we found that the orientation domains in macaque V1, V2, and V4 that processed orientation signals also served to process motion signals associated with the axis and speed of motion. In contrast, direction domains within the thick stripes of V2 demonstrated preferences that were independent of motion speed. The population responses encoding the orientation and motion axis could be precisely reproduced by a spatiotemporal energy model. Thus, our observation of orientation domains with dual functions in V1, V2, and V4 directly support the notion that the linear representation of the temporal series of retinotopic activations may serve as another motion processing strategy in primate ventral visual pathway, contributing directly to fine form and motion analysis. Our findings further reveal that different types of motion information are differentially processed in parallel and segregated compartments within primate early visual cortices, before these motion features are fully combined in high-tier visual areas.

Functional organization of envelope-responsive neurons in early visual cortex: organization of carrier tuning properties

It is well established that visual cortex neurons having similar selectivity for orientation, direction of motion, ocular dominance, and other properties of first-order (luminance-defined) stimuli are clustered into a columnar organization. However, the cortical architecture of neuronal responses to second-order (contrast/texture-defined) stimuli is poorly understood. A useful second-order stimulus is a contrast envelope, consisting of a finely detailed pattern (carrier) whose contrast varies on a coarse spatial scale (envelope). In this study, we analyzed the cortical organization of carrier tuning properties of neurons, which responded to contrast-modulated stimuli. We examined whether neurons tuned to similar carrier properties are clustered spatially and whether such spatial clusters are arranged in columns. To address these questions, we recorded single-unit activity, multiunit activity, and local field potentials simultaneously from area 18 of anesthetized cats, using single-channel microelectrodes and multielectrode arrays. Our data showed that neurons tuned to similar carrier spatial frequency are distributed in a highly clustered manner; neurons tuned to similar carrier orientation are also significantly clustered. Neurons along linear arrays perpendicular to the brain surface always exhibited similar optimal carrier spatial frequency, indicating a columnar organization. Multi-pronged tetrode recordings indicated that the diameter of these columns is ≥450 μm. Optimal carrier orientation was also significantly clustered but with finer-grain organization and greater scatter. These results indicate a fine anatomical structure of cortical organization of second-order information processing and suggest that there are probably more maps in cat area 18 than previously believed.

Motion-defined contour processing in the early visual cortex

From our daily experience, it is very clear that relative motion cues can contribute to correctly identifying object boundaries and perceiving depth. Motion-defined contours are not only generated by the motion of objects in a scene but also by the movement of an observer's head and body (motion parallax). However, the neural mechanism involved in detecting these contours is still unknown. To explore this mechanism, we extracellularly recorded visual responses of area 18 neurons in anesthetized and paralyzed cats. The goal of this study was to determine if motion-defined contours could be detected by neurons that have been previously shown to detect luminance-, texture-, and contrast-defined contours cue invariantly. Motion-defined contour stimuli were generated by modulating the velocity of high spatial frequency sinusoidal luminance gratings (carrier gratings) by a moving squarewave envelope. The carrier gratings were outside the luminance passband of a neuron, such that presence of the carrier alone within the receptive field did not elicit a response. Most neurons that responded to contrast-defined contours also responded to motion-defined contours. The orientation and direction selectivity of these neurons for motion-defined contours was similar to that of luminance gratings. A given neuron also exhibited similar selectivity for the spatial frequency of the carrier gratings of contrast- and motion-defined contours. These results suggest that different second-order contours are detected in a form-cue invariant manner, through a common neural mechanism in area 18.

Face categorization in visual scenes may start in a higher order area of the right fusiform gyrus: evidence from dynamic visual stimulation in neuroimaging

How a visual stimulus is initially categorized as a face by the cortical face-processing network remains largely unclear. In this study we used functional MRI to study the dynamics of face detection in visual scenes by using a paradigm in which scenes containing faces or cars are revealed progressively as they emerge from visual noise. Participants were asked to respond as soon as they detected a face or car during the noise sequence. Among the face-sensitive regions identified based on a standard localizer, a high-level face-sensitive area, the right fusiform face area (FFA), showed the earliest difference between face and car activation. Critically, differential activation in FFA was observed before differential activation in the more posteriorly located occipital face area (OFA). A whole brain analysis confirmed these findings, with a face-sensitive cluster in the right fusiform gyrus being the only cluster showing face preference before successful behavioral detection. Overall, these findings indicate that following generic low-level visual analysis, a face stimulus presented in a gradually revealed visual scene is first detected in the right middle fusiform gyrus, only after which further processing spreads to a network of cortical and subcortical face-sensitive areas (including the posteriorly located OFA). These results provide further evidence for a nonhierarchical organization of the cortical face-processing network.

Visual responses to contrast-defined contours with equally spatial-scaled carrier in cat area 18

Contrast-defined contours are one type of second-order contours, across which there are no differences in luminance. Although they can be always perceived, their responses have been only investigated when the spatial frequency of carrier, the background texture whose contrast is modulated to form contours, is much higher than that of contrast-defined contours, due to the interference of responses to luminance contours in other cases. In the present study, we examined visual responses in cat area 18 to the contrast-defined contours with carrier at same spatial frequency equal to neuron's preferred value for luminance contours, by establishing a control stimulus including all the luminance components but lack of the contrast contour information. Using single unit recording and intrinsic optical imaging, we demonstrated that contrast gratings with equally spatial-scaled carrier induced responses in a proportion of cat area 18 neurons with the preferred orientation similar to that for luminance contours, and the responses generated orientation maps similar to those for luminance contours. Our finding suggests that early visual cortex can process second-order contours regardless of the spatial frequency of carriers, in a way similar to the processing of luminance contours. This uniform manner of early visual processing might underlie the visual detection of both luminance contours and non-luminance second-order contours.

Orientation dependency of intrinsic optical signal dynamics in cat area 18

Imaging studies based on intrinsic optical signals have been primarily performed by analyses of the signal amplitude as opposed to using the temporal information of the intrinsic optical signals. The present study focused on the dynamics of these signals in cat area 18, and quantitatively compared the waveforms after presentation of different stimuli across the same cortical regions. Optical imaging based on intrinsic signals was conducted on 18 hemispheres of 9 cats. For the visual stimuli, gratings with orientations that changed from horizontal (0°) to 157.5° in 22.5° steps were used. The signal time course was examined at each pixel, with the peak delay defined as the amount of time required after the stimulus onset for the intrinsic optical signal to reach its negative maximum. In the area that showed significant orientation preference to 0° and 90° but not to their 22.5° separated nearby angles, the delays were 1.92 ± 0.22s and 1.99 ± 0.29s (mean ± SE, n = 18), respectively. Delays of 2.31 ± 0.20s and 2.28 ± 0.25s were observed in the cortical areas that selectively responded to the orientation gratings of 45°and 135° but not their nearby angles. Statistically, the delays in areas exhibiting oblique orientation preferences were significantly longer than those showing cardinal orientation preferences. These results demonstrate anisotropy for the intrinsic optical signal dynamics in the cat area 18. The possible neural mechanisms underlying were discussed.

Functional organization for color and orientation in macaque V4

Visual area V4 in the macaque monkey is a cortical area that is strongly involved in color and shape perception. However, fundamental questions about V4 are still debated. V4 was initially characterized as a color-processing area, but subsequent studies revealed that it contains a diverse complement of cells, including those with preference for color, orientation, disparity and higher-order feature preferences. This has led to disputes and uncertainty about the role of V4 in vision. Using intrinsic signal optical imaging methods in awake, behaving monkeys, we found that different feature preferences are functionally organized in V4. Optical images revealed that regions with preferential response to color were largely separate from orientation-selective regions. Our results help to resolve long-standing controversies regarding functional diversity and retinotopy in V4 and indicate the presence of spatially biased distribution of featural representation in V4 in the ventral visual pathway.

Orientation selectivity of motion-boundary responses in human visual cortex

Motion boundaries (local changes in visual motion direction) arise naturally when objects move relative to an observer. In human visual cortex, neuroimaging studies have identified a region (the kinetic occipital area [KO]) that responds more strongly to motion-boundary stimuli than to transparent-motion stimuli. However, some functional magnetic resonance imaging (fMRI) studies suggest that KO may encompass multiple visual areas and single-unit studies in macaque visual cortex have identified neurons selective for motion-boundary orientation in areas V2, V3, and V4, implying that motion-boundary selectivity may not be restricted to a single area. It is not known whether fMRI responses to motion boundaries are selective for motion-boundary orientation, as would be expected if these responses reflected the population activity of motion-boundary-selective neurons. We used an event-related fMRI adaptation protocol to measure orientation-selective responses to motion boundaries in human visual cortex. On each trial, we measured the response to a probe stimulus presented after an adapter stimulus (a vertical or horizontal motion-boundary grating). The probe stimulus was either a motion-boundary grating oriented parallel or orthogonal to the adapter stimulus or a transparent-motion stimulus. Orientation-selective adaptation for motion boundaries--smaller responses for trials in which test and adapter stimuli were parallel to each other--was observed in multiple extrastriate visual areas. The strongest adaptation, relative to the unadapted responses, was found in V3A, V3B, LO1, LO2, and V7. Most of the visual areas that exhibited orientation-selective adaptation in our data also showed response preference for motion boundaries over transparent motion, indicating that most of the human visual areas previously shown to respond to motion boundaries are also selective for motion-boundary orientation. These results suggest that neurons selective for motion-boundary orientation are distributed across multiple human visual cortical areas and argue against the existence of a single region or area specialized for motion-boundary processing.

Subcortical representation of non-Fourier image features

A fundamental goal of visual neuroscience is to identify the neural pathways representing different image features. It is widely argued that the early stages of these pathways represent linear features of the visual scene and that the nonlinearities necessary to represent complex visual patterns are introduced later in cortex. We tested this by comparing the responses of subcortical and cortical neurons to interference patterns constructed by summing sinusoidal gratings. Although a linear mechanism can detect the component gratings, a nonlinear mechanism is required to detect an interference pattern resulting from their sum. Consistent with in vitro retinal ganglion cell recordings, we found that interference patterns are represented subcortically by cat LGN Y-cells, but not X-cells. Linear and nonlinear tuning properties of LGN Y-cells were then characterized and compared quantitatively with those of cortical area 18 neurons responsive to interference patterns. This comparison revealed a high degree of similarity between the two neural populations, including the following: (1) the representation of similar spatial frequencies in both their linear and nonlinear responses, (2) comparable orientation selectivity for the high spatial frequency carrier of interference patterns, and (3) the same difference in their temporal frequency selectivity for drifting gratings versus the envelope of interference patterns. The present findings demonstrate that the nonlinear subcortical Y-cell pathway represents complex visual patterns and likely underlies cortical responses to interference patterns. We suggest that linear and nonlinear mechanisms important for encoding visual scenes emerge in parallel through distinct pathways originating at the retina.

Mapping of contextual modulation in the population response of primary visual cortex

We review the evidence of long-range contextual modulation in V1. Populations of neurons in V1 are activated by a wide variety of stimuli outside of their classical receptive fields (RF), well beyond their surround region. These effects generally involve extra-RF features with an orientation component. The population mapping of orientation preferences to the upper layers of V1 is well understood, as far as the classical RF properties are concerned, and involves organization into pinwheel-like structures. We introduce a novel hypothesis regarding the organization of V1's contextual response. We show that RF and extra-RF orientation preferences are mapped in related ways. Orientation pinwheels are the foci of both types of features. The mapping of contextual features onto the orientation pinwheel has a form that recapitulates the organization of the visual field: an iso-orientation patch within the pinwheel also responds to extra-RF stimuli of the same orientation. We hypothesize that the same form of mapping applies to other stimulus properties that are mapped out in V1, such as colour and contrast selectivity. A specific consequence is that fovea-like properties will be mapped in a systematic way to orientation pinwheels. We review the evidence that cytochrome oxidase blobs comprise the foci of this contextual remapping for colour and low contrasts. Neurodynamics and motion in the visual field are argued to play an important role in the shaping and maintenance of this type of mapping in V1.

Influence of parallel and orthogonal real lines on illusory contour perception

Real lines and illusory contours (ICs) have been reported to either interfere with or facilitate the perception of the other, depending on real line orientation and contrast. Here we investigate contextual effects of real lines on illusory contour perception. Curvature discrimination thresholds of Kanizsa-contours were measured for superimposed real lines of different sub- and suprathreshold contrasts. We find that parallel lines interfere with curvature discrimination at suprathreshold, whereas orthogonal lines interfere at subthreshold contrasts. We did not find stable facilitating effects of lines in any orientation or contrast. These results are discussed in relation to existing physiological and imaging data.

Three-dimensional shape from second-order orientation flows

In images of textured surfaces, orientation flows formed by perspective convergence invariably convey 3D shape. We show that orientation flows formed by contrast-modulated (CM) and illusory contours (IC) convey 3D shape, and that both stimulus types induce 3D shape aftereffects on CM and IC test stimuli. Adaptation to luminance-modulated (LM) orientation flows induce robust 3D shape aftereffects on CM and IC tests, however, aftereffects using CM/IC adapting stimuli on LM tests were substantially weaker. These results can be explained by the adaptation of 3D shape-selective neurons that invariantly extract first- and second-order orientation flows from striate and extra-striate signals, which receive stronger input from neurons selective for first-order orientation flows.

Models and Measurements of Functional Maps in V1

The organization of primary visual cortex has been heavily studied for nearly 50 years, and in the last 20 years functional imaging has provided high-resolution maps of its tangential organization. Recently, however, the usefulness of maps like those of orientation and spatial frequency (SF) preference has been called into question because they do not, by themselves, predict how moving images are represented in V1. In this review, we discuss a model for cortical responses (the spatiotemporal filtering model) that specifies the types of cortical maps needed to predict distributed activity within V1. We then review the structure and interrelationships of several of these maps, including those of orientation, SF, and temporal frequency preference. Finally, we discuss tests of the model and the sufficiency of the requisite maps in predicting distributed cortical responses. Although the spatiotemporal filtering model does not account for all responses within V1, it does, with reasonable accuracy, predict population responses to a variety of complex stimuli.

Neuronal response to texture- and contrast-defined boundaries in early visual cortex

Natural scenes contain a variety of visual cues that facilitate boundary perception (e.g., luminance, contrast, and texture). Here we explore whether single neurons in early visual cortex can process both contrast and texture cues. We recorded neural responses in cat A18 to both illusory contours formed by abutting gratings (ICs, texture-defined) and contrast-modulated gratings (CMs, contrast-defined). We found that if a neuron responded to one of the two stimuli, it also responded to the other. These neurons signaled similar contour orientation, spatial frequency, and movement direction of the two stimuli. A given neuron also exhibited similar selectivity for spatial frequency of the fine, stationary grating components (carriers) of the stimuli. These results suggest that the cue-invariance of early cortical neurons extends to different kinds of texture or contrast cues, and might arise from a common nonlinear mechanism.

Critical spatial frequencies for illusory contour processing in early visual cortex

Single neurons in primate V2 and cat A18 exhibit identical orientation tuning for sinewave grating and illusory contour stimuli. This cue invariance is also manifested in similar orientation maps to these stimuli, but in V1/A17 the illusory contour maps appear reversed. We hypothesized that this map reversal depends upon the spatial frequencies of the inducers in the illusory contours, relative to the spatial selectivities of these brain areas. We employed intrinsic signal optical imaging to measure orientation maps in cat A17/18 to illusory contours with inducers at spatial frequencies from 0.15 to 1.6 cpd. A17 illusory contour maps were indeed reversed compared with grating-driven maps for inducer spatial frequencies <1.3 cpd, whereas A18 maps were invariant. Simulations based on known neurophysiology demonstrated that map reversal can arise from linear filtering, and map invariance can be explained by a nonlinear (filter-rectify-filter) mechanism. The simulation also correctly predicted that A17 could show invariant maps when the inducer spatial frequency is sufficiently high (1.6 cpd), and that A18 maps could reverse at lower inducer frequencies (0.18 cpd). Thus, the map reversal or invariance to illusory contours depends critically on the relationship of the inducer spatial frequencies to the spatial filtering properties of neurons in each brain area.

Ecologically valid combinations of first- and second-order surface markings facilitate texture discrimination

Natural scenes contain localized variations in both first-order (luminance) and second-order (contrast and texture) information. There is much evidence that first- and second-order stimuli are detected by distinct mechanisms in the mammalian visual system. However, in natural scenes the two kinds of information tend to be spatially correlated. Do correlated and uncorrelated combinations of first- and second-order stimuli differentially affect perception? To address this question we employed orientation-modulated textures in which observers were required to discriminate the spatial frequency of the texture modulation. The textures consisted of micropatterns defined as either local variations in luminance (first-order) or luminance contrast (second-order). Performance was robust with textures composed of only first-order micropatterns, but impossible with only second-order micropatterns. However, when the second-order micropatterns were combined with the first-order micropatterns, they enhanced performance when the two were spatially correlated, but impaired performance when the two were spatially uncorrelated. We conclude that local second-order information may enhance texture modulation discrimination provided it is combined with first-order information in an ecologically valid manner.

Cue-invariant detection of centre-surround discontinuity by V1 neurons in awake macaque monkey

Visual perception of an object depends on the discontinuity between the object and its background, which can be defined by a variety of visual features, such as luminance, colour and motion. While human object perception is largely cue invariant, the extent to which neural mechanisms in the primary visual cortex contribute to cue-invariant perception has not been examined extensively. Here we report that many V1 neurons in the awake monkey are sensitive to the stimulus discontinuity between their classical receptive field (CRF) and non-classical receptive field (nCRF) regardless of the visual feature that defines the discontinuity. The magnitude of this sensitivity is strongly dependent on the strength of nCRF suppression of the cell. These properties of V1 neurons may contribute significantly to cue-invariant object perception.

Optical imaging of contrast response in Macaque monkey V1 and V2

Our studies on brightness information processing in Macaque monkey visual cortex suggest that the thin stripes in the secondary visual area (V2) are preferentially activated by brightness stimuli (such as full field luminance modulation and illusory edge-induced brightness modulation). To further examine this possibility, we used intrinsic signal optical imaging to examine contrast response of different functional domains in primary and secondary visual areas (V1 and V2). Color and orientation stimuli were used to map functional domains in V1 (color domains, orientation domains) and V2 (thin stripes, thick/pale stripes). To examine contrast response, sinusoidal gratings at different contrasts and spatial frequencies were presented. We find that, consistent with previous studies, the optical signal increased systematically with contrast level. Unlike single-unit responses, optical signals for both color domains and orientation domains in V1 exhibit linear contrast response functions, thereby providing a large dynamic range for V1 contrast response. In contrast to domains in V1, domains in V2 exhibit nonlinear responses, characterized by high gain at low contrasts, saturating at a mid-high contrast levels. At high contrasts, thin stripes exhibit increasing response, whereas thick/pale stripes saturate, consistent with a strong parvocellular input to thin stripes. These findings suggest that, with respect to contrast encoding, thin stripes have a larger dynamic range than thick/pale stripes and further support a role for thin stripes in processing of brightness information.

Cue-invariant networks for figure and background processing in human visual cortex

Lateral occipital cortical areas are involved in the perception of objects, but it is not clear how these areas interact with first tier visual areas. Using synthetic images portraying a simple texture-defined figure and an electrophysiological paradigm that allows us to monitor cortical responses to figure and background regions separately, we found distinct neuronal networks responsible for the processing of each region. The figure region of our displays was tagged with one temporal frequency (3.0 Hz) and the background region with another (3.6 Hz). Spectral analysis was used to separate the responses to the two regions during their simultaneous presentation. Distributed source reconstructions were made by using the minimum norm method, and cortical current density was measured in a set of visual areas defined on retinotopic and functional criteria with the use of functional magnetic resonance imaging. The results of the main experiments, combined with a set of control experiments, indicate that the figure region, but not the background, was routed preferentially to lateral cortex. A separate network extending from first tier through more dorsal areas responded preferentially to the background region. The figure-related responses were mostly invariant with respect to the texture types used to define the figure, did not depend on its spatial location or size, and mostly were unaffected by attentional instructions. Because of the emergent nature of a segmented figure in our displays, feedback from higher cortical areas is a likely candidate for the selection mechanism by which the figure region is routed to lateral occipital cortex.

Neural mechanisms mediating responses to abutting gratings: luminance edges vs. illusory contours

The discontinuities of phase-shifted abutting line gratings give rise to perception of an "illusory contour" (IC) along the line terminations. Neuronal responses to such ICs have been interpreted as evidence for a specialized visual mechanism, since such responses cannot be predicted from conventional linear receptive fields. However, when the spatial scale of the component gratings (carriers) is large compared to the neuron's luminance passband, these IC responses might be evoked simply by the luminance edges at the line terminations. Thus by presenting abutting gratings at a series of carrier spatial scales to cat A18 neurons, we were able to distinguish genuine nonlinear responses from those due to luminance edges. Around half of the neurons (both simple and complex types) showed a bimodal response pattern to abutting gratings: one peak at a low carrier spatial frequency range that overlapped with the luminance passband, and a second distinct peak at much higher frequencies beyond the neuron's grating resolution. For those bimodally responding neurons, the low-frequency responses were sensitive to carrier phase, but the high-frequency responses were phase-invariant. Thus the responses at low carrier spatial frequencies could be understood via a linear model, while the higher frequency responses represented genuine nonlinear IC processing. IC responsive neurons also demonstrated somewhat lower spatial preference to the periodic contours (envelopes) compared to gratings, but the optimal orientation and motion direction for both were quite similar. The nonlinear responses to ICs could be explained by the same energy mechanism underlying responses to second-order stimuli such as contrast-modulated gratings. Similar neuronal preferences for ICs and for gratings may contribute to the form-cue invariant perception of moving contours.