Surround suppression of V1 neurons mediates orientation-based representation of high-order visual features

Macaque V1 responses to 2nd-order contrast-modulated stimuli and the possible subcortical and cortical contributions

Natural images comprise contours and boundaries defined by 1st-order luminance-modulated (LM) cues that are readily encoded by V1 neurons, and 2nd-order contrast-modulated (CM) cues that carry local, but not over-the-space, luminance changes. The neurophysiological foundations for CM processing remain unsolved. Here we used two-photon calcium imaging to demonstrate that V1 superficial-layer neurons respond to both LM and CM gratings in awake, fixating, macaques, with overall LM responses stronger than CM responses. Furthermore, adaptation experiments revealed that LM responses were similarly suppressed by LM and CM adaptation, with moderately larger effects by iso-orientation adaptation than by orthogonal adaptation, suggesting that LM and CM orientation responses likely share a strong orientation-non-selective subcortical origin. In contrast, CM responses were substantially more suppressed by iso-orientation than by orthogonal LM and CM adaptation, likely suggesting stronger orientation-specific intracortical influences for CM responses than for LM responses, besides shared orientation-non-selective subcortical influences. These results thus may indicate a subcortical-to-V1 filter-rectify-filter mechanism for CM processing: Local luminance changes in CM stimuli are initially encoded by orientation-non-selective subcortical neurons, and the outputs are half-wave rectified, and then summed by V1 neurons to signal CM orientation, which may be further substantially refined by intracortical influences.

Population Models, Not Analyses, of Human Neuroscience Measurements

Selectivity for many basic properties of visual stimuli, such as orientation, is thought to be organized at the scale of cortical columns, making it difficult or impossible to measure directly with noninvasive human neuroscience measurement. However, computational analyses of neuroimaging data have shown that selectivity for orientation can be recovered by considering the pattern of response across a region of cortex. This suggests that computational analyses can reveal representation encoded at a finer spatial scale than is implied by the spatial resolution limits of measurement techniques. This potentially opens up the possibility to study a much wider range of neural phenomena that are otherwise inaccessible through noninvasive measurement. However, as we review in this article, a large body of evidence suggests an alternative hypothesis to this superresolution account: that orientation information is available at the spatial scale of cortical maps and thus easily measurable at the spatial resolution of standard techniques. In fact, a population model shows that this orientation information need not even come from single-unit selectivity for orientation tuning, but instead can result from population selectivity for spatial frequency. Thus, a categorical error of interpretation can result whereby orientation selectivity can be confused with spatial frequency selectivity. This is similarly problematic for the interpretation of results from numerous studies of more complex representations and cognitive functions that have built upon the computational techniques used to reveal stimulus orientation. We suggest in this review that these interpretational ambiguities can be avoided by treating computational analyses as models of the neural processes that give rise to measurement. Building upon the modeling tradition in vision science using considerations of whether population models meet a set of core criteria is important for creating the foundation for a cumulative and replicable approach to making valid inferences from human neuroscience measurements. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Stimulus vignetting and orientation selectivity in human visual cortex

Neural selectivity to orientation is one of the simplest and most thoroughly-studied cortical sensory features. Here, we show that a large body of research that purported to measure orientation tuning may have in fact been inadvertently measuring sensitivity to second-order changes in luminance, a phenomenon we term 'vignetting'. Using a computational model of neural responses in primary visual cortex (V1), we demonstrate the impact of vignetting on simulated V1 responses. We then used the model to generate a set of predictions, which we confirmed with functional MRI experiments in human observers. Our results demonstrate that stimulus vignetting can wholly determine the orientation selectivity of responses in visual cortex measured at a macroscopic scale, and suggest a reinterpretation of a well-established literature on orientation processing in visual cortex.

Contextual modulation of sensitivity to naturalistic image structure in macaque V2

The stimulus selectivity of neurons in V1 is well known, as is the finding that their responses can be affected by visual input to areas outside of the classical receptive field. Less well understood are the ways selectivity is modified as signals propagate to visual areas beyond V1, such as V2. We recently proposed a role for V2 neurons in representing the higher order statistical dependencies found in images of naturally occurring visual texture. V2 neurons, but not V1 neurons, respond more vigorously to "naturalistic" images that contain these dependencies than to "noise" images that lack them. In this work, we examine the dependency of these effects on stimulus size. For most V2 neurons, the preference for naturalistic over noise stimuli was modest when presented in small patches and gradually strengthened with increasing size, suggesting that the mechanisms responsible for this enhanced sensitivity operate over regions of the visual field that are larger than the classical receptive field. Indeed, we found that surround suppression was stronger for noise than for naturalistic stimuli and that the preference for large naturalistic stimuli developed over a delayed time course consistent with lateral or feedback connections. These findings are compatible with a spatially broad facilitatory mechanism that is absent in V1 and suggest that a distinct role for the receptive field surround emerges in V2 along with sensitivity for more complex image structure. NEW & NOTEWORTHY The responses of neurons in visual cortex are often affected by visual input delivered to regions of the visual field outside of the conventionally defined receptive field, but the significance of such contextual modulations are not well understood outside of area V1. We studied the importance of regions beyond the receptive field in establishing a novel form of selectivity for the statistical dependencies contained in natural visual textures that first emerges in area V2.

Bioplausible multiscale filtering in retino-cortical processing as a mechanism in perceptual grouping

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, 'Geometrical' and, in particular, 'Tilt Illusions' are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. The model is a variation of classical receptive field implementation for simple cells in early stages of vision with the scales tuned to the object/texture sizes in the pattern. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as 'Anchoring theory' and 'Perceptual grouping'.

Altered Balance of Receptive Field Excitation and Suppression in Visual Cortex of Amblyopic Macaque Monkeys

In amblyopia, a visual disorder caused by abnormal visual experience during development, the amblyopic eye (AE) loses visual sensitivity whereas the fellow eye (FE) is largely unaffected. Binocular vision in amblyopes is often disrupted by interocular suppression. We used 96-electrode arrays to record neurons and neuronal groups in areas V1 and V2 of six female macaque monkeys (Macaca nemestrina) made amblyopic by artificial strabismus or anisometropia in early life, as well as two visually normal female controls. To measure suppressive binocular interactions directly, we recorded neuronal responses to dichoptic stimulation. We stimulated both eyes simultaneously with large sinusoidal gratings, controlling their contrast independently with raised-cosine modulators of different orientations and spatial frequencies. We modeled each eye's receptive field at each cortical site using a difference of Gaussian envelopes and derived estimates of the strength of central excitation and surround suppression. We used these estimates to calculate ocular dominance separately for excitation and suppression. Excitatory drive from the FE dominated amblyopic visual cortex, especially in more severe amblyopes, but suppression from both the FE and AEs was prevalent in all animals. This imbalance created strong interocular suppression in deep amblyopes: increasing contrast in the AE decreased responses at binocular cortical sites. These response patterns reveal mechanisms that likely contribute to the interocular suppression that disrupts vision in amblyopes.SIGNIFICANCE STATEMENT Amblyopia is a developmental visual disorder that alters both monocular vision and binocular interaction. Using microelectrode arrays, we examined binocular interaction in primary visual cortex and V2 of six amblyopic macaque monkeys (Macaca nemestrina) and two visually normal controls. By stimulating the eyes dichoptically, we showed that, in amblyopic cortex, the binocular combination of signals is altered. The excitatory influence of the two eyes is imbalanced to a degree that can be predicted from the severity of amblyopia, whereas suppression from both eyes is prevalent in all animals. This altered balance of excitation and suppression reflects mechanisms that may contribute to the interocular perceptual suppression that disrupts vision in amblyopes.

Cross-orientation suppression in visual area V2

Object recognition relies on a series of transformations among which only the first cortical stage is relatively well understood. Already at the second stage, the visual area V2, the complexity of the transformation precludes a clear understanding of what specifically this area computes. Previous work has found multiple types of V2 neurons, with neurons of each type selective for multi-edge features. Here we analyse responses of V2 neurons to natural stimuli and find three organizing principles. First, the relevant edges for V2 neurons can be grouped into quadrature pairs, indicating invariance to local translation. Second, the excitatory edges have nearby suppressive edges with orthogonal orientations. Third, the resulting multi-edge patterns are repeated in space to form textures or texture boundaries. The cross-orientation suppression increases the sparseness of responses to natural images based on these complex forms of feature selectivity while allowing for multiple scales of position invariance.

V2 neurons exhibit complex and diverse selectivity for visual features. Here the authors use a statistical analytical framework to model V2 responses to natural stimuli and find three organizing principles, chief among them is the cross-orientation suppression that increases response selectivity.

Binocular functional architecture for detection of contrast-modulated gratings

Combination of signals from the two eyes is the gateway to stereo vision. To gain insight into binocular signal processing, we studied binocular summation for luminance-modulated gratings (L or LM) and contrast-modulated gratings (CM). We measured 2AFC detection thresholds for a signal grating (0.75c/deg, 216ms) shown to one eye, both eyes, or both eyes out-of-phase. For LM and CM, the carrier noise was in both eyes, even when the signal was monocular. Mean binocular thresholds for luminance gratings (L) were 5.4dB better than monocular thresholds - close to perfect linear summation (6dB). For LM and CM the binocular advantage was again 5-6dB, even when the carrier noise was uncorrelated, anti-correlated, or at orthogonal orientations in the two eyes. Binocular combination for CM probably arises from summation of envelope responses, and not from summation of these conflicting carrier patterns. Antiphase signals produced no binocular advantage, but thresholds were about 1-3dB higher than monocular ones. This is not consistent with simple linear summation, which should give complete cancellation and unmeasurably high thresholds. We propose a three-channel model in which noisy monocular responses to the envelope are binocularly combined in a contrast-weighted sum, but also remain separately available to perception via a max operator. Vision selects the largest of the three responses. With in-phase gratings the binocular channel dominates, but antiphase gratings cancel in the binocular channel and the monocular channels mediate detection. The small antiphase disadvantage might be explained by a subtle influence of background responses on binocular and monocular detection.

Mice Can Use Second-Order, Contrast-Modulated Stimuli to Guide Visual Perception

Visual processing along the primate ventral stream takes place in a hierarchy of areas, characterized by an increase in both complexity of neuronal preferences and invariance to changes of low-level stimulus attributes. A basic type of invariance is form-cue invariance, where neurons have similar preferences in response to first-order stimuli, defined by changes in luminance, and global features of second-order stimuli, defined by changes in texture or contrast. Whether in mice, a now popular model system for early visual processing, visual perception can be guided by second-order stimuli is currently unknown. Here, we probed mouse visual perception and neural responses in areas V1 and LM using various types of second-order, contrast-modulated gratings with static noise carriers. These gratings differ in their spatial frequency composition and thus in their ability to invoke first-order mechanisms exploiting local luminance features. We show that mice can transfer learning of a coarse orientation discrimination task involving first-order, luminance-modulated gratings to the contrast-modulated gratings, albeit with markedly reduced discrimination performance. Consistent with these behavioral results, we demonstrate that neurons in area V1 and LM are less responsive and less selective to contrast-modulated than to luminance-modulated gratings, but respond with broadly similar preferred orientations. We conclude that mice can, at least in a rudimentary form, use second-order stimuli to guide visual perception.

SIGNIFICANCE STATEMENT

To extract object boundaries in natural scenes, the primate visual system does not only rely on differences in local luminance but can also take into account differences in texture or contrast. Whether the mouse, which has a much simpler visual system, can use such second-order information to guide visual perception is unknown. Here we tested mouse perception of second-order, contrast-defined stimuli and measured their neural representations in two areas of visual cortex. We find that mice can use contrast-defined stimuli to guide visual perception, although behavioral performance and neural representations were less robust than for luminance-defined stimuli. These findings shed light on basic steps of feature extraction along the mouse visual cortical hierarchy, which may ultimately lead to object recognition.

Categorically distinct types of receptive fields in early visual cortex

In the visual cortex, distinct types of neurons have been identified based on cellular morphology, response to injected current, or expression of specific markers, but neurophysiological studies have revealed visual receptive field (RF) properties that appear to be on a continuum, with only two generally recognized classes: simple and complex. Most previous studies have characterized visual responses of neurons using stereotyped stimuli such as bars, gratings, or white noise and simple system identification approaches (e.g., reverse correlation). Here we estimate visual RF models of cortical neurons using visually rich natural image stimuli and regularized regression system identification methods and characterize their spatial tuning, temporal dynamics, spatiotemporal behavior, and spiking properties. We quantitatively demonstrate the existence of three functionally distinct categories of simple cells, distinguished by their degree of orientation selectivity (isotropic or oriented) and the nature of their output nonlinearity (expansive or compressive). In addition, these three types have differing average values of several other properties. Cells with nonoriented RFs tend to have smaller RFs, shorter response durations, no direction selectivity, and high reliability. Orientation-selective neurons with an expansive output nonlinearity have Gabor-like RFs, lower spontaneous activity and responsivity, and spiking responses with higher sparseness. Oriented RFs with a compressive nonlinearity are spatially nondescript and tend to show longer response latency. Our findings indicate multiple physiologically defined types of RFs beyond the simple/complex dichotomy, suggesting that cortical neurons may have more specialized functional roles rather than lying on a multidimensional continuum.

Contrast coding in the electrosensory system: parallels with visual computation

To identify and interact with moving objects, including other members of the same species, an animal's nervous system must correctly interpret patterns of contrast in the physical signals (such as light or sound) that it receives from the environment. In weakly electric fish, the motion of objects in the environment and social interactions with other fish create complex patterns of contrast in the electric fields that they produce and detect. These contrast patterns can extend widely over space and time and represent a multitude of relevant features, as is also true for other sensory systems. Mounting evidence suggests that the computational principles underlying contrast coding in electrosensory neural networks are conserved elements of spatiotemporal processing that show strong parallels with the vertebrate visual system.

The stabilized supralinear network: a unifying circuit motif underlying multi-input integration in sensory cortex

Neurons in sensory cortex integrate multiple influences to parse objects and support perception. Across multiple cortical areas, integration is characterized by two neuronal response properties: (1) surround suppression--modulatory contextual stimuli suppress responses to driving stimuli; and (2) "normalization"--responses to multiple driving stimuli add sublinearly. These depend on input strength: for weak driving stimuli, contextual influences facilitate or more weakly suppress and summation becomes linear or supralinear. Understanding the circuit operations underlying integration is critical to understanding cortical function and disease. We present a simple, general theory. A wealth of integrative properties, including the above, emerge robustly from four cortical circuit properties: (1) supralinear neuronal input/output functions; (2) sufficiently strong recurrent excitation; (3) feedback inhibition; and (4) simple spatial properties of intracortical connections. Integrative properties emerge dynamically as circuit properties, with excitatory and inhibitory neurons showing similar behaviors. In new recordings in visual cortex, we confirm key model predictions.

The efficiency of second order orientation coherence detection

Neurons in early visual cortex respond to both luminance- (1st order) and contrast-modulated (2nd order) local features in the visual field. In later extra-striate areas neurons with larger receptive fields integrate information across the visual field. For example, local luminance-defined features can be integrated into contours and shapes. Evidence for the global integration of features defined by contrast-modulation is less well established. While good performance in some shape tasks has been demonstrated with 2nd order stimuli, the integration of contours fails with 2nd order elements. Recently we developed a global orientation coherence task that is more basic than contour integration, bearing similarity to the well-established global motion coherence task. Similar to our previous 1st order result for this task, we find 2nd order coherence detection to be scale-invariant. There was a small but significant threshold elevation for 2nd order relative to 1st order. We used a noise masking approach to compare the efficiency of orientation integration for the 1st and 2nd order. We find a significant deficit for 2nd order detection at both the local and global level, however the small size of this effect stands in stark contrast against previous results from contour-integration experiments, which are almost impossible with 2nd order stimuli.

Form-cue invariant second-order neuronal responses to contrast modulation in primate area V2

A fundamental task of the visual system is to extract figure-ground boundaries between images of objects, which in natural scenes are often defined not only by luminance differences but also by "second-order" contrast or texture differences. Responses to contrast modulation (CM) and other second-order stimuli have been extensively studied in human psychophysics, but the neuronal substrates of second-order responses in nonhuman primates remain poorly understood. In this study, we have recorded single neurons in area V2 of macaque monkeys, using both CM patterns as well as conventional luminance modulation (LM) gratings. CM stimuli were constructed from stationary sine wave grating carrier patterns, which were modulated by drifting envelope gratings of a lower spatial frequency. We found approximately one-third of visually responsive V2 neurons responded to CM stimuli with a pronounced selectivity to carrier spatial frequencies, and often orientations, that were clearly outside the neurons' passbands for LM gratings. These neurons were "form-cue invariant" in that their tuning to CM envelope spatial frequency and orientation was very similar to that for LM gratings. Neurons were tuned to carrier spatial frequencies that were typically 2-4 octaves higher than their optimal envelope spatial frequencies, similar to results from human psychophysics. These results are distinct from CM responses arising from surround suppression, but could be understood in terms of a filter-rectify-filter model. Such neurons could provide a functionally useful and explicit representation of segmentation boundaries as well as a plausible neural substrate for human perception of second-order boundaries.

Possible functions of contextual modulations and receptive field nonlinearities: pop-out and texture segmentation

When analyzing a visual image, the brain has to achieve several goals quickly. One crucial goal is to rapidly detect parts of the visual scene that might be behaviorally relevant, while another one is to segment the image into objects, to enable an internal representation of the world. Both of these processes can be driven by local variations in any of several image attributes such as luminance, color, and texture. Here, focusing on texture defined by local orientation, we propose that the two processes are mediated by separate mechanisms that function in parallel. More specifically, differences in orientation can cause an object to "pop out" and attract visual attention, if its orientation differs from that of the surrounding objects. Differences in orientation can also signal a boundary between objects and therefore provide useful information for image segmentation. We propose that contextual response modulations in primary visual cortex (V1) are responsible for orientation pop-out, while a different kind of receptive field nonlinearity in secondary visual cortex (V2) is responsible for orientation-based texture segmentation. We review a recent experiment that led us to put forward this hypothesis along with other research literature relevant to this notion.

Surround suppression supports second-order feature encoding by macaque V1 and V2 neurons

Single neurons in areas V1 and V2 of macaque visual cortex respond selectively to luminance-modulated stimuli. These responses are often influenced by context, for example when stimuli extend outside the classical receptive field (CRF). These contextual phenomena, observed in many sensory areas, reflect a fundamental cortical computation and may inform perception by signaling second-order visual features which are defined by spatial relationships of contrast, orientation and spatial frequency. In the anesthetized, paralyzed macaque, we measured single-unit responses to a drifting preferred sinusoidal grating; low spatial frequency sinusoidal contrast modulations were applied to the grating, creating contrast-modulated, second-order forms. Most neurons responded selectively to the orientation of the contrast modulation of the preferred grating and were therefore second-order orientation-selective. Second-order selectivity was created by the asymmetric spatial organization of the excitatory CRF and suppressive extraclassical surround. We modeled these receptive field subregions using spatial Gaussians, sensitive to the modulation of contrast (not luminance) of the preferred carrier grating, that summed linearly and were capable of recovering asymmetrical receptive field organizations. Our modeling suggests that second-order selectivity arises both from elongated excitatory CRFs, asymmetrically organized extraclassical surround suppression, or both. We validated the model by successfully testing its predictions against conventional surround suppression measurements and spike-triggered analysis of second-order form responses. Psychophysical adaptation measurements on human observers revealed a pattern of second-order form selectivity consistent with neural response patterns. We therefore propose that cortical cells in primates do double duty, providing signals about both first- and second-order forms.

Orientation-cue invariant population responses to contrast-modulated and phase-reversed contour stimuli in macaque V1 and V2

Visual scenes can be readily decomposed into a variety of oriented components, the processing of which is vital for object segregation and recognition. In primate V1 and V2, most neurons have small spatio-temporal receptive fields responding selectively to oriented luminance contours (first order), while only a subgroup of neurons signal non-luminance defined contours (second order). So how is the orientation of second-order contours represented at the population level in macaque V1 and V2? Here we compared the population responses in macaque V1 and V2 to two types of second-order contour stimuli generated either by modulation of contrast or phase reversal with those to first-order contour stimuli. Using intrinsic signal optical imaging, we found that the orientation of second-order contour stimuli was represented invariantly in the orientation columns of both macaque V1 and V2. A physiologically constrained spatio-temporal energy model of V1 and V2 neuronal populations could reproduce all the recorded population responses. These findings suggest that, at the population level, the primate early visual system processes the orientation of second-order contours initially through a linear spatio-temporal filter mechanism. Our results of population responses to different second-order contour stimuli support the idea that the orientation maps in primate V1 and V2 can be described as a spatial-temporal energy map.

Pooling of first-order inputs in second-order vision

The processing of texture patterns has been characterized by a model that first filters the image to isolate one texture component, then applies a rectifying nonlinearity that converts texture variation into intensity variation, and finally processes the resulting pattern with mechanisms similar to those used in processing luminance-defined images (spatial-frequency- and orientation-tuned filters). This model, known as FRF for filter rectify filter, has the appeal of explaining sensitivity to second-order patterns in terms of mechanisms known to exist for processing first-order patterns. This model implies an unexpected interaction between the first and second stages of filtering; if the first-stage filter consists of narrowband mechanisms tuned to detect the carrier texture, then sensitivity to high-frequency texture modulations should be much lower than is observed in humans. We propose that the human visual system must pool over first-order channels tuned to a wide range of spatial frequencies and orientations to achieve texture demodulation, and provide psychophysical evidence for pooling in a cross-carrier adaptation experiment and in an experiment that measures modulation contrast sensitivity at very low first-order contrast.

Texture sparseness, but not local phase structure, impairs second-order segmentation

Texture boundary segmentation is typically thought to reflect a comparison of differences in Fourier energy (i.e. low-order texture statistics) on either side of a boundary. However in a previous study (Arsenault, Yoonessi, & Baker, 2011) we showed that the distribution of energy within a natural texture (i.e. its higher-order statistical structure) also influences segmentation of contrast boundaries. Here we examine the influence of specific higher-order texture statistics on segmentation of contrast- and orientation-defined boundaries. Using naturalistic synthetic textures to manipulate the sparseness, global phase structure, and local phase alignments of carrier textures, we measure segmentation thresholds based on forced-choice judgments of boundary orientation. We find a similar pattern of results for both contrast and orientation boundaries: (1) randomizing all structure by globally phase scrambling the texture reduces segmentation thresholds substantially, (2) decreasing sparseness also reduces thresholds, and (3) removing local phase alignments has little or no effect on segmentation thresholds. We show that a two-stage filter model with an intermediate compressive nonlinearity and expansive output nonlinearity can account for these data using synthetic textures. Furthermore, the model parameter fits obtained using synthetic textures also predict the segmentation thresholds presented in Arsenault, Yoonessi, and Baker (2011) for natural and phase-scrambled natural texture carriers.

A two-stage cascade model of BOLD responses in human visual cortex

Visual neuroscientists have discovered fundamental properties of neural representation through careful analysis of responses to controlled stimuli. Typically, different properties are studied and modeled separately. To integrate our knowledge, it is necessary to build general models that begin with an input image and predict responses to a wide range of stimuli. In this study, we develop a model that accepts an arbitrary band-pass grayscale image as input and predicts blood oxygenation level dependent (BOLD) responses in early visual cortex as output. The model has a cascade architecture, consisting of two stages of linear and nonlinear operations. The first stage involves well-established computations—local oriented filters and divisive normalization—whereas the second stage involves novel computations—compressive spatial summation (a form of normalization) and a variance-like nonlinearity that generates selectivity for second-order contrast. The parameters of the model, which are estimated from BOLD data, vary systematically across visual field maps: compared to primary visual cortex, extrastriate maps generally have larger receptive field size, stronger levels of normalization, and increased selectivity for second-order contrast. Our results provide insight into how stimuli are encoded and transformed in successive stages of visual processing.

Much has been learned about how stimuli are represented in the visual system from measuring responses to carefully designed stimuli. Typically, different studies focus on different types of stimuli. Making sense of the large array of findings requires integrated models that explain responses to a wide range of stimuli. In this study, we measure functional magnetic resonance imaging (fMRI) responses in early visual cortex to a wide range of band-pass filtered images, and construct a computational model that takes the stimuli as input and predicts the fMRI responses as output. The model has a cascade architecture, consisting of two stages of linear and nonlinear operations. A novel component of the model is a nonlinear operation that generates selectivity for second-order contrast, that is, variations in contrast-energy across the visual field. We find that this nonlinearity is stronger in extrastriate areas V2 and V3 than in primary visual cortex V1. Our results provide insight into how stimuli are encoded and transformed in the visual system.

Inconsistent channel bandwidth estimates suggest winner-take-all nonlinearity in second-order vision

The processing of texture patterns has been characterized by a model that postulates a first-stage linear filter to highlight a component texture, a pointwise rectification stage to convert contrast for the highlighted texture into mean response strength, followed by a second-stage linear filter to detect the texture-defined pattern. We estimated the spatial-frequency bandwidth of the second-stage filter mediating orientation discrimination of orientation-modulated second-order gratings by measuring threshold elevation in the presence of filtered noise added to the modulation signal. This experiment yielded no evidence for frequency tuning. A second experiment, in which subjects had to detect similar second-order gratings while judging their modulation frequency, produced bandwidth estimates of 1-1.5 octaves, similar to estimated bandwidths of first-order channels. We propose that an additional dominant-response-selection nonlinearity can account for these apparently contradictory results.

Natural versus synthetic stimuli for estimating receptive field models: a comparison of predictive robustness

An ultimate goal of visual neuroscience is to understand the neural encoding of complex, everyday scenes. Yet most of our knowledge of neuronal receptive fields has come from studies using simple artificial stimuli (e.g., bars, gratings) that may fail to reveal the full nature of a neuron's actual response properties. Our goal was to compare the utility of artificial and natural stimuli for estimating receptive field (RF) models. Using extracellular recordings from simple type cells in cat A18, we acquired responses to three types of broadband stimulus ensembles: two widely used artificial patterns (white noise and short bars), and natural images. We used a primary dataset to estimate the spatiotemporal receptive field (STRF) with two hold-back datasets for regularization and validation. STRFs were estimated using an iterative regression algorithm with regularization and subsequently fit with a zero-memory nonlinearity. Each RF model (STRF and zero-memory nonlinearity) was then used in simulations to predict responses to the same stimulus type used to estimate it, as well as to other broadband stimuli and sinewave gratings. White noise stimuli often elicited poor responses leading to noisy RF estimates, while short bars and natural image stimuli were more successful in driving A18 neurons and producing clear RF estimates with strong predictive ability. Natural image-derived RF models were the most robust at predicting responses to other broadband stimulus ensembles that were not used in their estimation and also provided good predictions of tuning curves for sinewave gratings.

Local sensitivity to stimulus orientation and spatial frequency within the receptive fields of neurons in visual area 2 of macaque monkeys

We used dynamic dense noise stimuli and local spectral reverse correlation methods to reveal the local sensitivities of neurons in visual area 2 (V2) of macaque monkeys to orientation and spatial frequency within their receptive fields. This minimized the potentially confounding assumptions that are inherent in stimulus selections. The majority of neurons exhibited a relatively high degree of homogeneity for the preferred orientations and spatial frequencies in the spatial matrix of facilitatory subfields. However, about 20% of all neurons showed maximum orientation differences between neighboring subfields that were greater than 25 deg. The neurons preferring horizontal or vertical orientations showed less inhomogeneity in space than the neurons preferring oblique orientations. Over 50% of all units also exhibited suppressive profiles, and those were more heterogeneous than facilitatory profiles. The preferred orientation and spatial frequency of suppressive profiles differed substantially from those of facilitatory profiles, and the neurons with suppressive subfields had greater orientation selectivity than those without suppressive subfields. The peak suppression occurred with longer delays than the peak facilitation. These results suggest that the receptive field profiles of the majority of V2 neurons reflect the orderly convergence of V1 inputs over space, but that a subset of V2 neurons exhibit more complex response profiles having both suppressive and facilitatory subfields. These V2 neurons with heterogeneous subfield profiles could play an important role in the initial processing of complex stimulus features.

Neuronal responses to texture-defined form in macaque visual area V2

Human and macaque observers can detect and discriminate visual forms defined by differences in texture. The neurophysiological correlates of visual texture perception are not well understood and have not been studied extensively at the single-neuron level in the primate brain. We used a novel family of texture patterns to measure the selectivity of neurons in extrastriate cortical area V2 of the macaque (Macaca nemestrina, Macaca fascicularis) for the orientation of texture-defined form, and to distinguish responses to luminance- and texture-defined form. Most V2 cells were selective for the orientation of luminance-defined form; they signaled the orientation of the component gratings that made up the texture patterns but not the overall pattern orientation. In some cells, these luminance responses were modulated by the direction or orientation of the texture envelope, suggesting an interaction of luminance and texture signals. We found little evidence for a "cue-invariant" representation in monkey V2. Few cells showed selectivity for the orientation of texture-defined form; they signaled the orientation of the texture patterns and not that of the component gratings. Small datasets recorded in monkey V1 and cat area 18 showed qualitatively similar patterns of results. Consistent with human functional imaging studies, our findings suggest that signals related to texture-defined form in primate cortex are most salient in areas downstream of V2. V2 may still provide the foundation for texture perception, through the interaction of luminance- and texture-based signals.

Human primary visual cortex (V1) is selective for second-order spatial frequency

A variety of cues can differentiate objects from their surrounds. These include "first-order" cues such as luminance modulations and "second-order" cues involving modulations of orientation and contrast. Human sensitivity to first-order modulations is well described by a computational model involving spatially localized filters that are selective for orientation and spatial frequency (SF). It is widely held that first-order modulations are represented by the firing rates of simple and complex cells ("first-order" neurons) in primary visual cortex (V1) that, likewise, have spatially localized receptive fields that are selective for orientation- and SF. Human sensitivity to second-order modulations is well described by a filter-rectify-filter (FRF) model, with first- and second-order filters selective for orientation and SF. However, little is known about how neuronal activity in visual cortex represents second-order modulations. We tested the FRF model by using an functional (f)MRI-adaptation protocol to characterize the selectivity of activity in visual cortex to second-order, orientation-defined gratings of two different SFs. fMRI responses throughout early visual cortex exhibited selective adaptation to these stimuli. The low-SF grating was a more effective adapter than the high-SF grating, incompatible with the FRF model. To explain the results, we extended the FRF model by incorporating normalization, yielding a filter-rectify-normalize-filter model, in which normalization enhances selectivity for second-order SF but only for low spatial frequencies. We conclude that neurons in human visual cortex are selective for second-order SF, that normalization (surround suppression) contributes to this selectivity, and that the selectivity in higher visual areas is simply fed forward from V1.

Development of sensitivity to visual texture modulation in macaque monkeys

In human and non-human primates, higher form vision matures substantially later than spatial acuity and contrast sensitivity, as revealed by performance on such tasks as figure-ground segregation and contour integration. Our goal was to understand whether delayed maturation on these tasks was intrinsically form-dependent or, rather, related to the nature of spatial integration necessary for extracting task-relevant cues. We used an intermediate-level form task that did not call for extensive spatial integration. We trained monkeys (6-201 weeks) to discriminate the orientation of pattern modulation in a two-alternative forced choice paradigm. We presented two families of form patterns, defined by texture or contrast variations, and luminance-defined patterns for comparison. Infant monkeys could discriminate texture- and contrast-defined form as early as 6 weeks; sensitivity improved up to 40 weeks. Surprisingly, sensitivity for texture- and contrast-defined form matured earlier than for luminance-defined form. These results suggest that intermediate-level form vision develops in concert with basic spatial vision rather than following sequentially. Comparison with earlier results reveals that different aspects of form vision develop over different time courses, with processes that depend on comparing local image content maturing earlier than those requiring "global" linking of multiple visual elements across a larger spatial extent.

A FPGA real-time model of single and multiple visual cortex neurons

Using a biologically realistic model of a single neuron can be very beneficial for visual physiologists to test their electrophysiology setups, train students in the laboratory, or conduct classroom-teaching demonstrations. Here we present a Field Programmable Gate Array (FPGA)-based spiking model of visual cortex neurons, which has the ability to simulate three independent neurons and output analog spike waveform signals in four channels. To realistically simulate multi-electrode (tetrode) recordings, the independently generated spikes of each simulated neuron has a distinct waveform, and each channel outputs a differentially weighted sum of these waveforms. The model can be easily constructed from a small number of inexpensive commercially available parts, and is straightforward to operate. In response to sinewave grating stimuli, the neurons exhibit biologically realistic simple-cell-like response properties, including highly modulated Poisson spike trains, orientation selectivity, spatial/temporal frequency selectivity, and space-time receptive fields. Users can customize their model neurons by downloading modifications to the FPGA with varying parameter values, particularly desired features, or qualitatively different models of their own design. The source code and documentation are provided to enable users to modify or extend the model's functionality according to their individual needs.

Suppression pathways saturate with contrast for parallel surrounds but not for superimposed cross-oriented masks

Contrast masking from parallel grating surrounds (doughnuts) and superimposed orthogonal masks have different characteristics. However, it is not known whether the saturation of the underlying suppression that has been found for parallel doughnut masks depends on (i) relative mask and target orientation, (ii) stimulus eccentricity or (iii) surround suppression. We measured contrast-masking functions for target patches of grating in the fovea and in the periphery for cross-oriented superimposed and doughnut masks and parallel doughnut masks. When suppression was evident, the factor that determined whether it accelerated or saturated was whether the mask stimulus was crossed or parallel. There are at least two interpretations of the asymptotic behaviour of the parallel surround mask. (1) Suppression arises from pathways that saturate with (mask) contrast. (2) The target is processed by a mechanism that is subject to surround suppression at low target contrasts, but a less sensitive mechanism that is immune from surround suppression 'breaks through' at higher target contrasts. If the mask can be made less potent, then masking functions should shift downwards, and sideways for the two accounts, respectively. We manipulated the potency of the mask by varying the size of the hole in a parallel doughnut mask. The results provided strong evidence for the first account but not the second. On the view that response compression becomes more severe progressing up the visual pathway, our results suggest that superimposed cross-orientation suppression precedes orientation tuned surround suppression. These results also reveal a previously unrecognized similarity between surround suppression and crowding (Pelli, Palomares, & Majaj, 2004).