Influence of the direction of elemental luminance gradients on the responses of V4 cells to textured surfaces

Pictorial depth cues elicit the perception of tridimensionality in dogs

The perception of tridimensionality is elicited by binocular disparity, motion parallax, and monocular or pictorial cues. The perception of tridimensionality arising from pictorial cues has been investigated in several non-human animal species. Although dogs can use and discriminate bidimensional images, to date there is no evidence of dogs' ability to perceive tridimensionality in pictures and/or through pictorial cues. The aim of the present study was to assess the perception of tridimensionality in dogs elicited by two pictorial cues: linear perspective and shading. Thirty-two dogs were presented with a tridimensional stimulus (i.e., a ball) rolling onto a planar surface until eventually falling into a hole (control condition) or until reaching and rolling over an illusory hole (test condition). The illusory hole corresponded to the bidimensional pictorial representation of the real hole, in which the pictorial cues of shading and linear perspective created the impression of tridimensionality. In a violation of expectation paradigm, dogs showed a longer looking time at the scene in which the unexpected situation of a ball rolling over an illusory hole occurred. The surprise reaction observed in the test condition suggests that the pictorial cues of shading and linear perspective in the bidimensional image of the hole were able to elicit the perception of tridimensionality in dogs.

Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks

Responses to natural stimuli in area V4-a mid-level area of the visual ventral stream-are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function that are not captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4's main functional role in semantic processing and suggesting that V4's selectivity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals.

Efficient coding of natural images in the mouse visual cortex

How the activity of neurons gives rise to natural vision remains a matter of intense investigation. The mid-level visual areas along the ventral stream are selective to a common class of natural images-textures-but a circuit-level understanding of this selectivity and its link to perception remains unclear. We addressed these questions in mice, first showing that they can perceptually discriminate between textures and statistically simpler spectrally matched stimuli, and between texture types. Then, at the neural level, we found that the secondary visual area (LM) exhibited a higher degree of selectivity for textures compared to the primary visual area (V1). Furthermore, textures were represented in distinct neural activity subspaces whose relative distances were found to correlate with the statistical similarity of the images and the mice's ability to discriminate between them. Notably, these dependencies were more pronounced in LM, where the texture-related subspaces were smaller than in V1, resulting in superior stimulus decoding capabilities. Together, our results demonstrate texture vision in mice, finding a linking framework between stimulus statistics, neural representations, and perceptual sensitivity-a distinct hallmark of efficient coding computations.

Second-order boundaries segment more easily when they are density-defined rather than feature-defined

Previous studies have demonstrated that density is an important perceptual aspect of textural appearance to which the visual system is highly attuned. Furthermore, it is known that density cues not only influence texture segmentation, but can enable segmentation by themselves, in the absence of other cues. A popular computational model of texture segmentation known as the "Filter-Rectify-Filter" (FRF) model predicts that density should be a second-order cue enabling segmentation. For a compound texture boundary defined by superimposing two single-micropattern density boundaries, a version of the FRF model in which different micropattern-specific channels are analyzed separately by different second-stage filters makes the prediction that segmentation thresholds should be identical in two cases: (1) Compound boundaries with an equal number of micropatterns on each side but different relative proportions of each variety (compound feature boundaries) and (2) Compound boundaries with different numbers of micropatterns on each side, but with each side having an identical number of each variety (compound density boundaries). We directly tested this prediction by comparing segmentation thresholds for second-order compound feature and density boundaries, comprised of two superimposed single-micropattern density boundaries comprised of complementary micropattern pairs differing either in orientation or contrast polarity. In both cases, we observed lower segmentation thresholds for compound density boundaries than compound feature boundaries, with identical results when the compound density boundaries were equated for RMS contrast. In a second experiment, we considered how two varieties of micropatterns summate for compound boundary segmentation. In the case where two single micro-pattern density boundaries are superimposed to form a compound density boundary, we find that the two channels combine via probability summation. By contrast, when they are superimposed to form a compound feature boundary, segmentation performance is worse than for either channel alone. From these findings, we conclude that density segmentation may rely on neural mechanisms different from those which underlie feature segmentation, consistent with recent findings suggesting that density comprises a separate psychophysical 'channel'.

A stimulus-driven approach reveals vertical luminance gradient as a stimulus feature that drives human cortical scene selectivity

Human neuroimaging studies have revealed a dedicated cortical system for visual scene processing. But what is a "scene"? Here, we use a stimulus-driven approach to identify a stimulus feature that selectively drives cortical scene processing. Specifically, using fMRI data from BOLD5000, we examined the images that elicited the greatest response in the cortical scene processing system, and found that there is a common "vertical luminance gradient" (VLG), with the top half of a scene image brighter than the bottom half; moreover, across the entire set of images, VLG systematically increases with the neural response in the scene-selective regions (Study 1). Thus, we hypothesized that VLG is a stimulus feature that selectively engages cortical scene processing, and directly tested the role of VLG in driving cortical scene selectivity using tightly controlled VLG stimuli (Study 2). Consistent with our hypothesis, we found that the scene-selective cortical regions-but not an object-selective region or early visual cortex-responded significantly more to images of VLG over control stimuli with minimal VLG. Interestingly, such selectivity was also found for images with an "inverted" VLG, resembling the luminance gradient in night scenes. Finally, we also tested the behavioral relevance of VLG for visual scene recognition (Study 3); we found that participants even categorized tightly controlled stimuli of both upright and inverted VLG to be a place more than an object, indicating that VLG is also used for behavioral scene recognition. Taken together, these results reveal that VLG is a stimulus feature that selectively engages cortical scene processing, and provide evidence for a recent proposal that visual scenes can be characterized by a set of common and unique visual features.

Visual Functions of Primate Area V4

Area V4-the focus of this review-is a mid-level processing stage along the ventral visual pathway of the macaque monkey. V4 is extensively interconnected with other visual cortical areas along the ventral and dorsal visual streams, with frontal cortical areas, and with several subcortical structures. Thus, it is well poised to play a broad and integrative role in visual perception and recognition-the functional domain of the ventral pathway. Neurophysiological studies in monkeys engaged in passive fixation and behavioral tasks suggest that V4 responses are dictated by tuning in a high-dimensional stimulus space defined by form, texture, color, depth, and other attributes of visual stimuli. This high-dimensional tuning may underlie the development of object-based representations in the visual cortex that are critical for tracking, recognizing, and interacting with objects. Neurophysiological and lesion studies also suggest that V4 responses are important for guiding perceptual decisions and higher-order behavior.

Deep neural networks capture texture sensitivity in V2

Deep convolutional neural networks (CNNs) trained on visual objects have shown intriguing ability to predict some response properties of visual cortical neurons. However, the factors (e.g., if the model is trained or not, receptive field size) and computations (e.g., convolution, rectification, pooling, normalization) that give rise to such ability, at what level, and the role of intermediate processing stages in explaining changes that develop across areas of the cortical hierarchy are poorly understood. We focused on the sensitivity to textures as a paradigmatic example, since recent neurophysiology experiments provide rich data pointing to texture sensitivity in secondary (but not primary) visual cortex (V2). We initially explored the CNN without any fitting to the neural data and found that the first two layers of the CNN showed qualitative correspondence to the first two cortical areas in terms of texture sensitivity. We therefore developed a quantitative approach to select a population of CNN model neurons that best fits the brain neural recordings. We found that the CNN could develop compatibility to secondary cortex in the second layer following rectification and that this was improved following pooling but only mildly influenced by the local normalization operation. Higher layers of the CNN could further, though modestly, improve the compatibility with the V2 data. The compatibility was reduced when incorporating random rather than learned weights. Our results show that the CNN class of model is effective for capturing changes that develop across early areas of cortex, and has the potential to help identify the computations that give rise to hierarchical processing in the brain (code is available in GitHub).

Modeling second-order boundary perception: A machine learning approach

Visual pattern detection and discrimination are essential first steps for scene analysis. Numerous human psychophysical studies have modeled visual pattern detection and discrimination by estimating linear templates for classifying noisy stimuli defined by spatial variations in pixel intensities. However, such methods are poorly suited to understanding sensory processing mechanisms for complex visual stimuli such as second-order boundaries defined by spatial differences in contrast or texture. We introduce a novel machine learning framework for modeling human perception of second-order visual stimuli, using image-computable hierarchical neural network models fit directly to psychophysical trial data. This framework is applied to modeling visual processing of boundaries defined by differences in the contrast of a carrier texture pattern, in two different psychophysical tasks: (1) boundary orientation identification, and (2) fine orientation discrimination. Cross-validation analysis is employed to optimize model hyper-parameters, and demonstrate that these models are able to accurately predict human performance on novel stimulus sets not used for fitting model parameters. We find that, like the ideal observer, human observers take a region-based approach to the orientation identification task, while taking an edge-based approach to the fine orientation discrimination task. How observers integrate contrast modulation across orientation channels is investigated by fitting psychophysical data with two models representing competing hypotheses, revealing a preference for a model which combines multiple orientations at the earliest possible stage. Our results suggest that this machine learning approach has much potential to advance the study of second-order visual processing, and we outline future steps towards generalizing the method to modeling visual segmentation of natural texture boundaries. This study demonstrates how machine learning methodology can be fruitfully applied to psychophysical studies of second-order visual processing.

Many naturally occurring visual boundaries are defined by spatial differences in features other than luminance, for example by differences in texture or contrast. Quantitative models of such “second-order” boundary perception cannot be estimated using the standard regression techniques (known as “classification images”) commonly applied to “first-order”, luminance-defined stimuli. Here we present a novel machine learning approach to modeling second-order boundary perception using hierarchical neural networks. In contrast to previous quantitative studies of second-order boundary perception, we directly estimate network model parameters using psychophysical trial data. We demonstrate that our method can reveal different spatial summation strategies that human observers utilize for different kinds of second-order boundary perception tasks, and can be used to compare competing hypotheses of how contrast modulation is integrated across orientation channels. We outline extensions of the methodology to other kinds of second-order boundaries, including those in natural images.

Computational Model for Human 3D Shape Perception From a Single Specular Image

In natural conditions the human visual system can estimate the 3D shape of specular objects even from a single image. Although previous studies suggested that the orientation field plays a key role for 3D shape perception from specular reflections, its computational plausibility, and possible mechanisms have not been investigated. In this study, to complement the orientation field information, we first add prior knowledge that objects are illuminated from above and utilize the vertical polarity of the intensity gradient. Then we construct an algorithm that incorporates these two image cues to estimate 3D shapes from a single specular image. We evaluated the algorithm with glossy and mirrored surfaces and found that 3D shapes can be recovered with a high correlation coefficient of around 0.8 with true surface shapes. Moreover, under a specific condition, the algorithm's errors resembled those made by human observers. These findings show that the combination of the orientation field and the vertical polarity of the intensity gradient is computationally sufficient and probably reproduces essential representations used in human shape perception from specular reflections.

Gradual Development of Visual Texture-Selective Properties Between Macaque Areas V2 and V4

Complex shape and texture representations are known to be constructed from V1 along the ventral visual pathway through areas V2 and V4, but the underlying mechanism remains elusive. Recent study suggests that, for processing of textures, a collection of higher-order image statistics computed by combining V1-like filter responses serves as possible representations of textures both in V2 and V4. Here, to gain a clue for how these image statistics are processed in the extrastriate visual areas, we compared neuronal responses to textures in V2 and V4 of macaque monkeys. For individual neurons, we adaptively explored their preferred textures from among thousands of naturalistic textures and fitted the obtained responses using a combination of V1-like filter responses and higher-order statistics. We found that, while the selectivity for image statistics was largely comparable between V2 and V4, V4 showed slightly stronger sensitivity to the higher-order statistics than V2. Consistent with that finding, V4 responses were reduced to a greater extent than V2 responses when the monkeys were shown spectrally matched noise images that lacked higher-order statistics. We therefore suggest that there is a gradual development in representation of higher-order features along the ventral visual hierarchy.

Joint coding of shape and blur in area V4

Edge blur, a prevalent feature of natural images, is believed to facilitate multiple visual processes including segmentation and depth perception. Furthermore, image descriptions that explicitly combine blur and shape information provide complete representations of naturalistic scenes. Here we report the first demonstration of blur encoding in primate visual cortex: neurons in macaque V4 exhibit tuning for both object shape and boundary blur, with observed blur tuning not explained by potential confounds including stimulus size, intensity, or curvature. A descriptive model wherein blur selectivity is cast as a distinct neural process that modulates the gain of shape-selective V4 neurons explains observed data, supporting the hypothesis that shape and blur are fundamental features of a sufficient neural code for natural image representation in V4.

Blurred edges of objects can aid in depth perception and segmentation, yet how it is combined with shape information in the visual pathway is unknown. Here the authors report that neurons in higher visual area V4 represent both object shape and boundary blur, controlling for stimulus size, intensity and curvature.

Feature-coding transitions to conjunction-coding with progression through human visual cortex

Identifying an object and distinguishing it from similar items depends upon the ability to perceive its component parts as conjoined into a cohesive whole, but the brain mechanisms underlying this ability remain elusive. The ventral visual processing pathway in primates is organized hierarchically: Neuronal responses in early stages are sensitive to the manipulation of simple visual features, whereas neuronal responses in subsequent stages are tuned to increasingly complex stimulus attributes. It is widely assumed that feature-coding dominates in early visual cortex whereas later visual regions employ conjunction-coding in which object representations are different from the sum of their simple feature parts. However, no study in humans has demonstrated that putative object-level codes in higher visual cortex cannot be accounted for by feature-coding and that putative feature codes in regions prior to ventral temporal cortex are not equally well characterized as object-level codes. Thus the existence of a transition from feature- to conjunction-coding in human visual cortex remains unconfirmed, and if a transition does occur its location remains unknown. By employing multivariate analysis of functional imaging data, we measure both feature-coding and conjunction-coding directly, using the same set of visual stimuli, and pit them against each other to reveal the relative dominance of one vs. the other throughout cortex. Our results reveal a transition from feature-coding in early visual cortex to conjunction-coding in both inferior temporal and posterior parietal cortices. This novel method enables the use of experimentally controlled stimulus features to investigate population-level feature and conjunction codes throughout human cortex.NEW & NOTEWORTHY We use a novel analysis of neuroimaging data to assess representations throughout visual cortex, revealing a transition from feature-coding to conjunction-coding along both ventral and dorsal pathways. Occipital cortex contains more information about spatial frequency and contour than about conjunctions of those features, whereas inferotemporal and parietal cortices contain conjunction coding sites in which there is more information about the whole stimulus than its component parts.

Biologically Inspired Model for Inference of 3D Shape from Texture

A biologically inspired model architecture for inferring 3D shape from texture is proposed. The model is hierarchically organized into modules roughly corresponding to visual cortical areas in the ventral stream. Initial orientation selective filtering decomposes the input into low-level orientation and spatial frequency representations. Grouping of spatially anisotropic orientation responses builds sketch-like representations of surface shape. Gradients in orientation fields and subsequent integration infers local surface geometry and globally consistent 3D depth. From the distributions in orientation responses summed in frequency, an estimate of the tilt and slant of the local surface can be obtained. The model suggests how 3D shape can be inferred from texture patterns and their image appearance in a hierarchically organized processing cascade along the cortical ventral stream. The proposed model integrates oriented texture gradient information that is encoded in distributed maps of orientation-frequency representations. The texture energy gradient information is defined by changes in the grouped summed normalized orientation-frequency response activity extracted from the textured object image. This activity is integrated by directed fields to generate a 3D shape representation of a complex object with depth ordering proportional to the fields output, with higher activity denoting larger distance in relative depth away from the viewer.

Perceptual Grouping in Shape from Shading

We report on the reversal of asymmetry in visual-search tasks with shaded items. Previous studies have suggested that the target of a bottom-lit disk among distractors of top-lit disks is detected in a rapid and parallel manner, but not vice versa. However, in this study, we have shown that the compound items of top-lit disks were searched more quickly than those composed of bottom-lit disks where the items had to be segregated from their background. By modulating the inter-element distances, we confirmed that the reversal of search asymmetry cannot be due to the grouping of items. Further, we showed that the regions of the top-lit disks were perceived as figure more consistently than those of bottom-lit disks. The results indicate that the boundary assignment to the compound items of the top-lit disks enhances the segregation of search items from the background, and that the search mechanism may access the relatively higher representation that includes figure – ground relations.

Linear and non-linear properties of feature selectivity in V4 neurons

Extrastriate area V4 is a critical component of visual form processing in both humans and non-human primates. Previous studies have shown that the tuning properties of V4 neurons demonstrate an intermediate level of complexity that lies between the narrow band orientation and spatial frequency tuning of neurons in primary visual cortex and the highly complex object selectivity seen in inferotemporal neurons. However, the nature of feature selectivity within this cortical area is not well understood, especially in the context of natural stimuli. Specifically, little is known about how the tuning properties of V4 neurons, measured in isolation, translate to feature selectivity within natural scenes. In this study, we assessed the degree to which preferences for natural image components can readily be inferred from classical orientation and spatial frequency tuning functions. Using a psychophysically-inspired method we isolated and identified the specific visual “driving features” occurring in natural scene photographs that reliably elicited spiking activity from single V4 neurons. We then compared the measured driving features to those predicted based on the spectral receptive field (SRF), estimated from responses to narrowband sinusoidal grating stimuli. This approach provided a quantitative framework for assessing the degree to which linear feature selectivity was preserved during natural vision. First, we found evidence of both spectrally and spatially tuned suppression within the receptive field, neither of which were present in the linear SRF. Second, we found driving features that were stable during translation of the image across the receptive field (due to small fixational eye movements). The degree of translation invariance fell along a continuum, with some cells showing nearly complete invariance across the receptive field and others exhibiting little to no position invariance. This form of limited translation invariance could indicate that a subset of V4 neurons are insensitive to small fixational eye movements, supporting perceptual stability during natural vision.

Image statistics underlying natural texture selectivity of neurons in macaque V4

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.

Selective responses to specular surfaces in the macaque visual cortex revealed by fMRI

The surface properties of objects, such as gloss, transparency and texture, provide important information about the material characteristics of objects in our visual environment. However, because there have been few reports on the neuronal responses to surface properties in primates, we still lack information about where and how surface properties are processed in the primate visual cortex. In this study, we used functional magnetic resonance imaging (fMRI) to examine the cortical responses to specular surfaces in the macaque visual cortex. Using computer graphics, we generated images of specular and matte objects and prepared scrambled images by locally randomizing the luminance phases of the images with specular and matte objects. In experiment 1, we contrasted the responses to specular images with those to matte and scrambled images. Activation was observed along the ventral visual pathway, including V1, V2, V3, V4 and the posterior inferior temporal (IT) cortex. In experiment 2, we manipulated the contrasts of images and found that the activation observed in these regions could not be explained solely by the global or local contrasts. These results suggest that image features related to specular surface are processed along the ventral visual pathway from V1 to specific regions in the IT cortex. This is consistent with previous human fMRI experiments that showed surface properties are processed in the ventral visual pathway.

The extraction of 3D shape in the visual system of human and nonhuman primates

Depth structure, the third dimension of object shape, is extracted from disparity, motion, texture, and shading in the optic array. Gradient-selective neurons play a key role in this process. Such neurons occur in CIP, AIP, TEs, and F5 (for first- or second-order disparity gradients), in MT/V5, in FST (for speed gradients), and in CIP and TEs (for texture gradients). Most of these regions are activated during magnetic resonance scanning in alert monkeys by comparing 3D conditions with the 2D controls for the different cues. Similarities in activation patterns of monkeys and humans tested with identical paradigms suggest that like gradient-selective neurons are found in corresponding human cortical areas. This view gains credence as the homologies between such areas become more evident. Furthermore, 3D shape-processing networks are similar in the two species, with the exception of the greater involvement of human posterior parietal cortex in the extraction of 3D shape from motion. Thus we can begin to understand how depth structure is extracted from motion, disparity, and texture in the primate brain, but the extraction of depth structure from shading and that of wire-like objects requires further scrutiny.

Transformation from image-based to perceptual representation of materials along the human ventral visual pathway

Every object in the world has its own surface quality that is a reflection of the material from which the object is made. We can easily identify and categorize materials (wood, metal, fabric etc.) at a glance, and this ability enables us to decide how to interact appropriately with these objects. Little is known, however, about how materials are represented in the brain, or how that representation is related to material perception or the physical properties of material surface. By combining multivoxel pattern analysis of functional magnetic resonance imaging data with perceptual and image-based physical measures of material properties, we found that the way visual information about materials is coded gradually changes from an image-based representation in early visual areas to a perceptual representation in the ventral higher-order visual areas. We suggest that meaningful information about multimodal aspects of real-world materials reside in the ventral cortex around the fusiform gyrus, where it can be utilized for categorization of materials.

Differential roles of frequency-following and frequency-doubling visual responses revealed by evoked neural harmonics

Frequency-following and frequency-doubling neurons are ubiquitous in both striate and extrastriate visual areas. However, responses from these two types of neural populations have not been effectively compared in humans because previous EEG studies have not successfully dissociated responses from these populations. We devised a light-dark flicker stimulus that unambiguously distinguished these responses as reflected in the first and second harmonics in the steady-state visual evoked potentials. These harmonics revealed the spatial and functional segregation of frequency-following (the first harmonic) and frequency-doubling (the second harmonic) neural populations. Spatially, the first and second harmonics in steady-state visual evoked potentials exhibited divergent posterior scalp topographies for a broad range of EEG frequencies. The scalp maximum was medial for the first harmonic and contralateral for the second harmonic, a divergence not attributable to absolute response frequency. Functionally, voluntary visual-spatial attention strongly modulated the second harmonic but had negligible effects on the simultaneously elicited first harmonic. These dissociations suggest an intriguing possibility that frequency-following and frequency-doubling neural populations may contribute complementary functions to resolve the conflicting demands of attentional enhancement and signal fidelity--the frequency-doubling population may mediate substantial top-down signal modulation for attentional selection, whereas the frequency-following population may simultaneously preserve relatively undistorted sensory qualities regardless of the observer's cognitive state.

Coding of shape from shading in area V4 of the macaque monkey

Background

The shading of an object provides an important cue for recognition, especially for determining its 3D shape. However, neuronal mechanisms that allow the recovery of 3D shape from shading are poorly understood. The aim of our study was to determine the neuronal basis of 3D shape from shading coding in area V4 of the awake macaque monkey.

Results

We recorded the responses of V4 cells to stimuli presented parafoveally while the monkeys fixated a central spot. We used a set of stimuli made of 8 different 3D shapes illuminated from 4 directions (from above, the left, the right and below) and different 2D controls for each stimulus. The results show that V4 neurons present a broad selectivity to 3D shape and illumination direction, but without a preference for a unique illumination direction. However, 3D shape and illumination direction selectivities are correlated suggesting that V4 neurons can use the direction of illumination present in complex patterns of shading present on the surface of objects. In addition, a vast majority of V4 neurons (78%) have statistically different responses to the 3D and 2D versions of the stimuli, while responses to 3D are not systematically stronger than those to 2D controls. However, a hierarchical cluster analysis showed that the different classes of stimuli (3D, 2D controls) are clustered in the V4 cells response space suggesting a coding of 3D stimuli based on the population response. The different illumination directions also tend to be clustered in this space.

Conclusion

Together, these results show that area V4 participates, at the population level, in the coding of complex shape from the shading patterns coming from the illumination of the surface of corrugated objects. Hence V4 provides important information for one of the steps of cortical processing of the 3D aspect of objects in natural light environment.

Organization of color-selective neurons in macaque visual area V4

Cortical area V4 in monkeys contains neurons that respond selectively to particular colors. It has been controversial how these color-selective neurons are spatially organized in V4. One view asserts that color-selective neurons are organized in columns with different colors orderly mapped across the cortex, whereas other studies have found no evidence for columnar organization or any other clustered structure. In the present study, we reexamined the functional organization of color-selective neurons in area V4 by quantitatively evaluating and comparing the color selectivity of nearby neurons as well as those encountered along electrode penetrations. Using a multiple single-unit recording technique, we recorded extracellular activities simultaneously from groups of nearby V4 neurons. Color discrimination and color preferences exhibited a moderate correlation between nearby neurons, consistent with neurons in a local region of V4 sharing similar responses to stimulus color. However, the degree of clustering was variable across recording sites. Some regions contained neurons with similar color preferences, whereas others contained neurons with diverse color preferences. Neurons in penetrations normal to the cortical surface responded to an overlapping range of colors and maintained a moderate correlation. Neurons in penetrations tangential to the cortical surface differed dramatically in their preferred color and exhibited a negative correlation. We conclude that neurons in area V4 are moderately clustered according to their color selectivity and that this weak clustering is columnar in structure.

Natural textures classification in area V4 of the macaque monkey

Natural texture of an object is an important cue for recognition. In real conditions, the incidence angle of light on natural textures leads to a complex pattern of micro-shading that modifies 3D rendering of surfaces. Little is known about visual processing of material properties. The present work aims to study the coding of natural textures by the neurons of area V4 of the awake macaque monkey. We used patches of natural textures issued from the CURET database and illuminated with two or three different angles with their corresponding controls (scrambled Fourier phase). We recorded the responses of V4 neurons to stimuli flashed in their receptive fields (RFs) while the macaques performed a simple fixation task. We show that a large majority of V4 neurons responded to texture patches with a strong modulation across stimuli. The analysis of those responses indicate that V4 neurons integrate first and second order parameters in the image (mean luminance, SNR, and energy), which may be used to achieve texture clustering in a multidimensional space. This clustering was comparable to that of a pyramid of Gabor filters and was not affected by illumination angles. Altogether, these results suggest that the V4 neuronal population acts as a set of filters able to classify textures independently of illumination angle. We conclude that area V4 contains mechanisms that are sensitive to the aspect of textured surfaces, even in an environment where illumination changes continuously.

The extraction of 3D shape from texture and shading in the human brain

We used functional magnetic resonance imaging to investigate the human cortical areas involved in processing 3-dimensional (3D) shape from texture (SfT) and shading. The stimuli included monocular images of randomly shaped 3D surfaces and a wide variety of 2-dimensional (2D) controls. The results of both passive and active experiments reveal that the extraction of 3D SfT involves the bilateral caudal inferior temporal gyrus (caudal ITG), lateral occipital sulcus (LOS) and several bilateral sites along the intraparietal sulcus. These areas are largely consistent with those involved in the processing of 3D shape from motion and stereo. The experiments also demonstrate, however, that the analysis of 3D shape from shading is primarily restricted to the caudal ITG areas. Additional results from psychophysical experiments reveal that this difference in neuronal substrate cannot be explained by a difference in strength between the 2 cues. These results underscore the importance of the posterior part of the lateral occipital complex for the extraction of visual 3D shape information from all depth cues, and they suggest strongly that the importance of shading is diminished relative to other cues for the analysis of 3D shape in parietal regions.

Perception of illuminance flow in the case of anisotropic rough surfaces

Human observers estimate the illumination direction of rough surfaces rather precisely. When surfaces are rough, the illumination generates visible "texture" from differential shading at the level of the roughness, whereas differential illumination at the level of significant global surface curvature leads to the more familiar "shading". The shading is used in conventional shape-from-shading (SFS) algorithms, which ignore the illumination texture cue. Because of this simplification, SFS algorithms are typically formulated as global problems (partial differential equations, etc.). Human observers are likely to apply different methods than do these conventional SFS algorithms, however. When the roughness is not isotropic, one expects systematic errors in the visual detection of illumination direction, conceivably giving rise to erroneous shape estimates. Here we addressed this issue through systematic psychophysics on illumination direction detection as a function of the roughness anisotropy. Our expectations were fully borne out, in that the observers committed the predicted systematic errors. These results are precise enough to allow the inference that illumination direction detection is based on second-order statistics--that is, of edge detector (rather than line detector) activity.

Depth from shading and disparity in humans and monkeys

A stimulus display was devised that enabled us to examine how effectively monkeys and humans can process shading and disparity cues for depth perception. The display allowed us to present these cues separately, in concert and in conflict with each other. An oddities discrimination task was used. Humans as well as monkeys were able to utilize both shading and disparity cues but shading cues were more effectively processed by humans. Humans and monkeys performed better and faster when the two cues were presented conjointly rather than singly. Performance was significantly degraded when the two cues were presented in conflict with each other suggesting that these cues are processed interactively at higher levels in the visual system. The fact that monkeys can effectively utilize depth information derived from shading and disparity indicates that they are a good animal model for the study of the neural mechanisms that underlie the processing of these two depth cues.

Facilitation of shape-from-shading perception by random textures

Most natural objects have a texture on their surface, so the segregation between shading and texture is crucial for the robust perception of three-dimensional structure: The visual system has to decide whether shading or texture evoked the luminance change. We found that the contextual pop-out that results from shading was not suppressed, but was even facilitated, when random texture was added to the luminance of the entire stimulus, indicating the functional segregation and facilitative interaction between shading and texture cues. The local contrast evoked by random texture within a figure or at a boundary was a major factor in the facilitation, suggesting the crucial role of early vision in the interaction between the cues.

A comparative study of shape representation in macaque visual areas v2 and v4

We compared aspects of shape representation in extrastriate visual areas V2 and V4, which are both implicated in shape processing and belong to different hierarchical levels. We recorded responses of cells in awake, fixating monkeys to matched sets of contour and grating stimuli of low or intermediate complexity. These included simple stimuli (bars and sinusoids) and more complex stimuli (angles, intersections, arcs, and non-Cartesian gratings), all scaled to receptive field size. The responses of cells within each area were substantially modulated by each shape characteristic tested, with substantial overlap between areas by many response measures. Our analyses revealed many clear and reliable differences between areas in terms of the effectiveness of, and response modulation by, various shape characteristics. Grating stimuli were on average more effective than contour stimuli in V2 and V4, but the difference was more pronounced in V4. As a population, V4 showed greater response modulation by some shape characteristics (including simple shape characteristics) and V2 showed greater response modulation by many others (including complex shape characteristics). Recordings from area V1 demonstrated complex shape selectivity in some cells and relatively modest population differences in comparison with V2. Altogether, the representation of 2-dimensional shape characteristics revealed by this analysis varies substantially among the 3 areas. But surprisingly, the differences revealed by our analyses, individually or collectively, do not parallel the stepwise organization of the anatomical hierarchy. Commonalities of visual shape representation across hierarchical levels may reflect the replication of neural circuits used in generating complex shape representations at multiple spatial scales.

Gradient representation and perception in the early visual system--a novel account of Mach band formation

Recent evidence suggests that object surfaces and their properties are represented at early stages in the visual system of primates. Most likely invariant surface properties are extracted to endow primates with robust object recognition capabilities. In real-world scenes, luminance gradients are often superimposed on surfaces. We argue that gradients should also be represented in the visual system, since they encode highly variable information, such as shading, focal blur, and penumbral blur. We present a neuronal architecture which was designed and optimized for segregating and representing luminance gradients in real-world images. Our architecture in addition provides a novel theory for Mach bands, whereby corresponding psychophysical data are predicted consistently.

Neural basis of shape representation in the primate brain

Visual shape recognition--the ability to recognize a wide variety of shapes regardless of their size, position, view, clutter and ambient lighting--is a remarkable ability essential for complex behavior. In the primate brain, this depends on information processing in a multistage pathway running from primary visual cortex (V1), where cells encode local orientation and spatial frequency information, to the inferotemporal cortex (IT), where cells respond selectively to complex shapes. A fundamental question yet to be answered is how the local orientation signals (in V1) are transformed into selectivity for complex shapes (in IT). To gain insights into the underlying mechanisms we investigated the neural basis of shape representation in area V4, an intermediate stage in this processing hierarchy. Theoretical considerations and psychophysical evidence suggest that contour features, i.e. angles and curves along an object contour, may serve as the basis of representation at intermediate stages of shape processing. To test this hypothesis we studied the response properties of single units in area V4 of primates. We first demonstrated that V4 neurons show strong systematic tuning for the orientation and acuteness of angles and curves when presented in isolation within the cells' receptive field. Next, we found that responses to complex shapes were dictated by the curvature at a specific boundary location within the shape. Finally, using basis function decoding, we demonstrated that an ensemble of V4 neurons could successfully encode complete shapes as aggregates of boundary fragments. These findings identify curvature as a basis of shape representation in area V4 and provide insights into the neurophysiological basis for the salience of convex curves in shape perception.

Differential contribution of early visual areas to the perceptual process of contour processing

We investigated contour processing and figure-ground detection within human retinotopic areas using event-related functional magnetic resonance imaging (fMRI) in 6 healthy and naïve subjects. A figure (6 degrees side length) was created by a 2nd-order texture contour. An independent and demanding foveal letter-discrimination task prevented subjects from noticing this more peripheral contour stimulus. The contour subdivided our stimulus into a figure and a ground. Using localizers and retinotopic mapping stimuli we were able to subdivide each early visual area into 3 eccentricity regions corresponding to 1) the central figure, 2) the area along the contour, and 3) the background. In these subregions we investigated the hemodynamic responses to our stimuli and compared responses with or without the contour defining the figure. No contour-related blood oxygenation level-dependent modulation in early visual areas V1, V3, VP, and MT+ was found. Significant signal modulation in the contour subregions of V2v, V2d, V3a, and LO occurred. This activation pattern was different from comparable studies, which might be attributable to the letter-discrimination task reducing confounding attentional modulation. In V3a, but not in any other retinotopic area, signal modulation corresponding to the central figure could be detected. Such contextual modulation will be discussed in light of the recurrent processing hypothesis and the role of visual awareness.

Three-dimensional orientation tuning in macaque area V4

Tuning for the orientation of elongated, linear image elements (edges, bars, gratings), first discovered by Hubel and Wiesel, is considered a key feature of visual processing in the brain. It has been studied extensively in two dimensions (2D) using frontoparallel stimuli, but in real life most lines, edges and contours are slanted with respect to the viewer. Here we report that neurons in macaque area V4, an intermediate stage in the ventral (object-related) pathway of visual cortex, were tuned for 3D orientation--that is,for specific slants as well as for 2D orientation. The tuning for 3D orientation was consistent across depth position (binocular disparity) and position within the 2D classical receptive field. The existence of 3D orientation signals in the ventral pathway suggests that the brain may use such information to interpret 3D shape.

Shape representation in area V4: position-specific tuning for boundary conformation

Visual shape recognition in primates depends on a multi-stage pathway running from primary visual cortex (V1) to inferotemporal cortex (IT). The mechanisms by which local shape signals from V1 are transformed into selectivity for abstract object categories in IT are unknown. One approach to this issue is to investigate shape representation at intermediate stages in the pathway, such as area V4. We studied 109 V4 cells that appeared sensitive to complex shape in preliminary tests. To achieve a more complete picture of shape representation in V4, we tested each cell with a set of 366 stimuli, constructed by systematically combining convex and concave boundary elements into closed shapes. Using this large, diverse stimulus set, we found that all the cells in our sample responded to a wide variety of shapes and did not appear to encode any single type of global shape. However, for most cells the shapes evoking strongest responses were characterized by a consistent type of boundary conformation at a specific position within the stimulus. For example, a given cell might be tuned for shapes containing concave curvature at the right, with other parts of the shape having little or no effect on responses. Many cells were tuned for more complex boundary configurations (e.g., a convex angle adjacent to a concave curve). We quantified this kind of shape tuning with Gaussian functions on a curvature x position domain. These tuning functions fit the neural responses much better than tuning functions based on edge or axis orientation. Thus individual V4 cells appear to encode moderately complex boundary information at specific locations within larger shapes. This finding suggests that, at intermediate stages in the V1-IT transformation, complex objects are represented at least partly in terms of the configurations and positions of their contour components.

Visual perception: sunny side up

Illumination, both natural and artificial, typically comes from above. Neurons in visual area V4--part of the object-processing pathway in the primate brain--may rely on this anisotropy to infer three-dimensional structure from shading cues.