Disparity-Tuning Characteristics of Neuronal Responses to Dynamic Random-Dot Stereograms in Macaque Visual Area V4

Hue and orientation pinwheels in macaque area V4

Area V4 is an intermediate-level area of the macaque visual cortical hierarchy that serves key functions in visual processing by integrating inputs from lower areas such as V1 and V2 and providing feedforward inputs to many higher cortical areas. Previous V4 imaging studies have focused on differential responses to color, orientation, disparity, and motion stimuli, but many details of the spatial organization of significant hue and orientation tuning have not been fully described. We used support vector machine (SVM) decoding of intrinsic cortical single-condition responses to generate high-resolution maps of hue and orientation tuning and to describe the organization of hue and orientation pinwheels in V4. Like V1 and V2, V4 contains maps of orientation that are organized as pinwheels. V4 also contains maps of hue that are organized as pinwheels, whose circular organization more closely represents the perception of hue than is observed in antecedent cortical areas. Unlike V1, where orientation is continuously mapped across the surface, V4 hue and orientation pinwheels are organized in limited numbers of pinwheel sequences. The organization of these sequences and the distance between pinwheels may provide insight into the functional organization of V4. Regions significantly tuned for hue occupy roughly four times that of the orientation, are largely separated from each other, and overlap by roughly 5%. This spatial organization is largely consistent with segregated inputs arising from V2 thin and interstripes. This modular organization of V4 suggests that further integration of color and shape might occur in higher areas in inferotemporal cortical.NEW & NOTEWORTHY The representation of hue and orientation in macaque monkey area V4 was determined by intrinsic cortical imaging of responses to isoluminant hues and achromatic grating stimuli. Vector summation of support vector machine (SVM) decoded single-condition responses was used to generate hue and orientation maps that, like V1 orientation maps, were both characterized by distinct pinwheel patterns. These data suggest that pinwheels are an important structure to represent different stimulus features across multiple visual cortical areas.

Asymmetric stimulus representations bias visual perceptual learning

The primate visual cortex contains various regions that exhibit specialization for different stimulus properties, such as motion, shape, and color. Within each region, there is often further specialization, such that particular stimulus features, such as horizontal and vertical orientations, are over-represented. These asymmetries are associated with well-known perceptual biases, but little is known about how they influence visual learning. Most theories would predict that learning is optimal, in the sense that it is unaffected by these asymmetries. However, other approaches to learning would result in specific patterns of perceptual biases. To distinguish between these possibilities, we trained human observers to discriminate between expanding and contracting motion patterns, which have a highly asymmetrical representation in the visual cortex. Observers exhibited biased percepts of these stimuli, and these biases were affected by training in ways that were often suboptimal. We simulated different neural network models and found that a learning rule that involved only adjustments to decision criteria, rather than connection weights, could account for our data. These results suggest that cortical asymmetries influence visual perception and that human observers often rely on suboptimal strategies for learning.

Transformations of sensory information in the brain suggest changing criteria for optimality

Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities-the slight differences in the retinal images of the two eyes-we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes.

Transformations of sensory information in the brain reflect a changing definition of optimality

Neurons throughout the brain modulate their firing rate lawfully in response to changes in sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization: neurons aim to efficiently and robustly represent sensory information under resource limitations. Our understanding of how this optimization varies across the brain, however, is still in its infancy. Here, we show that neural responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation to optimizing for perceptual discrimination. Focusing on binocular disparity - the slight differences in how objects project to the two eyes - we re-analyze measurements from neurons characterizing tuning curves in macaque monkey brain regions V1, V2, and MT, and compare these to measurements of the natural visual statistics of binocular disparity. The changes in tuning curve characteristics are computationally consistent with a shift in optimization goals from maximizing the information encoded about naturally occurring binocular disparities to maximizing the ability to support fine disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective regions of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings support a key re-framing of optimal coding in regions of the brain that contain sensory information, emphasizing the need to consider not just information preservation and neural resources, but also relevance to behavior.

Processing of visual statistics of naturalistic videos in macaque visual areas V1 and V4

Natural scenes are characterized by diverse image statistics, including various parameters of the luminance histogram, outputs of Gabor-like filters, and pairwise correlations between the filter outputs of different positions, orientations, and scales (Portilla–Simoncelli statistics). Some of these statistics capture the response properties of visual neurons. However, it remains unclear to what extent such statistics can explain neural responses to natural scenes and how neurons that are tuned to these statistics are distributed across the cortex. Using two-photon calcium imaging and an encoding-model approach, we addressed these issues in macaque visual areas V1 and V4. For each imaged neuron, we constructed an encoding model to mimic its responses to naturalistic videos. By extracting Portilla–Simoncelli statistics through outputs of both filters and filter correlations, and by computing an optimally weighted sum of these outputs, the model successfully reproduced responses in a subpopulation of neurons. We evaluated the selectivities of these neurons by quantifying the contributions of each statistic to visual responses. Neurons whose responses were mainly determined by Gabor-like filter outputs (low-level statistics) were abundant at most imaging sites in V1. In V4, the relative contribution of higher order statistics, such as cross-scale correlation, was increased. Preferred image statistics varied markedly across V4 sites, and the response similarity of two neurons at individual imaging sites gradually declined with increasing cortical distance. The results indicate that natural scene analysis progresses from V1 to V4, and neurons sharing preferred image statistics are locally clustered in V4.

Function-specific projections from V2 to V4 in macaques

Previous studies have revealed modular projections from area V2 to area V4 in macaques. Specifically, V2 neurons in cytochrome oxidase (CO)-rich thin and CO-sparse pale stripes project to distinct regions in V4. However, how these modular projections relate to the functional subcompartments of V4 remains unclear. In this study, we injected retrograde fluorescent tracers into V4 regions with different functional properties (color, orientation, and direction) that were identified with intrinsic signal optical imaging (ISOI). We examined the labeled neurons in area V2 and their locations with respect to the CO patterns. Covariation was observed between the functional properties of the V4 injection sites and the numbers of labeled neurons in particular CO stripes. This covariation indicates that the color domains in V4 mainly received inputs from thin stripes in V2, whereas V4 orientation domains received inputs from pale stripes. Although motion-sensitive domains are present in both V2 and V4, our results did not reveal a functional specific feedforward projection between them. These results confirmed previous findings of modular projections from V2 to V4 and provided functional specificity for these anatomical projections. Together, these findings indicate that color and form remain separate in ventral mid-level visual processing.

Specialized contributions of mid-tier stages of dorsal and ventral pathways to stereoscopic processing in macaque

The division of labor between the dorsal and ventral visual pathways has been well studied, but not often with direct comparison at the single-neuron resolution with matched stimuli. Here we directly compared how single neurons in MT and V4, mid-tier areas of the two pathways, process binocular disparity, a powerful cue for 3D perception and actions. We found that MT neurons transmitted disparity signals more quickly and robustly, whereas V4 or its upstream neurons transformed the signals into sophisticated representations more prominently. Therefore, signaling speed and robustness were traded for transformation between the dorsal and ventral pathways. The key factor in this tradeoff was disparity-tuning shape: V4 neurons had more even-symmetric tuning than MT neurons. Moreover, the tuning symmetry predicted the degree of signal transformation across neurons similarly within each area, implying a general role of tuning symmetry in the stereoscopic processing by the two pathways.

Early Emergence of Solid Shape Coding in Natural and Deep Network Vision

Area V4 is the first object-specific processing stage in the ventral visual pathway, just as area MT is the first motion-specific processing stage in the dorsal pathway. For almost 50 years, coding of object shape in V4 has been studied and conceived in terms of flat pattern processing, given its early position in the transformation of 2D visual images. Here, however, in awake monkey recording experiments, we found that roughly half of V4 neurons are more tuned and responsive to solid, 3D shape-in-depth, as conveyed by shading, specularity, reflection, refraction, or disparity cues in images. Using 2-photon functional microscopy, we found that flat- and solid-preferring neurons were segregated into separate modules across the surface of area V4. These findings should impact early shape-processing theories and models, which have focused on 2D pattern processing. In fact, our analyses of early object processing in AlexNet, a standard visual deep network, revealed a similar distribution of sensitivities to flat and solid shape in layer 3. Early processing of solid shape, in parallel with flat shape, could represent a computational advantage discovered by both primate brain evolution and deep-network training.

Asymmetries in Global Perception Are Represented in Near- versus Far-Preferring Clusters in Human Visual Cortex

Human perception is more “global” when stimuli are viewed within the lower (rather than the upper) visual field. This phenomenon is typically considered as a 2-D phenomenon, likely due to differential neural processing within dorsal versus ventral cortical areas that represent lower versus upper visual fields, respectively. Here we test a novel hypothesis that this vertical asymmetry in global processing is a 3-D phenomenon associated with (1) higher ecological relevance of low-spatial frequency (SF) components in encoding near (compared with far) visual objects and (2) the fact that near objects are more frequently found in lower rather than upper visual fields. Using high-resolution fMRI, collected within an ultra-high-field (7 T) scanner, we found that the extent of vertical asymmetry in global visual processing in human subjects (n = 10) was correlated with the fMRI response evoked by disparity-varying stimuli in human cortical area V3A. We also found that near-preferring clusters in V3A, located within stereoselective cortical columns, responded more selectively than far-preferring clusters, to low-SF features. These findings support the hypothesis that vertical asymmetry in global processing is a 3-D (not a 2-D) phenomenon, associated with the function of the stereoselective columns within visual cortex, especially those located within visual area V3A.

SIGNIFICANCE STATEMENT Here we test and confirm a new hypothesis: fine-scale neural mechanisms underlying the vertical asymmetry in global visual processing. According to this hypothesis, the asymmetry in global visual processing is a 3-D (rather than a 2-D) phenomenon, reflected in the function of fine-scale cortical structures (clusters and columns) underlying depth perception. Our findings highlight the importance of considering these structures, as regions of interest, in clarifying the neural mechanisms underlying visual perception. The results also highlight the importance of statistics of natural scenes in shaping human visual perception.

Population receptive field and connectivity properties of the early visual cortex in human albinism

In albinism, the pathological decussation of the temporal retinal afferents at the optic chiasm leads to superimposed representations of opposing hemifields in the visual cortex. Here, we assessed the equivalence of the two representations and the cortico-cortical connectivity of the early visual areas. Applying fMRI-based population receptive field (pRF)-mapping (both hemifield and bilateral mapping) and connective field (CF)-modeling, we investigated the early visual cortex in 6 albinotic participants and 4 controls. In albinism, superimposed retinotopic representations of the contra- and ipsilateral visual hemifield were observed on the hemisphere contralateral to the stimulated eye. This was confirmed by the observation of bilateral pRFs during bilateral mapping. Hemifield mapping revealed similar pRF-sizes for both hemifield representations throughout V1 to V3. The typical increase of V1-sampling extent for V3 compared to V2 was not found for the albinotic participants. The similarity of the pRF-sizes for opposing visual hemifield representations highlights the equivalence of the two maps in the early visual cortex. The altered V1-sampling extent in V3 might indicate the adaptation of cortico-cortical connections to visual pathway abnormalities in albinism. These findings thus suggest that conservative developmental mechanisms are complemented by alterations of the extrastriate cortico-cortical connectivity.

Single-cell responses to three-dimensional structure in a functionally defined patch in macaque area TEO

Both dorsal and ventral visual pathways harbor several areas sensitive to gradients of binocular disparity (i.e., higher-order disparity). Although a wealth of information exists about disparity processing in early visual (V1, V2, and V3) and end-stage areas, TE in the ventral stream, and the anterior intraparietal area (AIP) in the dorsal stream, little is known about midlevel area TEO in the ventral pathway. We recorded single-unit responses to disparity-defined curved stimuli in a functional magnetic resonance imaging (fMRI) activation elicited by curved surfaces compared with flat surfaces in the macaque area TEO. This fMRI activation contained a small proportion of disparity-selective neurons, with very few of them second-order disparity selective. Overall, this population of TEO neurons did not preserve its three-dimensional structure selectivity across positions in depth, indicating a lack of higher-order disparity selectivity, but showed stronger responses to flat surfaces than to curved surfaces, as predicted by the fMRI experiment. The receptive fields of the responsive TEO cells were relatively small and generally foveal. A linear support vector machine classifier showed that this population of disparity-selective TEO neurons contains reliable information about the sign of curvature and the position in depth of the stimulus. NEW & NOTEWORTHY We recorded in a part of the macaque area TEO that is activated more by curved surfaces than by flat surfaces at different disparities using the same stimuli. In contrast to previous studies, this functional magnetic resonance imaging-defined patch did not contain a large number of higher-order disparity-selective neurons. However, a linear support vector machine could reliably classify both the sign of the disparity gradient and the position in depth of the stimuli.

Simple reaction times to cyclopean stimuli reveal that the binocular system is tuned to react faster to near than to far objects

Binocular depth perception is an important mechanism to segregate the visual scene for mapping relevant objects in our environment. Convergent evidence from psychophysical and neurophysiological studies have revealed asymmetries between the processing of near (crossed) and far (uncrossed) binocular disparities. The aim of the present study was to test if near or far objects are processed faster and with higher contrast sensitivity in the visual system. We therefore measured the relationship between binocular disparity and simple reaction time (RT) as well as contrast gain based on the contrast-RT function in young healthy adults. RTs were measured to suddenly appearing cyclopean target stimuli, which were checkerboard patterns encoded by depth in dynamic random dot stereograms (DRDS). The DRDS technique allowed us to selectively study the stereoscopic processing system by eliminating all monocular cues. The results showed that disparity and contrast had significant effects on RTs. RTs as a function of disparity followed a U-shaped tuning curve indicating an optimum at around 15 arc min, where RTs were minimal. Surprisingly, the disparity tuning of RT was much less pronounced for far disparities. At the optimal disparity, we measured advantages of about 80 ms and 30 ms for near disparities at low (10%) and high (90%) contrasts, respectively. High contrast always reduced RTs as well as the disparity dependent differences. Furthermore, RT-based contrast gains were higher for near disparities in the range of disparities where RTs were the shortest. These results show that the sensitivity of the human visual system is biased for near versus far disparities and near stimuli can result in faster motor responses, probably because they bear higher biological relevance.

Weighted parallel contributions of binocular correlation and match signals to conscious perception of depth

Binocular disparity is detected in the primary visual cortex by a process similar to calculation of local cross-correlation between left and right retinal images. As a consequence, correlation-based neural signals convey information about false disparities as well as the true disparity. The false responses in the initial disparity detectors are eliminated at later stages in order to encode only disparities of the features correctly matched between the two eyes. For a simple stimulus configuration, a feed-forward nonlinear process can transform the correlation signal into the match signal. For human observers, depth judgement is determined by a weighted sum of the correlation and match signals rather than depending solely on the latter. The relative weight changes with spatial and temporal parameters of the stimuli, allowing adaptive recruitment of the two computations under different visual circumstances. A full transformation from correlation-based to match-based representation occurs at the neuronal population level in cortical area V4 and manifests in single-neuron responses of inferior temporal and posterior parietal cortices. Neurons in area V5/MT represent disparity in a manner intermediate between the correlation and match signals. We propose that the correlation and match signals in these areas contribute to depth perception in a weighted, parallel manner.This article is part of the themed issue 'Vision in our three-dimensional world'.

Human cortical neural correlates of visual fatigue during binocular depth perception: An fNIRS study

Functional near-infrared spectroscopy (fNIRS) was adopted to investigate the cortical neural correlates of visual fatigue during binocular depth perception for different disparities (from 0.1° to 1.5°). By using a slow event-related paradigm, the oxyhaemoglobin (HbO) responses to fused binocular stimuli presented by the random-dot stereogram (RDS) were recorded over the whole visual dorsal area. To extract from an HbO curve the characteristics that are correlated with subjective experiences of stereopsis and visual fatigue, we proposed a novel method to fit the time-course HbO curve with various response functions which could reflect various processes of binocular depth perception. Our results indicate that the parietal-occipital cortices are spatially correlated with binocular depth perception and that the process of depth perception includes two steps, associated with generating and sustaining stereovision. Visual fatigue is caused mainly by generating stereovision, while the amplitude of the haemodynamic response corresponding to sustaining stereovision is correlated with stereopsis. Combining statistical parameter analysis and the fitted time-course analysis, fNIRS could be a promising method to study visual fatigue and possibly other multi-process neural bases.

Influence of Test Distance on Stereoacuity in Intermittent Exotropia

AIMS

To investigate influence of test distance on stereoacuity in intermittent exotropia (X[T]) using the same test conditions for both near and far distances.

METHODS

Subjects were 38 consecutive patients with X(T). All the patients were between ages 6 and 15 years and had decimal visual acuity of 1.0 or better. Another inclusion criterion was presence of phoric condition at near and far distances. Stereoacuity was measured at a near distance of 40 cm and at a far distance of 5 m. The following test conditions were used for both test distances: separation of the two eyes using polarized glasses, and a target with a random dot pattern. All the stereograms had the same subtended angle of 2.5º, and binocular disparity of 480, 240, 120, and 60 arcsec. We used two stereogram types with crossed and uncrossed disparities.

RESULTS

Far stereoacuity of 38 subjects measured with the crossed disparity was significantly worse than near stereoacuity (P<0.05, Wilcoxon signed-ranks test), although 30 (78.9%) of the 38 subjects showed no differences in stereopsis between the near and far distances. Far stereoacuity of 38 cases measured with the uncrossed disparity was significantly worse than at near (P<0.05, Wilcoxon signed-ranks test), although 20 (52.6%) of the 38 subjects showed no differences between stereoacuity at near and far. In comparison of stereoacuity with crossed disparity and uncrossed disparity, stereoacuity with crossed disparity was significantly better than that with uncrossed disparity both at near and far (P<0.05, Wilcoxon signed-ranks test).

CONCLUSIONS

Stereoacuity in X(T) was different according to test distance when measured controlling subtended angle of stereogram at both distances. Far stereoacuity was significantly worse than near stereoacuity when measured using test targets with both crossed and uncrossed disparities. Additionally, stereoacuity measured with crossed disparity was better than that with uncrossed disparity at both distances.

Feature-location binding in 3D: Feature judgments are biased by 2D location but not position-in-depth

A fundamental aspect of human visual perception is the ability to recognize and locate objects in the environment. Importantly, our environment is predominantly three-dimensional (3D), but while there is considerable research exploring the binding of object features and location, it is unknown how depth information interacts with features in the object binding process. A recent paradigm called the spatial congruency bias demonstrated that 2D location is fundamentally bound to object features, such that irrelevant location information biases judgments of object features, but irrelevant feature information does not bias judgments of location or other features. Here, using the spatial congruency bias paradigm, we asked whether depth is processed as another type of location, or more like other features. We initially found that depth cued by binocular disparity biased judgments of object color. However, this result seemed to be driven more by the disparity differences than the depth percept: Depth cued by occlusion and size did not bias color judgments, whereas vertical disparity information (with no depth percept) did bias color judgments. Our results suggest that despite the 3D nature of our visual environment, only 2D location information - not position-in-depth - seems to be automatically bound to object features, with depth information processed more similarly to other features than to 2D location.

Basal Dendrites of Layer-III Pyramidal Neurons do not Scale with Changes in Cortical Magnification Factor in Macaque Primary Visual Cortex

Neurons in the mammalian primary visual cortex (V1) are systematically arranged across the cortical surface according to the location of their receptive fields (RFs), forming a visuotopic (or retinotopic) map. Within this map, the foveal visual field is represented by a large cortical surface area, with increasingly peripheral visual fields gradually occupying smaller cortical areas. Although cellular organization in the retina, such as the spatial distribution of ganglion cells, can partially account for the eccentricity-dependent differences in the size of cortical representation, whether morphological differences exist across V1 neurons representing different eccentricities is unclear. In particular, morphological differences in dendritic field diameter might contribute to the magnified representation of the central visual field. Here, we addressed this question by measuring the basal dendritic arbors of pyramidal neurons of layer-IIIC and adjoining layer III sublayers (in the Hassler's nomenclature) in macaque V1. We labeled layer-III pyramidal neurons at various retinotopic positions in V1 by injecting lightly fixed brain tissue with intracellular dye, and then compared dendritic morphology across regions in the retinotopic map representing 0-20° of eccentricity. The dendritic field area, total dendritic length, number of principal dendrites, branching complexity, spine density and total number of spines were all consistent across different retinotopic regions of V1. These results indicate that dendrites in layer-III pyramidal neurons are relatively homogeneous according to these morphometric parameters irrespective of their locations in this portion of the retinotopic map. The homogeneity of dendritic morphology in these neurons suggests that the emphasis of central visual field representation is not attributable to changes in the basal dendritic arbors of pyramidal neurons in layer III, but is likely the result of successive processes earlier in the retino-geniculo-striate pathway.

Visual field biases for near and far stimuli in disparity selective columns in human visual cortex

When visual objects are located in the lower visual field, human observers perceive objects to be nearer than their real physical location. Conversely, objects in the upper visual field are viewed farther than their physical location. This bias may be linked to the statistics of natural scenes, and perhaps the ecological relevance of objects in the upper and lower visual fields (Previc, 1990; Yang and Purves, 2003). However, the neural mechanisms underlying such perceptual distortions have remained unknown. To test for underlying brain mechanisms, we presented visual stimuli at different perceptual distances, while measuring high-resolution fMRI in human subjects. First, we localized disparity-selective thick stripes and thick-type columns in secondary and third visual cortical areas, respectively. Consistent with the perceptual bias, we found that the thick stripe/columns that represent the lower visual field also responded more selectively to near rather than far visual stimuli. Conversely, thick stripe/columns that represent the upper visual field show a complementary bias, i.e. selectively higher activity to far rather than near stimuli. Thus, the statistics of natural scenes may play a significant role in the organization of near- and far-selective neurons within V2 thick stripes and V3 thick-type columns.

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.

Using Functional Near Infrared Spectroscopy (fNIRS) to Study Dynamic Stereoscopic Depth Perception

The parietal cortex has been widely implicated in the processing of depth perception by many neuroimaging studies, yet functional near infrared spectroscopy (fNIRS) has been an under-utilised tool to examine the relationship of oxy- ([HbO]) and de-oxyhaemoglobin ([HbR]) in perception. Here we examine the haemodynamic response (HDR) to the processing of induced depth stimulation using dynamic random-dot-stereograms (RDS). We used fNIRS to measure the HDR associated with depth perception in healthy young adults (n = 13, mean age 24). Using a blocked design, absolute values of [HbO] and [HbR] were recorded across parieto-occipital and occipital cortices, in response to dynamic RDS. Control and test images were identical except for the horizontal shift in pixels in the RDS that resulted in binocular disparity and induced the percept of a 3D sine wave that ‘popped out’ of the test stimulus. The control stimulus had zero disparity and induced a ‘flat’ percept. All participants had stereoacuity within normal clinical limits and successfully perceived the depth in the dynamic RDS. Results showed a significant effect of this complex visual stimulation in the right parieto-occipital cortex (p < 0.01, η² = 0.54). The test stimulus elicited a significant increase in [HbO] during depth perception compared to the control image (p < 0.001, 99.99 % CI [0.008–0.294]). The similarity between the two stimuli may have resulted in the HDR of the occipital cortex showing no significant increase or decrease of cerebral oxygenation levels during depth stimulation. Cerebral oxygenation measures of [HbO] confirmed the strong association of the right parieto-occipital cortex with processing depth perception. Our study demonstrates the validity of fNIRS to investigate [HbO] and [HbR] during high-level visual processing of complex stimuli.

Pooled, but not single-neuron, responses in macaque V4 represent a solution to the stereo correspondence problem

Binocular disparity is an important cue for depth perception. To correctly represent disparity, neurons must find corresponding visual features between the left- and right-eye images. The visual pathway ascending from V1 to inferior temporal cortex solves the correspondence problem. An intermediate area, V4, has been proposed to be a critical stage in the correspondence process. However, the distinction between V1 and V4 is unclear, because accumulating evidence suggests that the process begins within V1. In this article, we report that the pooled responses in macaque V4, but not responses of individual neurons, represent a solution to the correspondence problem. We recorded single-unit responses of V4 neurons to random-dot stereograms of varying degrees of anticorrelation. To achieve gradual anticorrelation, we reversed the contrast of an increasing proportion of dots as in our previous psychophysical studies, which predicted that the neural correlates of the solution to correspondence problem should gradually eliminate their disparity modulation as the level of anticorrelation increases. Inconsistent with this prediction, the tuning amplitudes of individual V4 neurons quickly decreased to a nonzero baseline with small anticorrelation. By contrast, the shapes of individual tuning curves changed more gradually so that the amplitude of population-pooled responses gradually decreased toward zero over the entire range of graded anticorrelation. We explain these results by combining multiple energy-model subunits. From a comparison with the population-pooled responses in V1, we suggest that disparity representation in V4 is distinctly advanced from that in V1. Population readout of V4 responses provides disparity information consistent with the correspondence solution.

Computation of Object Size in Visual Cortical Area V4 as a Neural Basis for Size Constancy

Even when we view an object from different distances, so that the size of its projection onto the retina varies, we perceive its size to be relatively unchanged. In this perceptual phenomenon known as size constancy, the brain uses both distance and retinal image size to estimate the size of an object. Given that binocular disparity, the small positional difference between the retinal images in the two eyes, is a powerful visual cue for distance, we examined how it affects neuronal tuning to retinal image size in visual cortical area V4 of macaque monkeys. Depending on the imposed binocular disparity of a circular patch embedded in random dot stereograms, most neurons adjusted their preferred size in a manner consistent with size constancy. They preferred larger retinal image sizes when stimuli were stereoscopically presented nearer and preferred smaller retinal image sizes when stimuli were presented farther away. This disparity-dependent shift of preferred image size was not affected by the vergence angle, a cue for the fixation distance, suggesting that different V4 neurons compute object size for different fixation distances rather than that individual neurons adjust the shift based on vergence. This interpretation was supported by a simple circuit model, which could simulate the shift of preferred image size without any information about the fixation distance. We suggest that a population of V4 neurons encodes the actual size of objects, rather than simply the size of their retinal images, and that these neurons thereby contribute to size constancy.

SIGNIFICANCE STATEMENT

We perceive the size of an object to be relatively stable despite changes in the size of its retinal image that accompany changes in viewing distance. This phenomenon, called size constancy, is accomplished by combining retinal image size and distance information in our brain. We demonstrate that a large population of V4 neurons changes their size tuning depending on the perceived distance of a visual stimulus derived from binocular disparity. They prefer larger or smaller retinal image sizes when stimuli are stereoscopically presented nearer or farther away, respectively. This property makes V4 neurons suitable for encoding the actual size of objects, not simply the retinal image sizes, and providing a possible mechanism for perceptual size constancy.

Functional architecture for disparity in macaque inferior temporal cortex and its relationship to the architecture for faces, color, scenes, and visual field

Binocular disparity is a powerful depth cue for object perception. The computations for object vision culminate in inferior temporal cortex (IT), but the functional organization for disparity in IT is unknown. Here we addressed this question by measuring fMRI responses in alert monkeys to stimuli that appeared in front of (near), behind (far), or at the fixation plane. We discovered three regions that showed preferential responses for near and far stimuli, relative to zero-disparity stimuli at the fixation plane. These "near/far" disparity-biased regions were located within dorsal IT, as predicted by microelectrode studies, and on the posterior inferotemporal gyrus. In a second analysis, we instead compared responses to near stimuli with responses to far stimuli and discovered a separate network of "near" disparity-biased regions that extended along the crest of the superior temporal sulcus. We also measured in the same animals fMRI responses to faces, scenes, color, and checkerboard annuli at different visual field eccentricities. Disparity-biased regions defined in either analysis did not show a color bias, suggesting that disparity and color contribute to different computations within IT. Scene-biased regions responded preferentially to near and far stimuli (compared with stimuli without disparity) and had a peripheral visual field bias, whereas face patches had a marked near bias and a central visual field bias. These results support the idea that IT is organized by a coarse eccentricity map, and show that disparity likely contributes to computations associated with both central (face processing) and peripheral (scene processing) visual field biases, but likely does not contribute much to computations within IT that are implicated in processing color.

Stereopsis is adaptive for the natural environment

Humans and many animals have forward-facing eyes providing different views of the environment. Precise depth estimates can be derived from the resulting binocular disparities, but determining which parts of the two retinal images correspond to one another is computationally challenging. To aid the computation, the visual system focuses the search on a small range of disparities. We asked whether the disparities encountered in the natural environment match that range. We did this by simultaneously measuring binocular eye position and three-dimensional scene geometry during natural tasks. The natural distribution of disparities is indeed matched to the smaller range of correspondence search. Furthermore, the distribution explains the perception of some ambiguous stereograms. Finally, disparity preferences of macaque cortical neurons are consistent with the natural distribution.

Cross-matching: a modified cross-correlation underlying threshold energy model and match-based depth perception

Three-dimensional visual perception requires correct matching of images projected to the left and right eyes. The matching process is faced with an ambiguity: part of one eye's image can be matched to multiple parts of the other eye's image. This stereo correspondence problem is complicated for random-dot stereograms (RDSs), because dots with an identical appearance produce numerous potential matches. Despite such complexity, human subjects can perceive a coherent depth structure. A coherent solution to the correspondence problem does not exist for anticorrelated RDSs (aRDSs), in which luminance contrast is reversed in one eye. Neurons in the visual cortex reduce disparity selectivity for aRDSs progressively along the visual processing hierarchy. A disparity-energy model followed by threshold nonlinearity (threshold energy model) can account for this reduction, providing a possible mechanism for the neural matching process. However, the essential computation underlying the threshold energy model is not clear. Here, we propose that a nonlinear modification of cross-correlation, which we term “cross-matching,” represents the essence of the threshold energy model. We placed half-wave rectification within the cross-correlation of the left-eye and right-eye images. The disparity tuning derived from cross-matching was attenuated for aRDSs. We simulated a psychometric curve as a function of graded anticorrelation (graded mixture of aRDS and normal RDS); this simulated curve reproduced the match-based psychometric function observed in human near/far discrimination. The dot density was 25% for both simulation and observation. We predicted that as the dot density increased, the performance for aRDSs should decrease below chance (i.e., reversed depth), and the level of anticorrelation that nullifies depth perception should also decrease. We suggest that cross-matching serves as a simple computation underlying the match-based disparity signals in stereoscopic depth perception.

Impact of chiasma opticum malformations on the organization of the human ventral visual cortex

Congenital malformations of the optic chiasm, such as enhanced and reduced crossing of the optic nerve fibers, are evident in albinism and achiasma, respectively. In early visual cortex the resulting additional visual input from the ipsilateral visual hemifield is superimposed onto the normal retinotopic representation of the contralateral visual field, which is likely due to conservative geniculo-striate projections. Counterintuitively, this organization in early visual cortex does not have profound consequences on visual function. Here we ask, whether higher stages of visual processing provide a correction to the abnormal representation allowing for largely normal perception. To this end we assessed the organization patterns of early and ventral visual cortex in five albinotic, one achiasmic, and five control participants. In albinism and achiasma the mirror-symmetrical superposition of the ipsilateral and contalateral visual fields was evident not only in early visual cortex, but also in the higher areas of the ventral processing stream. Specifically, in the visual areas VO1/2 and PHC1/2 no differences in the extent, the degree of superposition, and the magnitude of the responses were evident in comparison to the early visual areas. Consequently, the highly atypical organization of the primary visual cortex was propagated downstream to highly specialized processing stages in an undiminished and unchanged manner. This indicates largely unaltered cortico-cortical connections in both types of misrouting, i.e., enhanced and reduced crossing of the optic nerves. It is concluded that main aspects of visual function are preserved despite sizable representation abnormalities in the ventral visual processing stream.

Learning to associate auditory and visual stimuli: behavioral and neural mechanisms

The ability to effectively combine sensory inputs across modalities is vital for acquiring a unified percept of events. For example, watching a hammer hit a nail while simultaneously identifying the sound as originating from the event requires the ability to identify spatio-temporal congruencies and statistical regularities. In this study, we applied a reaction time and hazard function measure known as capacity (e.g., Townsend and AshbyCognitive Theory 200-239, 1978) to quantify the extent to which observers learn paired associations between simple auditory and visual patterns in a model theoretic manner. As expected, results showed that learning was associated with an increase in accuracy, but more significantly, an increase in capacity. The aim of this study was to associate capacity measures of multisensory learning, with neural based measures, namely mean global field power (GFP). We observed a co-variation between an increase in capacity, and a decrease in GFP amplitude as learning occurred. This suggests that capacity constitutes a reliable behavioral index of efficient energy expenditure in the neural domain.

Neural activity in cortical area V4 underlies fine disparity discrimination

Primates are capable of discriminating depth with remarkable precision using binocular disparity. Neurons in area V4 are selective for relative disparity, which is the crucial visual cue for discrimination of fine disparity. Here, we investigated the contribution of V4 neurons to fine disparity discrimination. Monkeys discriminated whether the center disk of a dynamic random-dot stereogram was in front of or behind its surrounding annulus. We first behaviorally tested the reference frame of the disparity representation used for performing this task. After learning the task with a set of surround disparities, the monkey generalized its responses to untrained surround disparities, indicating that the perceptual decisions were generated from a disparity representation in a relative frame of reference. We then recorded single-unit responses from V4 while the monkeys performed the task. On average, neuronal thresholds were higher than the behavioral thresholds. The most sensitive neurons reached thresholds as low as the psychophysical thresholds. For subthreshold disparities, the monkeys made frequent errors. The variable decisions were predictable from the fluctuation in the neuronal responses. The predictions were based on a decision model in which each V4 neuron transmits the evidence for the disparity it prefers. We finally altered the disparity representation artificially by means of microstimulation to V4. The decisions were systematically biased when microstimulation boosted the V4 responses. The bias was toward the direction predicted from the decision model. We suggest that disparity signals carried by V4 neurons underlie precise discrimination of fine stereoscopic depth.

Toward a unified theory of visual area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.

Coding of stereoscopic depth information in visual areas V3 and V3A

The process of stereoscopic depth perception is thought to begin with the analysis of absolute binocular disparity, the difference in position of corresponding features in the left and right eye images with respect to the points of fixation. Our sensitivity to depth, however, is greater when depth judgments are based on relative disparity, the difference between two absolute disparities, compared to when they are based on absolute disparity. Therefore, the visual system is thought to compute relative disparities for fine depth discrimination. Functional magnetic resonance imaging studies in humans and monkeys have suggested that visual areas V3 and V3A may be specialized for stereoscopic depth processing based on relative disparities. In this study, we measured absolute and relative disparity-tuning of neurons in V3 and V3A of alert fixating monkeys, and we compared their basic tuning properties with those published previously for other visual areas. We found that neurons in V3 and V3A predominantly encode absolute, not relative, disparities. We also found that basic parameters of disparity-tuning in V3 and V3A are similar to those from other extrastriate visual areas. Finally, by comparing single-unit activity with multi-unit activity measured at the same recording site, we demonstrate that neurons with similar disparity selectivity are clustered in both V3 and V3A. We conclude that areas V3 and V3A are not particularly specialized for processing stereoscopic depth information compared to other early visual areas, at least with respect to the tuning properties that we have examined.

Disparity-tuned population responses from human visual cortex

We used source imaging of visual evoked potentials to measure neural population responses over a wide range of horizontal disparities (0.5-64 arcmin). The stimulus was a central disk that moved back and forth across the fixation plane at 2 Hz, surrounded either by binocularly uncorrelated dots (disparity noise) or by correlated dots presented in the fixation plane. Both disk and surround were composed of dynamic random dots to remove coherent monocular information. Disparity tuning was measured in five visual regions of interest (ROIs) [V1, human middle temporal area (hMT+), V4, lateral occipital complex (LOC), and V3A], defined in separate functional magnetic resonance imaging scans. The disparity tuning functions peaked between 2 and 16 arcmin for both types of surround in each ROI. Disparity tuning in the V1 ROI was unaffected by the type of surround, but surround correlation altered both the amplitude and phase of the disparity responses in the other ROIs. Response amplitude increased when the disk was in front of the surround in the V3A and LOC ROIs, indicating that these areas encode figure-ground relationships and object convexity. The correlated surround produced a consistent phase lag at the second harmonic in the hMT+ and V4 ROIs without a change in amplitude, while in the V3A ROI, both phase and amplitude effects were observed. Sensitivity to disparity context is thus widespread in visual cortex, but the dynamics of these contextual interactions differ across regions.

Organization of color-selective neurons in macaque visual area V4

Cortical area V4 in monkeys contains neurons that respond selectively to particular colors. It has been controversial how these color-selective neurons are spatially organized in V4. One view asserts that color-selective neurons are organized in columns with different colors orderly mapped across the cortex, whereas other studies have found no evidence for columnar organization or any other clustered structure. In the present study, we reexamined the functional organization of color-selective neurons in area V4 by quantitatively evaluating and comparing the color selectivity of nearby neurons as well as those encountered along electrode penetrations. Using a multiple single-unit recording technique, we recorded extracellular activities simultaneously from groups of nearby V4 neurons. Color discrimination and color preferences exhibited a moderate correlation between nearby neurons, consistent with neurons in a local region of V4 sharing similar responses to stimulus color. However, the degree of clustering was variable across recording sites. Some regions contained neurons with similar color preferences, whereas others contained neurons with diverse color preferences. Neurons in penetrations normal to the cortical surface responded to an overlapping range of colors and maintained a moderate correlation. Neurons in penetrations tangential to the cortical surface differed dramatically in their preferred color and exhibited a negative correlation. We conclude that neurons in area V4 are moderately clustered according to their color selectivity and that this weak clustering is columnar in structure.

Adaptive estimation of three-dimensional structure in the human brain

Perceiving the three-dimensional (3D) properties of the environment relies on the brain bringing together ambiguous cues (e.g., binocular disparity, shading, texture) with information gained from short- and long-term experience. Perceptual aftereffects, in which the perception of an ambiguous 3D stimulus is biased away from the shape of a previously viewed stimulus, provide a sensitive means of probing this process, yet little is known about their neural basis. Here, we investigate 3D aftereffects using psychophysical and functional MRI (fMRI) adaptation paradigms to gain insight into the cortical circuits that mediate the perceptual interpretation of ambiguous depth signals. Using two classic bistable stimuli (Mach card, kinetic depth effect), we test aftereffects produced by 3D shapes defined by binocular (disparity) or monocular (texture, shading) depth cues. We show that the processing of ambiguous 3D stimuli in dorsal visual cortical areas (V3B/KO, V7) and posterior parietal regions is modulated by adaptation in line with perceptual aftereffects. Similar behavioral and fMRI adaptation effects for the two types of bistable stimuli suggest common neural substrates for depth aftereffects independent of the inducing depth cues (disparity, texture, shading). In line with current thinking about the role of adaptation in sensory optimization, our findings provide evidence that estimation of 3D shape in dorsal cortical areas takes account of the adaptive context to resolve depth ambiguity and interpret 3D structure.

Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain

Processing of binocular disparity is thought to be widespread throughout cortex, highlighting its importance for perception and action. Yet the computations and functional roles underlying this activity across areas remain largely unknown. Here, we trace the neural representations mediating depth perception across human brain areas using multivariate analysis methods and high-resolution imaging. Presenting disparity-defined planes, we determine functional magnetic resonance imaging (fMRI) selectivity to near versus far depth positions. First, we test the perceptual relevance of this selectivity, comparing the pattern-based decoding of fMRI responses evoked by random dot stereograms that support depth perception (correlated RDS) with the decoding of stimuli containing disparities to which the perceptual system is blind (anticorrelated RDS). Preferential disparity selectivity for correlated stimuli in dorsal (visual and parietal) areas and higher ventral area LO (lateral occipital area) suggests encoding of perceptually relevant information, in contrast to early (V1, V2) and intermediate ventral (V3v, V4) visual cortical areas that show similar selectivity for both correlated and anticorrelated stimuli. Second, manipulating disparity parametrically, we show that dorsal areas encode the metric disparity structure of the viewed stimuli (i.e., disparity magnitude), whereas ventral area LO appears to represent depth position in a categorical manner (i.e., disparity sign). Our findings suggest that activity in both visual streams is commensurate with the use of disparity for depth perception but the neural computations may differ. Intriguingly, perceptually relevant responses in the dorsal stream are tuned to disparity content and emerge at a comparatively earlier stage than categorical representations for depth position in the ventral stream.

Higher order visual processing in macaque extrastriate cortex

The extrastriate cortex of primates encompasses a substantial portion of the cerebral cortex and is devoted to the higher order processing of visual signals and their dispatch to other parts of the brain. A first step towards the understanding of the function of this cortical tissue is a description of the selectivities of the various neuronal populations for higher order aspects of the image. These selectivities present in the various extrastriate areas support many diverse representations of the scene before the subject. The list of the known selectivities includes that for pattern direction and speed gradients in middle temporal/V5 area; for heading in medial superior temporal visual area, dorsal part; for orientation of nonluminance contours in V2 and V4; for curved boundary fragments in V4 and shape parts in infero-temporal area (IT); and for curvature and orientation in depth from disparity in IT and CIP. The most common putative mechanism for generating such emergent selectivity is the pattern of excitatory and inhibitory linear inputs from the afferent area combined with nonlinear mechanisms in the afferent and receiving area.

Spatial frequency integration for binocular correspondence in macaque area V4

Neurons in the primary visual cortex (V1) detect binocular disparity by computing the local disparity energy of stereo images. The representation of binocular disparity in V1 contradicts the global correspondence when the image is binocularly anticorrelated. To solve the stereo correspondence problem, this rudimentary representation of stereoscopic depth needs to be further processed in the extrastriate cortex. Integrating signals over multiple spatial frequency channels is one possible mechanism supported by theoretical and psychophysical studies. We examined selectivities of single V4 neurons for both binocular disparity and spatial frequency in two awake, fixating monkeys. Disparity tuning was examined with a binocularly correlated random-dot stereogram (RDS) as well as its anticorrelated counterpart, whereas spatial frequency tuning was examined with a sine wave grating or a narrowband noise. Neurons with broader spatial frequency tuning exhibited more attenuated disparity tuning for the anticorrelated RDS. Additional rectification at the output of the energy model does not likely account for this attenuation because the degree of attenuation does not differ among the various types of disparity-tuned neurons. The results suggest that disparity energy signals are integrated across spatial frequency channels for generating a representation of stereoscopic depth in V4.

Representation of stereoscopic depth based on relative disparity in macaque area V4

Stereoscopic vision is characterized by greater visual acuity when a background feature serves as a reference. When a reference is present, the perceived depth of an object is predominantly dependent on this reference. Neural representations of stereoscopic depth are expected to have a relative frame of reference. The conversion of absolute disparity encoded in area V1 to relative disparity begins in area V2, although the information encoded in this area appears to be insufficient for stereopsis. This study examines whether relative disparity is encoded in a higher cortical area. We recorded the responses of V4 neurons from macaque monkeys to various combinations of the absolute disparities of two features: the center patch and surrounding annulus of a dynamic random-dot stereogram. We analyzed the effects of the disparity of the surrounding annulus on the tuning for the disparity of the center patch; the tuning curves of relative-disparity-selective neurons for disparities of the center patch should shift with changes in the disparity of the surrounding annulus. Most V4 tuning curves exhibited significant shifts. The magnitudes of the shifts were larger than those reported for V2 neurons and smaller than that expected for an ideal relative-disparity-selective cell. No correlation was found between the shift magnitude and the degree of size suppression, suggesting that the two phenomena are not the result of a common mechanism. Our results suggest that the coding of relative disparity advances as information flows along the cortical pathway that includes areas V2 and V4.

Binocular depth perception and the cerebral cortex

Our ability to coordinate the use of our left and right eyes and to make use of subtle differences between the images received by each eye allows us to perceive stereoscopic depth, which is important for the visual perception of three-dimensional space. Binocular neurons in the visual cortex combine signals from the left and right eyes. Probing the roles of binocular neurons in different perceptual tasks has advanced our understanding of the stages within the visual cortex that lead to binocular depth perception.

Effects of perceptual learning on local stereopsis and neuronal responses of V1 and V2 in prism-reared monkeys

Visual performance improves with practice (perceptual learning). In this study, we sought to determine whether or not adult monkeys reared with early abnormal visual experience improve their stereoacuity by extensive psychophysical training and testing, and if so, whether alterations of neuronal responses in the primary visual cortex (V1) and/or visual area 2 (V2) are involved in such improvement. Strabismus was optically simulated in five macaque monkeys using a prism-rearing procedure between 4 and 14 wk of age. Around 2 yr of age, three of the prism-reared monkeys ("trained" monkeys) were tested for their spatial contrast sensitivity and stereoacuity. Two other prism-reared monkeys received no training or testing ("untrained" monkeys). Microelectrode experiments were conducted around 4 yr of age. All three prism-reared trained monkeys showed improvement in stereoacuity by a factor of 7 or better. However, final stereothresholds were still approximately 10-20 times worse than those in normal monkeys. In V1, disparity sensitivity was drastically reduced in both the trained and untrained prism-reared monkeys and behavioral training had no obvious effect. In V2, the disparity sensitivity in the trained monkeys was better by a factor of approximately 2.0 compared with that in the untrained monkeys. These data suggest that the observed improvement in stereoacuity of the trained prism-reared monkeys may have resulted from better retention of disparity sensitivity in V2 and/or from "learning" by upstream neurons to more efficiently attend to residual local disparity information in V1 and V2.

Extracting 3D structure from disparity

The neural mechanisms of stereoscopic 3D shape perception have only recently been investigated. Here we review the two cortical regions in which these mechanisms have been studied so far in macaques: a small subpart of inferotemporal cortex called TEs, and the caudal intraparietal (CIP) region. Neurons in TEs respond selectively to the orientation and curvature in depth of stereoscopic surfaces and this region provides a detailed 3D shape description of surface boundaries and surface content. This description is evoked only by binocular stimuli in which subjects see depth and it does not vary if depth is specified by different cues. Neurons in CIP are a selective for orientation in depth of surfaces and elongated objects, and their responses are also unaffected by changes in depth cues. Thus, stereoscopic 3D shape is processed in both the dorsal, occipito-parietal and the ventral, occipito-temporal streams.

Neural correlates of fine depth discrimination in monkey inferior temporal cortex

Binocular disparity is an important visual cue that gives rise to the perception of depth. Disparity signals are widely spread across the visual cortex, but their relative role is poorly understood. Here, we addressed the correlation between the responses of disparity-selective neurons in the occipitotemporal (ventral) visual pathway and the behavioral discrimination of stereoscopic depth. We recorded activity of disparity-selective neurons in the inferior temporal cortex (IT) while monkeys were engaged in a fine stereoscopic depth discrimination (stereoacuity) task. We found that trial-to-trial fluctuations in neuronal responses correlated with the monkey's perceptual choice. We suggest that disparity signals in the IT, located in the ventral visual pathway, are functionally linked to the discrimination of fine-grain depth.

Encoding of three-dimensional surface slant in cat visual areas 17 and 18

How are surface orientations of three-dimensional objects and scenes represented in the visual system? We have examined an idea that these surface orientations are encoded by neurons with a variety of tilts in their binocular receptive field (RF) structure. To examine whether neurons in the early visual areas are capable of encoding surface orientations, we have recorded from single neurons extracellularly in areas 17 and 18 of the cat using standard electrophysiological methods. Binocular RF structures are obtained using a binocular version of the reverse correlation technique. About 30% of binocularly responsive neurons have RFs with statistically significant tilts from the frontoparallel plane. The degree of tilts is sufficient for representing the range of surface slants found in typical visual environments. For a subset of neurons having significant RF tilts, the degrees of tilt are correlated with the preferred spatial frequency difference between the two eyes, indicating that a modified disparity energy model can account for the selectivity, at least partially. However, not all cases could be explained by this model, suggesting that multiple mechanisms may be responsible. Therefore an alternative hypothesis is also examined, where the tilt is generated by pooling of multiple disparity detectors whose preferred disparities progressively shift over space. Although there is evidence for extensive spatial pooling, this hypothesis was not satisfactory either, in that the neurons with extensive pooling tended to prefer an untilted surface. Our results suggest that encoding of surface orientations may begin with the binocular neurons in the early visual cortex.

Neural basis of shape representation in the primate brain

Visual shape recognition--the ability to recognize a wide variety of shapes regardless of their size, position, view, clutter and ambient lighting--is a remarkable ability essential for complex behavior. In the primate brain, this depends on information processing in a multistage pathway running from primary visual cortex (V1), where cells encode local orientation and spatial frequency information, to the inferotemporal cortex (IT), where cells respond selectively to complex shapes. A fundamental question yet to be answered is how the local orientation signals (in V1) are transformed into selectivity for complex shapes (in IT). To gain insights into the underlying mechanisms we investigated the neural basis of shape representation in area V4, an intermediate stage in this processing hierarchy. Theoretical considerations and psychophysical evidence suggest that contour features, i.e. angles and curves along an object contour, may serve as the basis of representation at intermediate stages of shape processing. To test this hypothesis we studied the response properties of single units in area V4 of primates. We first demonstrated that V4 neurons show strong systematic tuning for the orientation and acuteness of angles and curves when presented in isolation within the cells' receptive field. Next, we found that responses to complex shapes were dictated by the curvature at a specific boundary location within the shape. Finally, using basis function decoding, we demonstrated that an ensemble of V4 neurons could successfully encode complete shapes as aggregates of boundary fragments. These findings identify curvature as a basis of shape representation in area V4 and provide insights into the neurophysiological basis for the salience of convex curves in shape perception.

Role of primate visual area V4 in the processing of 3-D shape characteristics defined by disparity

We studied the responses of V4 neurons in awake, fixating monkeys to a diverse set of stereoscopic stimuli, including zero-order disparity (frontoparallel) stimuli, surfaces oriented in depth, and convex and concave shapes presented at various mean disparities. The responses of many V4 cells were significantly modulated across each of these stimulus subsets. In general, V4 cells were broadly tuned for zero-order disparity, and at any given disparity value, about four-fifths of the cells responded significantly above background. The response modulation by flat surfaces oriented in depth was significant for about one-quarter of cells, and the responses of about one-third of the cells were significantly modulated by convex or concave surfaces at various mean disparities. However, we encountered no cells that unambiguously distinguished a given three-dimensional (3-D) shape independent of mean disparity. Thus 3-D shapes defined by disparity are unlikely to be represented explicitly at the level of individual V4 cells. Nonetheless, V4 cells likely play an important role in the processing of 3-D shape characteristics defined by disparity as a part of a distributed network.