Encoding of binocular disparity by complex cells in the cat's visual cortex

Receptive-field nonlinearities in primary auditory cortex: a comparative perspective

Cortical processing of auditory information can be affected by interspecies differences as well as brain states. Here we compare multifeature spectro-temporal receptive fields (STRFs) and associated input/output functions or nonlinearities (NLs) of neurons in primary auditory cortex (AC) of four mammalian species. Single-unit recordings were performed in awake animals (female squirrel monkeys, female, and male mice) and anesthetized animals (female squirrel monkeys, rats, and cats). Neuronal responses were modeled as consisting of two STRFs and their associated NLs. The NLs for the STRF with the highest information content show a broad distribution between linear and quadratic forms. In awake animals, we find a higher percentage of quadratic-like NLs as opposed to more linear NLs in anesthetized animals. Moderate sex differences of the shape of NLs were observed between male and female unanesthetized mice. This indicates that the core AC possesses a rich variety of potential computations, particularly in awake animals, suggesting that multiple computational algorithms are at play to enable the auditory system's robust recognition of auditory events.

The influence of stereopsis on visual saliency in a proto-object based model of selective attention

Some animals including humans use stereoscopic vision which reconstructs spatial information about the environment from the disparity between images captured by eyes in two separate adjacent locations. Like other sensory information, such stereoscopic information is expected to influence attentional selection. We develop a biologically plausible model of binocular vision to study its effect on bottom-up visual attention, i.e., visual saliency. In our model, the scene is organized in terms of proto-objects on which attention acts, rather than on unbound sets of elementary features. We show that taking into account the stereoscopic information improves the performance of the model in the prediction of human eye movements with statistically significant differences.

Color and luminance processing in V1 complex cells and artificial neural networks

Object recognition by natural and artificial visual systems benefits from the identification of object boundaries. A useful cue for the detection of object boundaries is the superposition of luminance and color edges. To gain insight into the suitability of this cue for object recognition, we examined convolutional neural network models that had been trained to recognize objects in natural images. We focused specifically on units in the second convolutional layer whose activations are invariant to the spatial phase of a sinusoidal grating. Some of these units were tuned for a nonlinear combination of color and luminance, which is broadly consistent with a role in object boundary detection. Others were tuned for luminance alone, but very few were tuned for color alone. A literature review reveals that V1 complex cells have a similar distribution of tuning. We speculate that this pattern of sensitivity provides an efficient basis for object recognition, perhaps by mitigating the effects of lighting on luminance contrast polarity. The absence of a contrast polarity-invariant representation of chromaticity alone suggests that it is redundant with other representations.

Widespread and Multifaceted Binocular Integration in the Mouse Primary Visual Cortex

The brain combines two-dimensional images received from the two eyes to form a percept of three-dimensional surroundings. This process of binocular integration in the primary visual cortex (V1) serves as a useful model for studying how neural circuits generate emergent properties from multiple input signals. Here, we perform a thorough characterization of binocular integration using electrophysiological recordings in the V1 of awake adult male and female mice by systematically varying the orientation and phase disparity of monocular and binocular stimuli. We reveal widespread binocular integration in mouse V1 and demonstrate that the three commonly studied binocular properties-ocular dominance, interocular matching, and disparity selectivity-are independent of each other. For individual neurons, the responses to monocular stimulation can predict the average amplitude of binocular response but not its selectivity. Finally, the extensive and independent binocular integration of monocular inputs is seen across cortical layers in both regular-spiking and fast-spiking neurons, regardless of stimulus design. Our data indicate that the current model of simple feedforward convergence is inadequate to account for binocular integration in mouse V1, thus suggesting an indispensable role played by intracortical circuits in binocular computation.SIGNIFICANCE STATEMENT Binocular integration is an important step of visual processing that takes place in the visual cortex. Studying the process by which V1 neurons become selective for certain binocular disparities is informative about how neural circuits integrate multiple information streams at a more general level. Here, we systematically characterize binocular integration in mice. Our data demonstrate more widespread and complex binocular integration in mouse V1 than previously reported. Binocular responses cannot be explained by a simple convergence of monocular responses, contrary to the prevailing model of binocular integration. These findings thus indicate that intracortical circuits must be involved in the exquisite computation of binocular disparity, which would endow brain circuits with the plasticity needed for binocular development and processing.

Neural circuits for binocular vision: Ocular dominance, interocular matching, and disparity selectivity

The brain creates a single visual percept of the world with inputs from two eyes. This means that downstream structures must integrate information from the two eyes coherently. Not only does the brain meet this challenge effortlessly, it also uses small differences between the two eyes' inputs, i.e., binocular disparity, to construct depth information in a perceptual process called stereopsis. Recent studies have advanced our understanding of the neural circuits underlying stereoscopic vision and its development. Here, we review these advances in the context of three binocular properties that have been most commonly studied for visual cortical neurons: ocular dominance of response magnitude, interocular matching of orientation preference, and response selectivity for binocular disparity. By focusing mostly on mouse studies, as well as recent studies using ferrets and tree shrews, we highlight unresolved controversies and significant knowledge gaps regarding the neural circuits underlying binocular vision. We note that in most ocular dominance studies, only monocular stimulations are used, which could lead to a mischaracterization of binocularity. On the other hand, much remains unknown regarding the circuit basis of interocular matching and disparity selectivity and its development. We conclude by outlining opportunities for future studies on the neural circuits and functional development of binocular integration in the early visual system.

Strong tuning for stereoscopic depth indicates orientation-specific recurrent circuitry in tree shrew V1

Neurons in the primary visual cortex (V1) are tuned to specific disparities between the two retinal images, which form the neural substrate for stereoscopic vision. We show that V1 neurons in tree shrews, but not in mice, display highly selective responses to narrow ranges of disparity in random-dot stereograms. Surprisingly, V1 neurons in both species show similarly strong tuning to gratings of varying interocular phase differences. This stimulus-dependent dissociation of disparity tuning can be explained by a network model that combines both feedforward and recurrent connections. The features of the model connections are supported by cortical organizations specific to each species. We validate this model by identifying putative inhibitory neurons and confirming their predicted disparity tuning in both species. Together, our studies establish a foundation for using tree shrews in studying binocular vision and raise an exciting possibility of how cortical columns could be uniquely important in computing stereoscopic depth.

Pooling strategies in V1 can account for the functional and structural diversity across species

Neurons in the primary visual cortex are selective to orientation with various degrees of selectivity to the spatial phase, from high selectivity in simple cells to low selectivity in complex cells. Various computational models have suggested a possible link between the presence of phase invariant cells and the existence of orientation maps in higher mammals’ V1. These models, however, do not explain the emergence of complex cells in animals that do not show orientation maps. In this study, we build a theoretical model based on a convolutional network called Sparse Deep Predictive Coding (SDPC) and show that a single computational mechanism, pooling, allows the SDPC model to account for the emergence in V1 of complex cells with or without that of orientation maps, as observed in distinct species of mammals. In particular, we observed that pooling in the feature space is directly related to the orientation map formation while pooling in the retinotopic space is responsible for the emergence of a complex cells population. Introducing different forms of pooling in a predictive model of early visual processing as implemented in SDPC can therefore be viewed as a theoretical framework that explains the diversity of structural and functional phenomena observed in V1.

Stimulating both eyes with matching stimuli enhances V1 responses

Neurons in the primary visual cortex (V1) of primates play a key role in combining monocular inputs to form a binocular response. Although much has been gleaned from studying how V1 responds to discrepant (dichoptic) images, equally important is to understand how V1 responds to concordant (dioptic) images in the two eyes. Here, we investigated the extent to which concordant, balanced, zero-disparity binocular stimulation modifies V1 responses to varying stimulus contrast using intracranial multielectrode arrays. On average, binocular stimuli evoked stronger V1 activity than their monocular counterparts. This binocular facilitation scaled most proportionately with contrast during the initial transient. As V1 responses evolved, additional contrast-mediated dynamics emerged. Specifically, responses exhibited longer maintenance of facilitation for lower contrast and binocular suppression at high contrast. These results suggest that V1 processes concordant stimulation of both eyes in at least two sequential steps: initial response enhancement followed by contrast-dependent control of excitation.

Stereo slant discrimination of planar 3D surfaces: Frontoparallel versus planar matching

Binocular stereo cues are important for discriminating 3D surface orientation, especially at near distances. We devised a single-interval task where observers discriminated the slant of a densely textured planar test surface relative to a textured planar surround reference surface. Although surfaces were rendered with correct perspective, the stimuli were designed so that the binocular cues dominated performance. Slant discrimination performance was measured as a function of the reference slant and the level of uncorrelated white noise added to the test-plane images in the left and right eyes. We compared human performance with an approximate ideal observer (planar matching [PM]) and two subideal observers. The PM observer uses the image in one eye and back projection to predict a test image in the other eye for all possible slants, tilts, and distances. The estimated slant, tilt, and distance are determined by the prediction that most closely matches the measured image in the other eye. The first subideal observer (local planar matching [LPM]) applies PM over local neighborhoods and then pools estimates across the test plane. The second suboptimal observer (local frontoparallel matching [LFM]) uses only location disparity. We find that the ideal observer (PM) and the first subideal observer (LPM) outperforms the second subideal observer (LFM), demonstrating the additional benefit of pattern disparities. We also find that all three model observers can account for human performance, if two free parameters are included: a fixed small level of internal estimation noise, and a fixed overall efficiency scalar on slant discriminability.

Disparity in Context: Understanding how monocular image content interacts with disparity processing in human visual cortex

Horizontal disparities between the two eyes' retinal images are the primary cue for depth. Commonly used random ot tereograms (RDS) intentionally camouflage the disparity cue, breaking the correlations between monocular image structure and the depth map that are present in natural images. Because of the nonlinear nature of visual processing, it is unlikely that simple computational rules derived from RDS will be sufficient to explain binocular vision in natural environments. In order to understand the interplay between natural scene structure and disparity encoding, we used a depth-image-based-rendering technique and a library of natural 3D stereo pairs to synthesize two novel stereogram types in which monocular scene content was manipulated independent of scene depth information. The half-images of the novel stereograms comprised either random-dots or scrambled natural scenes, each with the same depth maps as the corresponding natural scene stereograms. Using these stereograms in a simultaneous Event-Related Potential and behavioral discrimination task, we identified multiple disparity-contingent encoding stages between 100 ~ 500 msec. The first disparity sensitive evoked potential was observed at ~100 msec after an earlier evoked potential (between ~50-100 msec) that was sensitive to the structure of the monocular half-images but blind to disparity. Starting at ~150 msec, disparity responses were stereogram-specific and predictive of perceptual depth. Complex features associated with natural scene content are thus at least partially coded prior to disparity information, but these features and possibly others associated with natural scene content interact with disparity information only after an intermediate, 2D scene-independent disparity processing stage.

Specialized contributions of mid-tier stages of dorsal and ventral pathways to stereoscopic processing in macaque

The division of labor between the dorsal and ventral visual pathways has been well studied, but not often with direct comparison at the single-neuron resolution with matched stimuli. Here we directly compared how single neurons in MT and V4, mid-tier areas of the two pathways, process binocular disparity, a powerful cue for 3D perception and actions. We found that MT neurons transmitted disparity signals more quickly and robustly, whereas V4 or its upstream neurons transformed the signals into sophisticated representations more prominently. Therefore, signaling speed and robustness were traded for transformation between the dorsal and ventral pathways. The key factor in this tradeoff was disparity-tuning shape: V4 neurons had more even-symmetric tuning than MT neurons. Moreover, the tuning symmetry predicted the degree of signal transformation across neurons similarly within each area, implying a general role of tuning symmetry in the stereoscopic processing by the two pathways.

Vision Changes the Cellular Composition of Binocular Circuitry during the Critical Period

High acuity stereopsis emerges during an early postnatal critical period when binocular neurons in the primary visual cortex sharpen their receptive field tuning properties. We find that this sharpening is achieved by dismantling the binocular circuit present at critical period onset and building it anew. Longitudinal imaging of receptive field tuning (e.g., orientation selectivity) of thousands of neurons reveals that most binocular neurons present in layer 2/3 at critical period onset are poorly tuned and are rendered monocular. In parallel, new binocular neurons are established by conversion of well-tuned monocular neurons as they gain matched input from the other eye. These improvements in binocular tuning in layer 2/3 are not inherited from layer 4 but are driven by the experience-dependent sharpening of ipsilateral eye responses. Thus, vision builds a new and more sharply tuned binocular circuit in layer 2/3 by cellular exchange and not by refining the original circuit.

An Alternative Theory of Binocularity

The fact that seeing with two eyes is universal among vertebrates raises a problem that has long challenged vision scientists: how do animals with overlapping visual fields combine non-identical right and left eye images to achieve fusion and the perception of depth that follows? Most theories address this problem in terms of matching corresponding images on the right and left retinas. Here we suggest an alternative theory of binocular vision based on anatomical correspondence that circumvents the correspondence problem and provides a rationale for ocular dominance.

Binocular Modulation of Monocular V1 Neurons

In humans and other primates, sensory signals from each eye remain separated until they arrive in the primary visual cortex (V1), but their exact meeting point is unknown. In V1, some neurons respond to stimulation of only one eye (monocular neurons), while most neurons respond to stimulation of either eye (binocular neurons). The main input layers of V1 contain most of the monocular neurons while binocular neurons dominate the layers above and below. This observation has given rise to the idea that the two eyes' signals remain separate until they converge outside V1's input layers. Here, we show that, despite responding to only one eye, monocular neurons in all layers, including the input layers, of V1 discriminate between stimulation of their driving eye alone and stimulation of both eyes. Some monocular V1 neurons' responses were significantly enhanced, or facilitated, when both eyes were stimulated. Binocular facilitation within V1's input layers tended to occur at the onset of the visual response, which could be explained by converging thalamocortical inputs. However, most V1 monocular neurons were significantly reduced, or suppressed, to binocular stimulation. In contrast to facilitation, binocular suppression occurred several milliseconds following the onset of the visual response, suggesting that the bulk of binocular modulation involves cortical inhibition. These findings, combined, suggest that binocular signals arise at an earlier processing stage than previously appreciated, as even so-called monocular neurons in V1's input layers encode what is shown to both eyes.

Emergence of Binocular Disparity Selectivity through Hebbian Learning

Neural selectivity in the early visual cortex strongly reflects the statistics of our environment (Barlow, 2001; Geisler, 2008). Although this has been described extensively in literature through various encoding hypotheses (Barlow and Földiák, 1989; Atick and Redlich, 1992; Olshausen and Field, 1996), an explanation as to how the cortex might develop the computational architecture to support these encoding schemes remains elusive. Here, using the more realistic example of binocular vision as opposed to monocular luminance-field images, we show how a simple Hebbian coincidence-detector is capable of accounting for the emergence of binocular, disparity selective, receptive fields. We propose a model based on spike timing-dependent plasticity, which not only converges to realistic single-cell and population characteristics, but also demonstrates how known biases in natural statistics may influence population encoding and downstream correlates of behavior. Furthermore, we show that the receptive fields we obtain are closer in structure to electrophysiological data reported in macaques than those predicted by normative encoding schemes (Ringach, 2002). We also demonstrate the robustness of our model to the input dataset, noise at various processing stages, and internal parameter variation. Together, our modeling results suggest that Hebbian coincidence detection is an important computational principle and could provide a biologically plausible mechanism for the emergence of selectivity to natural statistics in the early sensory cortex.SIGNIFICANCE STATEMENT Neural selectivity in the early visual cortex is often explained through encoding schemes that postulate that the computational aim of early sensory processing is to use the least possible resources (neurons, energy) to code the most informative features of the stimulus (information efficiency). In this article, using stereo images of natural scenes, we demonstrate how a simple Hebbian rule can lead to the emergence of a disparity-selective neural population that not only shows realistic single-cell and population tunings, but also demonstrates how known biases in natural statistics may influence population encoding and downstream correlates of behavior. Our approach allows us to view early neural selectivity, not as an optimization problem, but as an emergent property driven by biological rules of plasticity.

First- and second-order contributions to depth perception in anti-correlated random dot stereograms

The binocular energy model of neural responses predicts that depth from binocular disparity might be perceived in the reversed direction when the contrast of dots presented to one eye is reversed. While reversed-depth has been found using anti-correlated random-dot stereograms (ACRDS) the findings are inconsistent across studies. The mixed findings may be accounted for by the presence of a gap between the target and surround, or as a result of overlap of dots around the vertical edges of the stimuli. To test this, we assessed whether (1) the gap size (0, 19.2 or 38.4 arc min) (2) the correlation of dots or (3) the border orientation (circular target, or horizontal or vertical edge) affected the perception of depth. Reversed-depth from ACRDS (circular no-gap condition) was seen by a minority of participants, but this effect reduced as the gap size increased. Depth was mostly perceived in the correct direction for ACRDS edge stimuli, with the effect increasing with the gap size. The inconsistency across conditions can be accounted for by the relative reliability of first- and second-order depth detection mechanisms, and the coarse spatial resolution of the latter.

The effect of image position on the Independent Components of natural binocular images

Human visual performance degrades substantially as the angular distance from the fovea increases. This decrease in performance is found for both binocular and monocular vision. Although analysis of the statistics of natural images has provided significant insights into human visual processing, little research has focused on the statistical content of binocular images at eccentric angles. We applied Independent Component Analysis to rectangular image patches cut from locations within binocular images corresponding to different degrees of eccentricity. The distribution of components learned from the varying locations was examined to determine how these distributions varied across eccentricity. We found a general trend towards a broader spread of horizontal and vertical position disparity tunings in eccentric regions compared to the fovea, with the horizontal spread more pronounced than the vertical spread. Eccentric locations above the centroid show a strong bias towards far-tuned components, eccentric locations below the centroid show a strong bias towards near-tuned components. These distributions exhibit substantial similarities with physiological measurements in V1, however in common with previous research we also observe important differences, in particular distributions of binocular phase disparity which do not match physiology.

Linking normative models of natural tasks to descriptive models of neural response

Understanding how nervous systems exploit task-relevant properties of sensory stimuli to perform natural tasks is fundamental to the study of perceptual systems. However, there are few formal methods for determining which stimulus properties are most useful for a given natural task. As a consequence, it is difficult to develop principled models for how to compute task-relevant latent variables from natural signals, and it is difficult to evaluate descriptive models fit to neural response. Accuracy maximization analysis (AMA) is a recently developed Bayesian method for finding the optimal task-specific filters (receptive fields). Here, we introduce AMA–Gauss, a new faster form of AMA that incorporates the assumption that the class-conditional filter responses are Gaussian distributed. Then, we use AMA–Gauss to show that its assumptions are justified for two fundamental visual tasks: retinal speed estimation and binocular disparity estimation. Next, we show that AMA–Gauss has striking formal similarities to popular quadratic models of neural response: the energy model and the generalized quadratic model (GQM). Together, these developments deepen our understanding of why the energy model of neural response have proven useful, improve our ability to evaluate results from subunit model fits to neural data, and should help accelerate psychophysics and neuroscience research with natural stimuli.

Binocular depth processing in the ventral visual pathway

One of the most powerful forms of depth perception capitalizes on the small relative displacements, or binocular disparities, in the images projected onto each eye. The brain employs these disparities to facilitate various computations, including sensori-motor transformations (reaching, grasping), scene segmentation and object recognition. In accordance with these different functions, disparity activates a large number of regions in the brain of both humans and monkeys. Here, we review how disparity processing evolves along different regions of the ventral visual pathway of macaques, emphasizing research based on both correlational and causal techniques. We will discuss the progression in the ventral pathway from a basic absolute disparity representation to a more complex three-dimensional shape code. We will show that, in the course of this evolution, the underlying neuronal activity becomes progressively more bound to the global perceptual experience. We argue that these observations most probably extend beyond disparity processing per se, and pertain to object processing in the ventral pathway in general. We conclude by posing some important unresolved questions whose answers may significantly advance the field, and broaden its scope.This article is part of the themed issue 'Vision in our three-dimensional world'.

Disparity processing in primary visual cortex

The first step in binocular stereopsis is to match features on the left retina with the correct features on the right retina, discarding 'false' matches. The physiological processing of these signals starts in the primary visual cortex, where the binocular energy model has been a powerful framework for understanding the underlying computation. For this reason, it is often used when thinking about how binocular matching might be performed beyond striate cortex. But this step depends critically on the accuracy of the model, and real V1 neurons show several properties that suggest they may be less sensitive to false matches than the energy model predicts. Several recent studies provide empirical support for an extended version of the energy model, in which the same principles are used, but the responses of single neurons are described as the sum of several subunits, each of which follows the principles of the energy model. These studies have significantly improved our understanding of the role played by striate cortex in the stereo correspondence problem.This article is part of the themed issue 'Vision in our three-dimensional world'.

2015 Charles F. Prentice Medal Award Lecture: Neural Organization of Binocular Vision

During a Research Career Development Award from the National Eye Institute, I spent a year at the University of Cambridge doing research with John Robson. The goal was to use a visual stimulation approach that had not been previously attempted, with the intention of exploring fundamental organization principles of the neural basis of binocular vision. The idea was to use sinusoidal gratings that drifted before both eyes such that the relative phase for one eye was fixed while that of the other was varied. This provided binocular stimuli of variable relative phase, i.e. retinal disparity, to enable testing of binocular response characteristics. We were able to obtain different types of disparity tuning functions for neurons in the primary visual cortex. This work, followed by extended investigations in Berkeley, provided basic information regarding response characteristics of simple and complex cells. We have also shown for monocular deprivation, an approximate model for human amblyopia, that many neurons remain connected to the deprived eye, as demonstrated with dichoptic activation. A selected portion of this work is described here.

Effects of generalized pooling on binocular disparity selectivity of neurons in the early visual cortex

The key problem of stereoscopic vision is traditionally defined as accurately finding the positional shifts of corresponding object features between left and right images. Here, we demonstrate that the problem must be considered in a four-dimensional parameter space; with respect not only to shifts in space (X, Y), but also spatial frequency (SF) and orientation (OR). The proposed model sums outputs of binocular energy units linearly over the multi-dimensional V1 parameter space (X, Y, SF, OR). Theoretical analyses and physiological experiments show that many binocular neurons achieve sharp binocular tuning properties by pooling the output of multiple neurons with relatively broad tuning. Pooling in the space domain sharpens disparity-selective responses in the SF domain so that the responses to combinations of unmatched left-right SFs are attenuated. Conversely, pooling in the SF domain sharpens disparity selectivity in the space domain, reducing the possibility of false matches. Analogous effects are observed for the OR domain in that the spatial pooling sharpens the binocular tuning in the OR domain. Such neurons become selective to relative OR disparity. Therefore, pooling allows the visual system to refine binocular information into a form more desirable for stereopsis.This article is part of the themed issue 'Vision in our three-dimensional world'.

Binocular Depth Judgments on Smoothly Curved Surfaces

Binocular disparity is an important cue to depth, allowing us to make very fine discriminations of the relative depth of objects. In complex scenes, this sensitivity depends on the particular shape and layout of the objects viewed. For example, judgments of the relative depths of points on a smoothly curved surface are less accurate than those for points in empty space. It has been argued that this occurs because depth relationships are represented accurately only within a local spatial area. A consequence of this is that, when judging the relative depths of points separated by depth maxima and minima, information must be integrated across separate local representations. This integration, by adding more stages of processing, might be expected to reduce the accuracy of depth judgements. We tested this idea directly by measuring how accurately human participants could report the relative depths of two dots, presented with different binocular disparities. In the first, Two Dot condition the two dots were presented in front of a square grid. In the second, Three Dot condition, an additional dot was presented midway between the target dots, at a range of depths, both nearer and further than the target dots. In the final, Surface condition, the target dots were placed on a smooth surface defined by binocular disparity cues. In some trials, this contained a depth maximum or minimum between the target dots. In the Three Dot condition, performance was impaired when the central dot was presented with a large disparity, in line with predictions. In the Surface condition, performance was worst when the midpoint of the surface was at a similar distance to the targets, and relatively unaffected when there was a large depth maximum or minimum present. These results are not consistent with the idea that depth order is represented only within a local spatial area.

Binocular functional architecture for detection of contrast-modulated gratings

Combination of signals from the two eyes is the gateway to stereo vision. To gain insight into binocular signal processing, we studied binocular summation for luminance-modulated gratings (L or LM) and contrast-modulated gratings (CM). We measured 2AFC detection thresholds for a signal grating (0.75c/deg, 216ms) shown to one eye, both eyes, or both eyes out-of-phase. For LM and CM, the carrier noise was in both eyes, even when the signal was monocular. Mean binocular thresholds for luminance gratings (L) were 5.4dB better than monocular thresholds - close to perfect linear summation (6dB). For LM and CM the binocular advantage was again 5-6dB, even when the carrier noise was uncorrelated, anti-correlated, or at orthogonal orientations in the two eyes. Binocular combination for CM probably arises from summation of envelope responses, and not from summation of these conflicting carrier patterns. Antiphase signals produced no binocular advantage, but thresholds were about 1-3dB higher than monocular ones. This is not consistent with simple linear summation, which should give complete cancellation and unmeasurably high thresholds. We propose a three-channel model in which noisy monocular responses to the envelope are binocularly combined in a contrast-weighted sum, but also remain separately available to perception via a max operator. Vision selects the largest of the three responses. With in-phase gratings the binocular channel dominates, but antiphase gratings cancel in the binocular channel and the monocular channels mediate detection. The small antiphase disadvantage might be explained by a subtle influence of background responses on binocular and monocular detection.

Efficient Receptive Field Tiling in Primate V1

The primary visual cortex (V1) encodes a diverse set of visual features, including orientation, ocular dominance (OD), and spatial frequency (SF), whose joint organization must be precisely structured to optimize coverage within the retinotopic map. Prior experiments have only identified efficient coverage based on orthogonal maps. Here we used two-photon calcium imaging to reveal an alternative arrangement for OD and SF maps in macaque V1; their gradients run parallel but with unique spatial periods, whereby low-SF regions coincide with monocular regions. Next we mapped receptive fields and found surprisingly precise micro-retinotopy that yields a smaller point-image and requires more efficient inter-map geometry, thus underscoring the significance of map relationships. While smooth retinotopy is constraining, studies suggest that it improves both wiring economy and the V1 population code read downstream. Altogether, these data indicate that connectivity within V1 is finely tuned and precise at the level of individual neurons.

Demonstrations of Spatiotemporal Integration and what they Tell us about the Visual System

Five sets of displays are presented on the journal website to be viewed in conjunction with the text. We concentrate on the factors that give rise to the integration and disruption of the direction of apparent motion in two-dimensional and three-dimensional space. In the first set of displays we examine what factors contribute to the integration and disruption of apparent motion in the Ramachandran/Anstis clustered bistable quartets. In the second set we examine what factors give rise to the perception of the direction of motion in rotating two-dimensional wheels and dots. In the third and fourth sets we examine how the depth cues of shading and disparity contribute to the perception of apparent motion of opaque displays, and to the perception of rotating unoccluded displays, respectively. In the fifth set we examine how the depth cue of motion parallax influences the perception of apparent motion. Throughout, we make inferences about the roles which various parallel pathways and cortical areas play in the perceptions produced by the displays shown.

Ideal Binocular Disparity Detectors Learned Using Independent Subspace Analysis on Binocular Natural Image Pairs

An influential theory of mammalian vision, known as the efficient coding hypothesis, holds that early stages in the visual cortex attempts to form an efficient coding of ecologically valid stimuli. Although numerous authors have successfully modelled some aspects of early vision mathematically, closer inspection has found substantial discrepancies between the predictions of some of these models and observations of neurons in the visual cortex. In particular analysis of linear-non-linear models of simple-cells using Independent Component Analysis has found a strong bias towards features on the horoptor. In order to investigate the link between the information content of binocular images, mathematical models of complex cells and physiological recordings, we applied Independent Subspace Analysis to binocular image patches in order to learn a set of complex-cell-like models. We found that these complex-cell-like models exhibited a wide range of binocular disparity-discriminability, although only a minority exhibited high binocular discrimination scores. However, in common with the linear-non-linear model case we found that feature detection was limited to the horoptor suggesting that current mathematical models are limited in their ability to explain the functionality of the visual cortex.

Pooled, but not single-neuron, responses in macaque V4 represent a solution to the stereo correspondence problem

Binocular disparity is an important cue for depth perception. To correctly represent disparity, neurons must find corresponding visual features between the left- and right-eye images. The visual pathway ascending from V1 to inferior temporal cortex solves the correspondence problem. An intermediate area, V4, has been proposed to be a critical stage in the correspondence process. However, the distinction between V1 and V4 is unclear, because accumulating evidence suggests that the process begins within V1. In this article, we report that the pooled responses in macaque V4, but not responses of individual neurons, represent a solution to the correspondence problem. We recorded single-unit responses of V4 neurons to random-dot stereograms of varying degrees of anticorrelation. To achieve gradual anticorrelation, we reversed the contrast of an increasing proportion of dots as in our previous psychophysical studies, which predicted that the neural correlates of the solution to correspondence problem should gradually eliminate their disparity modulation as the level of anticorrelation increases. Inconsistent with this prediction, the tuning amplitudes of individual V4 neurons quickly decreased to a nonzero baseline with small anticorrelation. By contrast, the shapes of individual tuning curves changed more gradually so that the amplitude of population-pooled responses gradually decreased toward zero over the entire range of graded anticorrelation. We explain these results by combining multiple energy-model subunits. From a comparison with the population-pooled responses in V1, we suggest that disparity representation in V4 is distinctly advanced from that in V1. Population readout of V4 responses provides disparity information consistent with the correspondence solution.

Hierarchical Novelty-Familiarity Representation in the Visual System by Modular Predictive Coding

Predictive coding has been previously introduced as a hierarchical coding framework for the visual system. At each level, activity predicted by the higher level is dynamically subtracted from the input, while the difference in activity continuously propagates further. Here we introduce modular predictive coding as a feedforward hierarchy of prediction modules without back-projections from higher to lower levels. Within each level, recurrent dynamics optimally segregates the input into novelty and familiarity components. Although the anatomical feedforward connectivity passes through the novelty-representing neurons, it is nevertheless the familiarity information which is propagated to higher levels. This modularity results in a twofold advantage compared to the original predictive coding scheme: the familiarity-novelty representation forms quickly, and at each level the full representational power is exploited for an optimized readout. As we show, natural images are successfully compressed and can be reconstructed by the familiarity neurons at each level. Missing information on different spatial scales is identified by novelty neurons and complements the familiarity representation. Furthermore, by virtue of the recurrent connectivity within each level, non-classical receptive field properties still emerge. Hence, modular predictive coding is a biologically realistic metaphor for the visual system that dynamically extracts novelty at various scales while propagating the familiarity information.

Computation of Object Size in Visual Cortical Area V4 as a Neural Basis for Size Constancy

Even when we view an object from different distances, so that the size of its projection onto the retina varies, we perceive its size to be relatively unchanged. In this perceptual phenomenon known as size constancy, the brain uses both distance and retinal image size to estimate the size of an object. Given that binocular disparity, the small positional difference between the retinal images in the two eyes, is a powerful visual cue for distance, we examined how it affects neuronal tuning to retinal image size in visual cortical area V4 of macaque monkeys. Depending on the imposed binocular disparity of a circular patch embedded in random dot stereograms, most neurons adjusted their preferred size in a manner consistent with size constancy. They preferred larger retinal image sizes when stimuli were stereoscopically presented nearer and preferred smaller retinal image sizes when stimuli were presented farther away. This disparity-dependent shift of preferred image size was not affected by the vergence angle, a cue for the fixation distance, suggesting that different V4 neurons compute object size for different fixation distances rather than that individual neurons adjust the shift based on vergence. This interpretation was supported by a simple circuit model, which could simulate the shift of preferred image size without any information about the fixation distance. We suggest that a population of V4 neurons encodes the actual size of objects, rather than simply the size of their retinal images, and that these neurons thereby contribute to size constancy.

SIGNIFICANCE STATEMENT

We perceive the size of an object to be relatively stable despite changes in the size of its retinal image that accompany changes in viewing distance. This phenomenon, called size constancy, is accomplished by combining retinal image size and distance information in our brain. We demonstrate that a large population of V4 neurons changes their size tuning depending on the perceived distance of a visual stimulus derived from binocular disparity. They prefer larger or smaller retinal image sizes when stimuli are stereoscopically presented nearer or farther away, respectively. This property makes V4 neurons suitable for encoding the actual size of objects, not simply the retinal image sizes, and providing a possible mechanism for perceptual size constancy.

Integration of Multiple Spatial Frequency Channels in Disparity-Sensitive Neurons in the Primary Visual Cortex

For our vivid perception of a 3-D world, the stereoscopic function begins in our brain by detecting slight shifts of image features between the two eyes, called binocular disparity. The primary visual cortex is the first stage of this processing, and neurons there are tuned to a limited range of spatial frequencies (SFs). However, our visual world is generally highly complex, composed of numerous features at a variety of scales, thereby having broadband SF spectra. This means that binocular information signaled by individual neurons is highly incomplete, and combining information across multiple SF bands must be essential for the visual system to function in a robust and reliable manner. In this study, we investigated whether the integration of information from multiple SF channels begins in the cat primary visual cortex. We measured disparity-selective responses in the joint left-right SF domain using sequences of dichoptically flashed grating stimuli consisting of various combinations of SFs and phases. The obtained interaction map in the joint SF domain reflects the degree of integration across different SF channels. Our data are consistent with the idea that disparity information is combined from multiple SF channels in a substantial fraction of complex cells. Furthermore, for the majority of these neurons, the optimal disparity is matched across the SF bands. These results suggest that a highly specific SF integration process for disparity detection starts in the primary visual cortex.

SIGNIFICANCE STATEMENT

Our visual world is broadband, containing features with a wide range of object scales. On the other hand, single neurons in the primary visual cortex are narrow-band, being tuned narrowly for a specific scale. For robust visual perception, narrow-band information of single neurons must be integrated eventually at some stage. We have examined whether such an integration process begins in the primary visual cortex with respect to binocular processing. The results suggest that a subset of cells appear to combine binocular information across multiple scales. Furthermore, for the majority of these neurons, an optimal parameter of binocular tuning is matched across multiple scales, suggesting the presence of a highly specific neural integration mechanism.

Luminance, Colour, Viewpoint and Border Enhanced Disparity Energy Model

The visual cortex is able to extract disparity information through the use of binocular cells. This process is reflected by the Disparity Energy Model, which describes the role and functioning of simple and complex binocular neuron populations, and how they are able to extract disparity. This model uses explicit cell parameters to mathematically determine preferred cell disparities, like spatial frequencies, orientations, binocular phases and receptive field positions. However, the brain cannot access such explicit cell parameters; it must rely on cell responses. In this article, we implemented a trained binocular neuronal population, which encodes disparity information implicitly. This allows the population to learn how to decode disparities, in a similar way to how our visual system could have developed this ability during evolution. At the same time, responses of monocular simple and complex cells can also encode line and edge information, which is useful for refining disparities at object borders. The brain should then be able, starting from a low-level disparity draft, to integrate all information, including colour and viewpoint perspective, in order to propagate better estimates to higher cortical areas.

On the role of spatial phase and phase correlation in vision, illusion, and cognition

Numerous findings indicate that spatial phase bears an important cognitive information. Distortion of phase affects topology of edge structures and makes images unrecognizable. In turn, appropriately phase-structured patterns give rise to various illusions of virtual image content and apparent motion. Despite a large body of phenomenological evidence not much is known yet about the role of phase information in neural mechanisms of visual perception and cognition. Here, we are concerned with analysis of the role of spatial phase in computational and biological vision, emergence of visual illusions and pattern recognition. We hypothesize that fundamental importance of phase information for invariant retrieval of structural image features and motion detection promoted development of phase-based mechanisms of neural image processing in course of evolution of biological vision. Using an extension of Fourier phase correlation technique, we show that the core functions of visual system such as motion detection and pattern recognition can be facilitated by the same basic mechanism. Our analysis suggests that emergence of visual illusions can be attributed to presence of coherently phase-shifted repetitive patterns as well as the effects of acuity compensation by saccadic eye movements. We speculate that biological vision relies on perceptual mechanisms effectively similar to phase correlation, and predict neural features of visual pattern (dis)similarity that can be used for experimental validation of our hypothesis of “cognition by phase correlation.”

Solving stereo transparency with an extended coarse-to-fine disparity energy model

Modeling stereo transparency with physiologically plausible mechanisms is challenging because in such frameworks, large receptive fields mix up overlapping disparities, whereas small receptive fields can reliably compute only small disparities. It seems necessary to combine information across scales. A coarse-to-fine disparity energy model, with both position- and phase-shift receptive fields, has already been proposed. However, because each scale decodes only one disparity for each location and uses the decoded disparity to select cells at the next scale, this model cannot represent overlapping surfaces at different depths. We have extended the model to solve stereo transparency. First, we introduce multiplicative connections from cells at one scale to the next to implement coarse-to-fine computation. The connection is the strongest when the presynaptic cell's preferred disparity matches the postsynaptic cell's position-shift parameter, encouraging the next scale to encode residual disparities with the more reliable phase-shift mechanism. This modification not only eliminates the artificial decoding and selection steps of the original model but also enables maintenance of complete population responses throughout the coarse-to-fine process. Second, because of this modification, explicit decoding is no longer necessary but rather is for visualization only. We use a simple threshold criterion to decode multiple disparities from population energy responses instead of a single disparity in the original model. We demonstrate our model using simulations on a variety of transparent and nontransparent stereograms. The model also reproduces psychophysically observed disparity interactions (averaging, thickening, attraction, and repulsion) as the depth separation between two overlapping planes varies.

Neural representation of motion-in-depth in area MT

Neural processing of 2D visual motion has been studied extensively, but relatively little is known about how visual cortical neurons represent visual motion trajectories that include a component toward or away from the observer (motion in depth). Psychophysical studies have demonstrated that humans perceive motion in depth based on both changes in binocular disparity over time (CD cue) and interocular velocity differences (IOVD cue). However, evidence for neurons that represent motion in depth has been limited, especially in primates, and it is unknown whether such neurons make use of CD or IOVD cues. We show that approximately one-half of neurons in macaque area MT are selective for the direction of motion in depth, and that this selectivity is driven primarily by IOVD cues, with a small contribution from the CD cue. Our results establish that area MT, a central hub of the primate visual motion processing system, contains a 3D representation of visual motion.

An Augmented Two-Layer Model Captures Nonlinear Analog Spatial Integration Effects in Pyramidal Neuron Dendrites

In pursuit of the goal to understand and eventually reproduce the diverse functions of the brain, a key challenge lies in reverse engineering the peculiar biology-based "technology" that underlies the brain's remarkable ability to process and store information. The basic building block of the nervous system is the nerve cell, or "neuron," yet after more than 100 years of neurophysiological study and 60 years of modeling, the information processing functions of individual neurons, and the parameters that allow them to engage in so many different types of computation (sensory, motor, mnemonic, executive, etc.) remain poorly understood. In this paper, we review both historical and recent findings that have led to our current understanding of the analog spatial processing capabilities of dendrites, the major input structures of neurons, with a focus on the principal cell type of the neocortex and hippocampus, the pyramidal neuron (PN). We encapsulate our current understanding of PN dendritic integration in an abstract layered model whose spatially sensitive branch-subunits compute multidimensional sigmoidal functions. Unlike the 1-D sigmoids found in conventional neural network models, multidimensional sigmoids allow the cell to implement a rich spectrum of nonlinear modulation effects directly within their dendritic trees.

Delayed suppression shapes disparity selective responses in monkey V1

The stereo correspondence problem poses a challenge to visual neurons because localized receptive fields potentially cause false responses. Neurons in the primary visual cortex (V1) partially resolve this problem by combining excitatory and suppressive responses to encode binocular disparity. We explored the time course of this combination in awake, monkey V1 neurons using subspace mapping of receptive fields. The stimulus was a binocular noise pattern constructed from discrete spatial frequency components. We forward correlated the firing of the V1 neuron with the occurrence of binocular presentations of each spatial frequency component. The forward correlation yielded a complete set of response time courses to every combination of spatial frequency and interocular phase difference. Some combinations produced suppressive responses. Typically, if an interocular phase difference for a given spatial frequency produced strong excitation, we saw suppression in response to the opposite interocular phase difference at lower spatial frequencies. The suppression was delayed relative to the excitation, with a median difference in latency of 7 ms. We found that the suppressive mechanism explains a well-known mismatch of monocular and binocular signals. The suppressive components increased power at low spatial frequencies in disparity tuning, whereas they reduced the monocular response to low spatial frequencies. This long-recognized mismatch of binocular and monocular signals reflects a suppressive mechanism that helps reduce the response to false matches.

Optimal disparity estimation in natural stereo images

A great challenge of systems neuroscience is to understand the computations that underlie perceptual constancies, the ability to represent behaviorally relevant stimulus properties as constant even when irrelevant stimulus properties vary. As signals proceed through the visual system, neural states become more selective for properties of the environment, and more invariant to irrelevant features of the retinal images. Here, we describe a method for determining the computations that perform these transformations optimally, and apply it to the specific computational task of estimating a powerful depth cue: binocular disparity. We simultaneously determine the optimal receptive field population for encoding natural stereo images of locally planar surfaces and the optimal nonlinear units for decoding the population responses into estimates of disparity. The optimal processing predicts well-established properties of neurons in cortex. Estimation performance parallels important aspects of human performance. Thus, by analyzing the photoreceptor responses to natural images, we provide a normative account of the neurophysiology and psychophysics of absolute disparity processing. Critically, the optimal processing rules are not arbitrarily chosen to match the properties of neurophysiological processing, nor are they fit to match behavioral performance. Rather, they are dictated by the task-relevant statistical properties of complex natural stimuli. Our approach reveals how selective invariant tuning-especially for properties not trivially available in the retinal images-could be implemented in neural systems to maximize performance in particular tasks.

Stereoscopic depth perception using a model based on the primary visual cortex

This work describes an approach inspired by the primary visual cortex using the stimulus response of the receptive field profiles of binocular cells for disparity computation. Using the energy model based on the mechanism of log-Gabor filters for disparity encodings, we propose a suitable model to consistently represent the complex cells by computing the wide bandwidths of the cortical cells. This way, the model ensures the general neurophysiological findings in the visual cortex (V1), emphasizing the physical disparities and providing a simple selection method for the complex cell response. The results suggest that our proposed approach can achieve better results than a hybrid model with phase-shift and position-shift using position disparity alone.

A study on surface slant encoding in V1

Inter-ocular differences in spatial frequency occur during binocular viewing of a surface slanted in depth. Cortical cells with inter-ocular differences in preferred spatial frequency (dif-frequency cells) are expected to detect surfaces slanted in depth or vertical surface slant. Using our reaction-diffusion model, we obtain receptive fields and responses of simple cells in layer IV in cat V1. The dif-frequency cells in the model cortex have tilt in binocular receptive field but we show that tilt by itself does not indicate slant selectivity. We studied cell responses to binocular combination of spatial frequencies (SFs) by varying the SF ratio of the input gratings to the left and right eye in the range of 0.35-3. This range of SF ratio corresponds to surface slant variation of -85° to 85°. The mean binocular tuning hwhh (half width at half height) is 41°. Except for a small number (2.5%) of cells, most dif-frequency cells respond almost equally well for fronto-parallel surfaces. In the literature cells with inter-ocular difference in preferred orientation (IDPO) were expected to encode horizontal surface slant. In the model cat V1 mean hwhh in binocular orientation tuning curve for cells with IDPO is 39°. The wide binocular tuning width in dif-frequency cells and cells with IDPO imply that in cat V1 neither dif-frequency cells nor cells with IDPO detect surface slant.

Rapid, parallel path planning by propagating wavefronts of spiking neural activity

Efficient path planning and navigation is critical for animals, robotics, logistics and transportation. We study a model in which spatial navigation problems can rapidly be solved in the brain by parallel mental exploration of alternative routes using propagating waves of neural activity. A wave of spiking activity propagates through a hippocampus-like network, altering the synaptic connectivity. The resulting vector field of synaptic change then guides a simulated animal to the appropriate selected target locations. We demonstrate that the navigation problem can be solved using realistic, local synaptic plasticity rules during a single passage of a wavefront. Our model can find optimal solutions for competing possible targets or learn and navigate in multiple environments. The model provides a hypothesis on the possible computational mechanisms for optimal path planning in the brain, at the same time it is useful for neuromorphic implementations, where the parallelism of information processing proposed here can fully be harnessed in hardware.

Simulating the cortical 3D visuomotor transformation of reach depth

We effortlessly perform reach movements to objects in different directions and depths. However, how networks of cortical neurons compute reach depth from binocular visual inputs remains largely unknown. To bridge the gap between behavior and neurophysiology, we trained a feed-forward artificial neural network to uncover potential mechanisms that might underlie the 3D transformation of reach depth. Our physiologically-inspired 4-layer network receives distributed 3D visual inputs (1^st layer) along with eye, head and vergence signals. The desired motor plan was coded in a population (3^rd layer) that we read out (4^th layer) using an optimal linear estimator. After training, our network was able to reproduce all known single-unit recording evidence on depth coding in the parietal cortex. Network analyses predict the presence of eye/head and vergence changes of depth tuning, pointing towards a gain-modulation mechanism of depth transformation. In addition, reach depth was computed directly from eye-centered (relative) visual distances, without explicit absolute depth coding. We suggest that these effects should be observable in parietal and pre-motor areas.

Contributions of excitation and suppression in shaping spatial frequency selectivity of V1 neurons as revealed by binocular measurements

Neurons in the early visual cortex are generally highly sensitive to stimuli presented to the two eyes. However, the majority of studies on spatial and temporal aspects of neural responses were based on monocular measurements. To study neurons under more natural, i.e., binocular, conditions, we presented sinusoidal gratings of a variety of spatial frequencies (SF) dichoptically in rapid sequential flashes and analyzed the data using a binocular reverse correlation technique for neurons in cat area 17. The resulting set of data represents a frequency-domain binocular receptive field from which detailed selectivities, both monocular and binocular, could be obtained. Consistent with previous studies, the responses could generally be explained by linear summation of inputs from the two eyes. Suppressive responses were also observed and were delayed typically by 5–15 ms relative to excitatory responses. However, we have found more diverse nature of suppressive responses than those reported previously. The optimal suppressive frequency could be either higher or lower than that of the excitatory responses. The bandwidth of SF tuning of the suppressive responses was usually broader than that of the excitatory responses. Cells with lower optimal SFs for suppression tended to show high optimal SFs and sharp tuning curves. The dynamic shift of optimal SF from low to high SF was accompanied by suppression with earlier onset and higher peak SF or later onset and lower peak SF than excitation. These results suggest that the suppression plays an essential role in generating the temporal dynamics of SF selectivity.

Binocular disparity tuning and visual-vestibular congruency of multisensory neurons in macaque parietal cortex

Many neurons in the dorsal medial superior temporal (MSTd) and ventral intraparietal (VIP) areas of the macaque brain are multisensory, responding to both optic flow and vestibular cues to self-motion. The heading tuning of visual and vestibular responses can be either congruent or opposite, but only congruent cells have been implicated in cue integration for heading perception. Because of the geometric properties of motion parallax, however, both congruent and opposite cells could be involved in coding self-motion when observers fixate a world-fixed target during translation, if congruent cells prefer near disparities and opposite cells prefer far disparities. We characterized the binocular disparity selectivity and heading tuning of MSTd and VIP cells using random-dot stimuli. Most (70%) MSTd neurons were disparity selective with monotonic tuning, and there was no consistent relationship between depth preference and congruency of visual and vestibular heading tuning. One-third of disparity-selective MSTd cells reversed their depth preference for opposite directions of motion [direction-dependent disparity tuning (DDD)], but most of these cells were unisensory with no tuning for vestibular stimuli. Inconsistent with previous reports, the direction preferences of most DDD neurons do not reverse with disparity. By comparison to MSTd, VIP contains fewer disparity-selective neurons (41%) and very few DDD cells. On average, VIP neurons also preferred higher speeds and nearer disparities than MSTd cells. Our findings are inconsistent with the hypothesis that visual/vestibular congruency is linked to depth preference, and also suggest that DDD cells are not involved in multisensory integration for heading perception.

A reaction-diffusion model to capture disparity selectivity in primary visual cortex

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.

Adaptation changes stereoscopic depth selectivity in visual cortex

Exposure to specific visual stimuli causes a reduction in sensitivity to similar subsequent stimulation. This adaptation effect is observed behaviorally and for neurons in the primary visual cortex. Here, we explore the effects of adaptation on neurons that encode binocular depth discrimination in the cat's primary visual cortex. Our results show that neuronal preference for binocular depth is altered selectively with appropriate adaptation. At the preferred depth, adaptation causes substantial suppression of subsequent responses. Near the preferred depth, the same procedure causes a shift in depth preference. At the null depth, adaptation has little effect on binocular depth coding. These results demonstrate that prior exposure can change the depth selectivity of binocular neurons. The findings are relevant to the theoretical treatment of binocular depth processing. Specifically, the prevailing notion of binocular depth encoding based on the energy model requires modification.