Ocular dominance predicts neither strength nor class of disparity selectivity with random-dot stimuli in primate V1

Multisensory-inspired modeling and neural correlates for two key binocular interactions

Most binocular vision models assume that the two eyes sum incompletely. However, some facilitatory cortical neurons fire for only one eye, but amplify their firing rates if both eyes are stimulated. These 'binocular gate' neurons closely resemble subthreshold multisensory neurons. Binocular amplification for binocular gate neurons follows a power law, with a compressive exponent. Unexpectedly, this rule also applies to facilitatory true binocular neurons; although driven by either eye, binocular neurons are well modeled as gated amplifiers of their strongest monocular response, if both eyes are stimulated. Psychophysical data follows the same power law as the neural data, with a similar exponent; binocular contrast sensitivity can be modeled as a gated amplification of the more sensitive eye. These results resemble gated amplification phenomena in multisensory integration, and other non-driving modulatory interactions that affect sensory processing. Models of incomplete summation seem unnecessary for V1 facilitatory neurons or contrast sensitivity. However, binocular combination of clearly visible monocular stimuli follows Schrödinger's nonlinear magnitude-weighted average. We find that putatively suppressive binocular neurons closely follow Schrödinger's equation. Similar suppressive multisensory neurons are well documented but seldom studied. Facilitatory binocular neurons and mildly suppressive binocular neurons are likely neural correlates of binocular sensitivity and binocular appearance respectively.

Binocular luster elicited by isoluminant chromatic stimuli relies on mechanisms similar to those in the achromatic case

The phenomenon of binocular luster can be evoked by simple dichoptic center-surround stimuli showing a luminance contrast difference between the eyes. Previous findings support the idea that this phenomenon is mediated by a low-level conflict mechanism that integrates the monocular signals from different types of contrast detector cells. Also, isoluminant stimuli with different chromatic contrasts between eyes can trigger sensations of luster. Here, we investigate whether the lustrous impression in such purely chromatic stimuli depends on interocular contrast differences and in particular on interocular contrast polarity pairings in a similar way as in the achromatic case. In our experiments, we measured the magnitude of the lustrous response using a series of isoluminant dichoptic center-ring-surround stimuli with varying ring width whose chromatic properties were varied along the red-green and blue-yellow cardinal directions. The trends in the data were very similar to those of our former study with achromatic stimuli, indicating similar mechanisms in both cases. The empirical luster data could also be predicted fairly well by a chromatic version of our interocular conflict model (with overall R2 values between 0.577 and 0.639), for which two different receptive field models were used, simulating the behavior of color-sensitive double-opponent cells in V1.

A role for ocular dominance in binocular integration

Neurons in the primate primary visual cortex (V1) combine left- and right-eye information to form a binocular output. Controversy surrounds whether ocular dominance, the preference of these neurons for one eye over the other, is functionally relevant. Here, we demonstrate that ocular dominance impacts gain control during binocular combination. We recorded V1 spiking activity while monkeys passively viewed grating stimuli. Gratings were either presented to one eye (monocular), both eyes with the same contrasts (binocular balanced), or both eyes with different contrasts (binocular imbalanced). We found that contrast placed in a neuron's dominant eye was weighted more strongly than contrast placed in a neuron's non-dominant eye. This asymmetry covaried with neurons' ocular dominance. We then tested whether accounting for ocular dominance within divisive normalization improves the fit to neural data. We found that ocular dominance significantly improved model performance, with interocular normalization providing the best fits. These findings suggest that V1 ocular dominance is relevant for response normalization during binocular stimulation.

Widespread and Multifaceted Binocular Integration in the Mouse Primary Visual Cortex

The brain combines two-dimensional images received from the two eyes to form a percept of three-dimensional surroundings. This process of binocular integration in the primary visual cortex (V1) serves as a useful model for studying how neural circuits generate emergent properties from multiple input signals. Here, we perform a thorough characterization of binocular integration using electrophysiological recordings in the V1 of awake adult male and female mice by systematically varying the orientation and phase disparity of monocular and binocular stimuli. We reveal widespread binocular integration in mouse V1 and demonstrate that the three commonly studied binocular properties-ocular dominance, interocular matching, and disparity selectivity-are independent of each other. For individual neurons, the responses to monocular stimulation can predict the average amplitude of binocular response but not its selectivity. Finally, the extensive and independent binocular integration of monocular inputs is seen across cortical layers in both regular-spiking and fast-spiking neurons, regardless of stimulus design. Our data indicate that the current model of simple feedforward convergence is inadequate to account for binocular integration in mouse V1, thus suggesting an indispensable role played by intracortical circuits in binocular computation.SIGNIFICANCE STATEMENT Binocular integration is an important step of visual processing that takes place in the visual cortex. Studying the process by which V1 neurons become selective for certain binocular disparities is informative about how neural circuits integrate multiple information streams at a more general level. Here, we systematically characterize binocular integration in mice. Our data demonstrate more widespread and complex binocular integration in mouse V1 than previously reported. Binocular responses cannot be explained by a simple convergence of monocular responses, contrary to the prevailing model of binocular integration. These findings thus indicate that intracortical circuits must be involved in the exquisite computation of binocular disparity, which would endow brain circuits with the plasticity needed for binocular development and processing.

Neural circuits for binocular vision: Ocular dominance, interocular matching, and disparity selectivity

The brain creates a single visual percept of the world with inputs from two eyes. This means that downstream structures must integrate information from the two eyes coherently. Not only does the brain meet this challenge effortlessly, it also uses small differences between the two eyes' inputs, i.e., binocular disparity, to construct depth information in a perceptual process called stereopsis. Recent studies have advanced our understanding of the neural circuits underlying stereoscopic vision and its development. Here, we review these advances in the context of three binocular properties that have been most commonly studied for visual cortical neurons: ocular dominance of response magnitude, interocular matching of orientation preference, and response selectivity for binocular disparity. By focusing mostly on mouse studies, as well as recent studies using ferrets and tree shrews, we highlight unresolved controversies and significant knowledge gaps regarding the neural circuits underlying binocular vision. We note that in most ocular dominance studies, only monocular stimulations are used, which could lead to a mischaracterization of binocularity. On the other hand, much remains unknown regarding the circuit basis of interocular matching and disparity selectivity and its development. We conclude by outlining opportunities for future studies on the neural circuits and functional development of binocular integration in the early visual system.

A simple model of binocular luster

The dichoptic combination of simple center–surround stimuli showing a contrast difference between eyes can trigger a lustrous impression in the fused percept, particularly when the contrast polarities in the two input images are of opposite sign. Recent developments suggest that the phenomenon of binocular luster results from a neural conflict between ON and OFF visual pathways at an early binocular level. Support for this idea was found in a previous study in which the empirical luster judgments strongly correlated with the predictions of an interocular conflict model which was based on such ON–OFF pairings. However, our original model could not account for the fact that weaker lustrous sensations can also be evoked by stimuli showing contrast polarities of same sign between eyes. In the present study we present an improved model that also includes ON–ON and OFF–OFF pairings. The predictive power of this model was tested in a series of four experiments, using a total of about 500 different center–ring–surround configurations as test stimuli. We found that, overall, our modified version accounts for more than 80% of the variance in the empirical luster judgments and that the former problems could be largely resolved. Our results further suggest a nonlinear transducer function for the binocular conflict signals.

Binocular luster - A review

Binocular luster is a visual phenomenon that can be elicited by dichoptic stimuli showing an interocular difference in color or luminance contrast. For instance, when the two eyes are presented with simple center-surround stimuli in which the center patch in one eye is brighter and in the other eye darker than the common surround, the center patch in the fused percept assumes a lustrous appearance reminiscent of metal or graphite. Soon after the discovery of this phenomenon in the mid-19th century, it was intensively studied and several explanations were proposed. After this initial phase, however, research interest waned significantly. Stimulated by new insights into related phenomena and the underlying physiological mechanisms, the last 20 years have seen an increase in research activity in this field, which has considerably expanded our understanding of binocular luster. In this paper, we provide a detailed review of research on binocular luster over the past 170 years. We present and discuss the existing findings in a number of separate sections, dealing with 1) the phenomenology of binocular luster, 2) different theories that have been proposed, 3) several factors influencing the lustrous impression, 4) the relationship between binocular luster and binocular rivalry, 5) the current understanding of its neural basis, and 6) potential applications based on binocular luster.

Interocular suppression in primary visual cortex in strabismus: impact of staggering the presentation of stimuli to the eyes

Diplopia (double vision) in strabismus is prevented by suppression of the image emanating from one eye. In a recent study conducted in two macaques raised with exotropia (an outward ocular deviation) but having normal acuity in each eye, simultaneous display of stimuli to each eye did not induce suppression in V1 neurons. Puzzled by this negative result, we have modified our protocol to display stimuli in a staggered sequence, rather than simultaneously. Additional recordings were made in the same two macaques, following two paradigms. In trial type 1, the receptive field in one eye was stimulated with a sine-wave grating while the other eye was occluded. After 5 s, the occluder was removed and the neuron was stimulated for another 5 s. The effect of uncovering the eye, which potentially exposed the animal to diplopia, was quantified by the peripheral retinal interaction index (PRII). In trial type 2, the receptive field in the fixating eye was stimulated with a grating during binocular viewing. After 5 s, a second grating appeared in the receptive field of the nonfixating eye. The impact of the second grating, which had the potential to generate visual confusion, was quantified by the receptive field interaction index (RFII). For 82 units, the mean PRII was 0.48 ± 0.05 (0.50 = no suppression) and the mean RFII was 0.46 ± 0.08 (0.50 = no suppression). These values suggest mild suppression, but the modest decline in spike rate registered during the second epoch of visual stimulation might have been due to neuronal adaptation, rather than interocular suppression. In a few instances neurons showed unequivocal suppression, but overall, these recordings did not support the contention that staggered stimulus presentation is more effective than simultaneous stimulus presentation at evoking interocular suppression in V1 neurons.NEW & NOTEWORTHY In strabismus, double vision is prevented by interocular suppression. It has been reported that inhibition of neuronal firing in the primary visual cortex occurs only when stimuli are presented sequentially, rather than simultaneously. However, these recordings in alert macaques raised with exotropia showed, with rare exceptions, little evidence to support the concept that staggered stimulus presentation is more effective at inducing interocular suppression of V1 neurons.

Specialized contributions of mid-tier stages of dorsal and ventral pathways to stereoscopic processing in macaque

The division of labor between the dorsal and ventral visual pathways has been well studied, but not often with direct comparison at the single-neuron resolution with matched stimuli. Here we directly compared how single neurons in MT and V4, mid-tier areas of the two pathways, process binocular disparity, a powerful cue for 3D perception and actions. We found that MT neurons transmitted disparity signals more quickly and robustly, whereas V4 or its upstream neurons transformed the signals into sophisticated representations more prominently. Therefore, signaling speed and robustness were traded for transformation between the dorsal and ventral pathways. The key factor in this tradeoff was disparity-tuning shape: V4 neurons had more even-symmetric tuning than MT neurons. Moreover, the tuning symmetry predicted the degree of signal transformation across neurons similarly within each area, implying a general role of tuning symmetry in the stereoscopic processing by the two pathways.

Stimulus Feature-Specific Information Flow Along the Columnar Cortical Microcircuit Revealed by Multivariate Laminar Spiking Analysis

Most of the mammalian neocortex is comprised of a highly similar anatomical structure, consisting of a granular cell layer between superficial and deep layers. Even so, different cortical areas process different information. Taken together, this suggests that cortex features a canonical functional microcircuit that supports region-specific information processing. For example, the primate primary visual cortex (V1) combines the two eyes' signals, extracts stimulus orientation, and integrates contextual information such as visual stimulation history. These processes co-occur during the same laminar stimulation sequence that is triggered by the onset of visual stimuli. Yet, we still know little regarding the laminar processing differences that are specific to each of these types of stimulus information. Univariate analysis techniques have provided great insight by examining one electrode at a time or by studying average responses across multiple electrodes. Here we focus on multivariate statistics to examine response patterns across electrodes instead. Specifically, we applied multivariate pattern analysis (MVPA) to linear multielectrode array recordings of laminar spiking responses to decode information regarding the eye-of-origin, stimulus orientation, and stimulus repetition. MVPA differs from conventional univariate approaches in that it examines patterns of neural activity across simultaneously recorded electrode sites. We were curious whether this added dimensionality could reveal neural processes on the population level that are challenging to detect when measuring brain activity without the context of neighboring recording sites. We found that eye-of-origin information was decodable for the entire duration of stimulus presentation, but diminished in the deepest layers of V1. Conversely, orientation information was transient and equally pronounced along all layers. More importantly, using time-resolved MVPA, we were able to evaluate laminar response properties beyond those yielded by univariate analyses. Specifically, we performed a time generalization analysis by training a classifier at one point of the neural response and testing its performance throughout the remaining period of stimulation. Using this technique, we demonstrate repeating (reverberating) patterns of neural activity that have not previously been observed using standard univariate approaches.

Disparity Sensitivity and Binocular Integration in Mouse Visual Cortex Areas

Binocular disparity, the difference between the two eyes' images, is a powerful cue to generate the 3D depth percept known as stereopsis. In primates, binocular disparity is processed in multiple areas of the visual cortex, with distinct contributions of higher areas to specific aspects of depth perception. Mice, too, can perceive stereoscopic depth, and neurons in primary visual cortex (V1) and higher-order, lateromedial (LM) and rostrolateral (RL) areas were found to be sensitive to binocular disparity. A detailed characterization of disparity tuning across mouse visual areas is lacking, however, and acquiring such data might help clarifying the role of higher areas for disparity processing and establishing putative functional correspondences to primate areas. We used two-photon calcium imaging in female mice to characterize the disparity tuning properties of neurons in visual areas V1, LM, and RL in response to dichoptically presented binocular gratings, as well as random dot correlograms (RDC). In all three areas, many neurons were tuned to disparity, showing strong response facilitation or suppression at optimal or null disparity, respectively, even in neurons classified as monocular by conventional ocular dominance (OD) measurements. Neurons in higher areas exhibited broader and more asymmetric disparity tuning curves compared with V1, as observed in primate visual cortex. Finally, we probed neurons' sensitivity to true stereo correspondence by comparing responses to correlated RDC (cRDC) and anticorrelated RDC (aRDC). Area LM, akin to primate ventral visual stream areas, showed higher selectivity for correlated stimuli and reduced anticorrelated responses, indicating higher-level disparity processing in LM compared with V1 and RL.SIGNIFICANCE STATEMENT A major cue for inferring 3D depth is disparity between the two eyes' images. Investigating how binocular disparity is processed in the mouse visual system will not only help delineating the role of mouse higher areas for visual processing, but also shed light on how the mammalian brain computes stereopsis. We found that binocular integration is a prominent feature of mouse visual cortex, as many neurons are selectively and strongly modulated by binocular disparity. Comparison of responses to correlated and anticorrelated random dot correlograms (RDC) revealed that lateromedial area (LM) is more selective to correlated stimuli, while less sensitive to anticorrelated stimuli compared with primary visual cortex (V1) and rostrolateral area (RL), suggesting higher-level disparity processing in LM, resembling primate ventral visual stream areas.

An Alternative Theory of Binocularity

The fact that seeing with two eyes is universal among vertebrates raises a problem that has long challenged vision scientists: how do animals with overlapping visual fields combine non-identical right and left eye images to achieve fusion and the perception of depth that follows? Most theories address this problem in terms of matching corresponding images on the right and left retinas. Here we suggest an alternative theory of binocular vision based on anatomical correspondence that circumvents the correspondence problem and provides a rationale for ocular dominance.

The psychophysics of stereopsis can be explained without invoking independent ON and OFF channels

Early vision proceeds through distinct ON and OFF channels, which encode luminance increments and decrements respectively. It has been argued that these channels also contribute separately to stereoscopic vision. This is based on the fact that observers perform better on a noisy disparity discrimination task when the stimulus is a random-dot pattern consisting of equal numbers of black and white dots (a “mixed-polarity stimulus,” argued to activate both ON and OFF stereo channels), than when it consists of all-white or all-black dots (“same-polarity,” argued to activate only one). However, it is not clear how this theory can be reconciled with our current understanding of disparity encoding. Recently, a binocular convolutional neural network was able to replicate the mixed-polarity advantage shown by human observers, even though it was based on linear filters and contained no mechanisms which would respond separately to black or white dots. Here, we show that a subtle feature of the way the stimuli were constructed in all these experiments can explain the results. The interocular correlation between left and right images is actually lower for the same-polarity stimuli than for mixed-polarity stimuli with the same amount of disparity noise applied to the dots. Because our current theories suggest stereopsis is based on a correlation-like computation in primary visual cortex, this postulate can explain why performance was better for the mixed-polarity stimuli. We conclude that there is currently no evidence supporting separate ON and OFF channels in stereopsis.

Binocular Modulation of Monocular V1 Neurons

In humans and other primates, sensory signals from each eye remain separated until they arrive in the primary visual cortex (V1), but their exact meeting point is unknown. In V1, some neurons respond to stimulation of only one eye (monocular neurons), while most neurons respond to stimulation of either eye (binocular neurons). The main input layers of V1 contain most of the monocular neurons while binocular neurons dominate the layers above and below. This observation has given rise to the idea that the two eyes' signals remain separate until they converge outside V1's input layers. Here, we show that, despite responding to only one eye, monocular neurons in all layers, including the input layers, of V1 discriminate between stimulation of their driving eye alone and stimulation of both eyes. Some monocular V1 neurons' responses were significantly enhanced, or facilitated, when both eyes were stimulated. Binocular facilitation within V1's input layers tended to occur at the onset of the visual response, which could be explained by converging thalamocortical inputs. However, most V1 monocular neurons were significantly reduced, or suppressed, to binocular stimulation. In contrast to facilitation, binocular suppression occurred several milliseconds following the onset of the visual response, suggesting that the bulk of binocular modulation involves cortical inhibition. These findings, combined, suggest that binocular signals arise at an earlier processing stage than previously appreciated, as even so-called monocular neurons in V1's input layers encode what is shown to both eyes.

Emergence of Binocular Disparity Selectivity through Hebbian Learning

Neural selectivity in the early visual cortex strongly reflects the statistics of our environment (Barlow, 2001; Geisler, 2008). Although this has been described extensively in literature through various encoding hypotheses (Barlow and Földiák, 1989; Atick and Redlich, 1992; Olshausen and Field, 1996), an explanation as to how the cortex might develop the computational architecture to support these encoding schemes remains elusive. Here, using the more realistic example of binocular vision as opposed to monocular luminance-field images, we show how a simple Hebbian coincidence-detector is capable of accounting for the emergence of binocular, disparity selective, receptive fields. We propose a model based on spike timing-dependent plasticity, which not only converges to realistic single-cell and population characteristics, but also demonstrates how known biases in natural statistics may influence population encoding and downstream correlates of behavior. Furthermore, we show that the receptive fields we obtain are closer in structure to electrophysiological data reported in macaques than those predicted by normative encoding schemes (Ringach, 2002). We also demonstrate the robustness of our model to the input dataset, noise at various processing stages, and internal parameter variation. Together, our modeling results suggest that Hebbian coincidence detection is an important computational principle and could provide a biologically plausible mechanism for the emergence of selectivity to natural statistics in the early sensory cortex.SIGNIFICANCE STATEMENT Neural selectivity in the early visual cortex is often explained through encoding schemes that postulate that the computational aim of early sensory processing is to use the least possible resources (neurons, energy) to code the most informative features of the stimulus (information efficiency). In this article, using stereo images of natural scenes, we demonstrate how a simple Hebbian rule can lead to the emergence of a disparity-selective neural population that not only shows realistic single-cell and population tunings, but also demonstrates how known biases in natural statistics may influence population encoding and downstream correlates of behavior. Our approach allows us to view early neural selectivity, not as an optimization problem, but as an emergent property driven by biological rules of plasticity.

Neural architectures for stereo vision

Stereoscopic vision delivers a sense of depth based on binocular information but additionally acts as a mechanism for achieving correspondence between patterns arriving at the left and right eyes. We analyse quantitatively the cortical architecture for stereoscopic vision in two areas of macaque visual cortex. For primary visual cortex V1, the result is consistent with a module that is isotropic in cortical space with a diameter of at least 3 mm in surface extent. This implies that the module for stereo is larger than the repeat distance between ocular dominance columns in V1. By contrast, in the extrastriate cortical area V5/MT, which has a specialized architecture for stereo depth, the module for representation of stereo is about 1 mm in surface extent, so the representation of stereo in V5/MT is more compressed than V1 in terms of neural wiring of the neocortex. The surface extent estimated for stereo in V5/MT is consistent with measurements of its specialized domains for binocular disparity. Within V1, we suggest that long-range horizontal, anatomical connections form functional modules that serve both binocular and monocular pattern recognition: this common function may explain the distortion and disruption of monocular pattern vision observed in amblyopia.

This article is part of the themed issue ‘Vision in our three-dimensional world’.

Efficient Receptive Field Tiling in Primate V1

The primary visual cortex (V1) encodes a diverse set of visual features, including orientation, ocular dominance (OD), and spatial frequency (SF), whose joint organization must be precisely structured to optimize coverage within the retinotopic map. Prior experiments have only identified efficient coverage based on orthogonal maps. Here we used two-photon calcium imaging to reveal an alternative arrangement for OD and SF maps in macaque V1; their gradients run parallel but with unique spatial periods, whereby low-SF regions coincide with monocular regions. Next we mapped receptive fields and found surprisingly precise micro-retinotopy that yields a smaller point-image and requires more efficient inter-map geometry, thus underscoring the significance of map relationships. While smooth retinotopy is constraining, studies suggest that it improves both wiring economy and the V1 population code read downstream. Altogether, these data indicate that connectivity within V1 is finely tuned and precise at the level of individual neurons.

Ideal Binocular Disparity Detectors Learned Using Independent Subspace Analysis on Binocular Natural Image Pairs

An influential theory of mammalian vision, known as the efficient coding hypothesis, holds that early stages in the visual cortex attempts to form an efficient coding of ecologically valid stimuli. Although numerous authors have successfully modelled some aspects of early vision mathematically, closer inspection has found substantial discrepancies between the predictions of some of these models and observations of neurons in the visual cortex. In particular analysis of linear-non-linear models of simple-cells using Independent Component Analysis has found a strong bias towards features on the horoptor. In order to investigate the link between the information content of binocular images, mathematical models of complex cells and physiological recordings, we applied Independent Subspace Analysis to binocular image patches in order to learn a set of complex-cell-like models. We found that these complex-cell-like models exhibited a wide range of binocular disparity-discriminability, although only a minority exhibited high binocular discrimination scores. However, in common with the linear-non-linear model case we found that feature detection was limited to the horoptor suggesting that current mathematical models are limited in their ability to explain the functionality of the visual cortex.

Pooled, but not single-neuron, responses in macaque V4 represent a solution to the stereo correspondence problem

Binocular disparity is an important cue for depth perception. To correctly represent disparity, neurons must find corresponding visual features between the left- and right-eye images. The visual pathway ascending from V1 to inferior temporal cortex solves the correspondence problem. An intermediate area, V4, has been proposed to be a critical stage in the correspondence process. However, the distinction between V1 and V4 is unclear, because accumulating evidence suggests that the process begins within V1. In this article, we report that the pooled responses in macaque V4, but not responses of individual neurons, represent a solution to the correspondence problem. We recorded single-unit responses of V4 neurons to random-dot stereograms of varying degrees of anticorrelation. To achieve gradual anticorrelation, we reversed the contrast of an increasing proportion of dots as in our previous psychophysical studies, which predicted that the neural correlates of the solution to correspondence problem should gradually eliminate their disparity modulation as the level of anticorrelation increases. Inconsistent with this prediction, the tuning amplitudes of individual V4 neurons quickly decreased to a nonzero baseline with small anticorrelation. By contrast, the shapes of individual tuning curves changed more gradually so that the amplitude of population-pooled responses gradually decreased toward zero over the entire range of graded anticorrelation. We explain these results by combining multiple energy-model subunits. From a comparison with the population-pooled responses in V1, we suggest that disparity representation in V4 is distinctly advanced from that in V1. Population readout of V4 responses provides disparity information consistent with the correspondence solution.

Neuronal Representation of 3-D Space in the Primary Visual Cortex and Control of Eye Movements

The aim of this article is to consider the correlations between the structure of the primary visual cortical area V1 and control of coordinated movements of the two eyes. Using the anatomical data available, a schematic map of 3-D space representation in the layer IV of area V1 containing only monocular cells has been constructed. The analysis of this map revealed that binocular neurons of V1, which are formed by convergence of monocular cells, should encode the absolute disparity. Participation of monocular and binocular neurons of V1 in the control of convergence, divergence, and version eye movements is discussed. It is proposed that synchronous contraction of corresponding extraocular muscles of both eyes for vergence might be ensured by duplicated transmission of information from the central part of retina to visual cortex of both hemispheres.

Local Integration Accounts for Weak Selectivity of Mouse Neocortical Parvalbumin Interneurons

Dissecting the functional roles of excitatory and inhibitory neurons in cortical circuits is a fundamental goal in neuroscience. Of particular interest are their roles in emergent cortical computations such as binocular integration in primary visual cortex (V1). We measured the binocular response selectivity of genetically defined subpopulations of excitatory and inhibitory neurons. Parvalbumin (PV+) interneurons received strong inputs from both eyes but lacked selectivity for binocular disparity. Because broad selectivity could result from heterogeneous synaptic input from neighboring neurons, we examined how individual PV+ interneuron selectivity compared to that of the local neuronal network, which is primarily composed of excitatory neurons. PV+ neurons showed functional similarity to neighboring neuronal populations over spatial distances resembling measurements of synaptic connectivity. On the other hand, excitatory neurons expressing CaMKIIα displayed no such functional similarity with the neighboring population. Our findings suggest that broad selectivity of PV+ interneurons results from nonspecific integration within local networks. VIDEO ABSTRACT.

Strabismus disrupts binocular synaptic integration in primary visual cortex

Visual disruption early in development dramatically changes how primary visual cortex neurons integrate binocular inputs. The disruption is paradigmatic for investigating the synaptic basis of long-term changes in cortical function, because the primary visual cortex is the site of binocular convergence. The underlying alterations in circuitry by visual disruption remain poorly understood. Here we compare membrane potential responses, observed via whole-cell recordings in vivo, of primary visual cortex neurons in normal adult cats with those of cats in which strabismus was induced before the developmental critical period. In strabismic cats, we observed a dramatic shift in the ocular dominance distribution of simple cells, the first stage of visual cortical processing, toward responding to one eye instead of both, but not in complex cells, which receive inputs from simple cells. Both simple and complex cells no longer conveyed the binocular information needed for depth perception based on binocular cues. There was concomitant binocular suppression such that responses were weaker with binocular than with monocular stimulation. Our estimates of the excitatory and inhibitory input to single neurons indicate binocular suppression that was not evident in synaptic excitation, but arose de novo because of synaptic inhibition. Further constraints on circuit models of plasticity result from indications that the ratio of excitation to inhibition evoked by monocular stimulation decreased mainly for nonpreferred eye stimulation. Although we documented changes in synaptic input throughout primary visual cortex, a circuit model with plasticity at only thalamocortical synapses is sufficient to account for our observations.

A reaction-diffusion model to capture disparity selectivity in primary visual cortex

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.

Critical period plasticity matches binocular orientation preference in the visual cortex

Changes of ocular dominance in the visual cortex can be induced by visual manipulations during a critical period in early life. However, the role of critical period plasticity in normal development is unknown. Here we show that at the onset of this time window, the preferred orientations of individual cortical cells in the mouse are mismatched through the two eyes and the mismatch decreases and reaches adult levels by the end of the period. Deprivation of visual experience during this period irreversibly blocks the binocular matching of orientation preference, but has no effect in adulthood. The critical period of binocular matching can be delayed by long-term visual deprivation from birth, like that of ocular dominance plasticity. These results demonstrate that activity-dependent changes induced by normal visual experience during the well-studied critical period serve to match eye-specific inputs in the cortex, thus revealing a physiological role for critical period plasticity during normal development.

Disparity estimation by pooling evidence from energy neurons

In this paper, we propose an algorithm for disparity estimation from disparity energy neurons that seeks to maintain simplicity and biological plausibility, while also being based upon a formulation that enables us to interpret the model outputs probabilistically. We use the Bayes factor from statistical hypothesis testing to show that, in contradiction to the implicit assumption of many previously proposed biologically plausible models, a larger response from a disparity energy neuron does not imply more evidence for the hypothesis that the input disparity is close to the preferred disparity of the neuron. However, we find that the normalized response can be interpreted as evidence, and that information from different orientation channels can be combined by pooling the normalized responses. Based on this insight, we propose an algorithm for disparity estimation constructed out of biologically plausible operations. Our experimental results on real stereograms show that the algorithm outperforms a previously proposed coarse-to-fine model. In addition, because its outputs can be interpreted probabilistically, the model also enables us to identify occluded pixels or pixels with incorrect disparity estimates.

A micro-architecture for binocular disparity and ocular dominance in visual cortex

In invertebrate predators like the praying mantis and vertebrate predators such as wild cats, the ability to detect small differences in inter-ocular retinal disparities is a critical means for accurately determining the depth of moving objects such as prey1. In mammals, the first neurons along the visual pathway that encode binocular disparities are found in the visual cortex. However, a precise functional architecture for binocular disparity has never been demonstrated in any species, and coarse maps for disparity have been found in only one primate species2,3. Moreover, the dominant approach for assaying the developmental plasticity of binocular cortical neurons employed monocular tests of ocular dominance to infer binocular function4. The few studies that examined the relationship between ocular dominance and binocular disparity of individual cells used single-unit recordings and have provided conflicting results as to whether ocular dominance can predict the selectivity or sensitivity to binocular disparity5–9. Here we use two-photon calcium imaging to sample the response to monocular and binocular visual stimuli from nearly every adjacent neuron in a small region of the cat visual cortex, area 18. We show that local circuits for ocular dominance always have smooth and graded transitions from one apparently monocular functional domain to an adjacent binocular region. Most unexpectedly, we discovered a new map in the cat visual cortex that had a precise functional micro-architecture for binocular disparity selectivity. At the level of single cells, ocular dominance was unrelated to binocular disparity selectivity or sensitivity. When the local maps for ocular dominance and binocular disparity both had measurable gradients at a given cortical site, the two gradient directions were orthogonal to each other. Together, these results suggest that from the perspective of the spiking activity of individual neurons, ocular dominance cannot predict binocular disparity tuning. However, the precise local arrangement of ocular dominance and binocular disparity maps provide new clues on how monocular and binocular depth cues may be combined and decoded.

Mechanisms underlying the transformation of disparity signals from V1 to V2 in the macaque

Stereo vision relies on cortical signals that encode binocular disparity. In V1, the disparity energy model explains many features of binocular interaction, but it overestimates the responses to anticorrelated images. Combining the outputs of two, or more, energy model-like subunits [two-subunit (2SU) model] can resolve this discrepancy and provides an alternative explanation for disparity signals previously thought to indicate phase disparity between the receptive fields (RFs) of each eye. The 2SU model naturally explains how "near/far" (odd-symmetric) tuning becomes dominant in extrastriate cortex. To compare the energy and the 2SU models, we used a broadband compound grating and applied a common interocular phase difference to all spatial frequency components (a stimulus phase disparity), combined with a common spatial displacement (a stimulus position disparity). This produces binocular images that never occur in natural viewing, for which the 2SU model and the energy model make distinctively different predictions. Responses of neurons recorded from both V1 and V2 of awake rhesus macaques systematically deviated from the predictions of the energy model, in accordance with the 2SU model. These deviations correlated with the symmetry of the tuning curve, indicating that the 2SU mechanism is exploited to produce odd symmetry. Nonetheless, individual subunits also contain RF phase disparity that contributes to odd symmetry. The results suggest that neurons in V2 probably inherit phase disparity signals from V1 neurons, but systematically combine input from V1 neurons with different position disparities, in a way that elaborates odd-symmetric tuning and extends the range of disparities encoded by single neurons.

Adaptation to natural binocular disparities in primate V1 explained by a generalized energy model

Sensory processing in the brain is thought to have evolved to encode naturally occurring stimuli efficiently. We report an adaptation in binocular cortical neurons that reflects the tight constraints imposed by the geometry of 3D vision. We show that the widely used binocular energy model predicts that neurons dedicate part of their dynamic range to impossible combinations of left and right images. Approximately 42% of the neurons we record from V1 of awake monkeys behave in this way (a powerful confirmation of the model), while about 58% deviate from the model in a manner that concentrates more of their dynamic range on stimuli that obey the constraints of binocular geometry. We propose a simple extension of the energy model, using multiple subunits, that explains the adaptation we observe, as well as other properties of binocular neurons that have been hard to account for, such as the response to anti-correlated stereograms.

Psychophysically measured task strategy for disparity discrimination is reflected in V2 neurons

In perceptual tasks, subjects attempt to rely on their most informative cues. Such strategic choices should be reflected in the types of sensory neurons that are used. We investigated this in a binocular-disparity discrimination task. Using psychophysical reverse-correlation, also known as image classification, we identified the perceptual strategies of two macaques (Macaca mulatta). Correlations between reported disparity signs and disparity noise samples for each trial yielded detection 'filters'. Filter amplitude was greater at near disparities than at far disparities, indicating that the subjects relied more on near disparities. Recordings from both macaques' disparity-selective V2 neurons showed a correlation between neuronal responses and perceptual judgment in near-preferring, but not far-preferring, units, mirroring the psychophysically measured strategy. After one monkey learned to weight near and far disparities equally, activity in its far-preferring neurons correlated with choice. Thus, the pattern of correlations between neuronal activity and perceptual reports indicates how subjects use their neuronal signals.

Spatial frequency integration for binocular correspondence in macaque area V4

Neurons in the primary visual cortex (V1) detect binocular disparity by computing the local disparity energy of stereo images. The representation of binocular disparity in V1 contradicts the global correspondence when the image is binocularly anticorrelated. To solve the stereo correspondence problem, this rudimentary representation of stereoscopic depth needs to be further processed in the extrastriate cortex. Integrating signals over multiple spatial frequency channels is one possible mechanism supported by theoretical and psychophysical studies. We examined selectivities of single V4 neurons for both binocular disparity and spatial frequency in two awake, fixating monkeys. Disparity tuning was examined with a binocularly correlated random-dot stereogram (RDS) as well as its anticorrelated counterpart, whereas spatial frequency tuning was examined with a sine wave grating or a narrowband noise. Neurons with broader spatial frequency tuning exhibited more attenuated disparity tuning for the anticorrelated RDS. Additional rectification at the output of the energy model does not likely account for this attenuation because the degree of attenuation does not differ among the various types of disparity-tuned neurons. The results suggest that disparity energy signals are integrated across spatial frequency channels for generating a representation of stereoscopic depth in V4.

Sensors for impossible stimuli may solve the stereo correspondence problem

One of the fundamental challenges of binocular vision is that objects project to different positions on the two retinas (binocular disparity). Neurons in visual cortex show two distinct types of tuning to disparity, position and phase disparity, which are the results of differences in receptive field location and profile, respectively. Here, we point out that phase disparity does not occur in natural images. Why, then, should the brain encode it? We propose that phase-disparity detectors help to work out which feature in the left eye corresponds to a given feature in the right. This correspondence problem is plagued by false matches: regions of the image that look similar, but do not correspond to the same object. We show that phase-disparity neurons tend to be more strongly activated by false matches. Thus, they may act as 'lie detectors', enabling the true correspondence to be deduced by a process of elimination.

Macaque V2 neurons, but not V1 neurons, show choice-related activity

In the macaque extrastriate cortex, robust correlations between perceptual choice and neuronal response have been demonstrated, frequently quantified as choice probabilities (CPs). Such correlations are modest in early visual cortex, suggesting that CPs may depend on the position of a neuron in the hierarchy of visual processing. However, previous studies have not compared neurons with similar precision in equivalent tasks. We investigated the role of cortical hierarchy on CP using a task for which significant CPs have been described previously for middle temporal area (MT). We measured CPs in disparity-selective neurons from both V1 and V2. The stimulus was a dynamic random dot stereogram, presented with a near or a far disparity, masked by varying numbers of binocularly uncorrelated dots. Two macaque monkeys reported whether they perceived a circular patch in front or behind a surrounding annulus in a forced choice task. For V2 (n = 69), CP was on average 0.56, the first demonstration of systematic CPs in a visual area as early as V2. In V1 (n = 74), average CP was at chance level (0.51). The pattern was similar in a subgroup of neurons selected such that the statistical precision in the task was on average identical to that reported for MT (mean CP, 0.51 for V1, n = 33; 0.58 for V2, n = 54). This difference between V1 and V2 could not be explained by eye movements, stimulus size relative to the receptive field, or differences in disparity tuning. Rather, it seems to reflect a functional difference (at least in disparity processing) between striate and extrastriate cortex.

Monocular cells without ocular dominance columns

In many regions of the mammalian cerebral cortex, cells that share a common receptive field property are grouped into columns. Despite intensive study, the function of the cortical column remains unknown. In the squirrel monkey, the expression of ocular dominance columns is variable, with columns present in some animals and not in others. By searching for differences between animals with and without columns, it should be possible to infer how columns contribute to visual processing. Single-cell recordings outside layer 4C were made in nine squirrel monkeys, followed by labeling of ocular dominance columns in layer 4C. In the squirrel monkey, compared with the macaque, cells outside layer 4C were more likely to respond to stimulation of either eye whether ocular dominance columns were present or not. In three animals lacking ocular dominance columns, single cells were recorded from layer 4C. Remarkably, 20% of cells in layer 4C were monocular despite the absence of columns. This observation means that ocular dominance columns are not necessary for monocular cells to occur in striate cortex. In macaques each row of cytochrome oxidase (CO) patches is aligned with an ocular dominance column and receives koniocellular input serving one eye only. In squirrel monkeys this was not true: CO patches and ocular dominance columns had no spatial correlation and the koniocellular input to CO patches was binocular. Thus even when ocular dominance columns occur in the squirrel monkey, they do not transform the functional architecture to resemble that of the macaque.

Disparity-Tuning Characteristics of Neuronal Responses to Dynamic Random-Dot Stereograms in Macaque Visual Area V4

Stereo processing begins in the striate cortex and involves several extrastriate visual areas. We quantitatively analyzed the disparity-tuning characteristics of neurons in area V4 of awake, fixating monkeys. Approximately half of the analyzed V4 cells were tuned for horizontal binocular disparities embedded in dynamic random-dot stereograms (RDSs). Their response preferences were strongly biased for crossed disparities. To characterize the disparity-tuning profile, we fitted a Gabor function to the disparity-tuning data. The distribution of V4 cells showed a single dense cluster in a joint parameter space of the center and the phase parameters of the fitted Gabor function; most V4 neurons were maximally sensitive to fine stereoscopic depth increments near zero disparity. Comparing single-cell responses with background multiunit responses at the same sites showed that disparity-sensitive cells were clustered within V4 and that nearby cells possessed similar preferred disparities. Consistent with a recent report by Hegdé and Van Essen, the disparity tuning for an RDS drastically differed from that for a solid-figure stereogram (SFS). Disparity-tuning curves were generally broader for SFSs than for RDSs, and there was no correlation between the fitted Gabor functions' amplitudes, widths, or peaks for the two types of stereograms. The differences were partially attributable to shifts in the monocular images of an SFS. Our results suggest that the representation of stereoscopic depth in V4 is suited for detecting fine structural features protruding from a background. The representation is not generic and differs when the stimulus is broad-band noise or a solid figure.

The cortical column: a structure without a function

This year, the field of neuroscience celebrates the 50th anniversary of Mountcastle's discovery of the cortical column. In this review, we summarize half a century of research and come to the disappointing realization that the column may have no function. Originally, it was described as a discrete structure, spanning the layers of the somatosensory cortex, which contains cells responsive to only a single modality, such as deep joint receptors or cutaneous receptors. Subsequently, examples of columns have been uncovered in numerous cortical areas, expanding the original concept to embrace a variety of different structures and principles. A "column" now refers to cells in any vertical cluster that share the same tuning for any given receptive field attribute. In striate cortex, for example, cells with the same eye preference are grouped into ocular dominance columns. Unaccountably, ocular dominance columns are present in some species, but not others. In principle, it should be possible to determine their function by searching for species differences in visual performance that correlate with their presence or absence. Unfortunately, this approach has been to no avail; no visual faculty has emerged that appears to require ocular dominance columns. Moreover, recent evidence has shown that the expression of ocular dominance columns can be highly variable among members of the same species, or even in different portions of the visual cortex in the same individual. These observations deal a fatal blow to the idea that ocular dominance columns serve a purpose. More broadly, the term "column" also denotes the periodic termination of anatomical projections within or between cortical areas. In many instances, periodic projections have a consistent relationship with some architectural feature, such as the cytochrome oxidase patches in V1 or the stripes in V2. These tissue compartments appear to divide cells with different receptive field properties into distinct processing streams. However, it is unclear what advantage, if any, is conveyed by this form of columnar segregation. Although the column is an attractive concept, it has failed as a unifying principle for understanding cortical function. Unravelling the organization of the cerebral cortex will require a painstaking description of the circuits, projections and response properties peculiar to cells in each of its various areas.

Early computational processing in binocular vision and depth perception

Stereoscopic depth perception is a fascinating ability in its own right and also a useful model of perception. In recent years, considerable progress has been made in understanding the early cortical circuitry underlying this ability. Inputs from left and right eyes are first combined in primary visual cortex (V1), where many cells are tuned for binocular disparity. Although the observation of disparity tuning in V1, combined with psychophysical evidence that stereopsis must occur early in visual processing, led to initial suggestions that V1 was the neural correlate of stereoscopic depth perception, more recent work indicates that this must occur in higher visual areas. The firing of cells in V1 appears to depend relatively simply on the visual stimuli within local receptive fields in each retina, whereas the perception of depth reflects global properties of the stimulus. However, V1 neurons appear to be specialized in a number of respects to encode ecologically relevant binocular disparities. This suggests that they carry out essential pre-processing underlying stereoscopic depth perception in higher areas. This article reviews recent progress in developing accurate models of the computations carried out by these neurons. We seem close to achieving a mathematical description of the initial stages of the brain's stereo algorithm. This is important in itself--for instance, it may enable improved stereopsis in computer vision--and paves the way for a full understanding of how depth perception arises.

Understanding the cortical specialization for horizontal disparity

Because the eyes are displaced horizontally, binocular vision is inherently anisotropic. Recent experimental work has uncovered evidence of this anisotropy in primary visual cortex (V1): neurons respond over a wider range of horizontal than vertical disparity, regardless of their orientation tuning. This probably reflects the horizontally elongated distribution of two-dimensional disparity experienced by the visual system, but it conflicts with all existing models of disparity selectivity, in which the relative response range to vertical and horizontal disparities is determined by the preferred orientation. Potentially, this discrepancy could require us to abandon the widely held view that processing in V1 neurons is initially linear. Here, we show that these new experimental data can be reconciled with an initial linear stage; we present two physiologically plausible ways of extending existing models to achieve this. First, we allow neurons to receive input from multiple binocular subunits with different position disparities (previous models have assumed all subunits have identical position and phase disparity). Then we incorporate a form of divisive normalization, which has successfully explained many response properties of V1 neurons but has not previously been incorporated into a model of disparity selectivity. We show that either of these mechanisms decouples disparity tuning from orientation tuning and discuss how the models could be tested experimentally. This represents the first explanation of how the cortical specialization for horizontal disparity may be achieved.