Functional connectivity of disparity-tuned neurons in the visual cortex

Neural circuits for binocular vision: Ocular dominance, interocular matching, and disparity selectivity

The brain creates a single visual percept of the world with inputs from two eyes. This means that downstream structures must integrate information from the two eyes coherently. Not only does the brain meet this challenge effortlessly, it also uses small differences between the two eyes' inputs, i.e., binocular disparity, to construct depth information in a perceptual process called stereopsis. Recent studies have advanced our understanding of the neural circuits underlying stereoscopic vision and its development. Here, we review these advances in the context of three binocular properties that have been most commonly studied for visual cortical neurons: ocular dominance of response magnitude, interocular matching of orientation preference, and response selectivity for binocular disparity. By focusing mostly on mouse studies, as well as recent studies using ferrets and tree shrews, we highlight unresolved controversies and significant knowledge gaps regarding the neural circuits underlying binocular vision. We note that in most ocular dominance studies, only monocular stimulations are used, which could lead to a mischaracterization of binocularity. On the other hand, much remains unknown regarding the circuit basis of interocular matching and disparity selectivity and its development. We conclude by outlining opportunities for future studies on the neural circuits and functional development of binocular integration in the early visual system.

Operating principles of the cerebral cortex as a six-layered network in primates: beyond the classic canonical circuit model

The cerebral cortex performs its computations with many six-layered fundamental units, collectively spreading along the cortical sheet. What is the local network structure and the operating dynamics of such a fundamental unit? Previous investigations of primary sensory areas revealed a classic "canonical" circuit model, leading to an expectation of similar circuit organization and dynamics throughout the cortex. This review clarifies the different circuit dynamics at play in the higher association cortex of primates that implements computation for high-level cognition such as memory and attention. Instead of feedforward processing of response selectivity through Layers 4 to 2/3 that the classic canonical circuit stipulates, memory recall in primates occurs in Layer 5/6 with local backward projection to Layer 2/3, after which the retrieved information is sent back from Layer 6 to lower-level cortical areas for further retrieval of nested associations of target attributes. In this review, a novel "dynamic multimode module (D3M)" in the primate association cortex is proposed, as a new "canonical" circuit model performing this operation.

Suppressive mechanisms in monkey V1 help to solve the stereo correspondence problem

Neurons encode the depth in stereoscopic images by combining the signals from the receptive fields in the two eyes. Local variations in single images can activate neurons that do not signal the correct disparity (false matches), giving rise to the stereo correspondence problem. We used binocular white-noise stimuli to decompose the responses of monkey primary visual cortex V1 neurons into the elements of a linear-nonlinear model (via spike-triggered covariance analysis). In our population of disparity-selective neurons, we find both excitatory and suppressive elements in many of the neurons. Their binocular receptive fields were aligned in a specific push-pull manner for disparity. We demonstrate that this arrangement reduces the responses to false matches but preserves the responses to true matches. The responses of the cells to the noise stimuli were well explained by a linear summation of the elements, followed by a nonlinearity. This model also explained the shape of independently measured disparity-tuning curves, although it overestimated the response magnitude. This study constitutes the first direct physiological evidence for the contribution of suppressive mechanisms to disparity selectivity. This new mechanism contributes to solving the stereo correspondence problem.

Triphasic dynamics of stimulus-dependent information flow between single neurons in macaque inferior temporal cortex

The functional connectivity between cortical neurons is not static and is known to exhibit contextual modulations in terms of the coupling strength. Here we hypothesized that the information flow in a cortical local circuit exhibits complex forward-and-back dynamics, and conducted Granger causality analysis between the neuronal spike trains that were simultaneously recorded from macaque inferior temporal (IT) cortex while the animals performed a visual object discrimination task. Spikes from neuron pairs with a displaced peak on the cross-correlogram (CCG) showed Granger causality in the gamma-frequency range (30-80 Hz) with the dominance in the direction consistent with the CCG peak (forward direction). Although, in a classical view, the displaced CCG peak has been interpreted as an indicative of a pauci-synaptic serial linkage, temporal dynamics of the gamma Granger causality after stimulus onset exhibited a more complex triphasic pattern, with a transient forward component followed by a slowly developing backward component and subsequent reappearance of the forward component. These triphasic dynamics of causality were not explained by the firing rate dynamics and were not observed for cell pairs that exhibited a center peak on the CCG. Furthermore, temporal dynamics of Granger causality depended on the feature configuration within the presented object. Together, these results demonstrate that the classical view of functional connectivity could be expanded to incorporate more complex forward-and-back dynamics and also imply that multistage processing in the recognition of visual objects might be implemented by multiphasic dynamics of directional information flow between single neurons in a local circuit in the IT cortex.

Spatial frequency integration for binocular correspondence in macaque area V4

Neurons in the primary visual cortex (V1) detect binocular disparity by computing the local disparity energy of stereo images. The representation of binocular disparity in V1 contradicts the global correspondence when the image is binocularly anticorrelated. To solve the stereo correspondence problem, this rudimentary representation of stereoscopic depth needs to be further processed in the extrastriate cortex. Integrating signals over multiple spatial frequency channels is one possible mechanism supported by theoretical and psychophysical studies. We examined selectivities of single V4 neurons for both binocular disparity and spatial frequency in two awake, fixating monkeys. Disparity tuning was examined with a binocularly correlated random-dot stereogram (RDS) as well as its anticorrelated counterpart, whereas spatial frequency tuning was examined with a sine wave grating or a narrowband noise. Neurons with broader spatial frequency tuning exhibited more attenuated disparity tuning for the anticorrelated RDS. Additional rectification at the output of the energy model does not likely account for this attenuation because the degree of attenuation does not differ among the various types of disparity-tuned neurons. The results suggest that disparity energy signals are integrated across spatial frequency channels for generating a representation of stereoscopic depth in V4.

Dynamics of Receptive Field Size in Primary Visual Cortex

Recent studies have shown that the initial responses evoked by a stimulus in neurons of primary visual cortex are dominated by low spatial frequency information in the image, whereas finer spatial scales dominate later in the response. Such phenomena could arise from the dynamics of receptive field (RF) size at early stages of cortical processing. We measured changes in RF size in simple cells recorded from the primary visual cortex of anesthetized macaques by measuring their first-order spatio-temporal kernels and fitting them with two-dimensional Gabor functions at different time slices. We found that the width and length of the RF envelope and the period of the carrier tend to decrease during the time-course of the response. The most pronounced changes are seen in the width and spatial period of the RFs, which decrease by 15% during the central 20 ms of the response. These results show a novel form of spatio-temporal inseparability in simple cells and are consistent with the notion of a coarse-to-fine processing of information in early visual cortex.

Stereo dynamics are not scale-dependent

The experiments reported here focus on the temporal dynamics of stereopsis in an effort to shed light on how low level mechanisms might contribute to the execution of coarse-to-fine processing in the human stereo system. Because previous studies have used a variety of stimuli and configurations, we assess the effect of exposure duration on stereo thresholds using band-limited Gabor patches for a range of stimulus configurations. In preliminary studies, we found that the best stereo sensitivity-spatial frequency relationship was obtained when using configurations in which the size and target-reference spacing were consistent with spatially scaled stimuli. Sub-optimal stereo sensitivity as a function of spatial frequency was observed when the size and separation were fixed. Further, we found that the temporal properties of stereopsis were consistently sustained in nature irrespective of the stimulus spatial frequency content. This latter finding suggests that if coarse-to-fine stereo processing does occur it does not follow as a consequence of the dynamics of low-level disparity transduction.

Selective biasing of stereo correspondence in an ambiguous stereogram

In spite of numerous studies in stereoscopic perception, it is still not clear how the visual system matches features between the two eyes. One reason is that these previous studies used stimuli that presented little perceptual ambiguity, so the correspondence problem had only one solution. We present here a novel stimulus that presents a more complex correspondence problem. This stimulus is inspired by "wallpaper" stimuli and was specifically designed to put into conflict two possible constraints underlying stereo correspondence matching. These constraints are the nearest neighbour matching rule--that biases surfaces towards the horopter--and the nearest disparity rule--that biases surfaces to be smooth. By varying the contrast of adjacent image features in this stimulus, we were able to reveal and quantify a preference for nearest disparity matching. The magnitude of this preference is dependent upon the magnitude of possible disparities in the scene and is consistent with the idea that the visual system seeks to minimise local differences in disparity. We discuss these results with regard to the use of prior constraints in models of stereo matching.

Early computational processing in binocular vision and depth perception

Stereoscopic depth perception is a fascinating ability in its own right and also a useful model of perception. In recent years, considerable progress has been made in understanding the early cortical circuitry underlying this ability. Inputs from left and right eyes are first combined in primary visual cortex (V1), where many cells are tuned for binocular disparity. Although the observation of disparity tuning in V1, combined with psychophysical evidence that stereopsis must occur early in visual processing, led to initial suggestions that V1 was the neural correlate of stereoscopic depth perception, more recent work indicates that this must occur in higher visual areas. The firing of cells in V1 appears to depend relatively simply on the visual stimuli within local receptive fields in each retina, whereas the perception of depth reflects global properties of the stimulus. However, V1 neurons appear to be specialized in a number of respects to encode ecologically relevant binocular disparities. This suggests that they carry out essential pre-processing underlying stereoscopic depth perception in higher areas. This article reviews recent progress in developing accurate models of the computations carried out by these neurons. We seem close to achieving a mathematical description of the initial stages of the brain's stereo algorithm. This is important in itself--for instance, it may enable improved stereopsis in computer vision--and paves the way for a full understanding of how depth perception arises.

The derivation of direction selectivity in the striate cortex

In the central visual pathway of binocular animals, the property of directional selectivity (DS) is first exhibited in striate cortex. In this study, we sought to determine the neural circuitry underlying the transformation from non-DS neurons to DS cortical cells. In a well established model, DS receptive fields (RFs) are derived from the sum of two non-DS inputs with 90 degrees (quadrature) spatiotemporal phase differences. We explored possible input sources for this model, which include non-DS simple cells and lateral geniculate nucleus (LGN) neurons, by examination of spatiotemporal RFs of single cells and of pairs of cells. We find that distributions of non-DS simple RFs do not match the temporal predictions of the quadrature model because of a lack of long-latency responses. The long-latency inputs could potentially arise from lagged LGN afferents. However, analysis of cell pairs indicates that DS cells receive cortical input from non-DS simple cells for both short- and long-latency components, with temporal phase differences typically <90 degrees. Furthermore, the distribution of minimum phase differences needed to generate DS cells overlaps that exhibited by non-DS simple cells. Considered together, these results are consistent with a linear model whereby DS simple cells are formed from simple-cell inputs, with temporal phase differences often less than quadrature.