Temporal dynamics of binocular disparity processing with corticogeniculate interactions

A neural model is developed to probe how corticogeniculate feedback may contribute to the dynamics of binocular vision. Feedforward and feedback interactions among retinal, lateral geniculate, and cortical simple and complex cells are used to simulate psychophysical and neurobiological data concerning the dynamics of binocular disparity processing, including correct registration of disparity in response to dynamically changing stimuli, binocular summation of weak stimuli, and fusion of anticorrelated stimuli when they are delayed, but not when they are simultaneous. The model exploits dynamic rebounds between opponent ON and OFF cells that are due to imbalances in habituative transmitter gates. It shows how corticogeniculate feedback can carry out a top-down matching process that inhibits incorrect disparity responses and reduces persistence of previously correct responses to dynamically changing displays.

Impaired vertical phoria adaptation in patients with cerebellar dysfunction

PURPOSE

To determine whether phoria adaptation to a vertical prism disparity is altered in patients with cerebellar dysfunction.

METHODS

With a computer-aided haploscope, adaptive responses of fusion-free eye position to a 10- or 30-minute period was measured in subjects wearing a 3-prism diopter vertical prism over one eye. Thirteen patients with well-documented cerebellar diseases who did not have manifest ocular misalignment or limited versional eye movement and age-matched healthy subjects participated.

RESULTS

The mean +/- SD percentage of vertical phoria adaptation was 13% +/- 22% and 20% +/- 16% for the 10- and 30-minute adaptations, respectively. These levels were significantly smaller than the respective ones in the age-matched control group (P < 0.001, repeated measures MANOVA). Seven (54%) of 13 patients, including two with genetically confirmed pure cerebellar lesions (spinocerebellar ataxia type 6), showed markedly reduced responses to both the 10- and 30-minute adaptations. In all three patients with acute cerebellar ataxia, the adaptive response was improved at the same time as remission of cerebellum-associated neurologic deficits.

CONCLUSIONS

Phoria adaptation to vertical binocular disparity is frequently impaired in patients with cerebellar dysfunction. These results bolster the hypothesis that phoria adaptation is a cerebellar-dependent response.

Is vertical disparity used to determine azimuth?

The azimuth of a stimulus relative to the head can be determined from an extra-retinal, eye-position signal plus an estimate of the retinal eccentricity of the image. Alternatively, azimuth could be determined from retinal-image information alone. Specifically, stimulus azimuth could be estimated from two derivatives of vertical disparity: vertical size ratio (which varies with azimuth), and the horizontal gradient of vertical size ratio (a measure of distance). Here we examine the determinants of perceived azimuth in viewing conditions that, theoretically, should favor the use of vertical disparity. We find no evidence that vertical disparity is used. Perceived azimuth was determined completely by felt eye position and the retinal eccentricity of the image.

Disparity capture by flanking stimuli: a measure for the cooperative mechanism of stereopsis

In this work the range and scaling properties of the cooperative (contextual) interaction that was first proposed by Julesz [Foundations of Cyclopean Perception, University of Chicago Press, Chicago, 1971] to address the correspondence problem in stereopsis is measured. To this end the effect that flanking difference of Gaussians (DoG) patches produce on a perception of a target pair of patches is studied. The relative depth configuration of the target pair can switch from the small disparity gradient to a large disparity gradient state as a result of cooperative effects of the flanking stimuli. It is found that the interaction strength falls with distance. Its range varies for different subjects from 2 to 3 DoG patch sizes and scales proportionally to the size of the stimuli. The results suggest that a very localized cooperative interaction is in effect at a broad range of spatial scales.

[Synthesis of binocular disparity with motion parallax in depth slant perception: statistical efficiency approach]

Statistical efficiency approach is used in order to investigate sampling properties of both binocular disparity and motion parallax processes in depth slant perception and to examine the independent decisions model (Mulligan & Shaw, 1980) consisting of these processes. We carried out two experiments in which each cue was displayed solely (i.e., single condition) and both cues were displayed simultaneously (i.e., multiple condition). A two-alternative forced-choice (2AFC) task was used under the situation where a Gaussian noise was added to the stimulus value of depth slant. Statistical efficiencies were calculated in each experiment. The results showed that sampling size from binocular disparity was at least comparable or larger by a factor of 2.5-3.5 with that from motion parallax and that efficiencies of the multiple condition considerably exceeded those of the single condition. This suggests invalidity of the independent decisions model.

Integration of motion information during binocular rivalry

When two moving gratings are superimposed in normal viewing they often combine to form a pattern that moves with a single direction of motion. Here, we investigated whether the same mechanism underlies pattern motion when drifting gratings are presented independently to the two eyes. We report that, with relatively large circular grating patches (4 deg), there are periods of monocular dominance in which one eye's orientation alone is perceived, usually moving orthogonal to the contours (component motion). But, during the transitions from one monocular view to the other, a fluid mosaic is perceived, consisting of contiguous patches, each containing contours of only one of the gratings. This entire mosaic often appears to move in a single direction (pattern motion), just as when two gratings are literally superimposed. Although this implies that motion signals from the perceptually suppressed grating continue to influence the perception of motion, an alternative possibility is that it reflects a strategy that involves integrating directional information from the contiguous single-grating patches. To test between these possibilities, we performed a second experiment with very small grating stimuli that were about the same size as the contiguous single-grating patches in the mosaic (1-deg diameter). Despite the fact that the form of only one grating was perceived, we report that pattern motion was still perceived on about one third of trials. Moreover, a decrease in the occurrence of pattern motion was apparent when the contrast and spatial frequency of the gratings were made more different from each other. This phenomenon clearly demonstrates an independent binocular interaction for form and motion.

Luminance spatial scale and local stereo-sensitivity

Using filtered, broad band, fractal noise images we measured the dependence of D(min) and D(max) for stereo on luminance spatial frequency. D(min) was found to exhibit a simple dependence on the highest spatial frequency contained in the stimulus. D(max) depended on both image size and spatial frequency in a way that suggests an informational limit. Different rules govern D(min) and D(max) even for first order stereopsis, arguing against a common neural explanation based on independent access to the most pertinent spatial filter.

Double fusion does not occur in Panum's limiting case: evidence from orientation disparity

When two lines of different orientations are combined in regular stereograms, the orientation of the resulting line is different from those of the monocular lines. In this study we investigate the percept elicited by orientation disparity in Panum's limiting case. A variant of Panum's limiting case was designed to include orientation disparity. The single line in one half-image tilted leftwards. One of the double lines in the other half-image was parallel to the single line, while the other one tilted rightwards with obliquity one third that of the single line. In this stimulus configuration, if the single line in one half-image fuses with both lines in the other half-image at the same time, both of the two lines perceived after fusion should tilt leftwards. If double fusion does not happen, the two lines should tilt leftwards and rightwards respectively. The results of this study are in agreement with the latter prediction, which implies that double fusion does not occur in this variant of Panum's limiting case.

Quantitative analysis of the responses of V1 neurons to horizontal disparity in dynamic random-dot stereograms

Horizontal disparity tuning for dynamic random-dot stereograms was investigated for a large population of neurons (n = 787) in V1 of the awake macaque. Disparity sensitivity was quantified using a measure of the discriminability of the maximum and minimum points on the disparity tuning curve. This measure and others revealed a continuum of selectivity rather than separate populations of disparity- and nondisparity-sensitive neurons. Although disparity sensitivity was correlated with the degree of direction tuning, it was not correlated with other significant neuronal properties, including preferred orientation and ocular dominance. In accordance with the Gabor energy model, tuning curves for horizontal disparity were adequately described by Gabor functions when the neuron's orientation preference was near vertical. For neurons with orientation preferences near to horizontal, a Gaussian function was more frequently sufficient. The spatial frequency of the Gabor function that described the disparity tuning was weakly correlated with measurements of the spatial frequency and orientation preference of the neuron for drifting sinusoidal gratings. Energy models make several predictions about the relationship between the response rates to monocular and binocular dot patterns. Few of the predictions were fulfilled exactly, although the observations can be reconciled with the energy model by simple modifications. These same modifications also provide an account of the observed continuum in strength of disparity selectivity. A weak correlation between the disparity sensitivity of simultaneously recorded single- and multiunit data were revealed as well as a weak tendency to show similar disparity preferences. This is compatible with a degree of local clustering for disparity sensitivity in V1, although this is much weaker than that reported in area MT.

How context influences predominance during binocular rivalry

Variations in the predominance of an object engaged in binocular rivalry may arise from variations in the durations of dominance phases, suppression phases, or both. Earlier work has shown that the predominance of a binocular rival target is enhanced if that target fits well-via common color, orientation, or motion-with its surrounding objects. In the present experiments, the global context outside of the region of rivalry was changed during rivalry, to learn whether contextual information alters the ability to detect changes in a suppressed target itself. Results indicate that context will maintain the dominance of a rival target, but will not encourage a suppressed target to escape from suppression. Evidently, the fate of the suppressed stimulus is determined by neural events distinct from those responsible for global organization during dominance. To reconcile diverse findings concerning rivalry, it may be important to distinguish between processes responsible for selection of one eye's input for dominance from processes responsible for the implementation and maintenance of suppression.

Depth generalization from stereo to motion parallax in the owl

Although many sources of three-dimensional information have been isolated and demonstrated to contribute independently, to depth vision in animal studies, it is not clear whether these distinct cues are perceived to be perceptually equivalent. Such ability is observed in humans and would seem to be advantageous for animals as well in coping with the often co-varying (or ambiguous) information about the layout of physical space. We introduce the expression primary-depth-cue equivalence to refer to the ability to perceive mutually consistent information about differences in depth from either stereopsis or motion-parallax. We found that owls trained to detect relative depth as a perceptual category (objects versus holes) when specified by binocular disparity alone (stereopsis), immediately transferred this discrimination to novel stimuli where the equivalent depth categories were available only through differences in motion information produced by head movements (observer-produced motion-parallax). Motion-parallax discrimination did occur under monocular viewing conditions and reliable performance depended heavily on the amplitude of side-to-side head movements. The presence of primary-depth-cue equivalence in the visual system of the owl provides further conformation of the hypothesis that neural systems evolved to detect differences in either disparity or motion information are likely to share similar processing mechanisms.

Range and mechanism of encoding of horizontal disparity in macaque V1

The responses of single cortical neurons were measured as a function of the binocular disparity of dynamic random dot stereograms for a large sample of neurons (n = 787) from V1 of the awake macaque. From this sample, we selected 180 neurons whose tuning curves were strongly tuned for disparity, well sampled and well described by one-dimensional Gabor functions. The fitted parameters of the Gabor functions were used to resolve three outstanding issues in binocular stereopsis. First, we considered whether tuning curves can be meaningfully divided into discrete tuning types. Careful examination of the distributions of the Gabor parameters that determine tuning shape revealed no evidence for clustering. We conclude that a continuum of tuning types is present. Second, we investigated the mechanism of disparity encoding for V1 neurons. The shape of the disparity tuning function can be used to distinguish between position-encoding (in which disparity is encoded by an interocular shift in receptive field position) and phase-encoding (in which disparity is encoded by a difference in the receptive field profile in the 2 eyes). Both position and phase encoding were found to be common. This was confirmed by an independent assessment of disparity encoding based on the measurement of disparity sensitivity for sinusoidal luminance gratings of different spatial frequencies. The contributions of phase and position to disparity encoding were compared by estimating a population average of the rate of change in firing rate per degree of disparity. When this was calculated separately for the phase and position contributions, they were found to be closely similar. Third, we investigated the range of disparity tuning in V1 as a function of eccentricity in the parafoveal range. We find few cells which are selective for disparities greater than +/-1 degrees even at the largest eccentricity of approximately 5 degrees. The preferred disparity was correlated with the spatial scale of the tuning curve, and for most units lay within a +/-pi radians phase limit. Such a size-disparity correlation is potentially useful for the solution of the correspondence problem.

Model-based analysis of dynamics in vergence adaptation

We previously proposed a model to study the dynamics of disparity vergence responses. This model was based on known physiology and consisted of pulse and step neural control processes feeding a linear second-order oculomotor plant. Here, we apply a slightly modified version of that model to analyze the influence of short-term adaptation on vergence dynamics. This analysis showed that, unlike normal vergence responses, adapted responses could not be accurately simulated without a delay between the step and pulse components. Through simulations of normal vergence and adapted vergence responses, we found a strong correlation between delay of the step signal and the size of the movement overshoot. This correlation suggests a strong interaction between neural process generating the pulse and step motor control signals.

Motion rivalry impairs motion repulsion

In their classic study on motion repulsion, Marshak and Sekuler (Science 205 (1979) 1399) reported a repulsion of up to 10 degrees when two different directions of motion were presented dichoptically. However, subjects in that study did not experience binocular rivalry, presumably because of the brief presentation time. In the present study, we measured repulsion during binocular rivalry by requiring subjects to dichoptically view the stimuli until one direction of motion appeared to exclusively dominate the other (Blake, Yu, Lokey, & Norman (1998). J. Cogn. Neurosci., 10, 46-60). We found that motion repulsion was significantly reduced during exclusive dominance. Indeed, after controlling for reference repulsion--the misjudgment of a single direction of motion (Rauber & Treue (1998). Perception, 27, 393-402)--we found no significant motion repulsion during exclusive dominance. These data suggest that motion repulsion may require the perception, rather than merely the physical presence, of multiple directions.

Macaque inferior temporal neurons are selective for three-dimensional boundaries and surfaces

The lower bank of the superior temporal sulcus (TEs), part of the inferior temporal cortex, contains neurons selective for disparity-defined three-dimensional (3-D) shape. The large majority of these TEs neurons respond to the spatial variation of disparity, i.e., are higher-order disparity selective. To determine whether curved boundaries or curved surfaces by themselves are sufficient to elicit 3-D shape selectivity, we recorded the responses of single higher-order disparity-selective TEs neurons to concave and convex 3-D shapes in which the disparity varied either along the boundary of the shape, or only along its surface. For a majority of neurons, a 3-D boundary was sufficient for 3-D shape selectivity. At least as many neurons responded selectively to 3-D surfaces, and a number of neurons exhibited both surface and boundary selectivity. The second aim of this study was to determine whether TEs neurons can represent differences in second-order disparities along the horizontal axis. The results revealed that TEs neurons can also be selective for horizontal 3-D shapes and can code the direction of curvature (vertical or horizontal). Thus, TEs neurons represent both boundaries and surfaces curved in depth and can signal the direction of curvature along a surface. These results show that TEs neurons use not only boundary but also surface information to encode 3-D shape properties.

Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP

We investigated the effects of linear perspective and binocular disparity, as monocular and binocular depth cues, respectively, on the response of surface-orientation-selective (SOS) neurons in the caudal part of the lateral bank of the intraparietal sulcus (area CIP). During the single-unit recording, monkeys were required to perform the delayed-matching-to-sample (successive same/different discrimination) of discriminating surface orientation in stereoscopic computer graphics. Of 211 visually responsive neurons, 66 were intensively tested using the solid-figure stereogram (SFS) of a square plate with both disparity and perspective cues (D+P condition), and 62 of these were identified as SOS neurons for responding selectively to the orientation of stimuli. All these neurons were further tested using a solid figure with perspective cues alone (P-only condition), and 58% (36/62) of these showed selective response to the orientation of the stimuli. Of the 62 SOS neurons, 35 neurons were also tested using SFS with disparity cues alone (D-only condition) in addition to the D+P and P-only conditions. We classified these 35 neurons into four groups by comparing the response selectivity under the P-only and D-only conditions. More than one-half of these (19/35) were sensitive to both perspective and disparity cues (DP neurons), and nearly one-third (11/35) of these were sensitive to disparity cues alone (D neurons), but a few (2/35) were sensitive to perspective cues alone (P neurons). The remaining (3/35) neurons exhibited orientation selectivity only when both cues were present. In DP neurons, the preferred orientation under the D+P condition was correlated to those under the D-only and P-only conditions, and the response magnitude under the D+P condition was greater than those under the D-only and P-only conditions, suggesting the integration of both cues for the perception of surface orientation. However, in these neurons, the orientation tuning sharpness under the D+P and D-only conditions was higher than that under the P-only condition, suggesting the dominance of disparity cues. After the single-unit recording experiments, muscimol was microinjected into the recording site to temporarily inactivate its function. In all three effective cases out of six microinjection experiments, discrimination of a three-dimensional (3D) surface orientation was impaired when disparity cues alone were present. In only one effective case, when a relatively large amount of muscimol was microinjected, discrimination of a 3D surface orientation was impaired even when both disparity and perspective cues were present. These results suggest that linear perspective is an important cue for representations of a 3D surface of SOS neurons in area CIP, although it is less effective than binocular disparity, and that both of these depth cues may be integrated in area CIP for the perception of surface orientation in depth.

Adaptation of torsional eye alignment in relation to smooth pursuit and saccades

The long-term fusion of vertical or horizontal disparities by vergence eye movements is known to evoke persistent changes in vertical and horizontal eye alignment. Adaptive changes in response to torsional disparities have not been well studied. Torsional eye position was measured binocularly with a video system before and after 90 min training periods in which subjects attempted to fuse cyclodisparities. Subjects trained with either a single cyclodisparity presented at a single vertical eye position or with cyclodisparities that varied smoothly from an incyclodisparity to an excyclodisparity as a function of either vertical or horizontal eye position. All five subjects showed persistent changes in binocular torsional eye alignment following both types of training. Incyclodisparities were more easily fused during training and the training aftereffect was greater in that direction. The training aftereffect was observed in relation to both saccades and smooth pursuit under both open-loop and closed-loop viewing conditions. During saccades, the dynamics of the cyclovergence training aftereffect more closely resembled the dynamics of cyclofusional movements than the dynamics of the saccades with which they were associated.

Visual summation of luminance lines and illusory contours induced by pictorial, motion, and disparity cues

Illusory contours where no contrast exists in the image can be seen between pairs of spatially separate but aligned inducing real contours defined either by pictorial cues (luminance contrasts or offset gratings), kinetic contrast, or binocular disparity contrast. In previous studies it has been shown that the detection of a thin luminous line is facilitated when the line is superimposed on illusory contours and the inducing flanking elements are defined by luminance contrast. By using a spatial forced-choice technique I show that luminous lines summate with illusory contours induced by luminance contrast, offset gratings, motion contrast, and disparity contrast when the line is superimposed on the illusory contour. Control experiments show that the positional cues, offered by the inducing contours, are unable to account for these results. It is suggested that real luminous lines or edges and illusory contours activate common neural mechanisms in the brain irrespectively of the stimulus attributes that induce the illusory contour.

Effects of horizontal and vertical additive disparity noise on stereoscopic corrugation detection

Stereoscopic corrugation detection in the presence of horizontal- and vertical- additive disparity noise was examined using a signal detection paradigm. Random-dot stereograms either represented a 3-D square-wave surface with various amounts of Gaussian-distributed additive disparity noise or had the same disparity values randomly redistributed. Stereoscopic detection of 2 arcmin peak amplitude corrugations was found to tolerate significantly greater amplitudes of vertical-disparity noise than horizontal-disparity noise--irrespective of whether the corrugations were horizontally or vertically oriented. However, this directional difference in tolerance to disparity noise was found to reverse when the corrugation and noise amplitudes were increased (so as to produce equivalent signal-to-noise ratios). These results suggest that horizontal- and vertical-disparity noise pose different problems for dot-matching and post-matching surface reconstruction as corrugation and noise amplitudes increase.

Modeling V1 neuronal responses to orientation disparity

The contribution of interocular orientation differences to depth perception, at either the neuronal or the psychophysical level, is unclear. To understand the responses of binocular neurons to orientation disparity, we extended the energy model of Ohzawa et al. (1990) to incorporate binocular differences in receptive-field orientation. The responses of the model to grating stimuli with interocular orientation differences were examined, along with the responses to random dot stereograms (RDS) depicting slanted surfaces. The responses to combinations of stimulus orientations in the two eyes were left-right separable, which means there was no consistent response to the binocular orientation difference. All existing neuronal data concerning orientation disparity can be well described by this type of model (even a version with no disparity selectivity). The disparity sensitive model is nonetheless sensitive to changes in RDS slant, although it requires narrow orientation bandwidth to produce substantial modulation. The disparity-insensitive model shows no selectivity to slant in this stimulus. Several modifications to the model were attempted to improve its selectivity for orientation disparity and/or slant. A model built by summing several disparity-sensitive models showed left-right inseparable responses, responding maximally to a consistent orientation difference. Despite this property, the selectivity for slant in RDS stimuli was no better than the simple disparity-selective model. The range of models evaluated here demonstrate that interocular orientation differences are neither necessary nor sufficient for signaling slant. In contrast, within the framework of the energy model, positional disparity sensitivity appears to be both necessary and sufficient.

Stereoacuity thresholds in the presence of a reference surface

With isolated binocular targets, the best depth discrimination is found in the fixation plane (Blakemore, C., Journal of Physiology 211 (1970) 599). More recent studies have suggested that stereoscopic thresholds are not always a simple function of absolute disparity, but depend on the relative disparities in the stimulus. Here, we explored the effects of relative disparity in more detail, taking particular care to control for the possibility that subjects might change their binocular eye position or exploit monocular information provided by additional reference cues. Subjects judged the depth of a vertical target line presented above a comparison line in a blank window within a fronto-parallel reference surface composed of randomly positioned dots. On individual trials, the reference surface was presented at one of three disparities (-10, 0 and +10 arc min). To control for changes in binocular eye position, exposure duration was 150 ms, and experimental conditions with different disparities of the reference surface and comparison line were randomly interleaved. To control for monocular cues, changes in threshold were determined with respect to a disparity noise condition that was in all ways identical to the reference plane condition, except that the disparities of the dots were randomly assigned between 10 and +10 arc min. Stereo-thresholds were lowered by a factor of about 2 when the surface was at the same depth as the comparison line. Thresholds were also lowered when the comparison disparity was close to the same depth as the reference surface, but were often raised when the comparison disparity had the opposite disparity sign. These results provide unequivocal evidence that the fundamental sensitivity of the disparity detecting system can be influenced by relative disparity cues that are not related to the task.

Controlling binocular rivalry

Binocular rivalry is the alternating perception that occurs when the two eyes are presented with incompatible stimuli. We have developed a new method for controlling binocular rivalry and measuring its progress. One eye views a static grating while the fellow eye views a grating that smoothly and cyclically varies between two orientations, one the same as the static grating and the other orthogonal. Contrast sensitivity was tested monocularly a number of times during the stimulus cycle. When the eye viewing the static grating was tested, sensitivity varied between maximum and minimum values as the conditioning stimulus varied from binocularly compatible to incompatible. The interocular suppression thus demonstrated was limited to the eye viewing the static grating; variations in the fellow eye's sensitivity were due to interocular masking alone.

Spatial interactions minimize relative disparity between adjacent surfaces

Computational models of stereopsis employ a number of algorithms that constrain stereo matches to produce the smallest absolute disparity and to minimize the relative disparity between nearby features. In some natural scenes, such as large slanted textured surfaces, these two constraints lead to different matching solutions. The current study utilized a stimulus in which there was a large discrepancy in both the magnitude and direction of matches that solved for minimum absolute and minimum relative disparity. This discrepancy revealed a dominance for the minimum relative disparity over the minimum absolute disparity matching solution that increased with spatial proximity, spatial frequency and width of adjacent features. The likelihood of a minimum-relative-disparity matching solution also increased when the difference between the amplitudes of the alternative relative disparities was large. When alternative relative disparity matching solutions had similar amplitudes but opposite signs (crossed vs. uncrossed), an idiosyncratic depth bias served as a tie-breaker. The present results show that absolute disparity matches are constrained to minimize relative disparity between adjacent features.