Motion direction, speed and orientation in binocular matching

On the Aperture Problem of Binocular 3D Motion Perception

Like many predators, humans have forward-facing eyes that are set a short distance apart so that an extensive region of the visual field is seen from two different points of view. The human visual system can establish a three-dimensional (3D) percept from the projection of images into the left and right eye. How the visual system integrates local motion and binocular depth in order to accomplish 3D motion perception is still under investigation. Here, we propose a geometric-statistical model that combines noisy velocity constraints with a spherical motion prior to solve the aperture problem in 3D. In two psychophysical experiments, it is shown that instantiations of this model can explain how human observers disambiguate 3D line motion direction behind a circular aperture. We discuss the implications of our results for the processing of motion and dynamic depth in the visual system.

50 Years of Stereoblindness: Reconciliation of a Continuum of Disparity Detectors With Blindness for Disparity in Near or Far Depth

Whitman Richards (1932–2016) discovered some 50 years ago that about 30% of observers from the normal population exhibit stereoblindness: the disability to process binocular disparities in either far or near depth. We review the literature on stereoblindness entailing two insights. First, contemporary scholars in stereopsis undervalue the comprehension that disparity processing studies require precise assessments of observers’ stereoblindness. We argue that this frequently leads to suboptimal interpretations. Second, there is still an open conundrum: How can the established finding that disparity is processed by a continuum of detectors be reconciled with the disability of many observers to process a whole class of far or near disparities? We propose, based upon integration of literature, that an asymmetry between far and near disparity detection at birth—being present for a variety of reasons—can suppress the typical formation of binocular correlation during the critical period for the development of stereopsis early in life, thereby disabling a whole class of far or near disparities.

3D Displays

Creating realistic three-dimensional (3D) experiences has been a very active area of research and development, and this article describes progress and what remains to be solved. A very active area of technical development has been to build displays that create the correct relationship between viewing parameters and triangulation depth cues: stereo, motion, and focus. Several disciplines are involved in the design, construction, evaluation, and use of 3D displays, but an understanding of human vision is crucial to this enterprise because in the end, the goal is to provide the desired perceptual experience for the viewer. In this article, we review research and development concerning displays that create 3D experiences. And we highlight areas in which further research and development is needed.

Discriminability limits in spatio-temporal stereo block matching

Disparity estimation is a fundamental task in stereo imaging and is a well-studied problem. Recently, methods have been adapted to the video domain where motion is used as a matching criterion to help disambiguate spatially similar candidates. In this paper, we analyze the validity of the underlying assumptions of spatio-temporal disparity estimation, and determine the extent to which motion aids the matching process. By analyzing the error signal for spatio-temporal block matching under the sum of squared differences criterion and treating motion as a stochastic process, we determine the probability of a false match as a function of image features, motion distribution, image noise, and number of frames in the spatio-temporal patch. This performance quantification provides insight into when spatio-temporal matching is most beneficial in terms of the scene and motion, and can be used as a guide to select parameters for stereo matching algorithms. We validate our results through simulation and experiments on stereo video.

Mechanisms for similarity matching in disparity measurement

Early neural mechanisms for the measurement of binocular disparity appear to operate in a manner consistent with cross-correlation-like processes. Consequently, cross-correlation, or cross-correlation-like procedures have been used in a range of models of disparity measurement. Using such procedures as the basis for disparity measurement creates a preference for correspondence solutions that maximize the similarity between local left and right eye image regions. Here, we examine how observers’ perception of depth in an ambiguous stereogram is affected by manipulations of luminance and orientation-based image similarity. Results show a strong effect of coarse-scale luminance similarity manipulations, but a relatively weak effect of finer-scale manipulations of orientation similarity. This is in contrast to the measurements of depth obtained from a standard cross-correlation model. This model shows strong effects of orientation similarity manipulations and weaker effects of luminance similarity. In order to account for these discrepancies, the standard cross-correlation approach may be modified to include an initial spatial frequency filtering stage. The performance of this adjusted model most closely matches human psychophysical data when spatial frequency filtering favors coarser scales. This is consistent with the operation of disparity measurement processes where spatial frequency and disparity tuning are correlated, or where disparity measurement operates in a coarse-to-fine manner.

Screening and sampling in studies of binocular vision

Binocular deficits are relatively common within a typical sample of observers. This has implications for research on binocular vision, as a variety of stereo deficits can affect performance. Despite this, there is no agreed standard for testing stereo capabilities in observers and many studies do not report visual abilities at all. Within the stereo literature, failure to report screening and sampling has the potential to undermine the results of otherwise strictly controlled research. We reviewed research articles on binocular vision published in three journals between 2000 and 2008 to illustrate how screening for binocular deficits and sampling of participants is approached. Our results reveal that 44% of the studies do not mention screening for stereo deficits and 91% do not report selection of participants. The percentage of participants excluded from studies that report stereo screening amounts to 3.9% and 0.7% for studies that do not report stereo screening. These low numbers contrast with the exclusion of 17.6% of participants in studies that report screening for binocular deficits as well as selection of participants. We discuss various options for stereo testing and the need for stereo-motion testing with reference to recent research on binocular perception.

Integrating multiple cues to depth order at object boundaries

We examined the interaction between motion and stereo cues to depth order along object boundaries. Relative depth was conveyed by a change in the speed of image motion across a boundary (motion parallax), the disappearance of features on a surface moving behind an occluding object (motion occlusion), or a difference in the stereo disparity of adjacent surfaces. We compared the perceived depth orders for different combinations of cues, incorporating conditions with conflicting depth orders and conditions with varying reliability of the individual cues. We observed large differences in performance between subjects, ranging from those whose depth order judgments were driven largely by the stereo disparity cues to those whose judgments were dominated by motion occlusion. The relative strength of these cues influenced individual subjects' behavior in conditions of cue conflict and reduced reliability.

Decoding conjunctions of direction-of-motion and binocular disparity from human visual cortex

Motion and binocular disparity are two features in our environment that share a common correspondence problem. Decades of psychophysical research dedicated to understanding stereopsis suggest that these features interact early in human visual processing to disambiguate depth. Single-unit recordings in the monkey also provide evidence for the joint encoding of motion and disparity across much of the dorsal visual stream. Here, we used functional MRI and multivariate pattern analysis to examine where in the human brain conjunctions of motion and disparity are encoded. Subjects sequentially viewed two stimuli that could be distinguished only by their conjunctions of motion and disparity. Specifically, each stimulus contained the same feature information (leftward and rightward motion and crossed and uncrossed disparity) but differed exclusively in the way these features were paired. Our results revealed that a linear classifier could accurately decode which stimulus a subject was viewing based on voxel activation patterns throughout the dorsal visual areas and as early as V2. This decoding success was conditional on some voxels being individually sensitive to the unique conjunctions comprising each stimulus, thus a classifier could not rely on independent information about motion and binocular disparity to distinguish these conjunctions. This study expands on evidence that disparity and motion interact at many levels of human visual processing, particularly within the dorsal stream. It also lends support to the idea that stereopsis is subserved by early mechanisms also tuned to direction of motion.

Depth-cue integration in grasp programming: no evidence for a binocular specialism

When we grasp with one eye covered, the finger and thumb are typically opened wider than for binocularly guided grasps, as if to build a margin-for-error into the movement. Also, patients with visual form agnosia can have profound deficits in their (otherwise relatively normal) grasping when binocular information is removed. One interpretation of these findings is that there is a functional specialism for binocular vision in the control of grasping. Alternatively, cue-integration theory suggests that binocular and monocular depth cues are combined in the control of grasping, and so impaired performance reflects not the loss of 'critical' binocular cues, but increased uncertainty per se. Unfortunately, removing binocular information confounds removing particular (binocular) depth cues with an overall reduction in the available information, and so such experiments cannot distinguish between these alternatives. We measured the effects on visually open-loop grasping of selectively removing monocular (texture) or binocular depth cues. To allow meaningful comparisons, we made psychophysical measurements of the uncertainty in size estimates in each case, so that the informativeness of binocular and monocular cues was known in each condition. Consistent with cue-integration theory, removing either binocular or monocular cues resulted in similar increases in grip apertures. In a separate experiment, we also confirmed that changes in uncertainty per se (keeping the same depth cues available) resulted in larger grip apertures. Overall, changes in the margin-for-error in grasping movements were determined by the uncertainty in size estimates and not by the presence or absence of particular depth cues. Our data therefore argue against a binocular specialism for grasp programming. Instead, grip apertures were smaller when binocular and monocular cues were available than with either cue alone, providing strong evidence that the visuo-motor system exploits the redundancy available in multiple sources of information, and integrates binocular and monocular cues to improve grasping performance.

On the inverse problem of binocular 3D motion perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy.

Humans and many other predators have two eyes that are set a short distance apart so that an extensive region of the world is seen simultaneously by both eyes from slightly different points of view. Although the images of the world are essentially two-dimensional, we vividly see the world as three-dimensional. This is true for static as well as dynamic images. Here we elaborate on how the visual system may establish 3D motion perception from local input in the left and right eye. Using tools from analytic geometry we show that existing 3D motion models offer no general solution to the inverse optics problem of 3D motion perception. We suggest a flexible framework of motion and depth processing and suggest default strategies for local 3D motion estimation. Our results on the aperture and inverse problem of 3D motion are likely to stimulate computational, behavioral, and neuroscientific studies because they address the fundamental issue of how 3D motion is represented in the visual system.

Autonomous Development of Vergence Control Driven by Disparity Energy Neuron Populations

We present a simple optimization criterion that leads to autonomous development of a sensorimotor feedback loop driven by the neural representation of the depth in the mammalian visual cortex. Our test bed is an active stereo vision system where the vergence angle between the two eyes is controlled by the output of a population of disparity-selective neurons. By finding a policy that maximizes the total response across the neuron population, the system eventually tracks a target as it moves in depth. We characterized the tracking performance of the resulting policy using objects moving both sinusoidally and randomly in depth. Surprisingly, the system can even learn how to track based on stimuli it cannot track: even though the closed loop 3 dB tracking bandwidth of the system is 0.3 Hz, correct tracking policies are learned for input stimuli moving as fast as 0.75 Hz.

Cooperative and competitive interactions facilitate stereo computations in macaque primary visual cortex

Inferring depth from binocular disparities is a difficult problem for the visual system because local features in the left- and right-eye images must be matched correctly to solve this "stereo correspondence problem." Cortical architecture and computational studies suggest that lateral interactions among neurons could help resolve local uncertainty about disparity encoded in individual neurons by incorporating contextual constraints. We found that correlated activity among pairs of neurons in primary visual cortex depended both on disparity-tuning relationships and the stimuli displayed within the receptive fields of the neurons. Nearby pairs of neurons with distinct disparity tuning exhibited a decrease in spike correlation at competing disparities soon after response onset. Distant neuronal pairs of similar disparity tuning exhibited an increase in spike correlation at mutually preferred disparities. The observed correlated activity and response dynamics suggests that local competitive and distant cooperative interactions improve disparity tuning of individual neurons over time. Such interactions could represent a neural substrate for the principal constraints underlying cooperative stereo algorithms.

Perceptual incongruence influences bistability and cortical activation

We employed a parametric psychophysical design in combination with functional imaging to examine the influence of metric changes in perceptual incongruence on perceptual alternation rates and cortical responses. Subjects viewed a bistable stimulus defined by incongruent depth cues; bistability resulted from incongruence between binocular disparity and monocular perspective cues that specify different slants (slant rivalry). Psychophysical results revealed that perceptual alternation rates were positively correlated with the degree of perceived incongruence. Functional imaging revealed systematic increases in activity that paralleled the psychophysical results within anterior intraparietal sulcus, prior to the onset of perceptual alternations. We suggest that this cortical activity predicts the frequency of subsequent alternations, implying a putative causal role for these areas in initiating bistable perception. In contrast, areas implicated in form and depth processing (LOC and V3A) were sensitive to the degree of slant, but failed to show increases in activity when these cues were in conflict.

The effects of interocular correlation and contrast on stereoscopic depth magnitude estimation

PURPOSE

Decreasing the interocular correlation in random dot stereograms elevates disparity detection thresholds. Whether decorrelation also affects perceived depth from suprathreshold disparity magnitudes is unknown. The present study investigated the effects of interocular correlation and contrast on the magnitude of perceived depth in suprathreshold random dot stereograms.

METHODS

Stereoscopic depth magnitude estimation as a function of percent interocular correlation of dynamic random dot stimuli was measured for five human subjects with clinically normal binocular vision. Each trial's stimulus was randomly assigned one of two magnitudes of either crossed or uncrossed relative disparity. Subjects verbally reported the direction and magnitude of perceived relative depth for each trial using a modulus-free scale. Normalized depth magnitude estimations as a function of the percent interocular correlation demonstrated the relationship between perceived depth, interocular correlation and contrast within subjects. Inter-subject variability was examined with comparisons of data across subjects.

RESULTS

The depth magnitude perceived for a given magnitude of disparity declined as the percent of correlation of elements between the eyes decreased for both crossed and uncrossed directions. The effect generally was greater for uncrossed disparities and lower contrast. Some subjects demonstrated asymmetries in perceived depth for crossed vs. uncrossed disparities of the same magnitude.

CONCLUSIONS

Magnitude estimation of suprathreshold stimuli provided a method of studying performance characteristics of stereoscopic depth perception across the range of functional disparities. Differences found in depth magnitude estimation as a function of the sign of disparity suggest that the neural mechanisms underlying depth perception from uncrossed disparity are more sensitive to image decorrelation, particularly at low contrast, than the mechanisms underlying depth estimation from crossed disparity. These results could occur from differences in near and far disparity-sensitive neurons, from the geometrical relationship between disparity and physical distance in normal viewing, or from the response measure independent of perception.

Stimulus motion propels traveling waves in binocular rivalry

State transitions in the nervous system often take shape as traveling waves, whereby one neural state is replaced by another across space in a wave-like manner. In visual perception, transitions between the two mutually exclusive percepts that alternate when the two eyes view conflicting stimuli (binocular rivalry) may also take shape as traveling waves. The properties of these waves point to a neural substrate of binocular rivalry alternations that have the hallmark signs of lower cortical areas. In a series of experiments, we show a potent interaction between traveling waves in binocular rivalry and stimulus motion. The course of the traveling wave is biased in the motion direction of the suppressed stimulus that gains dominance by means of the wave-like transition. Thus, stimulus motion may propel the traveling wave across the stimulus to the extent that the stimulus motion dictates the traveling wave's direction completely. Using a computational model, we show that a speed-dependent asymmetry in lateral inhibitory connections between retinotopically organized and motion-sensitive neurons can explain our results. We argue that such a change in suppressive connections may play a vital role in the resolution of dynamic occlusion situations.

Stereoscopic depth discrimination with contrast windowed stimuli

Depth discrimination with a shifted contrast window was compared to that with a fixed contrast window. Stereoscopic performance with the fixed window was limited to small disparities and varied with spatial frequency. Performance with the shifted window extended to larger disparities and was more similar for low and high spatial frequencies. The results depended upon window shape, indicating that edge blur is an important factor. Stereoscopic performance with shifted patterns was supported at disparities larger than a phase disparity model might predict, suggesting that a combination of position and phase disparity computations are used for the perception of stereoscopic depth.

Temporal dynamics of stereo correspondence bi-stability

Periodic stereoscopic stimuli offer multiple viable solutions to the stereo correspondence problem. When viewing such stimuli for prolonged periods of time, observers continually switch their perceptual state between alternative correspondence solutions. We examine the temporal dynamics of this correspondence bi-stability. Participants were presented with an ambiguous stereogram comprised of regularly spaced dots. This stimulus was perceived as a fronto-parallel plane situated either behind or in front of fixation, depending on the achieved correspondence solution. The stimulus was presented continuously for one minute, with participants instructed to report the sign of the perceived depth at the sound of an auditory prompt presented, on average, every 2 s. Inter-ocular contrast and available disparities were varied so as to manipulate preferred correspondence. We find that participants are initially biased to perceive the stimulus as having an uncrossed disparity. Furthermore, we find that following an initial period of change, perceptual preference and perceptual stability (measured as the probability of an observer's percept changing between consecutive responses) remain constant over the presentation period. Finally, we find that manipulations of matching preference affect both the transient preference for, and stability of, one percept over another. Our results suggest two distinct phases of biasing in the correspondence matching process, one early, the other sustained.

Pushing the limits of transparent-motion detection with binocular disparity

When transparent motion is defined purely by direction differences, observers fail to detect more than two signal directions simultaneously [Edwards, M., & Greenwood, J.A. (2005). The perception of motion transparency: A signal-to-noise limit. Vision Research, 45, 1877-1884]. This limit is strongly related to signal-detection thresholds for transparent motion, which are several times higher than uni-directional thresholds. When the effective signal intensities are elevated by speed differences that drive independent global-motion systems, the transparent-motion limit can be extended to allow detection of three signals [Greenwood, J.A., & Edwards, M. (2006). An extension of transparent-motion detection limit using speed-tuned global-motion systems. Vision Research, 46, 1440-1449]. Because there are independent disparity-tuned global-motion systems, distributing transparent-motion signals across distinct depth planes also allows an increase in their effective signal intensity. In the present study, the addition of depth differences enabled the simultaneous detection of three signals. However, as with the addition of speed differences, observers were not able to detect four signals, which would be predicted if signal intensity were the sole constraint on transparent-motion detection. The combination of depth and speed produced similar results, suggesting that there is a strict higher-order limit, possibly related to attention, restricting the maximum number of signals that can be detected simultaneously to three.

Slant perception, and its voluntary control, do not govern the slant aftereffect: multiple slant signals adapt independently

Although it is known that high-level spatial attention affects adaptation for a variety of stimulus features (including binocular disparity), the influence of voluntary attentional control-and the associated awareness-on adaptation has remained unexplored. We developed an ambiguous surface slant adaptation stimulus with conflicting monocular and binocular slant signals that instigated two mutually exclusive surface percepts with opposite slants. Using intermittent stimulus removal, subjects were able to voluntarily select one of the two rivaling slant percepts for extended adaptation periods, enabling us to dissociate slant adaptation due to awareness from stimulus-induced slant adaptation. We found that slant aftereffects (SAE) for monocular and binocular test patterns had opposite signs when measured simultaneously. There was no significant influence of voluntarily controlled perceptual state during adaptation on SAEs of monocular or binocular signals. In addition, the magnitude of the binocular SAE did not correlate with the magnitude of perceived slant. Using adaptation to one slant cue, and testing with the other cue, we demonstrated that multiple slant signals adapt independently. We conclude that slant adaptation occurs before the level of slant awareness. Our findings place the site of stereoscopic slant adaptation after disparity and eye posture are interpreted for slant [as demonstrated by Berends et al. (Berends, E. M., Liu, B., & Schor, C. M. (2005). Stereo-slant adaptation is high level and does not involve disparity coding. Journal of Vision 5 (1), 71-80), using that disparity scales with distance], but before other slant signals are integrated for the resulting awareness of the presented slant stimulus.

Pooling signals from vertically and non-vertically orientation-tuned disparity mechanisms in human stereopsis

To understand the role that orientation-tuned disparity-sensitive mechanisms play in the perception of stereoscopic depth, we measured stereothresholds using two sets of random-dot stimuli that produce identical stimulation of disparity mechanisms tuned to vertical orientation but dissimilar stimulation of disparity mechanisms tuned to non-vertical orientations. Either 1 or 1.5D of astigmatic blur was simulated in the random-dot images presented to both eyes, using two axis configurations. In the parallel-axis conditions, the axis of simulated astigmatic blur was same in the two eyes (0, 45 or 135 o[rientation]deg). In the orthogonal-axis conditions, the axes of astigmatic blur were orthogonal in the two eyes (LE: 180, RE: 90; LE: 90, RE: 180; LE: 45, RE: 135; and LE: 135, RE: 45). Whereas the stimulation of disparity mechanisms tuned to near-vertical orientations should be similar in the oblique parallel- and orthogonal-axis conditions, the stimulation of non-vertically tuned disparity mechanisms should be dissimilar. Measured stereothresholds were higher in the orthogonal compared to the parallel-axis condition by factors of approximately 2 and 5, for 1 and 1.5D of simulated oblique astigmatic blur, respectively. Further, for comparable magnitudes of simulated astigmatic blur, stereothresholds in the (LE: 180, RE: 90 and LE: 90, RE: 180) conditions were similar to those in the (LE: 45, RE: 135 and LE: 135, RE: 45) conditions. These results suggest that the computation of horizontal disparity includes substantial contributions from disparity mechanisms tuned to non-vertical orientations. Simulations using a modified version of a disparity-energy model [Qian, N., & Zhu, Y. (1997). Physiological computation of binocular disparity. Vision Research, 37, 1811-1827], show (1) that pooling across disparity mechanisms tuned to vertical and non-vertical orientations is required to account for our data and (2) that this pooling can provide the spatial resolution needed to encode spatially changing horizontal disparities.

Colour helps to solve the binocular matching problem

The spatial differences between the two retinal images, called binocular disparities, can be used to recover the three-dimensional (3D) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this binocular matching problem is an important issue in research on stereoscopic vision. The role of colour in binocular vision is controversial and it has been argued that colour is ineffective in achieving binocular vision. In the current experiment subjects were required to indicate the amount of perceived depth. The stimulus consisted of an array of fronto-parallel bars uniformly distributed in a constant sized volume. We studied the perceived depth in those 3D stimuli by manipulating both colour (monochrome, trichrome) and luminance (congruent, incongruent). Our results demonstrate that the amount of perceived depth was influenced by colour, indicating that the visual system uses colour to achieve binocular matching. Physiological data have revealed cortical cells in macaque V2 that are tuned both to binocular disparity and to colour. We suggest that one of the functional roles of these cells may be to help solve the binocular matching problem.

Dynamics of perceptual bi-stability for stereoscopic slant rivalry and a comparison with grating, house-face, and Necker cube rivalry

A way to study conscious perception is to expose the visual system to an ambiguous stimulus that instigates bi-stable perception. This provides the opportunity to study neural underpinnings related to the percepts rather than to the stimulus. We have recently developed a slant-rivalry paradigm that has beneficial metrical (quantitative) aspects and that exhibits temporal aspects of perceptual reversals that seemed to be under considerable voluntary control of the observer. Here we examined a range of different aspects of the temporal dynamics of the perceptual reversals of slant rivalry and we compared these with the dynamics of orthogonal grating rivalry, house-face rivalry, and Necker cube rivalry. We found that slant rivalry exhibits a qualitatively similar pattern of dynamics. The drift of the perceptual reversal rate, both across successive experimental repetitions, and across successive 35-s portions of data were similar. The sequential dependence of the durations of perceptual phases, too, revealed very similar patterns. The main quantitative difference, which could make slant rivalry a useful stimulus for future neurophysiological studies, is that the percept durations are relatively long compared to the other rivalry stimuli. In the paper that accompanies this paper [van Ee, R., van Dam, L. C. J., Brouwer, G. J. (2005). Voluntary control and the dynamics of perceptual bi-stability. Vision Research,] we focused on the role of voluntary control in the dynamics of perceptual reversals.

The Effect of Monocularly and Binocularly Induced Astigmatic Blur on Depth Discrimination Is Orientation Dependent

PURPOSE

Naturally occurring astigmatism varies according to the age of the person. Although uncorrected astigmatism may be associated with meridional amblyopia, there is little information of its effect on stereopsis. The purpose of this study was to determine the effect of astigmatism on depth discrimination and whether this was dependent on the axis of the astigmatism.

METHODS

Astigmatic blur was induced in four healthy subjects (mean age, 31.5 years; range, 22 to 42 years) using plain cylinders (-8.75 D to +11.5 D) for orientation control and Jackson cross-cylinders (0 to 12 D) for spherical neutrality. Horizontal, vertical, and oblique astigmatism was induced with five monocular and three binocular axis steps. Depth discrimination was recorded at near using Frisby, TNO, and Titmus stereoacuity tests and at distance (4 m) using the variable distance stereoacuity test. Visual acuity was recorded at 0.4 m and 4 m.

RESULTS

Visual acuity and depth discrimination degraded with increasing astigmatic blur. The effect of monocular astigmatic blur on depth discrimination and visual acuity was not dependent on the axis of orientation. For binocular astigmatic blur, the reduction in depth discrimination was dependent on the axis of the induced astigmatism (p < 0.01). The maximum effect occurred with orthogonal-oblique orientations (x45 left; x135 right), followed by against-the-rule (ATR) astigmatism; with-the-rule (WTR) astigmatism had the least effect (p < 0.001).

CONCLUSIONS

The lesser effect of WTR compared with ATR astigmatic blur on depth discrimination may reflect the contribution of horizontal compared with nonhorizontal disparity processing in stereopsis. The pronounced effect of oblique astigmatic blur may be because of the effects on horizontal and nonhorizontal disparity and interocular differential image blur.

Stereoscopic matching and the aperture problem

In order to perceive stereoscopic depth, the visual system must define binocular disparities. Consider an oblique line seen through an aperture formed by flanking occluders. Because the line is perceived behind the aperture, the line must have disparity relative to the aperture. What is the assigned disparity of the line in this aperture problem? To answer this question five observers adjusted the horizontal disparity of a probe until it was perceived at the same depth as the disparate line behind the aperture. The results show that, when both the horizontal and the vertical disparities of the occluders are well-defined, the probe must have the same horizontal disparity as the horizontal separation between the line half-images. However, when the horizontal and vertical disparities of the occluders are ill-defined, the intersections of the line and the occluder borders can determine the matching direction. In the latter case, the matching direction varies with the aperture orientation and there is considerable variability across observers.

Binocular correlation does not improve coherence detection for fronto-parallel motion

We studied the low-level interactions between motion coherence detection and binocular correlation detection. It is well-established that e.g. depth information from motion parallax and from binocular disparities is effectively integrated. The question we aimed to answer is whether such interactions also exist at the very first correlation level that both mechanisms might have in common. First we quantitatively compared motion coherence detection and binocular correlation detection using similar stimuli (random pixels arrays, RPAs) and the same noise masking paradigm (luminance signal to noise ratio, LSNR). This showed that human observers are much more sensitive to motion than to binocular correlation. Adding noise therefore has a much stronger effect on binocular correlation than on motion detection. Next we manipulated the shape of the stimulus aperture to equalize LSNR thresholds for motion and binocular correlation. Motion sensitivity could be progressively reduced by shortening the length of the motion path, while keeping the aperture area constant. Changing the shape of the aperture did not affect binocular correlation sensitivity. A 'balanced' stimulus, one with equal strengths of motion and binocular correlation signals was then used to study the mutual interactions. In accordance with previous results, motion was found to greatly facilitate binocular correlation. Binocular correlation, however did not facilitate motion detection. We conclude that interactions are asymmetrical; fronto-parallel motion is primarily detected monocularly and this information can then be used to facilitate binocular correlation, but binocular correlation cannot improve motion sensitivity.

Stereoscopic depth perception from oblique phase disparities

In order to understand the role of oblique retinal image disparities in the perception of stereoscopic depth, we measured the depth perceived from random dot stereograms in which phase disparities were introduced in a selected band of stimulus orientations. A band of orientation was defined by a center orientation that ranged from 7.5 (near vertical) to 82.5 o[rientation]deg and by a bandwidth that was defined as the difference between the highest and the lowest orientation in the band. The bandwidths tested were 15, 30 and 45 odeg. A constant phase disparity of 90 p[hase]deg was introduced in all of the oriented spatial frequency components within the orientation band and the perceived depth of each stimulus was matched using a small square binocular probe. For each bandwidth, perceived depth increased with an increase in the center orientation up to approximately 60 odeg. This suggests that the human stereovision system derives a large proportion of information about perceived stereoscopic depth from oblique phase disparities. Simulations using an energy model of stereoscopic depth perception indicate that oblique phase disparities are unlikely to be processed by neural mechanisms tuned to near-vertical orientations within the stimulus. Our results therefore suggest that oblique retinal disparities are initially detected as oblique phase disparities by binocular mechanisms tuned to oblique orientations. Because the perceived depth from oblique phase disparities is consistent with the trigonometrically determined equivalent horizontal disparities, we presume that the information from oblique phase disparities is included in the visual system's computation of the horizontal retinal disparity.

Bayesian modeling of cue interaction: bistability in stereoscopic slant perception

Our two eyes receive different views of a visual scene, and the resulting binocular disparities enable us to reconstruct its three-dimensional layout. However, the visual environment is also rich in monocular depth cues. We examined the resulting percept when observers view a scene in which there are large conflicts between the surface slant signaled by binocular disparities and the slant signaled by monocular perspective. For a range of disparity-perspective cue conflicts, many observers experience bistability: They are able to perceive two distinct slants and to flip between the two percepts in a controlled way. We present a Bayesian model that describes the quantitative aspects of perceived slant on the basis of the likelihoods of both perspective and disparity slant information combined with prior assumptions about the shape and orientation of objects in the scene. Our Bayesian approach can be regarded as an overarching framework that allows researchers to study all cue integration aspects-including perceptual decisions--in a unified manner.

Correlation between stereoanomaly and perceived depth when disparity and motion interact in binocular matching

The aim of this study was to find out to what extent binocular matching is facilitated by motion when stereoanomalous and normal subjects estimate the perceived depth of a 3-D stimulus containing excessive matching candidates. Thirty subjects viewed stimuli that consisted of bars uniformly distributed inside a volume. They judged the perceived depth-to-width ratio of the volume by adjusting the aspect ratio of an outline rectangle (a metrical 3-D task). Although there were large inter-subject differences in the depth perceived, the experimental results yielded a good correlation with stereoanomaly (the inability to distinguish disparities of different magnitudes and/or signs in part of the disparity spectrum). The results cannot be explained solely by depth-cue combination. Since up to 30% of the population is stereoanomalous, stereoscopic experiments would yield more informative results if subjects were first characterized with regard to their stereo capacities. Intriguingly, it was found that motion does not help to define disparities in subjects who are able to perceive depth-from-disparity in half of the disparity spectrum. These stereoanomalous subjects were found to rely completely on the motion signals. This suggests that the perception of volumetric depth in subjects with normal stereoscopic vision requires the joint processing of crossed and uncrossed disparities.

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.

The influence of cyclovergence on unconstrained stereoscopic matching

In order to perceive depth from binocular disparities the visual system has to identify matching features of the two retinal images. Normally, the assigned disparity is unambiguously determined by monocularly visible matching constraints. The assigned disparity is ambiguous when matching is unconstrained, such as when we view an isolated long oblique disparate line. Recently we found that in order to perceive a depth probe at the same depth as the oblique line, the probe needs to have the same horizontal disparity as the line (i.e. matching occurs along horizontal "search-zones" [Vis. Res. 40 (2000) 151]). Here we examined whether the depth probe disparity in unconstrained matching of long lines is influenced by cyclovergence, by cyclorotation between stereogram half-images, or by combinations of the two. We measured retinal rotation (>6 deg in cyclovergence conditions). We found that in those conditions in which the retinal images were the same (a condition with, say, both zero cyclovergence and zero cyclorotation between the half-images, creates the same retinal images as a condition with both 6 deg cyclovergence and 6 deg cyclorotation) assigned depth was the same too, i.e. independent of cyclovergence. Thus, the assigned depth of the test-line seems to be determined solely by the retinal test-line orientation, implying that the binocular matching algorithm does not seem to incorporate the eyes' cyclovergence when matching is unconstrained.

Poor visibility of motion in depth is due to early motion averaging

Under a variety of conditions, motion in depth from binocular cues is harder to detect than lateral motion in the frontoparallel plane. This is surprising, as the nasal-temporal motion in the left eye associated with motion in depth is easily detectable, as is the nasal-temporal motion in the right eye. It is only when the two motions are combined in binocular viewing that detection can become difficult. We previously suggested that the visibility of motion-in-depth is low because early stereomotion detectors average left and right retinal motions. For motion in depth, a neural averaging process would produce a motion signal close to zero. Here we tested the averaging hypothesis further. Specifically we asked, could the reduced visibility observed in previous experiments be associated with depth and layout in the stimuli, rather than motion averaging? We used anti-correlated random dot stereograms to show that, despite no depth being perceived, it is still harder to detect motion when it is presented in opposite directions in the two eyes than when motion is presented in the same direction in the two eyes. This suggests that the motion in depth signal is lost due to early motion averaging, rather than due to the presence of noise from the perceived depth patterns in the stimulus.

A planar and a volumetric test for stereoanomaly

Stereoanomaly is the failure to see differences in depth when the viewer is presented with stimuli having different magnitudes of stereoscopic disparity. In the absence of eye movements, everyone suffers from stereoanomaly for extremely large disparities. Typically, such disparities are seen at the same depth as monocular stimuli. However, about 30%, of the population exhibit some form of stereoanomaly even for very small disparities, provided eye movements are avoided. In some cases, the sign of the disparity will be confused, and the perceived depth will be incorrectly seen as 'behind' rather than 'in front of' the fixation point, for example. Because anomalies provide useful information about perceptual mechanisms, tests that measure and quantify the extent of a blindness are important investigative tools for research. Here we offer two easy-to-administer tests for stereoanomaly. The first test is based on depth judgments of two bars relative to a fixation point. The second test involves judgments of volumetric stimuli, seen stereoscopically. In each case, subjects indicate depth by setting a rectangle (with fixed base) to match the perceived depth. Although both tests are correlated, some differences in stereo processing are seen, depending upon whether or not the stimuli are presented near the point of fixation.