A computational model of depth perception based on headcentric disparity

It is now well established that depth is coded by local horizontal disparity and global vertical disparity. We present a computational model which explains how depth is extracted from these two types of disparities. The model uses the two (one for each eye) headcentric directions of binocular targets, derived from retinal signals and oculomotor signals. Headcentric disparity is defined as the difference between headcentric directions of corresponding features in the left and right eye's images. Using Helmholtz's coordinate systems we decompose headcentric disparity into azimuthal and elevational disparity. Elevational disparities of real objects are zero if the signals which contribute to headcentric disparity do not contain any errors. Azimuthal headcentric disparity is a 1D quantity from which an exact equation relating distance and disparity can be derived. The equation is valid for all headcentric directions and for all binocular fixation positions. Such an equation does not exist if disparity is expressed in retinal coordinates. Possible types of errors in oculomotor signals (six) produce global elevational disparity fields which are characterised by different gradients in the azimuthal and elevational directions. Computations show that the elevational disparity fields uniquely characterise both the type and size of the errors in oculomotor signals. Our model uses a measure of the global elevational disparity field together with local azimuthal disparity to accurately derive headcentric distance throughout the visual field. The model explains existing data on whole-field disparity transformations as well as hitherto unexplained aspects of stereoscopic depth perception.

Sensitivity to disparity corrugations in peripheral vision

Disparity discrimination thresholds are known to increase with both retinal eccentricity and distance from the horopter. However, little is known about how the detectability of cyclopean gratings varies with retinal position. Thresholds for disparity corrugations were measured as a function of corrugation frequency for different visual eccentricities. Subjects viewed annular displays of random dot stereograms, and judged in which of two intervals a circumferential disparity modulation was present. For any given eccentricity, visual sensitivity to disparity corrugations was bandpass. As eccentricity increased from 3.5 to 21.0 degrees, peak-to-trough thresholds were found to increase, the optimal corrugation frequency for detection decreased, and the upper cutoff corrugation frequency also decreased. The M-Scaling functions of Rovamo and Virsu were used to replot the data in terms of cycles per unit cortical distance. Peak detection frequency was constant at 0.8 cycles per mm of cortex after this rescaling, demonstrating that acuity for disparity modulations is approximately M-scaled beyond the fovea.

Dynamic asymmetries in disparity convergence eye movements

Vergence eye movements have traditionally been considered the product of a single neural control center and are usually studied by combining the movements of each eye into a single 'vergence' response. In the present experiment, disparity-driven eye movements were produced by symmetrical step stimuli, and the dynamic properties of each eye movement were analyzed separately. Although the final positions of the two eyes were symmetrical, large dynamic asymmetries often occurred. The timing between the two eyes showed fair synchrony as they attained maximum velocity at approximately the same time. Since the final static positions were symmetrical, asymmetries occurring during the initial dynamic component must necessarily be compensated by offsetting asymmetries in the latter portion of the response.

The perception of scale-dependent and scale-independent surface structure from binocular disparity, texture, and shading

The integration of binocular disparity, shading, and texture was measured for two different aspects of three-dimensional structure: (1) shape index, which is a measure of scale-independent structure, and (2) curvedness, which is a measure of scale-dependent structure. Binocular disparity was found to contribute significantly more to judged shape index than it does to judged curvedness, and shading and texture were both found to contribute more to judged curvedness than to judged shape index. These results demonstrate that different cues do not contribute equally to different aspects of perceived surface structure. This finding suggests that, for the case of linear integration, multiple cues to three-dimensional structure do not combine on the basis of a single type of representation shared by all the 'shape-from-X' processes in the visual system.

Depth interactions between inclined and slanted surfaces in vertical and horizontal orientations

Depth interactions between a frontal test surface and an adjacent induction surface were measured as a function of the type of disparity in the induction surface and of the vertical/horizontal orientation of the boundary between the surfaces. The types of disparity were 4 degrees horizontal-shear disparity, 4 degrees vertical-shear disparity, and 4 degrees rotation disparity; 4% horizontal-size disparity, 4% vertical-size disparity, and 4% overall-size disparity. Depth contrast in a frontal surface was produced by surfaces containing horizontal-size disparity but not by those containing horizontal-shear disparity. Vertical-shear and vertical-size disparities produced induced effects in both the induction and the test surface, which is here explained in terms of deformation-disparity processing. Effects of rotation disparity on the test surface can be accounted for in terms of cyclovergence, deformation disparity, and perhaps also depth contrast. The fact that horizontal-size disparity produced more depth contrast than horizontal-shear disparity is due to an anisotropy of disparity processing rather than the relative orientation of the surfaces. Ground surfaces appeared more slanted than ceiling surfaces. Surfaces containing horizontal disparities produced a sharp boundary with the test surface because horizontal disparities are processed locally. Surfaces with vertical disparities produced a gradual boundary with the test surface because vertical disparities are processed over a wider area.

Vergence eye movements elicited by stimuli without corresponding features

We have observed quantitative depth perception with a dichoptic stimulus which possessed no contrast-defined binocular corresponding features (phantom stereogram). The depth perception can be the result of appreciation of a partial-occlusion situation depicted by the stimulus, or the result of activities of low-level disparity detectors which are capable of combining dissimilar local features in the stimulus. Although both mechanisms predict similar depth perception, they predict different vergence eye-movement outputs, especially in the vertical dimension. To identify the underlying mechanisms of the phantom stereopsis, we recorded vergence tracking eye movements to four types of dichoptic stimuli: (a) conventional stereogram with horizontal disparity (HD); (b) horizontal phantom stereogram (HP); (c) conventional stereogram with vertical disparity (VD); and (d) vertical phantom stereogram (VP). We found that HD, HP, and VD stimuli could elicit robust vergence tracking eye movements but VP stimulus could not. While the success of HP stimulus in eliciting vergence tracking may be explained by proximal vergence, the failure of VP stimulus in eliciting vergence tracking clearly indicates that phantom stereogram could not elicit coherent responses among low-level disparity detectors. Partial occlusion, therefore, has to play an important role in the depth perception from the phantom stereogram.

Fixation disparity, accommodation, dark vergence and dark focus during inclined gaze

We investigated several oculomotor functions at different angles of vertical inclination of the gaze direction from 15 deg upwards to 45 deg downwards. The mean accommodative resting state (measured in a dark visual field) increased when the eyes or the head were declined from 0 to 45 deg. Fixation disparity (the vergence error in minutes of arc relative to the principle visual directions) became more eso when a fusion target at a viewing distance of 40 cm was lowered: declining the gaze by 45 deg changed mean fixation disparity by 1.8 min arc with eye inclination (keeping the head upright), and by 0.9 min arc with head inclination (with eye position unchanged relative to the head). When the eyes were lowered, the individual rate of eso change in fixation disparity was correlated with the amount of the subjects' near shifts in the resting position of vergence, measured in darkness. Significant test-retest correlations between repeated measurements showed that the effects of eye inclination on vergence varied in a reproducible way among individuals with good binocular vision.

The visual effects of head-mounted display (HMD) are not distinguishable from those of desk-top computer display

Concerns about potentially harmful effects on the visual system due to the use of head mounted displays (HMDs) in general, and stereoscopic systems in particular, have been raised in the literature. Most of the concerns were based on studies measuring visual function changes following short-term use of HMDs. This study measured functional changes in binocular vision, accommodation, and resolution following 30 min use of HMD in both stereoscopic- and non-stereoscopic modes, and compared them to changes following the same task performed on a desk-top CRT display. No functional differences were found between HMD and CRT and most measured changes were too small to be considered clinically meaningful. An evaluation of subjective comfort found a statistically significant difference in the impression of comfort between the CRT and the HMD in stereoscopic mode, with the latter being less comfortable. It can be concluded that the functional changes reported following short term use of HMDs are not specific to stereoscopic presentation and do not differ from those caused by desk-top CRT display.

Assimilation: central and peripheral effects

Assimilation and contrast have opposite effects: Contrast leads to an increase of perceived differences between neighbouring fields, whereas assimilation leads to a reduction. It is relatively easy to demonstrate these effects, but the precise localisation of these effects in the perceptual system is not yet possible. In an experiment the strength of assimilation effects was modified by adding spatial noise. By varying the localisation in perceived space of the added noise (by presentation of the noise pattern with different binocular disparities) the masking effect of this noise can be influenced. Masking caused by binocularly disparate noise is less than masking caused by binocularly non-disparate noise. It is concluded that the effect at least partly occurs beyond the (binocular) locus of separation in different depth planes. A similar approach, involving moving noise, is also presented. Finally, several demonstrations show that images that are peripherally similar can give rise to differences in the perceived amount of assimilation. These effects further indicate that a central mechanism is involved in assimilation.

Large scale stereopsis and optic flow: depth enhanced by speed and opponent-motion

To understand better the range of conditions supporting stereoscopic vision, we explored the effects of speed, as well as specific optic flow patterns, on judgments of the depth, near or far of fixation, of large targets briefly presented in the upper periphery. They had large disparities (1-6 deg) and moved at high speeds (20-100 deg/sec). Motion was either vertical or horizontal, as well as either unidirectional or layered in bands of alternating directions (opponent-motion). High stimulus speeds can extend dmax. The effects are explained by models having linear filters that signal both faster speeds and larger disparities. Stereo depth localization can also be enhanced by opponent-motion even when kinetic depth itself is not apparent. Improvements are greatest with wide-field, horizontal opponent-motion. The results imply functions such as vection, posture-control, and vergence may benefit from disparity information enhanced by optic flow patterns that are commonly available to a moving, binocular observer.

Optic flow and the metric of the visual ground plane

A theory is developed in which the optic flow of an observer translating over the ground plane determines the metric of egocentric visual space. Optic flow is used to operationalize the equality of spatial intervals not unlike physicists use time to compare spatial intervals. The theory predicts empirical matching ratios for collinear, sagittal intervals to within 2% of the mean (eight subjects, standard error also 2%). The theory predicts that frontoparallel intervals on the ground plane will match sagittal intervals if their relative image motions match, which was found empirically. It is suggested that the optic flow metric serves to calibrate static depth cues such as angular elevation and binocular parallax.

Stereoscopic segregation of transparent surfaces and the effect of motion contrast

Stereoscopic segregation in depth was studied using two superimposed frontoparallel surfaces displayed in dynamic random dot stereograms. The two patterns were positioned symmetrically in front of and behind a binocular fixation point. They were either stationary, or they could move relative to each other. Sensitivity for segregation was established by adding gaussian distributed disparity noise to the disparities specifying the two planes, and finding the noise amplitude that gave threshold segregation performance. Observers easily segregate the two surfaces for disparity differences between approximately 6 and 30-40 arcmin. Motion contrast, which by itself provides no cue to perform the task, greatly improves sensitivity for segregation. Noise tolerance rises by a factor of two or more when the patterns move at different speeds, or in different (frontoparallel) directions. The effect increases with directional difference, but the optimal directional difference deviated from 180 deg. The optimal speed varies with disparity difference. Thus, motion and disparity must interact in order to resolve the two transparent planes.

Display duration and stereoscopic depth discrimination

We investigated the role of display duration in stereoscopic depth perception. The display consisted of a dynamic random-dot stereogram, with two disparity-defined squares (1.9 degrees x 1.9 degrees), one on the left and one on the right of a central (Nonius) fixation stimulus. The sign of the disparity (crossed or uncrossed) was always the same for both squares, and the magnitude of disparity was 0.25 degree for one square and either 0.125 degree or 0.375 degree for the other square. Participants indicated which square appeared closer. The display duration was varied adaptively between 20 and 1000 ms until participants performed at 75% accuracy. Results confirmed large individual differences in the display duration required for stereoscopic depth perception. Approximately half of the 100 naive participants were able to perform the task at 20 ms, while the remaining participants required up to 1000 ms to perform at criterion. The present study shows that display duration is a critical variable in explaining wide differences in reported abilities of individuals to process stereoscopic depth information.

dmax for stereopsis and motion in random dot displays

The upper displacement limit for motion was compared with the upper disparity limit for stereopsis using two-frame random dot kinematograms or briefly presented stereograms. dmax (the disparity/displacement at which subjects make 20% errors in a forced-choice paradigm) was found to be very similar for motion and stereo at all dot densities, and to fall with increasing dot density (0.006% or two dots to 50%) according to a power law (exponent -0.2). If dmax is limited by the spacing of false targets, this pattern of results suggests that the spatial primitives in the input to the correspondence process may be derived from multiple spatial scales. A model using MIRAGE centroids provides a good fit to the data.

Interactions between global motion and local binocular rivalry

Binocular rivalry is thought to arise from a low-level cortical site. Experiment 1 evaluates this claim with respect to local and global motion processing by using a multiple-aperture motion stimulus and measuring the predominance of global coherence while one of the component gratings is engaged in rivalry. Results show that rivalry suppression of the component grating precludes global coherence. Presumable, suppression prevents the component motion signal from advancing to higher-level global motion areas, suggesting rivalry occurs between local and global motion processing. However, feedback from higher-level mechanisms might exert an influence on binocular rivalry and thus Experiment 2 measures how the predominance of a local target engaged in binocular rivalry with a competing local stimulus is affected when the target forms part of a globally coherent motion stimulus. The augmented level of target predominance during global motion relative to local motion indicates that higher-level motion mechanisms can feedback and influence the binocular rivalry process. Together, these data imply a looping hierarchy of motion processing stages, with rivalry suppression transpiring at an intermediate level and subject to feedback from higher-level motion areas.

Spatial interactions modulate stereoscopic processing of horizontal and vertical disparities

Stereoscopic processing of horizontal and vertical disparities was assessed by measuring how the stereoscopic appearance of test dots near the fixation point was influenced by inducing stimuli in the near periphery. The inducing stimuli were differentially magnified in the two eyes and varied in horizontal eccentricity. As expected, when the inducers were horizontally magnified, the test dots exhibited depth contrast, slanting in depth in a direction opposite the slant of the inducing dots. When the inducers were vertically magnified, the test dots slanted in depth around a vertical axis toward the eye with the larger vertical image (the induced-size effect). However, two lines of evidence suggested that an eccentricity-dependent weighted average of horizontal and vertical components of inducer-dot magnification determined the slant of the test dots. First, as the horizontal eccentricity of the inducing dots was varied, the trend of test-dot slants measured with vertical inducer magnifications was predicted by the trend of test-dot slants measured with horizontal inducer magnifications. Second, test-dot slants measured with a combination of both horizontal and vertical inducer magnification could be predicted by simply adding test-dot slants measured with either horizontal or vertical inducer magnification alone.

Luminance contrast affects motion coherency in plaid patterns by acting as a depth-from-occlusion cue

Moving plaid patterns composed of component gratings that differ in luminance contrast tend not to cohere perceptually. Plaid patterns configured to mimic one occlusive grating overlying another also fail to cohere. We hypothesized that plaids constructed of components with different luminance contrasts fail to cohere because these components are interpreted as occlusive surfaces lying in different depth planes. It is known that when depth-from-occlusion and depth-from-binocular disparity cues support the same depth-ordering, both segregation in depth and motion non-coherency are more likely to be perceived than when these two cues conflict. We exploited this interaction and tested our hypothesis by introducing horizontal binocular disparity between two superimposed component gratings of different luminance contrasts. We found that both depth segregation and motion non-coherency were much more likely when the high-contrast grating was stereoscopically in front of the low-contrast grating. From these results we infer that luminance contrast acts as a depth-cue in plaid patterns, with higher contrast gratings appearing to lie in front of lower contrast gratings. Perceptual motion coherency parallels these depth-ordering judgments. We conclude that luminance contrast affects motion coherency by acting as a depth-from-occlusion cue.

Synchronization, decrease in strength of pattern, illusory contours, and neon-color effect

When certain elements in a pattern are replaced by colored lines, a spreading of neon-like color can be seen around the lines. Using a variety of neon displaying patterns, we hypothesize that two conditions are critical to the neon effect. One is the occurrence of illusory contours and another a decrease in strength of pattern of the colored lines. On the basis of the two conditions, various phenomena of the neon effect are discussed. Finally, examining the striking characteristics of the neon effect, the vagueness of the colored lines and the spreading of the color of the colored lines over an illusory area wherein the color stimulus does not exist, we conclude that the neon effect is caused by synchronization of strength of pattern.

Fixation disparity at different viewing distances and the preferred viewing distance in a laboratory near-vision task

Previous research has shown that subjects with normal binocular vision differ reliably in the extent to which their fixation disparity changes in the exo direction when the viewing distance is shortened from 100 to 20 cm. Since fixation disparity can lead to asthenopic complaints, the present study investigates whether an exo fixation disparity induced by proximity may cause subjects to move to a longer viewing distance during a near-vision task in order to reduce exo fixation disparity. In two optometric sessions, fixation disparity and accommodation were tested at 60, 40, and 30 cm viewing distance. In a further session, subjects were required to begin a one-hour near-vision task at about 40 cm viewing distance, at which the text characters subtended a comfortable visual angle of 21 min arc. Later, the subjects were free to adopt any viewing distance. In the initial phase of the task, subjects moved back from the screen to a greater or lesser extent that was correlated with the amount of proximal exo fixation disparity: the more a subject's fixation disparity changed to exo when the viewing distance had been shortened from 60 to 30 cm the more he or she moved to longer viewing distances in the course of the near-vision task. Further, the more distant the resting position of vergence (dark vergence), the more visual complaints the subjects indicated after the task relative to before.

Anomalous correspondence--the cause or consequence of strabismus?

I hypothesize that the functional role of binocular correspondence, whether normal or anomalous, is that of detecting retinal disparity stimuli. Normal correspondence involves disparity stimulus detection that is quantitatively normal and stable, whereas anomalous correspondence involves disparity detection that is quantitatively abnormal and variable. Normal disparity stimuli typically produce motor fusion between normally corresponding biretinal areas. The result is orthotropia, which permits normal single binocular vision and acute stereopsis. In contrast, abnormal disparity stimuli typically produce motor fusion between anomalously corresponding areas. The result is strabismus, usually coupled with anomalous single binocular vision (harmonious anomalous correspondence). Accordingly, corrective prism or muscle surgery only produces vergence movements that restore anomalous motor fusion, rather than eliminating the heterotropia. The inference is that anomalous correspondence involves a neurological disorder in the disparity detection mechanism, either on the convergent side (estropia with anomalous correspondence) or divergent side (exotropia with anomalous correspondence).

Extra-retinal and perspective cues cause the small range of the induced effect

With a horizontal magnifier before one eye, a frontoparallel surface appears rotated about a vertical axis (geometric effect). With a vertical magnifier, apparent rotation is opposite in direction (induced effect); to restore appearance of frontoparallelism, the surface must be rotated away from the magnified eye. The induced effect is interesting because it was thought until recently that vertical disparities do not play an important role in surface perception. As with the geometric effect, the required rotation for the induced effect increases linearly to approximately equal to 4% magnification; unlike the geometric effect, it plateaus at approximately 8%. Current theory explains the linear portion: vertical size ratios (VSRs) are used to compensate for changes in horizontal size ratios (HSRs) that accompany eccentric gaze, so changes in VSR cause changes in perceived slant. The theory does not explain the plateau. We demonstrate that it results from differing slant estimates obtained by use of various retinal and extra-retinal signals. When perspective cues to slant are minimized or sensed eye position is consistent with VSR, the induced and geometric effects have similar magnitudes even at large magnifications.

Sequential stereopsis using high-pass spatial frequency filtered textures

Enright [(1995). Perception, 24 (suppl.), 32-33; (1996). Vision Research, 36, 307-312] described a simple piece of equipment for demonstrating a perceptual mechanism he called sequential stereopsis. The equipment requires an observer to set two textured targets seen behind a pair of small viewing ports to appear equi-distant. The principle upon which the apparatus depends is the use of textures whose elements cannot be resolved in peripheral vision at the eccentricity determined by the target separation. Enright used a fine sandpaper for this purpose. We have conducted two similar experiments using high bandpass filtered textures which eliminate any possibility that the low spatial frequency content of sandpaper textures could play a role. Our results corroborate Enright's general conclusions on sequential stereopsis, while at the same time showing that high-pass textures do not give wholly similar results to sandpaper.

Monocular discrimination of rigidly and nonrigidly moving objects

We measured thresholds for the monocular discrimination of rigidly and nonrigidly moving objects defined by motion parallax. The retinal projections of rigidly moving objects are subject to certain constraints. By applying smooth 2-D transformations to the projections of rigidly moving objects, we created stimuli in which these constraints were affected. Thresholds for (generic) nonrigid transformations that in theory can be detected from rigid ones by processing pairs of views depended not only on the extent to which the rigidity constraints were affected, but also on the structure and the movement of the simulated object. Nonrigid transformations under which every three successive views had a rigid interpretation were not discriminable from rigid transformations, except in cases where the distortions were very large. Under the rigidity assumption, this would mean that a large class of nonrigidly moving objects is erroneously perceived as rigidly moving.

Motion processing for saccadic eye movements during the visually induced sensation of ego-motion in humans

During ego-motion an observer is often faced with the task of controlling his heading direction while simultaneously registering the movement of objects in order to avoid possible obstacles. Psychophysical experiments have shown that the detection of moving objects is impaired by concurrent ego-motion. We investigated the interaction between ego-motion and object-motion by examining the latencies of saccades executed to moving targets under a visually induced sensation of ego-motion. Saccadic latencies increased during this sensation, with a global or non-retinotopic effect of optic flow on motion detection. Furthermore, separating stereoscopically the moving target and the optic flow into foreground and background, respectively, still resulted in increased latencies. We propose that an inhibitory influence of the perception of self-motion exists on the perception of object-motion. These results support a model of space constancy which strives to create a stable world during locomotion.