Cue conflict between disparity change and looming in the perception of motion in depth

Minimal theory of 3D vision: new approach to visual scale and visual shape

Since Kepler and Descartes in the early-1600s, vision science has been committed to a triangulation model of stereo vision. But in the early-1800s, we realized that disparities are responsible for stereo vision. And we have spent the past 200 years trying to shoe-horn disparities back into the triangulation account. The first part of this article argues that this is a mistake, and that stereo vision is a solution to a different problem: the eradication of rivalry between the two retinal images, rather than the triangulation of objects in space. This leads to a 'minimal theory of 3D vision', where 3D vision is no longer tied to estimating the scale, shape, and direction of objects in the world. The second part of this article then asks whether the other aspects of 3D vision, which go beyond stereo vision, really operate at the same level of visual experience as stereo vision? I argue they do not. Whilst we want a theory of real-world 3D vision, the literature risks giving us a theory of picture perception instead. And I argue for a two-stage theory, where our purely internal 'minimal' 3D percept (from stereo vision) is linked to the world through cognition. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Image statistics determine the integration of visual cues to motion-in-depth

Motion-in-depth perception is critical in enabling animals to avoid hazards and respond to potential threats. For humans, important visual cues for motion-in-depth include changing disparity (CD) and changing image size (CS). The interpretation and integration of these cues depends upon multiple scene parameters, such as distance moved, object size and viewing distance, posing a significant computational challenge. We show that motion-in-depth cue integration depends upon sensitivity to the joint probabilities of the scene parameters determining these signals, and on the probability of CD and CS signals co-occurring. Models that took these factors into account predicted human performance in speed-in-depth and cue conflict discrimination tasks, where standard linear integration models could not. These results suggest that cue integration is affected by both the uncertainty of sensory signals and the mapping of those signals to real-world properties. Evidence of a role for such mappings demonstrates the importance of scene and image statistics to the processes underpinning cue integration and the perception of motion-in-depth.

Does Vergence Affect Perceived Size?

Since Kepler (1604) and Descartes (1637), it has been suggested that 'vergence' (the angular rotation of the eyes) plays a key role in size constancy. However, this has never been tested divorced from confounding cues such as changes in the retinal image. In our experiment, participants viewed a target which grew or shrank in size over 5 s. At the same time, the fixation distance specified by vergence was reduced from 50 to 25 cm. The question was whether this change in vergence affected the participants' judgements of whether the target grew or shrank in size? We found no evidence of any effect, and therefore no evidence that eye movements affect perceived size. If this is correct, then our finding has three implications. First, perceived size is much more reliant on cognitive influences than previously thought. This is consistent with the argument that visual scale is purely cognitive in nature (Linton, 2017; 2018). Second, it leads us to question whether the vergence modulation of V1 contributes to size constancy. Third, given the interaction between vergence, proprioception, and the retinal image in the Taylor illusion, it leads us to ask whether this cognitive approach could also be applied to multisensory integration.

The Effect of Motion Direction and Eccentricity on Vection, VR Sickness and Head Movements in Virtual Reality

Virtual Reality (VR) experienced through head-mounted displays often leads to vection, discomfort and sway in the user. This study investigated the effect of motion direction and eccentricity on these three phenomena using optic flow patterns displayed using the Valve Index. Visual motion stimuli were presented in the centre, periphery or far periphery and moved either in depth (back and forth) or laterally (left and right). Overall vection was stronger for motion in depth compared to lateral motion. Additionally, eccentricity primarily affected stimuli moving in depth with stronger vection for more peripherally presented motion patterns compared to more central ones. Motion direction affected the various aspects of VR sickness differently and modulated the effect of eccentricity on VR sickness. For stimuli moving in depth far peripheral presentation caused more discomfort, whereas for lateral motion the central stimuli caused more discomfort. Stimuli moving in depth led to more head movements in the anterior-posterior direction when the entire visual field was stimulated. Observers demonstrated more head movements in the anterior-posterior direction compared to the medio-lateral direction throughout the entire experiment independent of motion direction or eccentricity of the presented moving stimulus. Head movements were elicited on the same plane as the moving stimulus only for stimuli moving in depth covering the entire visual field. A correlation showed a positive relationship between dizziness and vection duration and between general discomfort and sway. Identifying where in the visual field motion presented to an individual causes the least amount of VR sickness without losing vection and presence can guide development for Virtual Reality games, training and treatment programmes.

Size matters: How reaching and vergence movements are influenced by the familiar size of stereoscopically presented objects

The knowledge about the usual size of objects-familiar size-is known to be a taken into account for distance perception. The influence of familiar size on action programming is less clear and has not yet been tested with regard to vergence eye movements. In two experiments, we stereoscopically presented everyday objects, such as a credit card or a package of paper tissues, and varied the distance as specified by binocular disparity and the distance as specified by familiar size. Participants had to fixate the shown object and subsequently reach towards it either with open or with closed eyes. When binocular disparity and familiar size were in conflict, reaching movements revealed a combination of the two depth cues with individually different weights. The influence of familiar size was larger when no visual feedback was available during the reaching movement. Vergence movements closely followed binocular disparity and were largely unaffected by familiar size. In sum, the results suggest that in this experimental setting familiar size is taken into account for programming and executing reaching movements while vergence movements are primarily based on binocular disparity.

Impending Collision Judgment from an Egocentric Perspective in Real and Virtual Environments: A Review

The human egocentric perception of approaching objects and the related perceptual processes have been of interest to researchers for several decades. This article gives a literature review on numerous studies that investigated the phenomenon when an object approaches an observer (or the other way around) with the goal to single out factors that influence the perceptual process. A taxonomy of metrics is followed by a breakdown of different experimental measurement methods. Thereinafter, potential factors affecting the judgment of approaching objects are compiled and debated while divided into human factors (e.g., gender, age, and driving experience), compositional factors (e.g., approaching velocity, spatial distance, and observation time), and technical factors (e.g., field of view, stereoscopy, and display contrast). Experimental findings are collated, juxtaposed, and critically discussed. With virtual-reality devices having taken a tremendous developmental leap forward in the past few years, they have been able to gain ground in experimental research. Therefore, special attention in this article is also given to the perception of approaching objects in virtual environments and put in contrast to the perception in reality.

Temporal Audiovisual Motion Prediction in 2D- vs. 3D-Environments

Predicting motion is essential for many everyday life activities, e.g., in road traffic. Previous studies on motion prediction failed to find consistent results, which might be due to the use of very different stimulus material and behavioural tasks. Here, we directly tested the influence of task (detection, extrapolation) and stimulus features (visual vs. audiovisual and three-dimensional vs. non-three-dimensional) on temporal motion prediction in two psychophysical experiments. In both experiments a ball followed a trajectory toward the observer and temporarily disappeared behind an occluder. In audiovisual conditions a moving white noise (congruent or non-congruent to visual motion direction) was presented concurrently. In experiment 1 the ball reappeared on a predictable or a non-predictable trajectory and participants detected when the ball reappeared. In experiment 2 the ball did not reappear after occlusion and participants judged when the ball would reach a specified position at two possible distances from the occluder (extrapolation task). Both experiments were conducted in three-dimensional space (using stereoscopic screen and polarised glasses) and also without stereoscopic presentation. Participants benefitted from visually predictable trajectories and concurrent sounds during detection. Additionally, visual facilitation was more pronounced for non-3D stimulation during detection task. In contrast, for a more complex extrapolation task group mean results indicated that auditory information impaired motion prediction. However, a post hoc cross-validation procedure (split-half) revealed that participants varied in their ability to use sounds during motion extrapolation. Most participants selectively profited from either near or far extrapolation distances but were impaired for the other one. We propose that interindividual differences in extrapolation efficiency might be the mechanism governing this effect. Together, our results indicate that both a realistic experimental environment and subject-specific differences modulate the ability of audiovisual motion prediction and need to be considered in future research.

Audiovisual Integration of Time-to-Contact Information for Approaching Objects

Previous studies of time-to-collision (TTC) judgments of approaching objects focused on effectiveness of visual TTC information in the optical expansion pattern (e.g., visual tau, disparity). Fewer studies examined effectiveness of auditory TTC information in the pattern of increasing intensity (auditory tau), or measured integration of auditory and visual TTC information. Here, participants judged TTC of an approaching object presented in the visual or auditory modality, or both concurrently. TTC information provided by the modalities was jittered slightly against each other, so that auditory and visual TTC were not perfectly correlated. A psychophysical reverse correlation approach was used to estimate the influence of auditory and visual cues on TTC estimates. TTC estimates were shorter in the auditory than the visual condition. On average, TTC judgments in the audiovisual condition were not significantly different from judgments in the visual condition. However, multiple regression analyses showed that TTC estimates were based on both auditory and visual information. Although heuristic cues (final sound pressure level, final optical size) and more reliable information (relative rate of change in acoustic intensity, optical expansion) contributed to auditory and visual judgments, the effect of heuristics was greater in the auditory condition. Although auditory and visual information influenced judgments, concurrent presentation of both did not result in lower response variability compared to presentation of either one alone; there was no multimodal advantage. The relative weightings of heuristics and more reliable information differed between auditory and visual TTC judgments, and when both were available, visual information was weighted more heavily.

Perceived rigidity in motion-in-depth increases with contour perspective

When observers are asked to match the depth of an object according to its height, they often report systematic errors depending on viewing distance. Systematic biases can also arise while vergence distances are induced by binocular disparities. Observers of stereoscopic images tend to overestimate the depth of objects displayed in front of the screen, while the depth of objects displayed behind the screen plane is underestimated. This phenomenon creates a serious problem in that veridicality in depth perception appears distorted when one attempts to render the metrics of a captured 3-D world. These distortions could also subsist with structure-from-motion information and during motion-in-depth. Observers judged the circularity of transparent rotating cylinders that were either static or moving in depth. Crossed results show that participants could precisely retrieve the best modulation between presented depth and width. As this effect could be amplified with stimuli containing stronger perspective cues (ie contour perspective), participants judged the rigidity of spinning cubes, moving along the line of sight, which were either edges-defined or defined by randomly textured surfaces (dots). The results showed that, although depth constancy was not improved by contour perspective, perceived rigidity was increased by perspective when the best scaling estimate was displayed. This finding suggests that appropriate binocular disparity information in combination to monocular signal is necessary for stereoscopic depth perception.

Unaffected smooth pursuit but impaired motion perception in monocularly enucleated observers

The objective of this paper was to study the characteristics of closed-loop smooth pursuit eye movements of 15 unilaterally eye enucleated individuals and 18 age-matched controls and to compare them to their performance in two tests of motion perception: relative motion and motion coherence. The relative motion test used a brief (150 ms) small stimulus with a continuously present fixation target to preclude pursuit eye movements. The duration of the motion coherence trials was 1s, which allowed a brief pursuit of the stimuli. Smooth pursuit data were obtained with a step-ramp procedure. Controls were tested both monocularly and binocularly. The data showed worse performance by the enucleated observers in the relative motion task but no statistically significant differences in motion coherence between the two groups. On the other hand, the smooth pursuit gain of the enucleated participants was as good as that of controls for whom we found no binocular advantage. The data show that enucleated observers do not exhibit deficits in the afferent or sensory pathways or in the efferent or motor pathways of the steady-state smooth pursuit system even though their visual processing of motion is impaired.

Psychophysical point: a disc tends to become a point when Weber's law fails

In three experiments, we showed that the visual system treats a dot somewhat like a geometrical point, which has a location but no area. We represented a "point" or "dot" with a small disc (diameter of 0.08º of visual angle), and a "disc" with a larger disc (diameter of 1.5º). The Weber fraction of the dot was larger than that of the disc. In Experiment 1, the relative retinal image size cues for depth for the dot and the disc were placed in conflict with the motion parallax cue. We found that the dot indicated the positions defined by the motion parallax cue better than the disc did. In Experiment 2, we placed a constant retinal image size in conflict with convergence eye movements. We found that a binocularly fused dot appeared to move in depth with convergence eye movement, whereas a fused disc appeared to move less. In Experiment 3, we examined the apparent sizes of the afterimages of a dot and a disc and found that Emmert's law failed for the dot afterimage; the apparent size of the dot afterimage changed very little for different distances-as though it had no area-whereas the apparent size of the disc afterimage changed by an extent predicted by Emmert's law. The differences in the dot and disc conditions could not be explained by the differences in the Weber fractions alone.

Replicating and extending Bourdon's (1902) experiment on motion parallax

Bourdon conducted the first laboratory experiment on observer-produced motion parallax as a cue to depth. In three experiments, we replicated and extended Bourdon's experiment. In experiment 1, we reproduced his finding: when the two cues, motion parallax and relative height, were combined, accuracy of depth perception was high, and when the two cues were in conflict, accuracy was lower. In experiment 2, the relative height cue was replaced with relative retinal image size. As in experiment 1, when the two cues (motion parallax and relative retinal image size) were combined, accuracy was high, but when they were in conflict, it was lower. In experiment 3, the stimuli from experiments 1 and 2 were viewed monocularly with head movement and binocularly without head movement. In the binocular conditions, accuracy, certainty, and the extent of perceived depth were higher than in the monocular condition. In the conflict conditions, accuracy, certainty, and the extent of perceived depth were lower than in the no-conflict condition, but the extent of perceived motion was larger. These results are discussed in terms of recent findings about the effectiveness of motion parallax as a cue for depth.

Where do the eyes really go in the hollow-face illusion?

The hollow-face illusion refers to the finding that people typically perceive a concave (hollow) mask as being convex, despite the presence of binocular disparity cues that indicate the contrary. Unlike other illusions of depth, recent research has suggested that the eyes tend to converge at perceived, rather than actual, depths. However, technical and methodological limitations prevented one from knowing whether disparity cues may still have influenced vergence. In the current study, we presented participants with virtual normal or hollow masks and asked them to fixate the tip of the face's nose until they had indicated whether they perceived it as pointing towards or away from them. The results showed that the direction of vergence was indeed determined by perceived depth, although vergence responses were both somewhat delayed and of smaller amplitude (by a factor of about 0.5) for concave than convex masks. These findings demonstrate how perceived depth can override disparity cues when it comes to vergence, albeit not entirely.

Accuracy of stereomotion speed perception with persisting and dynamic textures

It has been established that the motion in depth of stimuli visible to both eyes may be signalled binocularly either by a change of disparity over time or by the difference in the velocity of the images projected on each retina, known as an interocular velocity difference. A two-interval forced-choice stereomotion speed discrimination experiment was performed on four participants to ascertain the relative speed of a persistent random dot stereogram (RDS) and a dynamic RDS undergoing directly approaching or receding motion in depth. While the persistent RDS pattern involved identical dot patterns translating in opposite directions in each eye, and hence included both changing disparity and interocular velocity difference cues, the dynamic RDS pattern (which contains no coherent monocular motion signals) specified motion in depth through changing disparity, but no motion through interocular velocity difference. Despite an interocular velocity difference speed signal of zero motion in depth, the dynamic RDS stimulus appeared to move more rapidly. These observations are consistent with a scheme in which cues that rely on coherent monocular motion signals (such as looming and the interocular velocity difference cue) are less influential in dynamic stimuli due to their lack of reliability (i.e., increased noise). While dynamic RDS stimuli may be relatively unaffected by the contributions of such cues when they signal that the stimulus did not move in depth, the persistent RDS stimulus may retain a significant and conflicting contribution from the looming cue, resulting in a lower perceived speed.