Stereo algorithm that extracts a depth cue from interocularly unpaired points

Deadeye Visualization Revisited: Investigation of Preattentiveness and Applicability in Virtual Environments

Visualizations rely on highlighting to attract and guide our attention. To make an object of interest stand out independently from a number of distractors, the underlying visual cue, e.g., color, has to be preattentive. In our prior work, we introduced Deadeye as an instantly recognizable highlighting technique that works by rendering the target object for one eye only. In contrast to prior approaches, Deadeye excels by not modifying any visual properties of the target. However, in the case of 2D visualizations, the method requires an additional setup to allow dichoptic presentation, which is a considerable drawback. As a follow-up to requests from the community, this paper explores Deadeye as a highlighting technique for 3D visualizations, because such stereoscopic scenarios support dichoptic presentation out of the box. Deadeye suppresses binocular disparities for the target object, so we cannot assume the applicability of our technique as a given fact. With this motivation, the paper presents quantitative evaluations of Deadeye in VR, including configurations with multiple heterogeneous distractors as an important robustness challenge. After confirming the preserved preattentiveness (all average accuracies above 90%) under such real-world conditions, we explore VR volume rendering as an example application scenario for Deadeye. We depict a possible workflow for integrating our technique, conduct an exploratory survey to demonstrate benefits and limitations, and finally provide related design implications.

Perceptual asymmetry reveals neural substrates underlying stereoscopic transparency

We describe a perceptual asymmetry found in stereoscopic perception of overlaid random-dot surfaces. Specifically, the minimum separation in depth needed to perceptually segregate two overlaid surfaces depended on the distribution of dots across the surfaces. With the total dot density fixed, significantly larger inter-plane disparities were required for perceptual segregation of the surfaces when the front surface had fewer dots than the back surface compared to when the back surface was the one with fewer dots. We propose that our results reflect an asymmetry in the signal strength of the front and back surfaces due to the assignment of the spaces between the dots to the back surface by disparity interpolation. This hypothesis was supported by the results of two experiments designed to reduce the imbalance in the neuronal response to the two surfaces. We modeled the psychophysical data with a network of inter-neural connections: excitatory within-disparity and inhibitory across disparity, where the spread of disparity was modulated according to figure-ground assignment. These psychophysical and computational findings suggest that stereoscopic transparency depends on both inter-neural interactions of disparity-tuned cells and higher-level processes governing figure ground segregation.

The role of transparency in da Vinci stereopsis

The majority of natural scenes contains zones that are visible to one eye only. Past studies have shown that these monocular regions can be seen at a precise depth even though there are no binocular disparities that uniquely constrain their locations in depth. In the so-called da Vinci stereopsis configuration, the monocular region is a vertical line placed next to a binocular rectangular occluder. The opacity of the occluder has been mentioned to be a necessary condition to obtain da Vinci stereopsis. However, this opacity constraint has never been empirically tested. In the present study, we tested whether da Vinci stereopsis and perceptual transparency can interact using a classical da Vinci configuration in which the opacity of the occluder varied. We used two different monocular objects: a line and a disk. We found no effect of the opacity of the occluder on the perceived depth of the monocular object. A careful analysis of the distribution of perceived depth revealed that the monocular object was perceived at a depth that increased with the distance between the object and the occluder. The analysis of the skewness of the distributions was not consistent with a double fusion explanation, favoring an implication of occlusion geometry in da Vinci stereopsis. A simple model that includes the geometry of the scene could account for the results. In summary, the mechanism responsible to locate monocular regions in depth is not sensitive to the material properties of objects, suggesting that da Vinci stereopsis is solved at relatively early stages of disparity processing.

The role of monocularly visible regions in depth and surface perception

The mainstream of binocular vision research has long been focused on understanding how binocular disparity is used for depth perception. In recent years, researchers have begun to explore how monocular regions in binocularly viewed scenes contribute to our perception of the three-dimensional world. Here we review the field as it currently stands, with a focus on understanding the extent to which the role of monocular regions in depth perception can be understood using extant theories of binocular vision.

Solving da Vinci stereopsis with depth-edge-selective V2 cells

We propose a new model for da Vinci stereopsis based on a coarse-to-fine disparity energy computation in V1 and disparity-boundary-selective units in V2. Unlike previous work, our model contains only binocular cells, relies on distributed representations of disparity, and has a simple V1-to-V2 feedforward structure. We demonstrate with random-dot stereograms that the V2 stage of our model is able to determine the location and the eye-of-origin of monocularly occluded regions, and improve disparity map computation. We also examine a few related issues. First, we argue that since monocular regions are binocularly defined, they cannot generally be detected by monocular cells. Second, we show that our coarse-to-fine V1 model for conventional stereopsis explains double matching in Panum's limiting case. This provides computational support to the notion that the perceived depth of a monocular bar next to a binocular rectangle may not be da Vinci stereopsis per se [Gillam, B., Cook, M., & Blackburn, S. (2003). Monocular discs in the occlusion zones of binocular surfaces do not have quantitative depth--a comparison with Panum's limiting case. Perception 32, 1009-1019.]. Third, we demonstrate that some stimuli previously deemed invalid have simple, valid geometric interpretations. Our work suggests that studies of da Vinci stereopsis should focus on stimuli more general than the bar-and-rectangle type and that disparity-boundary-selective V2 cells may provide a simple physiological mechanism for da Vinci stereopsis.

An integrative model of binocular vision: a stereo model utilizing interocularly unpaired points produces both depth and binocular rivalry

Half-occluded points (visible only in one eye) are perceived at a certain depth behind the occluding surface without binocular rivalry, even though no disparity is defined at such points. Here we propose a stereo model that reconstructs 3D structures not only from disparity information of interocularly paired points but also from unpaired points. Starting with an array of depth detection cells, we introduce cells that detect unpaired points visible only in the left eye or the right eye (left and right unpaired point detection cells). They interact cooperatively with each other based on optogeometrical constraints (such as uniqueness, cohesiveness, occlusion) to recover the depth and the border of 3D objects. Since it is contradictory for monocularly visible regions to be visible in both eyes, we introduce mutual inhibition between left and right unpaired point detection cells. When input images satisfy occlusion geometry, the model outputs the depth of unpaired points properly. An interesting finding is that when we input two unmatched images, the model shows an unstable output that alternates between interpretations of monocularly visible regions for the left and the right eyes, thereby reproducing binocular rivalry. The results suggest that binocular rivalry arises from the erroneous output of a stereo mechanism that estimates the depth of half-occluded unpaired points. In this sense, our model integrates stereopsis and binocular rivalry, which are usually treated separately, into a single framework of binocular vision. There are two general theories for what the "rivals" are during binocular rivalry: the two eyes, or representations of two stimulus patterns. We propose a new hypothesis that bridges these two conflicting hypotheses: interocular inhibition between representations of monocularly visible regions causes binocular rivalry. Unlike the traditional eye theory, the level of the interocular inhibition introduced here is after binocular convergence at the stage solving the correspondence problem, and thus open to pattern-specific mechanisms.

Monocular gap stereopsis: manipulation of the outer edge disparity and the shape of the gap

A binocular stimulus that arises when two black frontal plane surfaces located at different depths have a gap between them for one eye but not for the other eye is interesting since the gap is monocular--it has no matching contours in the other eye--and yet binocular processes resolve a depth step effortlessly (Vision Research, 39, 493). In two experiments we investigate the processes and constraints underlying this depth resolution by varying the width of the solid image (the one without the gap) and the shape of the gap. The results show that the processes underlying monocular gap stereopsis can handle a situation in which the images of two surfaces in depth are effectively overlapping for one eye's view with the other eye seeing between them and that binocular depth is seen even when there is no disparity present. We also show that under ecologically appropriate conditions, depth curvature and warping can result when the monocular gap has a curved or warped edge. Both these experiments imply that the visual system responds to the ambiguity of the stimulus by adopting a minimum slant constraint.

Pre-attentive segmentation and correspondence in stereo

Traditional stereo grouping models have focused on the problem of stereo correspondence between monocular inputs. Recent physiological data revealed that the disparity selective V2 cells increase their responses when (random-dot stereograms) stimuli within their receptive fields are at or near the boundary of a depth surface. Such highlights to depth (non-luminance) edges are seemingly not computationally required for the correspondence problem. Computationally, these highlights make the boundaries of a depth surface more salient, serving pre-attentive segmentation (between depth planes) and attracting visual attention. In special cases, they enable the psychophysically observed perceptual pop-out of a target from a background of visually identical distractors at a different depth. To achieve the highlights, mutual inhibition between disparity selective cells that are tuned to the same or similar depths is required. However, such mutual inhibition would impede the computation for the correspondence problem, which requires mutual excitation between the same cells. In this work, I introduce a computational model that, I believe, is the first to address both stereo correspondence and pre-attentive stereo segmentation. The computational mechanisms in the model are based on intracortical interactions in V2. I will demonstrate that the model captures the following physiological and psychophysical phenomena: (i) depth-edge highlighting; (ii) disparity capture; (iii) pop-out; and (iv) transparency.