The (In)Effectiveness of Simulated Blur for Depth Perception in Naturalistic Images

Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technology

Stereoscopic imagery often aims to evoke three-dimensional (3-D) percepts that are accurate and realistic-looking. The "gap" between 3-D imagery and real scenes is small, but focus cues typically remain incorrect because images are displayed on a single focal plane. Research has concentrated on the resulting vergence-accommodation conflicts. Yet, incorrect focus cues may also affect the appearance of 3-D imagery. We investigated whether incorrect focus cues reduce perceived realism of 3-D structure ("depth realism"). Experiment 1 used a multiple-focal-planes display to compare depth realism with correct focus cues vs. conventional stereo presentation. The stimuli were random-dot stereograms, which isolated the role of focus cues. Depth realism was consistently lower with incorrect focus cues, providing proof-of-principle evidence that they contribute to perceptual realism. Experiments 2 and 3 examined whether focus cues play a similar role with realistic objects, presented with an almost complete set of visual cues using a high-resolution, high-dynamic-range multiple-focal-planes display. We also examined the efficacy of approximating correct focus cues via gaze-contingent depth-of-field rendering. Improvements in depth realism with correct focus cues were less clear in more realistic scenes, indicating that the role of focus cues in depth realism depends on scene content. Rendering-based approaches, if anything, reduced depth realism, which we attribute to their inability to present higher-order aspects of blur correctly. Our findings suggest future general 3-D display solutions may need to present focus cues correctly to maximise perceptual realism.

Despite Impaired Binocular Function, Binocular Disparity Integration Across the Visual Field Is Spared in Normal Aging and Glaucoma

Purpose

To examine how binocularly asymmetric glaucomatous visual field damage affects binocular disparity processing across the visual field.

Methods

We recruited 18 patients with primary open-angle glaucoma, 16 age-matched controls, and 13 young controls. Participants underwent standard clinical assessments of binocular visual acuity, binocular contrast sensitivity, stereoacuity, and perimetry. We employed a previously validated psychophysical procedure to measure how sensitivity to binocular disparity varied across spatial frequencies and visual field sectors (i.e., with full-field stimuli spanning the central 21° of the visual field and with stimuli restricted to annular regions spanning 0°-3°, 3°-9°, or 9°-21°). We employed measurements with annular stimuli to model different possible scenarios regarding how disparity information is combined across visual field sectors. We adjudicated between potential mechanisms by comparing model predictions to the patterns observed with full-field stimuli.

Results

Perimetry confirmed that patients with glaucoma exhibited binocularly asymmetric visual field damage (P < 0.001). Across participant groups, foveal regions preferentially processed disparities at finer spatial scales, whereas periphery regions were tuned for coarser scales (P < 0.001). Disparity sensitivity also decreased from fovea to periphery (P < 0.001) and across participant groups (Ps < 0.01). Finally, similar to controls, patients with glaucoma exhibited near-optimal disparity integration, specifically at low spatial frequencies (P < 0.001).

Conclusions

Contrary to the conventional view that glaucoma spares central vision, we find that glaucomatous damage causes a widespread loss of disparity sensitivity across both foveal and peripheral regions. Despite these losses, cortical integration mechanisms appear to be well preserved, suggesting that patients with glaucoma make the best possible use of their remaining binocular function.

Blurring the boundary between models and reality: Visual perception of scale assessed by performance

One of the primary jobs of visual perception is to build a three-dimensional representation of the world around us from our flat retinal images. These are a rich source of depth cues but no single one of them can tell us about scale (i.e., absolute depth and size). For example, the pictorial depth cues in a (perfect) scale model are identical to those in the real scene that is being modelled. Here we investigate image blur gradients, which derive naturally from the limited depth of field available for any optical device and can be used to help estimate visual scale. By manipulating image blur artificially to produce what is sometimes called fake tilt shift miniaturization, we provide the first performance-based evidence that human vision uses this cue when making forced-choice judgements about scale (identifying which of an image pair was a photograph of a full-scale railway scene, and which was a 1:76 scale model). The orientation of the blur gradient (relative to the ground plane) proves to be crucial, though its rate of change is less important for our task, suggesting a fairly coarse visual analysis of this image parameter.

When virtual and real worlds coexist: Visualization and visual system affect spatial performance in augmented reality

New visualization approaches are being actively developed aiming to mitigate the effect of vergence-accommodation conflict in stereoscopic augmented reality; however, high interindividual variability in spatial performance makes it difficult to predict user gain. To address this issue, we investigated the effects of consistent and inconsistent binocular and focus cues on perceptual matching in the stereoscopic environment of augmented reality using a head-mounted display that was driven in multifocal and single focal plane modes. Participants matched the distance of a real object with images projected at three viewing distances, concordant with the display focal planes when driven in the multifocal mode. As a result, consistency of depth cues facilitated faster perceptual judgments on spatial relations. Moreover, the individuals with mild binocular and accommodative disorders benefited from the visualization of information on the focal planes corresponding to image planes more than individuals with normal vision, which was reflected in performance accuracy. Because symptoms and complaints may be absent when the functionality of the sensorimotor system is reduced, the results indicate the need for a detailed assessment of visual functions in research on spatial performance. This study highlights that the development of a visualization system that reduces visual stress and improves user performance should be a priority for the successful implementation of augmented reality displays.

Factors affecting depth perception and comparison of depth perception measured by the three-rods test in monocular and binocular vision

Purpose

The purpose of this study was to explore the effects of factors affecting depth perception of moving objects using a modified three-rods test, which can be used at longer distances than the conventional one, and to compare differences in the results between binocular and monocular vision.

Methods

This study included 24 volunteers (10 women, 14 men; mean age, 35.2 years; standard deviation, 6.8 years; range, 22–56 years). We measured depth perception using a modified three-rods test under eight different conditions and investigated the factors affecting depth perception using a linear-effect model.

Results

The results identified test distance, binocularity, masking, and direction of movement as significant factors affecting depth perception of a moving object.

Conclusions

The current study successfully determined factors affecting depth perception using the three-rods test with a moving object and the results should contribute to further clinical and social applications of the three-rods test.

Near-optimal combination of disparity across a log-polar scaled visual field

The human visual system is foveated: we can see fine spatial details in central vision, whereas resolution is poor in our peripheral visual field, and this loss of resolution follows an approximately logarithmic decrease. Additionally, our brain organizes visual input in polar coordinates. Therefore, the image projection occurring between retina and primary visual cortex can be mathematically described by the log-polar transform. Here, we test and model how this space-variant visual processing affects how we process binocular disparity, a key component of human depth perception. We observe that the fovea preferentially processes disparities at fine spatial scales, whereas the visual periphery is tuned for coarse spatial scales, in line with the naturally occurring distributions of depths and disparities in the real-world. We further show that the visual system integrates disparity information across the visual field, in a near-optimal fashion. We develop a foveated, log-polar model that mimics the processing of depth information in primary visual cortex and that can process disparity directly in the cortical domain representation. This model takes real images as input and recreates the observed topography of human disparity sensitivity. Our findings support the notion that our foveated, binocular visual system has been moulded by the statistics of our visual environment.

AR Displays: Next-Generation Technologies to Solve the Vergence–Accommodation Conflict

Augmenting reality (AR) holds many benefits in how people perceive information and use it in their workflow or leisure activities. A cohesive AR experience has many components; nevertheless, the key is display technologies. The current industry standard for the core solution is still conventional stereoscopy, which has proven to be inadequate for near-work due to the caused vergence–accommodation conflict and the inability to precisely overlay the 3D content on the real world. To overcome this, next-generation technologies have been proposed. While the holographic method holds the highest potential of being the ultimate solution, its current level of maturity is not sufficient to yield a practical product. Consequently, the next solution for near-work-capable AR displays will be of another type. LightSpace Technologies have developed a static multifocal display architecture based on stacked liquid crystal-based optical diffuser elements and a synchronized high-refresh rate image projector. A stream of 2D image depth planes comprising a 3D scene is projected onto respective physically-separated diffuser elements, causing the viewer to perceive a scene as continuous and having all relevant physical as well as psychological depth cues. A system with six image depth planes yielding 6 cpd resolution and 72° horizontal field-of-view has been demonstrated to provide perceptually continuous accommodation over 3.2 Diopter range. A further optimization by using a conventional image combiner resulted in the compact and practical design of the AR display.

Defocus Discrimination in Video: Motion in Depth

We perform two psychophysics experiments to investigate a viewer's ability to detect defocus in video; in particular, the defocus that arises in video during motion in depth when the camera does not maintain sharp focus throughout the motion. The first experiment demonstrates that blur sensitivity during viewing is affected by the speed at which the target moves towards the camera. The second experiment measures a viewer's ability to notice momentary defocus and shows that the threshold of blur detection in arc minutes decreases significantly as the duration of the blur increases. Our results suggest that it is important to have good control of focus while recording video and that momentary defocus should be kept as short as possible so it goes unnoticed.

Vergence driven accommodation with simulated disparity in myopia and emmetropia

The formation of focused and corresponding foveal images requires a close synergy between the accommodation and vergence systems. This linkage is usually decoupled in virtual reality systems and may be dysfunctional in people who are at risk of developing myopia. We study how refractive error affects vergence-accommodation interactions in stereoscopic displays. Vergence and accommodative responses were measured in 21 young healthy adults (n=9 myopes, 22-31 years) while subjects viewed naturalistic stimuli on a 3D display. In Step 1, vergence was driven behind the monitor using a blurred, non-accommodative, uncrossed disparity target. In Step 2, vergence and accommodation were driven back to the monitor plane using naturalistic images that contained structured depth and focus information from size, blur and/or disparity. In Step 1, both refractive groups converged towards the stereoscopic target depth plane, but the vergence-driven accommodative change was smaller in emmetropes than in myopes (F_1,19=5.13, p=0.036). In Step 2, there was little effect of peripheral depth cues on accommodation or vergence in either refractive group. However, vergence responses were significantly slower (F_1,19=4.55, p=0.046) and accommodation variability was higher (F_1,19=12.9, p=0.0019) in myopes. Vergence and accommodation responses are disrupted in virtual reality displays in both refractive groups. Accommodation responses are less stable in myopes, perhaps due to a lower sensitivity to dioptric blur. Such inaccuracies of accommodation may cause long-term blur on the retina, which has been associated with a failure of emmetropization.

Blur perception throughout the visual field in myopia and emmetropia

We evaluated the ability of emmetropic and myopic observers to detect and discriminate blur across the retina under monocular or binocular viewing conditions. We recruited 39 young (23-30 years) healthy adults (n = 19 myopes) with best-corrected visual acuity 0.0 LogMAR (20/20) or better in each eye and no binocular or accommodative dysfunction. Monocular and binocular blur discrimination thresholds were measured as a function of pedestal blur using naturalistic stimuli with an adaptive 4AFC procedure. Stimuli were presented in a 46° diameter window at 40 cm. Gaussian blur pedestals were confined to an annulus at either 0°, 4°, 8°, or 12° eccentricity, with a blur increment applied to only one quadrant of the image. The adaptive procedure efficiently estimated a dipper shaped blur discrimination threshold function with two parameters: intrinsic blur and blur sensitivity. The amount of intrinsic blur increased for retinal eccentricities beyond 4° (p < 0.001) and was lower in binocular than monocular conditions (p < 0.001), but was similar across refractive groups (p = 0.47). Blur sensitivity decreased with retinal eccentricity (p < 0.001) and was highest for binocular viewing, but only for central vision (p < 0.05). Myopes showed worse blur sensitivity than emmetropes monocularly (p < 0.05) but not binocularly (p = 0.66). As expected, blur perception worsens in the visual periphery and binocular summation is most evident in central vision. Furthermore, myopes exhibit a monocular impairment in blur sensitivity that improves under binocular conditions. Implications for the development of myopia are discussed.

Effect of apparent depth cues on accommodation in a Badal optometer

BACKGROUND

The aim was to analyse the effect of peripheral depth cues on accommodation in Badal optometers.

METHODS

Monocular refractions at 0.17 and 5.00 D of accommodative stimulus were measured with the PowerRef II autorefractor (Plusoptix Inc., Atlanta, Georgia, USA). Subjects looked (randomly) at four different scenes, one real scene comprising familiar objects at different depth planes (Real) and three virtual scenes comprising different two-dimensional pictures seen through a Badal lens. The first image consisted of a photograph of the real scene taken in conditions that closely mimic a healthy standard human eye performance (out-of-focus [OoF] blur); the second image was the same photograph rendered with a depth of focus to infinity (OoF sharpness); and finally the third image consisted of a fixation target and a even white surrounding (White). In all cases the field of view was 25.0° and the fixation target was a Maltese cross subtending to two degrees.

RESULTS

Twenty-eight right eyes from healthy young subjects were measured. The achieved statistical power was 0.9. At 5.00 D of accommodative stimulus, the repeated measures analysis of variance was statistically significant (p < 0.05) and the corresponding Bonferroni post hoc tests showed the following mean accommodative response differences and standard deviation (p-value) between the real and the virtual scenes: real-white =-0.66 ± 0.92 D (p < 0.01); real-OoF sharpness = -0.43 ± 0.88 D (p = 0.07); real-OoF blur =-0.25 ± 0.93 D (p = 0.89).

CONCLUSIONS

A stimulus poor in depth cues inaccurately stimulates accommodation in Badal optometers; however, accommodation can be significantly improved in the same Badal optometer, when displaying a realistic image rich in peripheral depth cues, even though these peripheral cues (also referred to as retinal blur cues) are shown in the same plane as the fixation target. These results have important implications in stereoscopic virtual reality systems that fail to represent appropriately retinal blur.

The natural statistics of blur

Blur from defocus can be both useful and detrimental for visual perception: It can be useful as a source of depth information and detrimental because it degrades image quality. We examined these aspects of blur by measuring the natural statistics of defocus blur across the visual field. Participants wore an eye-and-scene tracker that measured gaze direction, pupil diameter, and scene distances as they performed everyday tasks. We found that blur magnitude increases with increasing eccentricity. There is a vertical gradient in the distances that generate defocus blur: Blur below the fovea is generally due to scene points nearer than fixation; blur above the fovea is mostly due to points farther than fixation. There is no systematic horizontal gradient. Large blurs are generally caused by points farther rather than nearer than fixation. Consistent with the statistics, participants in a perceptual experiment perceived vertical blur gradients as slanted top-back whereas horizontal gradients were perceived equally as left-back and right-back. The tendency for people to see sharp as near and blurred as far is also consistent with the observed statistics. We calculated how many observations will be perceived as unsharp and found that perceptible blur is rare. Finally, we found that eye shape in ground-dwelling animals conforms to that required to put likely distances in best focus.