Learning to see in depth

Distinct rich and diverse clubs regulate coarse and fine binocular disparity processing: Evidence from stereoscopic task-based fMRI

While cortical regions involved in processing binocular disparities have been studied extensively, little is known on how the human visual system adapts to changing disparity magnitudes. In this paper, we investigate causal mechanisms of coarse and fine binocular disparity processing using fMRI with a clinically validated, custom anaglyph-based stimulus. We make use of Granger causality and graph measures to reveal the existence of distinct rich and diverse clubs across different disparity magnitudes. We demonstrate that Middle Temporal area (MT) plays a specialized role with overlapping rich and diverse characteristics. Next, we show that subtle interhemispheric differences exist across various brain regions, despite an overall right hemisphere dominance. Finally, we pass the graph measures through the decision tree and found that the diverse clubs outperform rich clubs in decoding disparity magnitudes. Our study sets the stage for conducting further investigations on binocular disparity processing, particularly in the context of neuro-ophthalmic disorders with binocular impairments.

Binocular receptive-field construction in the primary visual cortex

ON and OFF thalamic afferents from the two eyes converge in the primary visual cortex to form binocular receptive fields. The receptive fields need to be diverse to sample our visual world but also similar across eyes to achieve binocular fusion. It is currently unknown how the cortex balances these competing needs between receptive-field diversity and similarity. Our results demonstrate that receptive fields in the cat visual cortex are binocularly matched with exquisite precision for retinotopy, orientation/direction preference, orientation/direction selectivity, response latency, and ON-OFF polarity/structure. Specifically, the average binocular mismatches in retinotopy and ON-OFF structure are tightly restricted to 1/20 and 1/5 of the average receptive-field size but are still large enough to generate all types of binocular disparity tuning. Based on these results, we conclude that cortical receptive fields are binocularly matched with the high precision needed to facilitate binocular fusion while allowing restricted mismatches to process visual depth.

Toward a theory of perspective perception in pictures

This paper reviews projection models and their perception in realistic pictures, and proposes hypotheses for three-dimensional (3D) shape and space perception in pictures. In these hypotheses, eye fixations, and foveal vision play a central role. Many past theories and experimental studies focus solely on linear perspective. Yet, these theories fail to explain many important perceptual phenomena, including the effectiveness of nonlinear projections. Indeed, few classical paintings strictly obey linear perspective, nor do the best distortion-avoidance techniques for wide-angle computational photography. The hypotheses here employ a two-stage model for 3D human vision. When viewing a picture, the first stage perceives 3D shape for the current gaze. Each fixation has its own perspective projection, but, owing to the nature of foveal and peripheral vision, shape information is obtained primarily for a small region of the picture around the fixation. As a viewer moves their eyes, the second stage continually integrates some of the per-gaze information into an overall interpretation of a picture. The interpretation need not be geometrically stable or consistent over time. It is argued that this framework could explain many disparate pictorial phenomena, including different projection styles throughout art history and computational photography, while being consistent with the constraints of human 3D vision. The paper reviews open questions and suggests new studies to explore these hypotheses.

Quantifying the Relationship Between Unilateral Induced Metamorphopsia and Stereopsis Impairment

Purpose

To investigate the relationship between unilateral metamorphopsia, characterized by visual distortions in one eye, and impaired stereopsis.

Methods

Utilizing both near and distance measurements through advanced testing systems, including 4K smartphones and an active shutter three-dimensional system, we simulated varying degrees of unilateral metamorphopsia in 30 healthy young adults aged between 21 and 29 years. Two types of contour-based stereotest symbols, lines and squares, were developed. Distortions were classified into six distinct patterns, each further divided into eight grades of severity. Participants were tasked with identifying visual targets, and their stereothresholds were determined under different conditions of induced distortion. Stereopsis was measured within a range of 2.9 to 1.0 log arcsec, at 0.2 log arcsec intervals. Stereopsis changes under different distortion scenarios were analyzed using the generalized estimating equations, with a sequential Bonferroni adjustment applied for pairwise comparisons.

Results

A direct and quantifiable correlation was observed between the severity of metamorphopsia and reductions in stereopsis. As the degree of visual distortion increased, notably in both frequency and amplitude, there was a corresponding decline in stereopsis. This relationship held true in both near and distance measurements of stereopsis. Statistical analyses further reinforced these findings, highlighting a significant detrimental effect of distortion components on stereoacuity.

Conclusions

The findings highlight the clinical significance of understanding the interplay between unilateral metamorphopsia and stereopsis. Early interventions in conditions leading to metamorphopsia might be critical to maintaining optimal stereopsis.

New Approaches to 3D Vision

New approaches to 3D vision are enabling new advances in artificial intelligence and autonomous vehicles, a better understanding of how animals navigate the 3D world, and new insights into human perception in virtual and augmented reality. Whilst traditional approaches to 3D vision in computer vision (SLAM: simultaneous localization and mapping), animal navigation (cognitive maps), and human vision (optimal cue integration) start from the assumption that the aim of 3D vision is to provide an accurate 3D model of the world, the new approaches to 3D vision explored in this issue challenge this assumption. Instead, they investigate the possibility that computer vision, animal navigation, and human vision can rely on partial or distorted models or no model at all. This issue also highlights the implications for artificial intelligence, autonomous vehicles, human perception in virtual and augmented reality, and the treatment of visual disorders, all of which are explored by individual articles. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Applications and implications for extended reality to improve binocular vision and stereopsis

Extended reality (XR) devices, including virtual reality (VR), augmented reality (AR), and mixed reality (MR) devices, are immersive technologies that can swap or merge the natural environment with virtual content (e.g., videogames, movies, or other content). Although these devices are widely used for playing videogames and other applications, they have one distinct feature that makes them potentially very useful for the measurement and treatment of binocular vision anomalies-they can deliver different content to the two eyes simultaneously. Indeed, horizontally shifting the images in the two eyes (thereby creating binocular disparity) can provide the user with a compelling percept of depth through stereopsis. Because these devices are stereoscopic, they can also be used as high-tech synoptophores, in which the images to the two eyes differ in contrast, luminance, size, position, and content for measuring and treating binocular anomalies. The inclusion of eye tracking in VR adds an additional dimension to its utility in measuring and treating binocular vision anomalies, as well as other conditions. This paper describes the essential requirements for testing and treating binocular anomalies and reviews current studies in which XR devices have been used to measure and treat binocular vision anomalies.