Encoding and estimation of first- and second-order binocular disparity in natural images

Initial postoperative plasticity as a predictor of mid-term stereoacuity outcome after surgery for intermittent exotropia

BACKGROUND

Intermittent exotropia (IXT) would cause different degrees of damage to stereopsis. We aimed to introduce a visual perception plasticity score (VPPS) that reflects initial postoperative plasticity and evaluate its effectiveness in predicting the mid-term surgical outcome in IXT patients.

METHODS

A total of 149 patients with intermittent exotropia who underwent surgery in November 2018 and October 2019 were recruited. All subjects underwent detailed ocular examinations before and after surgery. VPPS were calculated based on visual perception examination system at one week postoperatively. Demographic, angle of deviation and stereopsis were collected and analyzed with regard to the VPPSs preoperatively and at one week, one month, three months, six months postoperatively. Predictive performances of VPPS were assessed using receiver operating characteristic (ROC) curves, the area under the curve (AUC) and cut-offs were obtained.

RESULTS

Of the 149 patients, the average deviation was 43^Δ at distance and 46^Δ at near. The average rate of normal stereopsis before surgery was 22.81% at distance and 29.53% at near. Higher VPPS was associated with preoperative better near stereoacuity (r = 0.362, p = 0.000), less angle of deviation at distance (r=-0.164, p = 0.046), and better near (r = 0.400, p = 0.000) and distant stereoacuity (r = 0.321, p = 0.000) during the early postoperative period (1 week). The areas under the curves suggested that VPPS could be an effective predictor of sensory outcome(AUC>0.6). Cut-off values of 50 and 80 were calculated for VPPS using ROC curve analysis.

CONCLUSION

Higher VPPSs were associated with a greater possibility of stereopsis improvement in patients with IXT. VPPS is a potentially promising indicator to predict the mid-term surgical outcome of intermittent exotropia.

Robust natural depth for anticorrelated random dot stereogram for edge stimuli, but minimal reversed depth for embedded circular stimuli, irrespective of eccentricity

The small differences between the images formed in our left and right eyes are an important cue to the three-dimensional structure of scenes. These disparities are encoded by binocular neurons in the visual cortex. At the earliest stage of processing, these respond to binocular correlation between images. We assessed the perception of depth in anticorrelated stimuli, in which the contrast polarity in one eye is reversed, as a function of their location in the retinal image, and their depth configuration (a horizontal edge or a circle surrounded by an annulus) We found that, regardless of stimulus eccentricity, participants perceived depth in the natural direction for edge stimuli, and weakened, reversed depth for circular stimuli.

A unified model for binocular fusion and depth perception

We describe a new unified model to explain both binocular fusion and depth perception, over a broad range of depths. At each location, the model consists of an array of paired spatial frequency filters, with different relative horizontal shifts (position disparity) and interocular phase disparities of 0, 90, ±180, or -90°. The paired filters with different spatial profiles (non-zero phase disparity) compute interocular misalignment and provide phase-disparity energy (binocular fusion energy) to drive selection of the appropriate filters along the position disparity space until the misalignment is eliminated and sensory fusion is achieved locally. The paired filters with identical spatial profiles (0 phase disparity) compute the position-disparity energy. After sensory fusion, the combination of position and possible residual phase disparity energies is calculated for binocular depth perception. Binocular fusion occurs at multiple scales following a coarse-to-fine process. At a given location, the apparent depth is the weighted sum of fusion shifts combined with residual phase disparity in all spatial-frequency channels, and the weights depend on stimulus spatial frequency and stimulus contrast. To test the theory, we measured disparity minimum and maximum thresholds (Dmin and Dmax) at three spatial frequencies and with different intraocular contrast levels. The stimuli were Random-Gabor-Patch (RGP) stereograms consisting of Gabor patches with random positions and phases, but with a fixed spatial frequency. The two eyes viewed identical arrays of patches except that one eye's array could be shifted horizontally and could differ in contrast. Our experiments and modeling reveal two contrast normalization mechanisms: (1) Energy Normalization (EN): Binocular energy is normalized with monocular energy after the site of binocular combination. This predicts constant Dmin thresholds when varying stimulus contrast in the two eyes; (2) DSKL model Interocular interactions: Monocular contrasts are normalized before the binocular combination site through interocular contrast gain-control and gain-enhancement mechanisms. This predicts contrast dependent Dmax thresholds. We tested a range of models and found that a model consisting of a second-order pathway with DSKL interocular interactions and a first-order pathway with EN at each spatial-frequency band can account for both the Dmin and Dmax data very well. Simulations show that the model makes reasonable predictions of suprathreshold depth perception.

Impairment of cyclopean surface processing by disparity-defined masking stimuli

Binocular disparity signals allow for the estimation of three-dimensional shape, even in the absence of monocular depth cues. The perception of such disparity-defined form depends, however, on the linkage of multiple disparity measurements over space. Performance limitations in cyclopean tasks thus inform us about errors arising in disparity measurement and difficulties in the linkage of such measurements. We used a cyclopean orientation discrimination task to examine the perception of disparity-defined form. Participants were presented with random-dot sinusoidal modulations in depth and asked to report whether they were clockwise or counter-clockwise rotated. To assess the effect of different noise structures on measurement and linkage processes, task performance was measured in the presence of binocular, random-dot masks, structured as either antiphase depth sinusoids, or as random distributions of dots in depth. For a fixed number of surface dots, the ratio of mask-to-surface dots was varied to obtain thresholds for orientation discrimination. Antiphase masks were found to be more effective than random depth masks, requiring a lower mask-to-surface dot ratio to inhibit performance. For antiphase masks, performance improved with decreased cyclopean frequency, increased disparity amplitude, and/or an increase in the total number of stimulus dots. Although a cross-correlation model of disparity measurement could account for antiphase mask performance, random depth masking effects were consistent with limitations in relative disparity processing. This suggests that performance is noise-limited for antiphase masks and complexity-limited for random masks. We propose that use of differing mask types may prove effective in understanding these distinct forms of impairment.

Near-optimal combination of disparity across a log-polar scaled visual field

The human visual system is foveated: we can see fine spatial details in central vision, whereas resolution is poor in our peripheral visual field, and this loss of resolution follows an approximately logarithmic decrease. Additionally, our brain organizes visual input in polar coordinates. Therefore, the image projection occurring between retina and primary visual cortex can be mathematically described by the log-polar transform. Here, we test and model how this space-variant visual processing affects how we process binocular disparity, a key component of human depth perception. We observe that the fovea preferentially processes disparities at fine spatial scales, whereas the visual periphery is tuned for coarse spatial scales, in line with the naturally occurring distributions of depths and disparities in the real-world. We further show that the visual system integrates disparity information across the visual field, in a near-optimal fashion. We develop a foveated, log-polar model that mimics the processing of depth information in primary visual cortex and that can process disparity directly in the cortical domain representation. This model takes real images as input and recreates the observed topography of human disparity sensitivity. Our findings support the notion that our foveated, binocular visual system has been moulded by the statistics of our visual environment.

Binocular Matching Model Based on Hierarchical V1 and V2 Receptive Fields With Color, Orientation, and Region Feature Information

Binocular matching models serve as the core component in most stereo visual aid systems developed for people with visual impairments. However, purely computational models lack a neuro-biological basis for explaining the phenomena observed in neuro-biology, and therefore offer no support for the development of bioengineering applications, and are overly complex for hardware implementation. In contrast, existing neurobiological models suffer from low matching calculation accuracy. Therefore, the present work proposes a novel binocular matching model based on the receptive field of simple cells rather than on image pixels, and thereby incorporates neurobiological structure, reduces hardware complexity, has enough accuracy and can be used in visual aid system. The proposed model is employed to calculate and optimize the binocular disparity via a cost function. Specifically, we simulate the functions and structures of V1 and V2 neurons according to the discoveries of modern neurobiology. Accordingly, the receptive fields of V1 layer neurons are aggregated to obtain the receptive fields of the V2 layer, and the disparity is obtained in the V2 layer. The accuracy of the proposed model is verified by comparisons of the disparity results obtained using the proposed model with those obtained using other neurobiological model, and thereby demonstrates that the model can guide the design of visual aid systems.

First- and second-order contributions to depth perception in anti-correlated random dot stereograms

The binocular energy model of neural responses predicts that depth from binocular disparity might be perceived in the reversed direction when the contrast of dots presented to one eye is reversed. While reversed-depth has been found using anti-correlated random-dot stereograms (ACRDS) the findings are inconsistent across studies. The mixed findings may be accounted for by the presence of a gap between the target and surround, or as a result of overlap of dots around the vertical edges of the stimuli. To test this, we assessed whether (1) the gap size (0, 19.2 or 38.4 arc min) (2) the correlation of dots or (3) the border orientation (circular target, or horizontal or vertical edge) affected the perception of depth. Reversed-depth from ACRDS (circular no-gap condition) was seen by a minority of participants, but this effect reduced as the gap size increased. Depth was mostly perceived in the correct direction for ACRDS edge stimuli, with the effect increasing with the gap size. The inconsistency across conditions can be accounted for by the relative reliability of first- and second-order depth detection mechanisms, and the coarse spatial resolution of the latter.

Binocular Depth Judgments on Smoothly Curved Surfaces

Binocular disparity is an important cue to depth, allowing us to make very fine discriminations of the relative depth of objects. In complex scenes, this sensitivity depends on the particular shape and layout of the objects viewed. For example, judgments of the relative depths of points on a smoothly curved surface are less accurate than those for points in empty space. It has been argued that this occurs because depth relationships are represented accurately only within a local spatial area. A consequence of this is that, when judging the relative depths of points separated by depth maxima and minima, information must be integrated across separate local representations. This integration, by adding more stages of processing, might be expected to reduce the accuracy of depth judgements. We tested this idea directly by measuring how accurately human participants could report the relative depths of two dots, presented with different binocular disparities. In the first, Two Dot condition the two dots were presented in front of a square grid. In the second, Three Dot condition, an additional dot was presented midway between the target dots, at a range of depths, both nearer and further than the target dots. In the final, Surface condition, the target dots were placed on a smooth surface defined by binocular disparity cues. In some trials, this contained a depth maximum or minimum between the target dots. In the Three Dot condition, performance was impaired when the central dot was presented with a large disparity, in line with predictions. In the Surface condition, performance was worst when the midpoint of the surface was at a similar distance to the targets, and relatively unaffected when there was a large depth maximum or minimum present. These results are not consistent with the idea that depth order is represented only within a local spatial area.