Image Segmentation Based on Relative Motion and Relative Disparity Cues in Topographically Organized Areas of Human Visual Cortex

Two Disparity Channels in Human Visual Cortex With Different Contrast and Blur Sensitivity

Purpose

Our goal is to describe the contrast and blur sensitivity of multiple horizontal disparity subsystems and to relate them to the contrast and spatial sensitivities of their monocular inputs.

Methods

Steady-state visual evoked potential (SSVEP) amplitudes were recorded in response to dynamic random dot stereograms (DRDSs) alternating at 2 Hz between zero disparity and varying magnitudes of crossed disparity for disparity plane and disparity grating stimuli. Half-image contrasts ranged between 2.5% and 80% and over a range of Gaussian blurs from 1.4 to 12 arcmin. Separate experiments measured contrast and blur sensitivity for the monocular half-images.

Results

The first and second harmonics disparity responses were maximal for disparity gratings and for the disparity plane condition, respectively. The first harmonic of the disparity grating response was more affected by both contrast and blur than was the second harmonic of the disparity plane response, which had higher contrast sensitivity than the first harmonic.

Conclusions

The corrugation frequency, contrast, and blur tuning of the first harmonic suggest that it reflects activity of neurons tuned to higher luminance spatial frequencies that are selective for relative disparity, whereas the second harmonic reflects the activity of neurons sensitive to absolute disparity that are driven by low monocular spatial frequencies.

Translational Relevance

SSVEPs to DRDSs provide two objective neural measures of disparity processing, the first harmonic-whose stimulus preferences are similar to those of behavioral stereoacuity-and the second harmonic that represents an independent disparity-specific but not necessarily stereoscopic mechanism.

The role of binocular disparity in the neural representation of multiple moving stimuli in the visual cortex

A fundamental process of vision is to segment visual scenes into distinct objects and surfaces. Stereoscopic depth and visual motion cues are particularly important for segmentation. However, how the primate visual system uses depth and motion cues to segment multiple surfaces in 3-dimensional space is not well understood. We investigated how neurons in the middle temporal (MT) cortex represented two overlapping surfaces located at different depths and moved simultaneously in different directions. We recorded neuronal activities in MT of three male macaque monkeys while they performed discrimination tasks under different attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the horizontal disparity of one of the two surfaces. For all animals, the disparity bias in response to two surfaces was positively correlated with the disparity preference of the neurons to single surfaces. For two animals, neurons that preferred the near disparities of single surfaces (near neurons) showed a near bias to overlapping stimuli, and neurons that preferred the far disparities (far neurons) showed a far bias. For the third animal, both near and far neurons showed a near bias, although the near neurons showed a stronger near bias than the far neurons. Interestingly, for all three animals, both near and far neurons showed an initial near bias relative to the average of the responses to the individual surfaces. Although attention can modulate neuronal response to better represent the attended surface, the disparity bias was still present when attention was directed away from the visual stimuli, indicating that the disparity bias cannot be explained by an attention bias. We also found that the effect of attention modulation of MT responses was consistent with object-based rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. Our model is a novel extension of the standard normalization model and provides a unified explanation of the disparity bias across animals. Our results revealed the neural encoding rule for multiple moving stimuli located at different depths and showed new evidence of response modulation by object-based attention in MT. The disparity bias would allow subgroups of neurons to preferentially represent individual surfaces at different depths of multiple stimuli and therefore facilitate segmentation. Attention can further select a surface and enhance its neural representation.

Cognitive penetrability of scene representations based on horizontal image disparities

The structure of natural scenes is signaled by many visual cues. Principal amongst them are the binocular disparities created by the laterally separated viewpoints of the two eyes. Disparity cues are believed to be processed hierarchically, first in terms of local measurements of absolute disparity and second in terms of more global measurements of relative disparity that allow extraction of the depth structure of a scene. Psychophysical and oculomotor studies have suggested that relative disparities are particularly relevant to perception, whilst absolute disparities are not. Here, we compare neural responses to stimuli that isolate the absolute disparity cue with stimuli that contain additional relative disparity cues, using the high temporal resolution of EEG to determine the temporal order of absolute and relative disparity processing. By varying the observers' task, we assess the extent to which each cue is cognitively penetrable. We find that absolute disparity is extracted before relative disparity, and that task effects arise only at or after the extraction of relative disparity. Our results indicate a hierarchy of disparity processing stages leading to the formation of a proto-object representation upon which higher cognitive processes can act.

Dynamics of absolute and relative disparity processing in human visual cortex

Cortical processing of binocular disparity is believed to begin in V1 where cells are sensitive to absolute disparity, followed by the extraction of relative disparity in higher visual areas. While much is known about the cortical distribution and spatial tuning of disparity-selective neurons, the relationship between their spatial and temporal properties is less well understood. Here, we use steady-state Visual Evoked Potentials and dynamic random dot stereograms to characterize the temporal dynamics of spatial mechanisms in human visual cortex that are primarily sensitive to either absolute or relative disparity. Stereograms alternated between disparate and non-disparate states at 2 Hz. By varying the disparity-defined spatial frequency content of the stereograms from a planar surface to corrugated ones, we biased responses towards absolute vs. relative disparities. Reliable Components Analysis was used to derive two dominant sources from the 128 channel EEG records. The first component (RC1) was maximal over the occipital pole. In RC1, first harmonic responses were sustained, tuned for corrugation frequency, and sensitive to the presence of disparity references, consistent with prior psychophysical sensitivity measurements. By contrast, the second harmonic, associated with transient processing, was not spatially tuned and was indifferent to references, consistent with it being generated by an absolute disparity mechanism. Thus, our results reveal a duplex coding strategy in the disparity domain, where relative disparities are computed via sustained mechanisms and absolute disparities are computed via transient mechanisms.

Associations Between Binocular Depth Perception and Performance Gains in Laparoscopic Skill Acquisition

The ability to perceive differences in depth is important in many daily life situations. It is also of relevance in laparoscopic surgical procedures that require the extrapolation of three-dimensional visual information from two-dimensional planar images. Besides visual-motor coordination, laparoscopic skills and binocular depth perception are demanding visual tasks for which learning is important. This study explored potential relations between binocular depth perception and individual variations in performance gains during laparoscopic skill acquisition in medical students naïve of such procedures. Individual differences in perceptual learning of binocular depth discrimination when performing a random dot stereogram (RDS) task were measured as variations in the slope changes of the logistic disparity psychometric curves from the first to the last blocks of the experiment. The results showed that not only did the individuals differ in their depth discrimination; the extent with which this performance changed across blocks also differed substantially between individuals. Of note, individual differences in perceptual learning of depth discrimination are associated with performance gains from laparoscopic skill training, both with respect to movement speed and an efficiency score that considered both speed and precision. These results indicate that learning-related benefits for enhancing demanding visual processes are, in part, shared between these two tasks. Future studies that include a broader selection of task-varying monocular and binocular cues as well as visual-motor coordination are needed to further investigate potential mechanistic relations between depth perceptual learning and laparoscopic skill acquisition. A deeper understanding of these mechanisms would be important for applied research that aims at designing behavioral interventions for enhancing technology-assisted laparoscopic skills.

Disparity in Context: Understanding how monocular image content interacts with disparity processing in human visual cortex

Horizontal disparities between the two eyes' retinal images are the primary cue for depth. Commonly used random ot tereograms (RDS) intentionally camouflage the disparity cue, breaking the correlations between monocular image structure and the depth map that are present in natural images. Because of the nonlinear nature of visual processing, it is unlikely that simple computational rules derived from RDS will be sufficient to explain binocular vision in natural environments. In order to understand the interplay between natural scene structure and disparity encoding, we used a depth-image-based-rendering technique and a library of natural 3D stereo pairs to synthesize two novel stereogram types in which monocular scene content was manipulated independent of scene depth information. The half-images of the novel stereograms comprised either random-dots or scrambled natural scenes, each with the same depth maps as the corresponding natural scene stereograms. Using these stereograms in a simultaneous Event-Related Potential and behavioral discrimination task, we identified multiple disparity-contingent encoding stages between 100 ~ 500 msec. The first disparity sensitive evoked potential was observed at ~100 msec after an earlier evoked potential (between ~50-100 msec) that was sensitive to the structure of the monocular half-images but blind to disparity. Starting at ~150 msec, disparity responses were stereogram-specific and predictive of perceptual depth. Complex features associated with natural scene content are thus at least partially coded prior to disparity information, but these features and possibly others associated with natural scene content interact with disparity information only after an intermediate, 2D scene-independent disparity processing stage.