Image Segmentation Based on Relative Motion and Relative Disparity Cues in Topographically Organized Areas of Human Visual Cortex

Characterising human disparity tuning properties using population receptive field mapping

Our visual percept of small differences in depth is largely informed by binocular stereopsis, the ability to decode depth from the horizontal offset between the retinal images in each eye. While multiple cortical areas are associated with stereoscopic processing, it is unclear how tuning to specific binocular disparities is organised across human visual cortex. We used 3T functional magnetic resonance imaging to generate population receptive fields in response to modulation of binocular disparity to characterise the neural tuning to disparity. We also used psychophysics to measure stereoacuity thresholds compared to backgrounds at different depths (pedestal disparity). Ten human participants (7 female) observed correlated or anticorrelated random-dot stereograms with disparity ranging from -0.3° to 0.3°, and responses were modelled as 1-dimensional tuning curves along the depth dimension. First, we demonstrate that lateral and dorsal visual areas show the greatest proportion of vertices selective for binocular disparity. Second, with binocularly correlated stimuli, we show a polynomial relationship between preferred disparity and tuning curve width, with sharply tuned disparity responses at near-zero disparities, and broader disparity tuning profiles at near or far disparities. This relationship held across visual areas and was not present for anticorrelated stimuli. Finally, the individual thresholds for psychophysical stereoacuity at the 3 different pedestal disparities were broadly related to population receptive field tuning width in area V1, suggesting a possible limit for fine stereopsis at the earliest level of cortical processing. Together, these findings point to heterogeneity of disparity processing across human visual areas, comparable to non-human primates.Significance Statement Binocular disparity arises from the horizonal separation of the two eyes and provides information for determining depth and 3D structure. We used functional magnetic resonance imaging and population receptive field mapping to measure tuning of multiple visual areas to binocular disparity in the human visual cortex. We additionally measured psychophysical thresholds for detecting binocular disparity and correlated these with the neural measures. The width of the disparity tuning was related to the preferred disparity across all visual areas. Disparity tuning widths in V1 were also related to psychophysical thresholds. These findings in the human are broadly comparable to non-human primates.

MRI free targeting of deep brain structures based on facial landmarks

Emerging neurostimulation methods aim to selectively modulate deep brain structures. Guiding these therapies has presented a substantial challenge, since imaging modalities such as MRI limit the spectrum of beneficiaries. In this study, we assess the guidance accuracy of a neuronavigation method that does not require taking MRI scans. The method is based on clearly identifiable anatomical landmarks of each subject's face. We compared this technique to the ideal case, MRI-based nonlinear brain registration, and evaluated the accuracy of both methods across ten targets located in deep brain structures: 7 targets in the anterior cingulate cortex as well as the anterior commissure and posterior commissure. Compared with the ideal case, the average localization error of the MRI-free method was 5.75 ± 2.98 mm (mean ± sd). These findings suggest that this method may provide sufficient compromise between practicality and the accuracy of targeting deep brain structures.

The role of binocular disparity and attention in the neural representation of multiple moving stimuli in the visual cortex

Segmenting visual scenes into distinct objects and surfaces is a fundamental visual process, with stereoscopic depth and motion serving as crucial cues. However, how the visual system uses these cues to segment multiple objects is not fully understood. We investigated how neurons in the middle-temporal (MT) cortex of macaque monkeys represent overlapping surfaces at different depths, moving in different directions. Neuronal activity was recorded from three male monkeys during discrimination tasks under varying attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the binocular disparity of one surface over the other. The disparity bias of a neuron was positively correlated with the neuron's disparity preference for a single surface. In two animals, neurons preferring near disparities of single surfaces (near neurons) showed a near bias for overlapping stimuli, while neurons preferring far disparities (far neurons) showed a far bias. In the third animal, both near and far neurons displayed a near bias, though the near neurons showed a stronger near bias. All three animals exhibited an initial near bias across neurons relative to the average of the responses to the individual surfaces. Although attention modulated neuronal responses, the disparity bias was not caused by attention. We also found that the effect of attention was consistent with object-based, rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. This model is a novel extension of the standard normalization model and provides a unified explanation for the disparity bias across animals. Our results reveal how MT neurons encode multiple stimuli moving at different depths and present new evidence of response modulation by object-based attention. The disparity bias allows subgroups of neurons to preferentially represent individual surfaces of multiple stimuli at different depths, thereby facilitating segmentation.

Two Disparity Channels in Human Visual Cortex With Different Contrast and Blur Sensitivity

Purpose

Our goal is to describe the contrast and blur sensitivity of multiple horizontal disparity subsystems and to relate them to the contrast and spatial sensitivities of their monocular inputs.

Methods

Steady-state visual evoked potential (SSVEP) amplitudes were recorded in response to dynamic random dot stereograms (DRDSs) alternating at 2 Hz between zero disparity and varying magnitudes of crossed disparity for disparity plane and disparity grating stimuli. Half-image contrasts ranged between 2.5% and 80% and over a range of Gaussian blurs from 1.4 to 12 arcmin. Separate experiments measured contrast and blur sensitivity for the monocular half-images.

Results

The first and second harmonics disparity responses were maximal for disparity gratings and for the disparity plane condition, respectively. The first harmonic of the disparity grating response was more affected by both contrast and blur than was the second harmonic of the disparity plane response, which had higher contrast sensitivity than the first harmonic.

Conclusions

The corrugation frequency, contrast, and blur tuning of the first harmonic suggest that it reflects activity of neurons tuned to higher luminance spatial frequencies that are selective for relative disparity, whereas the second harmonic reflects the activity of neurons sensitive to absolute disparity that are driven by low monocular spatial frequencies.

Translational Relevance

SSVEPs to DRDSs provide two objective neural measures of disparity processing, the first harmonic-whose stimulus preferences are similar to those of behavioral stereoacuity-and the second harmonic that represents an independent disparity-specific but not necessarily stereoscopic mechanism.

Cognitive penetrability of scene representations based on horizontal image disparities

The structure of natural scenes is signaled by many visual cues. Principal amongst them are the binocular disparities created by the laterally separated viewpoints of the two eyes. Disparity cues are believed to be processed hierarchically, first in terms of local measurements of absolute disparity and second in terms of more global measurements of relative disparity that allow extraction of the depth structure of a scene. Psychophysical and oculomotor studies have suggested that relative disparities are particularly relevant to perception, whilst absolute disparities are not. Here, we compare neural responses to stimuli that isolate the absolute disparity cue with stimuli that contain additional relative disparity cues, using the high temporal resolution of EEG to determine the temporal order of absolute and relative disparity processing. By varying the observers' task, we assess the extent to which each cue is cognitively penetrable. We find that absolute disparity is extracted before relative disparity, and that task effects arise only at or after the extraction of relative disparity. Our results indicate a hierarchy of disparity processing stages leading to the formation of a proto-object representation upon which higher cognitive processes can act.

Dynamics of absolute and relative disparity processing in human visual cortex

Cortical processing of binocular disparity is believed to begin in V1 where cells are sensitive to absolute disparity, followed by the extraction of relative disparity in higher visual areas. While much is known about the cortical distribution and spatial tuning of disparity-selective neurons, the relationship between their spatial and temporal properties is less well understood. Here, we use steady-state Visual Evoked Potentials and dynamic random dot stereograms to characterize the temporal dynamics of spatial mechanisms in human visual cortex that are primarily sensitive to either absolute or relative disparity. Stereograms alternated between disparate and non-disparate states at 2 Hz. By varying the disparity-defined spatial frequency content of the stereograms from a planar surface to corrugated ones, we biased responses towards absolute vs. relative disparities. Reliable Components Analysis was used to derive two dominant sources from the 128 channel EEG records. The first component (RC1) was maximal over the occipital pole. In RC1, first harmonic responses were sustained, tuned for corrugation frequency, and sensitive to the presence of disparity references, consistent with prior psychophysical sensitivity measurements. By contrast, the second harmonic, associated with transient processing, was not spatially tuned and was indifferent to references, consistent with it being generated by an absolute disparity mechanism. Thus, our results reveal a duplex coding strategy in the disparity domain, where relative disparities are computed via sustained mechanisms and absolute disparities are computed via transient mechanisms.

Associations Between Binocular Depth Perception and Performance Gains in Laparoscopic Skill Acquisition

The ability to perceive differences in depth is important in many daily life situations. It is also of relevance in laparoscopic surgical procedures that require the extrapolation of three-dimensional visual information from two-dimensional planar images. Besides visual-motor coordination, laparoscopic skills and binocular depth perception are demanding visual tasks for which learning is important. This study explored potential relations between binocular depth perception and individual variations in performance gains during laparoscopic skill acquisition in medical students naïve of such procedures. Individual differences in perceptual learning of binocular depth discrimination when performing a random dot stereogram (RDS) task were measured as variations in the slope changes of the logistic disparity psychometric curves from the first to the last blocks of the experiment. The results showed that not only did the individuals differ in their depth discrimination; the extent with which this performance changed across blocks also differed substantially between individuals. Of note, individual differences in perceptual learning of depth discrimination are associated with performance gains from laparoscopic skill training, both with respect to movement speed and an efficiency score that considered both speed and precision. These results indicate that learning-related benefits for enhancing demanding visual processes are, in part, shared between these two tasks. Future studies that include a broader selection of task-varying monocular and binocular cues as well as visual-motor coordination are needed to further investigate potential mechanistic relations between depth perceptual learning and laparoscopic skill acquisition. A deeper understanding of these mechanisms would be important for applied research that aims at designing behavioral interventions for enhancing technology-assisted laparoscopic skills.

Disparity in Context: Understanding how monocular image content interacts with disparity processing in human visual cortex

Horizontal disparities between the two eyes' retinal images are the primary cue for depth. Commonly used random ot tereograms (RDS) intentionally camouflage the disparity cue, breaking the correlations between monocular image structure and the depth map that are present in natural images. Because of the nonlinear nature of visual processing, it is unlikely that simple computational rules derived from RDS will be sufficient to explain binocular vision in natural environments. In order to understand the interplay between natural scene structure and disparity encoding, we used a depth-image-based-rendering technique and a library of natural 3D stereo pairs to synthesize two novel stereogram types in which monocular scene content was manipulated independent of scene depth information. The half-images of the novel stereograms comprised either random-dots or scrambled natural scenes, each with the same depth maps as the corresponding natural scene stereograms. Using these stereograms in a simultaneous Event-Related Potential and behavioral discrimination task, we identified multiple disparity-contingent encoding stages between 100 ~ 500 msec. The first disparity sensitive evoked potential was observed at ~100 msec after an earlier evoked potential (between ~50-100 msec) that was sensitive to the structure of the monocular half-images but blind to disparity. Starting at ~150 msec, disparity responses were stereogram-specific and predictive of perceptual depth. Complex features associated with natural scene content are thus at least partially coded prior to disparity information, but these features and possibly others associated with natural scene content interact with disparity information only after an intermediate, 2D scene-independent disparity processing stage.