Stereoscopic depth cues can segment motion information

The role of binocular disparity and attention in the neural representation of multiple moving stimuli in the visual cortex

A fundamental process of vision involves segmenting visual scenes into distinct objects and surfaces. Stereoscopic depth and visual motion are important cues for segmentation. However, the mechanisms by which the visual system utilizes depth and motion cues to segment multiple objects are not fully understood. We investigated how neurons in the middle-temporal (MT) cortex of macaque monkeys represented overlapping surfaces located at different depths and moved simultaneously in different directions. We recorded neuronal activities in the MT of three male monkeys while they performed discrimination tasks under different attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the horizontal binocular disparity of one of the two surfaces. Across all animals, the disparity bias of a neuron in response to two surfaces positively correlated with the neuron's disparity preference for a single surface. For two animals, neurons that preferred near disparities of single surfaces (near neurons) showed a near bias to overlapping stimuli, whereas neurons that preferred far disparities (far neurons) showed a far bias. For the third animal, both near and far neurons displayed a near bias, although the near neurons showed a stronger near bias than the far neurons. Interestingly, for all three animals, both near and far neurons exhibited an initial near bias relative to the average of the responses to the individual surfaces. Although attention can modulate neuronal response to better represent the attended surface, the disparity bias was not due to attention. We also found that the effect of attention modulation on MT responses was consistent with object-based rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. This model is a novel extension of the standard normalization model and provides a unified explanation for the disparity bias observed across animals. Our results revealed the neural encoding rule for multiple stimuli located at different depths and presented new evidence of response modulation by object-based attention in MT. The disparity bias allows subgroups of neurons to preferentially represent individual surfaces of multiple stimuli at different depths, thereby facilitating segmentation.

Evidence for strictly monocular processing in visual motion opponency and Glass pattern perception

When presented with locally paired dots moving in opposite directions, motion selective neurons in the middle temporal cortex (MT) reduce firing while neurons in V1 are unaffected. This physiological effect is known as motion opponency. The current study used psychophysics to investigate the neural circuit underlying motion opponency. We asked whether opposing motion signals could arrive from different eyes into the receptive field of a binocular neuron while still maintaining motion opponency. We took advantage of prior findings that orientation discrimination of the motion axis (along which paired dots oscillate) is harder when dots move counter-phase than in-phase, an effect associated with motion opponency. We found that such an effect disappeared when paired dots originated from different eyes. This suggests that motion opponency, at some point, involves strictly monocular processing. This does not mean that motion opponency is entirely monocular. Further, we found that the effect of a Glass pattern disappeared under similar viewing conditions, suggesting that Glass pattern perception also involves some strictly monocular processing.

Directional Limits on Motion Transparency Assessed Through Colour-Motion Binding

Motion-defined transparency is the perception of two or more distinct moving surfaces at the same retinal location. We explored the limits of motion transparency using superimposed surfaces of randomly positioned dots defined by differences in motion direction and colour. In one experiment, dots were red or green and we varied the proportion of dots of a single colour that moved in a single direction ('colour-motion coherence') and measured the threshold direction difference for discriminating between two directions. When colour-motion coherences were high (e.g., 90% of red dots moving in one direction), a smaller direction difference was required to correctly bind colour with direction than at low coherences. In another experiment, we varied the direction difference between the surfaces and measured the threshold colour-motion coherence required to discriminate between them. Generally, colour-motion coherence thresholds decreased with increasing direction differences, stabilising at direction differences around 45°. Different stimulus durations were compared, and thresholds were higher at the shortest (150 ms) compared with the longest (1,000 ms) duration. These results highlight different yet interrelated aspects of the task and the fundamental limits of the mechanisms involved: the resolution of narrowly separated directions in motion processing and the local sampling of dot colours from each surface.

Binocular summation of second-order global motion signals in human vision

Although many studies have examined the principles governing first-order global motion perception, the mechanisms that mediate second-order global motion perception remain unresolved. This study investigated the existence, nature and extent of the binocular advantage for encoding second-order (contrast-defined) global motion. Motion coherence thresholds (79.4% correct) were assessed for determining the direction of radial, rotational and translational second-order motion trajectories as a function of local element modulation depth (contrast) under monocular and binocular viewing conditions. We found a binocular advantage for second-order global motion processing for all motion types. This advantage was mainly one of enhanced modulation sensitivity, rather than of motion-integration. However, compared to findings for first-order motion where the binocular advantage was in the region of a factor of around 1.7 (Hess et al., 2007), the binocular advantage for second-order global motion was marginal, being in the region of around 1.2. This weak enhancement in sensitivity with binocular viewing is considerably less than would be predicted by conventional models of either probability summation or neural summation.

Adaptation state of the local-motion-pooling units determines the nature of the motion aftereffect to transparent motion

When observers adapt to a transparent-motion stimulus, the resulting motion aftereffect (MAE) is typically in the direction opposite to the vector average of the component directions. It has been proposed that the reason for this is that it is the adaptation state at the local-level (i.e. of the local-motion-pooling units) that determines the nature of the MAE (Vidnyanszky et al. Trends in Cognitive Sciences, 6(4), 157-161). The adapting stimuli used in these experiments typically consisted of random-dot kinematograms, with each dot being able to move over the entire viewing aperture. Here we used spatially-localised global-plaid stimuli which enabled us, over the course of adaptation, to present either one of both motion directions at each local region. A unidirectional MAE was perceived when two motion directions were presented at each location and a transparent MAE was perceived when a single direction was presented. These results support the notion that it is the adaptation state at the local-motion-pooling level that determines the nature of the MAE to transparent motion stimuli.

Population anisotropy in area MT explains a perceptual difference between near and far disparity motion segmentation

Segmentation of the visual scene into relevant object components is a fundamental process for successfully interacting with our surroundings. Many visual cues, including motion and binocular disparity, support segmentation, yet the mechanisms using these cues are unclear. We used a psychophysical motion discrimination task in which noise dots were displaced in depth to investigate the role of segmentation through disparity cues in visual motion stimuli (experiment 1). We found a subtle, but significant, bias indicating that near disparity noise disrupted the segmentation of motion more than equidistant far disparity noise. A control experiment showed that the near-far difference could not be attributed to attention (experiment 2). To account for the near-far bias, we constructed a biologically constrained model using recordings from neurons in the middle temporal area (MT) to simulate human observers' performance on experiment 1. Performance of the model of MT neurons showed a near-disparity skew similar to that shown by human observers. To isolate the cause of the skew, we simulated performance of a model containing units derived from properties of MT neurons, using phase-modulated Gabor disparity tuning. Using a skewed-normal population distribution of preferred disparities, the model reproduced the elevated motion discrimination thresholds for near-disparity noise, whereas a skewed-normal population of phases (creating individually asymmetric units) did not lead to any performance skew. Results from the model suggest that the properties of neurons in area MT are computationally sufficient to perform disparity segmentation during motion processing and produce similar disparity biases as those produced by human observers.

The visual processing of motion-defined transparency

Our understanding of how the visual system processes motion transparency, the phenomenon by which multiple directions of motion are perceived to coexist in the same spatial region, has grown considerably in the past decade. There is compelling evidence that the process is driven by global-motion mechanisms. Consequently, although transparently moving surfaces are readily segmented over an extended space, the visual system cannot separate two motion signals that coexist in the same local region. A related issue is whether the visual system can detect transparently moving surfaces simultaneously or whether the component signals encounter a serial 'bottleneck' during their processing. Our initial results show that, at sufficiently short stimulus durations, observers cannot accurately detect two superimposed directions; yet they have no difficulty in detecting one pattern direction in noise, supporting the serial-bottleneck scenario. However, in a second experiment, the difference in performance between the two tasks disappears when the component patterns are segregated. This discrepancy between the processing of transparent and non-overlapping patterns may be a consequence of suppressed activity of global-motion mechanisms when the transparent surfaces are presented in the same depth plane. To test this explanation, we repeated our initial experiment while separating the motion components in depth. The marked improvement in performance leads us to conclude that transparent motion signals are represented simultaneously.

Binocular influences on global motion processing in the human visual system

This study investigates four key issues concerning the binocular properties of the mechanisms that encode global motion in human vision: (1) the extent of any binocular advantage; (2) the possible site of this binocular summation; (3) whether or not purely monocular inputs exist for global motion perception; (4) the extent of any dichoptic interaction. Global motion coherence thresholds were measured using random-dot-kinematograms as a function of the dot modulation depth (contrast) for translational, radial and circular flow fields. We found a marked binocular advantage of approximately 1.7, comparable for all three types of motion and the performance benefit was due to a contrast rather than a global motion enhancement. In addition, we found no evidence for any purely monocular influences on global motion detection. The results suggest that the site of binocular combination for global motion perception occurs prior to the extra-striate cortex where motion integration occurs. All cells involved are binocular and exhibit dichoptic interactions, suggesting the existence of a neural mechanism that involves more than just simple summation of the two monocular inputs.

Pushing the limits of transparent-motion detection with binocular disparity

When transparent motion is defined purely by direction differences, observers fail to detect more than two signal directions simultaneously [Edwards, M., & Greenwood, J.A. (2005). The perception of motion transparency: A signal-to-noise limit. Vision Research, 45, 1877-1884]. This limit is strongly related to signal-detection thresholds for transparent motion, which are several times higher than uni-directional thresholds. When the effective signal intensities are elevated by speed differences that drive independent global-motion systems, the transparent-motion limit can be extended to allow detection of three signals [Greenwood, J.A., & Edwards, M. (2006). An extension of transparent-motion detection limit using speed-tuned global-motion systems. Vision Research, 46, 1440-1449]. Because there are independent disparity-tuned global-motion systems, distributing transparent-motion signals across distinct depth planes also allows an increase in their effective signal intensity. In the present study, the addition of depth differences enabled the simultaneous detection of three signals. However, as with the addition of speed differences, observers were not able to detect four signals, which would be predicted if signal intensity were the sole constraint on transparent-motion detection. The combination of depth and speed produced similar results, suggesting that there is a strict higher-order limit, possibly related to attention, restricting the maximum number of signals that can be detected simultaneously to three.

An extension of the transparent-motion detection limit using speed-tuned global-motion systems

When transparent motion is defined purely by direction differences, no more than two signal directions can be detected simultaneously. This limit appears to occur because higher signal intensities are required to detect transparent motion compared with uni-directional motion (Edwards, M., & Greenwood, J. A. (2005). The perception of motion transparency: A signal-to-noise limit. Vision Research, 45, 1877-1884). Increasing the effective signal intensities should therefore increase the number of signals that can be detected. We achieved this by adding speed differences, dividing transparent-motion signals between two speed-tuned global-motion systems. When some signals moved at appropriate low speeds and others at high speeds, up to three signals were detected. This is consistent, at least in part, with the signal-to-noise processing basis of the transparency limit. Differences in contrast polarity were also used to assess whether the limit could be extended using stimulus features without independent global-motion systems. A modest improvement in performance was obtained, suggesting that there may be multiple routes to extending the transparent-motion limit.

Colour, polarity, disparity, and texture contributions to motion segregation

We measured how different cues are combined in motion-segregation processes by using motion stimuli where randomly distributed target dots were organised in global revolving motion while the remaining noise dots performed random motion. Target dots were cued with a different colour, polarity, disparity depth, or texture orientation than the noise dots, or they were the same as the noise dots. The stimuli were presented with a prolonged static cue preview which provided position cues to target dots or, briefly with static pre-target and post-target noise frames, which provided false position cues (no preview). All cues efficiently facilitated global motion segregation in cued-preview conditions. Colour completely failed to facilitate global motion segregation in no-preview conditions. Polarity and disparity facilitated segregation in no-preview conditions, although sensitivities were lower than in the preview conditions. Remarkably, texture orientation largely facilitated motion segregation by the same amount in both cued-preview and no-preview conditions. So, colour provides only position cues to the motion-segregation task whereas texture orientation, disparity, and to a lesser extent polarity are integrated with the segregation process.

The perception of motion transparency: A signal-to-noise limit

A number of studies were conducted to determine how many transparent motion signals observers could simultaneously perceive. It was found that that the limit was two. However, observers required a signal intensity of about 42% in order to perceive a bi-directional transparent stimulus. This signal level was about three times that required to detect a uni-directional motion signal, and higher than was physically possible to achieve in a tri-directional stimulus (in a stimulus in which the different transparent signals are defined only by direction). These results indicate that signal intensity plays an important role in establishing the transparency limit and, as a consequence, implicates the global-motion area (V5/MT) in this process.

Binocular correlation does not improve coherence detection for fronto-parallel motion

We studied the low-level interactions between motion coherence detection and binocular correlation detection. It is well-established that e.g. depth information from motion parallax and from binocular disparities is effectively integrated. The question we aimed to answer is whether such interactions also exist at the very first correlation level that both mechanisms might have in common. First we quantitatively compared motion coherence detection and binocular correlation detection using similar stimuli (random pixels arrays, RPAs) and the same noise masking paradigm (luminance signal to noise ratio, LSNR). This showed that human observers are much more sensitive to motion than to binocular correlation. Adding noise therefore has a much stronger effect on binocular correlation than on motion detection. Next we manipulated the shape of the stimulus aperture to equalize LSNR thresholds for motion and binocular correlation. Motion sensitivity could be progressively reduced by shortening the length of the motion path, while keeping the aperture area constant. Changing the shape of the aperture did not affect binocular correlation sensitivity. A 'balanced' stimulus, one with equal strengths of motion and binocular correlation signals was then used to study the mutual interactions. In accordance with previous results, motion was found to greatly facilitate binocular correlation. Binocular correlation, however did not facilitate motion detection. We conclude that interactions are asymmetrical; fronto-parallel motion is primarily detected monocularly and this information can then be used to facilitate binocular correlation, but binocular correlation cannot improve motion sensitivity.

Timing accuracy in motion extrapolation: reverse effects of target size and visible extent of motion at low and high speeds

By varying target size, speed, and extent of visible motion we examined the timing accuracy in motion extrapolation. Small or large targets (0.2 or 0.8 deg) moved at either 2.5, 5, or 10 deg s(-1) across a horizontal path (2.5 or 10 deg) and then vanished behind an occluder. Observers responded when they judged that the target had reached a randomly specified position between 0 and 12 deg. With higher speeds, the timing accuracy (the reverse of absolute error) was better for small than for large targets, and for long than for short visible extents. With low speed, these effects were reversed. In addition, while long visible extents yielded a greater accuracy at high than at low speeds, for short extents the accuracy was much better with the low speed. The findings suggest that, when extrapolating motion with targets and visible extents of different sizes, the visual system implements different scaling algorithms depending on target speed. At higher speeds, processing of visible and occluded motion is likely to share a common scaling mechanism based on velocity transposition. Reverse effects for target size and extent of visible motion at low and high speeds converge with the assumption of two distinct speed-tuned motion-processing mechanisms in human vision.

Comparison of the spatial-frequency selectivity of local and global motion detectors

Convergent physiological and behavioral evidence indicates that the initial receptive fields responsible for motion detection are spatially localized. Consequently, the perception of global patterns of movement (such as expansion) requires that the output of these local mechanisms be integrated across visual space. We have differentiated local and global motion processes, with mixtures of coherent and incoherent moving patterns composed of bandpass filtered dots, and have measured their spatial-frequency selectivity. We report that local motion detectors show narrow-band spatial-frequency tuning (i.e., they respond only to a narrow range of spatial frequencies) but that global motion detectors show broadband spatial-frequency tuning (i.e., they integrate across a broad range of spatial frequencies), with a preference for low spatial frequencies.

Biophysical vision model and learning paradigms about vision: review

A learning paradigm of a new biophysical vision model (BVM) is presented. It incorporates anatomical and physiological evidence from micro- and macroscopic research on vision as reported in the literature during the past five years. Anatomical and physiological vision research tends to drift away from the technological foundations of encoding and reproducing size-defined images of real ongoing life scenarios. White and color light waves reflecting life scenarios are converted by the retina to encoded electrical train pulses with attached real information to be decoded by cortical vision neurons. The BVM paradigm is based on the ideas that: (1) cinema technology reproduces real-life scenes just as the human eye sees them; (2) virtual reality and robotics are computerized replications of categorized human vision faculties in operation. We believe that vision-related technology may extend our knowledge about vision and direct vision research into new horizons. The biophysical vision model has three prerequisites: (1) The faculties of human vision must be categorized. (2) Logic circuits of the 'hardware' of neuronal vision must be present. (3) Vision faculties are operated by self-induced 'software'. Vision research may be enhanced with devices constructed according to BVM that would enable biophysical vision experiments in both humans and animals.

Techniques for fast stereoscopic MRI

Stereoscopic MRI can impart 3D perception with only two image acquisitions. This economy over standard multiplanar 3D volume renderings allows faster frame rates, which are needed for real-time imaging applications. Real-time 3D perception may enhance the appreciation of complex anatomical structures, and may improve hand-eye coordination while manipulating a medical device during an image-guided interventional procedure. To this goal, a system is being developed to acquire and display stereoscopic MR images in real-time. A clinically used, fast gradient-recalled echo-train sequence has been modified to produce stereo image pairs. Features have been added for depth cueing, view sharing, and bulk signal suppression. A workstation was attached to a clinical MR scanner for fast data extraction, image reconstruction and stereoscopic image display.

Speed perception is affected by the Ebbinghaus-Titchener illusion

We examined whether the apparent extent of motion affects speed perception. On the first presentation of each trial, a light dot travelled horizontally across a central circle of one of the Ebbinghaus configurations (with either small or large inducing elements). On the second presentation, observers adjusted the speed of a dot moving within the central circle alone so as to match the speed perceived in the first presentation. For all stimulus speeds (1.3, 2.1, and 5.5 deg s-1), the matched speed with small inducing circles was systematically less than that with large inducing circles. The findings indicate that the perceived speed depends on the apparent extent of motion: the larger the apparent size of a frame, the slower the apparent speed. These results are consistent with the predictions of transposition effects in visual motion.

Cue combination in the motion correspondence problem

Image motion is a primary source of visual information about the world. However, before this information can be used the visual system must determine the spatio-temporal displacements of the features in the dynamic retinal image, which originate from objects moving in space. This is known as the motion correspondence problem. We investigated whether cross-cue matching constraints contribute to the solution of this problem, which would be consistent with physiological reports that many directionally selective cells in the visual cortex also respond to additional visual cues. We measured the maximum displacement limit (Dmax) for two-frame apparent motion sequences. Dmax increases as the number of elements in such sequences decreases. However, in our displays the total number of elements was kept constant while the number of a subset of elements, defined by a difference in contrast polarity, binocular disparity or colour, was varied. Dmax increased as the number of elements distinguished by a particular cue was decreased. Dmax was affected by contrast polarity for all observers, but only some observers were influenced by binocular disparity and others by colour information. These results demonstrate that the human visual system exploits local, cross-cue matching constraints in the solution of the motion correspondence problem.

Does binocular disparity facilitate the detection of transparent motion?

Recent physiological studies have established that cortical cells that are tuned for the direction of motion may also exhibit tuning for binocular disparity. This tuning does not appear to provide any advantage in discriminating the direction of global motion in random-dot kinematograms. Here we investigated the possibility that this tuning may be important in the perception of transparent motion. Random-dot kinematograms were presented which contained coherent motion in a single direction or in two opposing directions. A greater proportion of signal dots was required for the detection of transparent motion than of motion in a single direction. This difference vanished when the two opposite directions of motion were presented with different disparities. These results suggest that the direction of global motion can be computed separately for surfaces which are clearly segregated in depth.