Decoding Neural Responses to Motion-in-Depth Using EEG

Two stereoscopic cues that underlie the perception of motion-in-depth (MID) are changes in retinal disparity over time (CD) and interocular velocity differences (IOVD). These cues have independent spatiotemporal sensitivity profiles, depend upon different low-level stimulus properties, and are potentially processed along separate cortical pathways. Here, we ask whether these MID cues code for different motion directions: do they give rise to discriminable patterns of neural signals, and is there evidence for their convergence onto a single "motion-in-depth" pathway? To answer this, we use a decoding algorithm to test whether, and when, patterns of electroencephalogram (EEG) signals measured from across the full scalp, generated in response to CD- and IOVD-isolating stimuli moving toward or away in depth can be distinguished. We find that both MID cue type and 3D-motion direction can be decoded at different points in the EEG timecourse and that direction decoding cannot be accounted for by static disparity information. Remarkably, we find evidence for late processing convergence: IOVD motion direction can be decoded relatively late in the timecourse based on a decoder trained on CD stimuli, and vice versa. We conclude that early CD and IOVD direction decoding performance is dependent upon fundamentally different low-level stimulus features, but that later stages of decoding performance may be driven by a central, shared pathway that is agnostic to these features. Overall, these data are the first to show that neural responses to CD and IOVD cues that move toward and away in depth can be decoded from EEG signals, and that different aspects of MID-cues contribute to decoding performance at different points along the EEG timecourse.

Simulation of Driving in Low-Visibility Conditions: Does Stereopsis Improve Speed Perception?

Laboratory-based studies of perceived speed show that, under most circumstances, perceived speed is reduced as a function of contrast. However, a recent investigation of perceived vehicular speed while driving around a closed road circuit showed no such effect (Owens, Wood, & Carberry, 2010, Perception, 39: , 1199-1215). We sought to probe the source of this discrepancy, asking whether the presence or absence of stereoscopic motion information might account for the difference in results. In a two-alternative forced-choice psychophysical speed-discrimination task, observers compared the speed of high- and low-contrast driving clips filmed with a 3-D camera and presented either stereoscopically (3-D) or monoscopically (2-D). Although perceived speed was reduced at low contrast, the size of this misperception was equivalent for 2-D and 3-D presentations. However, the inclusion of stereoscopic cues to vehicular speed caused significant improvements in the precision of speed judgments. It is concluded that although stereopsis can provide access to valuable information on perceived speed, contrast-independent speed estimation as demonstrated by Owens et al. (2010) is more likely to reflect the use of the full visual field in a real driving situation (compared with limited field of view simulations), or the additional contributions of nonvisual cues rather than stereopsis.

A Distinct Mechanism of Temporal Integration for Motion through Depth

Temporal integration of visual motion has been studied extensively within the frontoparallel plane (i.e., 2D). However, the majority of motion occurs within a 3D environment, and it is unknown whether the principles from 2D motion processing generalize to more realistic 3D motion. We therefore characterized and compared temporal integration underlying 2D (left/right) and 3D (toward/away) direction discrimination in human observers, varying motion coherence across a range of viewing durations. The resulting discrimination-versus-duration functions followed three stages, as follows: (1) a steep improvement during the first ∼150 ms, likely reflecting early sensory processing; (2) a subsequent, more gradual benefit of increasing duration over several hundreds of milliseconds, consistent with some form of temporal integration underlying decision formation; and (3) a final stage in which performance ceased to improve with duration over ∼1 s, which is consistent with an upper limit on integration. As previously found, improvements in 2D direction discrimination with time were consistent with near-perfect integration. In contrast, 3D motion sensitivity was lower overall and exhibited a substantial departure from perfect integration. These results confirm that there are overall differences in sensitivity for 2D and 3D motion that are consistent with a sensory difference between binocular and dichoptic sensory mechanisms. They also reveal a difference at the integration stage, in which 3D motion is not accumulated as perfectly as in the 2D motion model system.

Area MT encodes three-dimensional motion

We use visual information to determine our dynamic relationship with other objects in a three-dimensional (3D) world. Despite decades of work on visual motion processing, it remains unclear how 3D directions-trajectories that include motion toward or away from the observer-are represented and processed in visual cortex. Area MT is heavily implicated in processing visual motion and depth, yet previous work has found little evidence for 3D direction sensitivity per se. Here we use a rich ensemble of binocular motion stimuli to reveal that most neurons in area MT of the anesthetized macaque encode 3D motion information. This tuning for 3D motion arises from multiple mechanisms, including different motion preferences in the two eyes and a nonlinear interaction of these signals when both eyes are stimulated. Using a novel method for functional binocular alignment, we were able to rule out contributions of static disparity tuning to the 3D motion tuning we observed. We propose that a primary function of MT is to encode 3D motion, critical for judging the movement of objects in dynamic real-world environments.

Aftereffect of motion-in-depth based on binocular cues: Effects of adaptation duration, interocular correlation, and temporal correlation

There are at least two possible binocular cues to motion-in-depth, namely disparity change over time and interocular velocity differences. There has been significant controversy about their relative contributions to the perception of motion-in-depth. In the present study, we used the technique of selective adaptation to address this question. In Experiment 1, we found that adaptation to motion-in-depth depicted by temporally correlated random-dot stereograms, which contained coherent interocular velocity difference, produced motion aftereffect in the depth direction irrespective of the adaptors' interocular correlation for any adaptation duration tested. This suggests that coherent changing disparity does not contribute to motion-in-depth adaptation. Because the aftereffect duration did not saturate in the tested range of adaptation duration, it is unlikely that the lack of the effect of changing disparity was due to a ceiling effect. In Experiment 2, we used a novel adaptor that contained a unidirectional coherent interocular velocity difference signal and a bidirectional changing disparity signal that should not induce a motion aftereffect in depth. Following the adaptation, motion aftereffect in depth occurred in the opposite direction to the adaptor's motion-in-depth based on interocular velocity difference. Experiment 3 demonstrated that these results generalized in 12 untrained subjects. These experiments suggest that the contribution of interocular velocity difference to the perception of motion-in-depth is substantial, while that of changing disparity is very limited (if any), at least at the stages of direction-selective mechanisms subject to an aftereffect phenomenon.

Dynamics and cortical distribution of neural responses to 2D and 3D motion in human

The perception of motion-in-depth is important for avoiding collisions and for the control of vergence eye-movements and other motor actions. Previous psychophysical studies have suggested that sensitivity to motion-in-depth has a lower temporal processing limit than the perception of lateral motion. The present study used functional MRI-informed EEG source-imaging to study the spatiotemporal properties of the responses to lateral motion and motion-in-depth in human visual cortex. Lateral motion and motion-in-depth displays comprised stimuli whose only difference was interocular phase: monocular oscillatory motion was either in-phase in the two eyes (lateral motion) or in antiphase (motion-in-depth). Spectral analysis was used to break the steady-state visually evoked potentials responses down into even and odd harmonic components within five functionally defined regions of interest: V1, V4, lateral occipital complex, V3A, and hMT+. We also characterized the responses within two anatomically defined regions: the inferior and superior parietal cortex. Even harmonic components dominated the evoked responses and were a factor of approximately two larger for lateral motion than motion-in-depth. These responses were slower for motion-in-depth and were largely independent of absolute disparity. In each of our regions of interest, responses at odd-harmonics were relatively small, but were larger for motion-in-depth than lateral motion, especially in parietal cortex, and depended on absolute disparity. Taken together, our results suggest a plausible neural basis for reduced psychophysical sensitivity to rapid motion-in-depth.

Three-dimensional motion aftereffects reveal distinct direction-selective mechanisms for binocular processing of motion through depth

Motion aftereffects are historically considered evidence for neuronal populations tuned to specific directions of motion. Despite a wealth of motion aftereffect studies investigating 2D (frontoparallel) motion mechanisms, there is a remarkable dearth of psychophysical evidence for neuronal populations selective for the direction of motion through depth (i.e., tuned to 3D motion). We compared the effects of prolonged viewing of unidirectional motion under dichoptic and monocular conditions and found large 3D motion aftereffects that could not be explained by simple inheritance of 2D monocular aftereffects. These results (1) demonstrate the existence of neurons tuned to 3D motion as distinct from monocular 2D mechanisms, (2) show that distinct 3D direction selectivity arises from both interocular velocity differences and changing disparities over time, and (3) provide a straightforward psychophysical tool for further probing 3D motion mechanisms.