Integration of motion and stereopsis in middle temporal cortical area of macaques

The role of binocular disparity and attention in the neural representation of multiple moving stimuli in the visual cortex

Segmenting visual scenes into distinct objects and surfaces is a fundamental visual process, with stereoscopic depth and motion serving as crucial cues. However, how the visual system uses these cues to segment multiple objects is not fully understood. We investigated how neurons in the middle-temporal (MT) cortex of macaque monkeys represent overlapping surfaces at different depths, moving in different directions. Neuronal activity was recorded from three male monkeys during discrimination tasks under varying attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the binocular disparity of one surface over the other. The disparity bias of a neuron was positively correlated with the neuron's disparity preference for a single surface. In two animals, neurons preferring near disparities of single surfaces (near neurons) showed a near bias for overlapping stimuli, while neurons preferring far disparities (far neurons) showed a far bias. In the third animal, both near and far neurons displayed a near bias, though the near neurons showed a stronger near bias. All three animals exhibited an initial near bias across neurons relative to the average of the responses to the individual surfaces. Although attention modulated neuronal responses, the disparity bias was not caused by attention. We also found that the effect of attention was consistent with object-based, rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. This model is a novel extension of the standard normalization model and provides a unified explanation for the disparity bias across animals. Our results reveal how MT neurons encode multiple stimuli moving at different depths and present new evidence of response modulation by object-based attention. The disparity bias allows subgroups of neurons to preferentially represent individual surfaces of multiple stimuli at different depths, thereby facilitating segmentation.

Motion opponency at the middle temporal cortex: Preserved motion information and the effect of perceptual learning

Motion opponency, first observed within the primate middle temporal cortex (MT), refers to the suppressing effect of opposite motion directions on neuronal activity. Namely, when opposing motion directional signals stimulate an MT neuron's receptive field, this neuron's response is comparable with that induced by flicker noise. Under such suppression, it is unknown whether any directional information is still represented at MT. In this study, we applied support vector machine (SVM) learning to human functional magnetic resonance imaging data to investigate if any motion defined orientation information was still available from suppressed MT. We found that, at least at the level of ±45° discrimination, such orientation information was still available. Interestingly, after behavioural perceptual learning that improved human discrimination of fine orientation discrimination (e.g. 42° vs. 48°) using the MT-suppressive motion stimuli, the SVM discrimination of ±45° worsened when functional magnetic resonance imaging (fMRI) signals at post-learning MT were used. This result is consistent with findings in Thompson et al. (2013) that, post-perceptual learning, MT suppression was not released, suggesting that motion opponency was perhaps functionally too important for perceptual learning to overcome.

Abnormal effective connectivity in visual cortices underlies stereopsis defects in amblyopia

•

Abnormal effective connectivity inherent stereopsis defects in amblyopia was studied.

•

A weakened connection from V2v to LO2 relates to stereopsis defects in amblyopia.

•

Higher-order visual cortices may serve as key nodes to the stereopsis defects.

•

An independent longitudinal dataset was used to validate the obtained results.

The neural basis underlying stereopsis defects in patients with amblyopia remains unclear, which hinders the development of clinical therapy. This study aimed to investigate visual network abnormalities in patients with amblyopia and their associations with stereopsis function. Spectral dynamic causal modeling methods were employed for resting-state functional magnetic resonance imaging data to investigate the effective connectivity (EC) among 14 predefined regions of interest in the dorsal and ventral visual pathways. We adopted two independent datasets, including a cross-sectional and a longitudinal dataset. In the cross-sectional dataset, we compared group differences in EC between 31 patients with amblyopia (mean age: 26.39 years old) and 31 healthy controls (mean age: 25.71 years old) and investigated the association between EC and stereoacuity. In addition, we explored EC changes after perceptual learning in a novel longitudinal dataset including 9 patients with amblyopia (mean age: 15.78 years old). We found consistent evidence from the two datasets indicating that the aberrant EC from V2v to LO2 is crucial for the stereoscopic deficits in the patients with amblyopia: it was weaker in the patients than in the controls, showed a positive linear relationship with the stereoscopic function, and increased after perceptual learning in the patients. In addition, higher-level dorsal (V3d, V3A, and V3B) and ventral areas (LO1 and LO2) were important nodes in the network of abnormal ECs associated with stereoscopic deficits in the patients with amblyopia. Our research provides insights into the neural mechanism underlying stereopsis deficits in patients with amblyopia and provides candidate targets for focused stimulus interventions to enhance the efficacy of clinical treatment for the improvement of stereopsis deficiency.

Evidence for strictly monocular processing in visual motion opponency and Glass pattern perception

When presented with locally paired dots moving in opposite directions, motion selective neurons in the middle temporal cortex (MT) reduce firing while neurons in V1 are unaffected. This physiological effect is known as motion opponency. The current study used psychophysics to investigate the neural circuit underlying motion opponency. We asked whether opposing motion signals could arrive from different eyes into the receptive field of a binocular neuron while still maintaining motion opponency. We took advantage of prior findings that orientation discrimination of the motion axis (along which paired dots oscillate) is harder when dots move counter-phase than in-phase, an effect associated with motion opponency. We found that such an effect disappeared when paired dots originated from different eyes. This suggests that motion opponency, at some point, involves strictly monocular processing. This does not mean that motion opponency is entirely monocular. Further, we found that the effect of a Glass pattern disappeared under similar viewing conditions, suggesting that Glass pattern perception also involves some strictly monocular processing.

Motion perception and its disorders

As we live in a dynamic world, motion is a fundamental aspect of our visual experience. The advent of computerized stimuli has allowed controlled study of a wide array of motion phenomena, including global integration and segmentation, speed and direction discrimination, motion aftereffects, the optic flow that accompanies self-motion, perception of object form derived from motion cues, and point-light biological motion. Animal studies first revealed the existence of a motion-selective region, the middle temporal (MT) area, also known as V5, located in the lateral occipitotemporal cortex, followed by areas such as V5A (also known as MST, the middle superior temporal area), V6/V6A, the ventral intraparietal area, and others. In humans there are rare cases of bilateral lesions of the V5/V5A complex causing cerebral akinetopsia, a severe impairment of motion perception. Unilateral V5/V5A lesions are more common but cause milder asymptomatic deficits, often limited to the contralateral hemifield, while parietal lesions can impair perception of point-light biological motion or high-level motion tasks that are attentionally demanding. Impairments of motion perception have also been described in optic neuropathy, particularly glaucoma, as well as Alzheimer's disease, Parkinson's disease with dementia, and dementia with Lewy body disease. Prematurity with or without periventricular leukomalacia and developmental syndromes such as Williams' syndrome, autism, and dyslexia have also been associated with impaired motion perception, suggesting a developmental vulnerability of the dorsal pathway.

Idiosyncratic preferences in transparent motion and binocular rivalry are dissociable

Previous studies revealed that there are idiosyncratic preferences to perceive certain motion directions in front during motion transparency depth rivalry (Mamassian & Wallace, 2010; Schütz, 2014). Meanwhile, other studies reported idiosyncratic preferences in binocular rivalry during the onset stage (Carter & Cavanagh, 2007; Stanley, Carter, & Forte, 2011). Here we investigated the relationship of idiosyncratic preferences in transparent motion and binocular rivalry. We presented two dot clouds that were moving in opposite directions. In the transparent motion condition, both dot clouds were presented to both eyes and participants had to report the dot cloud they perceived in front. In the binocular rivalry condition, the dot clouds were presented to different eyes and participants had to report the dominant dot cloud. There were strong idiosyncratic directional preferences in transparent motion and rather weak directional preferences in binocular rivalry. In general, binocular rivalry was dominated by biases in contrast polarity, whereas transparent motion was dominated by biases in motion direction. A circular correlation analysis showed no correlation between directional preferences in transparent motion and binocular rivalry. These findings show that idiosyncratic preferences in a visual feature can be dissociated at different stages of processing.

Directional Limits on Motion Transparency Assessed Through Colour-Motion Binding

Motion-defined transparency is the perception of two or more distinct moving surfaces at the same retinal location. We explored the limits of motion transparency using superimposed surfaces of randomly positioned dots defined by differences in motion direction and colour. In one experiment, dots were red or green and we varied the proportion of dots of a single colour that moved in a single direction ('colour-motion coherence') and measured the threshold direction difference for discriminating between two directions. When colour-motion coherences were high (e.g., 90% of red dots moving in one direction), a smaller direction difference was required to correctly bind colour with direction than at low coherences. In another experiment, we varied the direction difference between the surfaces and measured the threshold colour-motion coherence required to discriminate between them. Generally, colour-motion coherence thresholds decreased with increasing direction differences, stabilising at direction differences around 45°. Different stimulus durations were compared, and thresholds were higher at the shortest (150 ms) compared with the longest (1,000 ms) duration. These results highlight different yet interrelated aspects of the task and the fundamental limits of the mechanisms involved: the resolution of narrowly separated directions in motion processing and the local sampling of dot colours from each surface.

Low-level mediation of directionally specific motion aftereffects: Motion perception is not necessary

Previous psychophysical experiments with normal human observers have shown that adaptation to a moving dot stream causes directionally specific repulsion in the perceived angle of a subsequently viewed moving probe. In this study, we used a two-alternative forced choice task with roving pedestals to determine the conditions that are necessary and sufficient for producing directionally specific repulsion with compound adaptors, each of which contains two oppositely moving, differently colored component streams. Experiment 1 provided a demonstration of repulsion between single-component adaptors and probes moving at approximately 90° or 270°. In Experiment 2, oppositely moving dots in the adaptor were paired to preclude the appearance of motion. Nonetheless, repulsion remained strong when the angle between each probe stream and one component was approximately 30°. In Experiment 3, adapting dot pairs were kept stationary during their limited lifetimes. Their orientation content alone proved insufficient for producing repulsion. In Experiments 4–6, the angle between the probe and both adapting components was approximately 90° or 270°. Directional repulsion was found when observers were asked to visually track one of the adapting components (Exp. 6), but not when they were asked to attentionally track it (Exp. 5), nor while they passively viewed the adaptor (Exp. 4). Our results are consistent with a low-level mechanism for motion adaptation. This mechanism is not selective for stimulus color and is not susceptible to attentional modulation. The most likely cortical locus of adaptation is area V1.

Interocular contrast difference drives illusory 3D percept

Any processing delay between the two eyes can result in illusory 3D percepts for moving objects because of either changes in the pure disparities over time for disparity sensors or by changes to sensors that encode motion/disparity conjointly. This is demonstrated by viewing a fronto-parallel pendulum through a neutral density (ND) filter placed over one eye, resulting in the illusory 3D percept of the pendulum following an elliptical orbit in depth, the so-called Pulfrich phenomenon. Here we use a paradigm where a cylinder rotating in depth, defined by moving Gabor patches is presented at different interocular phases, generating strong to ambiguous depth percepts. This paradigm allows one to manipulate independently the contrast and the luminance of the patches to determine their influence on perceived motion-in-depth. Thus we show psychophysically that an interocular contrast difference can itself result in a similar illusory 3D percept of motion-in-depth. We argue that contrast, like luminance (ND filter) can modify the dynamics of visual neurons resulting in an interocular processing delay or an interocular velocity difference.

A Model of Binocular Motion Integration in MT Neurons

Primate cortical area MT plays a central role in visual motion perception, but models of this area have largely overlooked the binocular integration of motion signals. Recent electrophysiological studies tested binocular integration in MT and found surprisingly that MT neurons lose their hallmark "pattern motion" selectivity when stimuli are presented dichoptically and that many neurons are selective for motion-in-depth (MID). By unifying these novel observations with insights from monocular, frontoparallel motion studies concurrently in a binocular MT motion model, we generated clear, testable predictions about the circuitry and mechanisms underlying visual motion processing. We built binocular models in which signals from left- and right-eye streams could be integrated at various stages from V1 to MT, attempting to create the simplest plausible circuits that accounted for the physiological range of pattern motion selectivity, that explained changes across this range for dichoptic stimulus presentation, and that spanned the spectrum of MID selectivity observed in MT. Our successful models predict that motion-opponent suppression is the key mechanism to account for the striking loss of pattern motion sensitivity with dichoptic plaids, that opponent suppression precedes binocular integration, and that opponent suppression will be stronger in inputs to pattern cells than to component cells. We also found an unexpected connection between circuits for pattern motion selectivity and MID selectivity, suggesting that these two separately studied phenomena could be related. These results also hold in models that include binocular disparity computations, providing a platform for future exploration of binocular response properties in MT.

SIGNIFICANCE STATEMENT

The neural pathways underlying our sense of visual motion are among the most studied and well-understood parts of the primate cerebral cortex. Nevertheless, our understanding is incomplete because electrophysiological research has focused mainly on motion in the 2D frontoparallel plane, even though real-world motion often occurs in three dimensions, involving a change in distance from the viewer. Recent studies have revealed a specialization for sensing 3D motion in area MT, the cortical area most tightly linked to the processing and perception of visual motion. Our study provides the first model to explain how 3D motion sensitivity can arise in MT neurons and predicts how essential features of 2D motion integration may relate to 3D motion processing.

Integration of motion energy from overlapping random background noise increases perceived speed of coherently moving stimuli

The perception of visual motion can be profoundly influenced by visual context. To gain insight into how the visual system represents motion speed, we investigated how a background stimulus that did not move in a net direction influenced the perceived speed of a center stimulus. Visual stimuli were two overlapping random-dot patterns. The center stimulus moved coherently in a fixed direction, whereas the background stimulus moved randomly. We found that human subjects perceived the speed of the center stimulus to be significantly faster than its veridical speed when the background contained motion noise. Interestingly, the perceived speed was tuned to the noise level of the background. When the speed of the center stimulus was low, the highest perceived speed was reached when the background had a low level of motion noise. As the center speed increased, the peak perceived speed was reached at a progressively higher background noise level. The effect of speed overestimation required the center stimulus to overlap with the background. Increasing the background size within a certain range enhanced the effect, suggesting spatial integration. The speed overestimation was significantly reduced or abolished when the center stimulus and the background stimulus had different colors, or when they were placed at different depths. When the center- and background-stimuli were perceptually separable, speed overestimation was correlated with perceptual similarity between the center- and background-stimuli. These results suggest that integration of motion energy from random motion noise has a significant impact on speed perception. Our findings put new constraints on models regarding the neural basis of speed perception.

Contribution of color signals to ocular following responses

Ocular following responses (OFRs) are elicited at ultra-short latencies (< 60 ms) by sudden movements of the visual scene. In this study, we investigated the roles of color signals in OFRs in monkeys. To make physiologically isoluminant sinusoidal color gratings, we estimated the physiologically isoluminant points using OFRs and found that the physiologically isoluminant points were nearly independent of the spatiotemporal frequency of the gratings. We recorded OFRs induced by the motion of physiologically isoluminant color gratings and found that OFRs elicited by the motion of color gratings had different spatiotemporal frequency tuning from those elicited by the motion of luminance gratings. Additionally, OFRs to isoluminant color gratings had smaller peak responses, suggesting that color signals weakly contribute to OFRs compared with luminance signals. OFRs to the motion of stimuli composed of luminance and color signals were also examined. We found that color signals largely contributed to OFRs under low luminance signals regardless of whether color signals moved in the same or opposite direction to luminance signals. These results provide evidence of the multichannel visual computations underlying motor responses. We conclude that, in everyday situations, color information contributes cooperatively with luminance information to the generation of ocular tracking behaviors.

Absolute Depth Sensitivity in Cat Primary Visual Cortex under Natural Viewing Conditions

Mechanisms of 3D perception, investigated in many laboratories, have defined depth either relative to the fixation plane or to other objects in the visual scene. It is obvious that for efficient perception of the 3D world, additional mechanisms of depth constancy could operate in the visual system to provide information about absolute distance. Neurons with properties reflecting some features of depth constancy have been described in the parietal and extrastriate occipital cortical areas. It has also been shown that, for some neurons in the visual area V1, responses to stimuli of constant angular size differ at close and remote distances. The present study was designed to investigate whether, in natural free gaze viewing conditions, neurons tuned to absolute depths can be found in the primary visual cortex (area V1). Single-unit extracellular activity was recorded from the visual cortex of waking cats sitting on a trolley in front of a large screen. The trolley was slowly approaching the visual scene, which consisted of stationary sinusoidal gratings of optimal orientation rear-projected over the whole surface of the screen. Each neuron was tested with two gratings, with spatial frequency of one grating being twice as high as that of the other. Assuming that a cell is tuned to a spatial frequency, its maximum response to the grating with a spatial frequency twice as high should be shifted to a distance half way closer to the screen in order to attain the same size of retinal projection. For hypothetical neurons selective to absolute depth, location of the maximum response should remain at the same distance irrespective of the type of stimulus. It was found that about 20% of neurons in our experimental paradigm demonstrated sensitivity to particular distances independently of the spatial frequencies of the gratings. We interpret these findings as an indication of the use of absolute depth information in the primary visual cortex.

A massively asynchronous, parallel brain

Whether the visual brain uses a parallel or a serial, hierarchical, strategy to process visual signals, the end result appears to be that different attributes of the visual scene are perceived asynchronously--with colour leading form (orientation) by 40 ms and direction of motion by about 80 ms. Whatever the neural root of this asynchrony, it creates a problem that has not been properly addressed, namely how visual attributes that are perceived asynchronously over brief time windows after stimulus onset are bound together in the longer term to give us a unified experience of the visual world, in which all attributes are apparently seen in perfect registration. In this review, I suggest that there is no central neural clock in the (visual) brain that synchronizes the activity of different processing systems. More likely, activity in each of the parallel processing-perceptual systems of the visual brain is reset independently, making of the brain a massively asynchronous organ, just like the new generation of more efficient computers promise to be. Given the asynchronous operations of the brain, it is likely that the results of activities in the different processing-perceptual systems are not bound by physiological interactions between cells in the specialized visual areas, but post-perceptually, outside the visual brain.

Reward modulates the effect of visual cortical microstimulation on perceptual decisions

Effective perceptual decisions rely upon combining sensory information with knowledge of the rewards available for different choices. However, it is not known where reward signals interact with the multiple stages of the perceptual decision-making pathway and by what mechanisms this may occur. We combined electrical microstimulation of functionally specific groups of neurons in visual area V5/MT with performance-contingent reward manipulation, while monkeys performed a visual discrimination task. Microstimulation was less effective in shifting perceptual choices towards the stimulus preferences of the stimulated neurons when available reward was larger. Psychophysical control experiments showed this result was not explained by a selective change in response strategy on microstimulated trials. A bounded accumulation decision model, applied to analyse behavioural performance, revealed that the interaction of expected reward with microstimulation can be explained if expected reward modulates a sensory representation stage of perceptual decision-making, in addition to the better-known effects at the integration stage.

DOI:http://dx.doi.org/10.7554/eLife.07832.001

Identifying how an object is moving in three-dimensional (3D) space depends upon a brain region known as V5/MT. The neurons that make up area V5/MT form groups that each have a ‘preference’ for a particular direction of movement and a particular 3D depth. If a group of neurons detects its preferred direction of movement and 3D depth, it will become highly active. The brain can assess which groups of neurons are active, in a process known as integration. This information can then be used to work out the object's movement in space.

The process of integration can be influenced by whether a rewarding outcome is expected to result from identifying the 3D movement correctly. This allows the brain to increase its likelihood of success in situations where a large reward is on offer. Until now, it was thought that the activity in area V5/MT, which takes place before integration, was not affected by the likelihood of receiving a reward.

As well as being ‘naturally’ stimulated by moving objects, the V5/MT neurons can also be ‘artificially’ activated by a technique called microstimulation, which uses a tiny electrode to electrically stimulate groups of neurons. Microstimulation can bias visual perception towards the movement and 3D depth ‘preference’ of the artificially activated neurons. If the V5/MT neurons do receive information about potential rewards from other areas of the brain, we would expect rewards to affect naturally and artificially stimulated neural activity in different ways. On the other hand, if the V5/MT neurons do not receive any information about reward, then it will not matter whether their activity is natural or artificial; the signal that they produce will be the same.

Cicmil et al. gave two monkeys a task in which they could receive rewards for correctly identifying a three-dimensional cylinder's direction of rotation, and applied microstimulation to specific groups of V5/MT neurons on some of the trials. When a larger reward was available, microstimulation was less able to bias the monkeys' choices about the rotation direction of the 3D cylinders.

Overall, Cicmil et al.'s results suggest that the V5/MT neurons are able to incorporate information about reward, before integration occurs. The next step will be to record the activity of area V5/MT to investigate exactly how this happens.

DOI:http://dx.doi.org/10.7554/eLife.07832.002

Opponent backgrounds reduce discrimination sensitivity to competing motions: effects of different vertical motions on horizontal motion perception

We examined the relationship between two distinct motion phenomena. First, locally balanced stimuli in which opposing motion signals are presented spatially near one another fail to cause a robust firing pattern in brain area MT. The brain's response to this motion is effectively suppressed, a phenomenon known as opponency. Second, past research has found that discrimination sensitivity to a target motion is negatively affected by a superimposed irrelevant motion signal - a process we call "perceptual suppression." In the current study, we examined how opponency affects the strength of perceptual suppression. We found unexpected results: a target motion embedded within an opponent background was harder to discriminate than a target motion embedded within a non-opponent background. We argue that this pattern of results runs contrary to the clear prediction stemming from the current understanding of the role of opponency in motion processing and tentatively offer an explanation based on recent MT physiology.

Area MT encodes three-dimensional motion

We use visual information to determine our dynamic relationship with other objects in a three-dimensional (3D) world. Despite decades of work on visual motion processing, it remains unclear how 3D directions-trajectories that include motion toward or away from the observer-are represented and processed in visual cortex. Area MT is heavily implicated in processing visual motion and depth, yet previous work has found little evidence for 3D direction sensitivity per se. Here we use a rich ensemble of binocular motion stimuli to reveal that most neurons in area MT of the anesthetized macaque encode 3D motion information. This tuning for 3D motion arises from multiple mechanisms, including different motion preferences in the two eyes and a nonlinear interaction of these signals when both eyes are stimulated. Using a novel method for functional binocular alignment, we were able to rule out contributions of static disparity tuning to the 3D motion tuning we observed. We propose that a primary function of MT is to encode 3D motion, critical for judging the movement of objects in dynamic real-world environments.

Neural representation of motion-in-depth in area MT

Neural processing of 2D visual motion has been studied extensively, but relatively little is known about how visual cortical neurons represent visual motion trajectories that include a component toward or away from the observer (motion in depth). Psychophysical studies have demonstrated that humans perceive motion in depth based on both changes in binocular disparity over time (CD cue) and interocular velocity differences (IOVD cue). However, evidence for neurons that represent motion in depth has been limited, especially in primates, and it is unknown whether such neurons make use of CD or IOVD cues. We show that approximately one-half of neurons in macaque area MT are selective for the direction of motion in depth, and that this selectivity is driven primarily by IOVD cues, with a small contribution from the CD cue. Our results establish that area MT, a central hub of the primate visual motion processing system, contains a 3D representation of visual motion.

Unaffected smooth pursuit but impaired motion perception in monocularly enucleated observers

The objective of this paper was to study the characteristics of closed-loop smooth pursuit eye movements of 15 unilaterally eye enucleated individuals and 18 age-matched controls and to compare them to their performance in two tests of motion perception: relative motion and motion coherence. The relative motion test used a brief (150 ms) small stimulus with a continuously present fixation target to preclude pursuit eye movements. The duration of the motion coherence trials was 1s, which allowed a brief pursuit of the stimuli. Smooth pursuit data were obtained with a step-ramp procedure. Controls were tested both monocularly and binocularly. The data showed worse performance by the enucleated observers in the relative motion task but no statistically significant differences in motion coherence between the two groups. On the other hand, the smooth pursuit gain of the enucleated participants was as good as that of controls for whom we found no binocular advantage. The data show that enucleated observers do not exhibit deficits in the afferent or sensory pathways or in the efferent or motor pathways of the steady-state smooth pursuit system even though their visual processing of motion is impaired.

Perceptual asynchrony for motion

Psychophysical experiments show that two different visual attributes, color and motion, processed in different areas of the visual brain, are perceived at different times relative to each other (Moutoussis and Zeki, 1997a). Here we demonstrate psychophysically that two variants of the same attribute, motion, which have the same temporal structure and are processed in the same visual areas, are also processed asynchronously. When subjects were asked to pair up–down motion of dots in one half of their hemifield with up-right motion in the other, they perceived the two directions of motion asynchronously, with the advantage in favor of up-right motion; when they were asked to pair the motion of white dots moving against a black background with that of red dots moving against an equiluminant green background, they perceived the luminant motion first, thus demonstrating a perceptual advantage of luminant over equiluminant motion. These results were not affected by motion speed or perceived motion “streaks.” We thus interpret these results to reflect the different processing times produced by luminant and equiluminant motion stimuli or by different degrees of motion direction change, thus adding to the evidence that processing time within the visual system is a major determinant of perceptual time.

Global motion perception in 2-year-old children: a method for psychophysical assessment and relationships with clinical measures of visual function

PURPOSE

We developed and validated a technique for measuring global motion perception in 2-year-old children, and assessed the relationship between global motion perception and other measures of visual function.

METHODS

Random dot kinematogram (RDK) stimuli were used to measure motion coherence thresholds in 366 children at risk of neurodevelopmental problems at 24 ± 1 months of age. RDKs of variable coherence were presented and eye movements were analyzed offline to grade the direction of the optokinetic reflex (OKR) for each trial. Motion coherence thresholds were calculated by fitting psychometric functions to the resulting datasets. Test-retest reliability was assessed in 15 children, and motion coherence thresholds were measured in a group of 10 adults using OKR and behavioral responses. Standard age-appropriate optometric tests also were performed.

RESULTS

Motion coherence thresholds were measured successfully in 336 (91.8%) children using the OKR technique, but only 31 (8.5%) using behavioral responses. The mean threshold was 41.7 ± 13.5% for 2-year-old children and 3.3 ± 1.2% for adults. Within-assessor reliability and test-retest reliability were high in children. Children's motion coherence thresholds were significantly correlated with stereoacuity (LANG I & II test, ρ = 0.29, P < 0.001; Frisby, ρ = 0.17, P = 0.022), but not with binocular visual acuity (ρ = 0.11, P = 0.07). In adults OKR and behavioral motion coherence thresholds were highly correlated (intraclass correlation = 0.81, P = 0.001).

CONCLUSIONS

Global motion perception can be measured in 2-year-old children using the OKR. This technique is reliable and data from adults suggest that motion coherence thresholds based on the OKR are related to motion perception. Global motion perception was related to stereoacuity in children.

Spatial warping by oriented line detectors can counteract neural delays

The slow speed of neural transmission necessitates that cortical visual information from dynamic scenes will lag reality. The “perceiving the present” (PTP) hypothesis suggests that the visual system can mitigate the effect of such delays by spatially warping scenes to look as they will in ~100 ms from now (Changizi, 2001). We here show that the Hering illusion, in which straight lines appear bowed, can be induced by a background of optic flow, consistent with the PTP hypothesis. However, importantly, the bowing direction is the same whether the flow is inward or outward. This suggests that if the warping is meant to counteract latencies, it is accomplished by a simple strategy that is insensitive to motion direction, and that works only under typical (forward-moving) circumstances. We also find that the illusion strengthens with longer pulses of optic flow, demonstrating motion integration over ~80 ms. The illusion is identical whether optic flow precedes or follows the flashing of bars, exposing the spatial warping to be equally postdictive and predictive, i.e., peri-dictive. Additionally, the illusion is diminished by cues which suggest the bars are independent of the background movement. Collectively, our findings are consistent with a role for networks of visual orientation-tuned neurons (e.g., simple cells in primary visual cortex) in spatial warping. We conclude that under the common condition of forward ego-motion, spatial warping counteracts the disadvantage of neural latencies. It is not possible to prove that this is the purpose of spatial warping, but our findings at minimum place constraints on the PTP hypothesis, demonstrating that any spatial warping for the purpose of counteracting neural delays is not a precise, on-the-fly computation, but instead a heuristic achieved by a simple mechanism that succeeds under normal circumstances.

Joint representation of depth from motion parallax and binocular disparity cues in macaque area MT

Perception of depth is based on a variety of cues, with binocular disparity and motion parallax generally providing more precise depth information than pictorial cues. Much is known about how neurons in visual cortex represent depth from binocular disparity or motion parallax, but little is known about the joint neural representation of these depth cues. We recently described neurons in the middle temporal (MT) area that signal depth sign (near vs far) from motion parallax; here, we examine whether and how these neurons also signal depth from binocular disparity. We find that most MT neurons in rhesus monkeys (Macaca Mulatta) are selective for depth sign based on both disparity and motion parallax cues. However, the depth-sign preferences (near or far) are not always aligned: 56% of MT neurons have matched depth-sign preferences ("congruent" cells) whereas the remaining 44% of neurons prefer near depth from motion parallax and far depth from disparity, or vice versa ("opposite" cells). For congruent cells, depth-sign selectivity increases when disparity cues are added to motion parallax, but this enhancement does not occur for opposite cells. This suggests that congruent cells might contribute to perceptual integration of depth cues. We also found that neurons are clustered in MT according to their depth tuning based on motion parallax, similar to the known clustering of MT neurons for binocular disparity. Together, these findings suggest that area MT is involved in constructing a representation of 3D scene structure that takes advantage of multiple depth cues available to mobile observers.

Joint tuning for direction of motion and binocular disparity in macaque MT is largely separable

Neurons in sensory cortical areas are tuned to multiple dimensions, or features, of their sensory space. Understanding how single neurons represent multiple features is of great interest for determining the informative dimensions of the neurons' response, the decoding algorithms appropriate for extracting this information from the neuronal population, and for determining where specific transformations occur along the visual hierarchy. Despite the established role of cortical area MT in judgments of motion and depth, it is not known how individual neurons jointly encode the two dimensions. We investigated the joint tuning of individual MT neurons for two visual features: direction of motion and binocular disparity, an important depth cue. We found that a separable, multiplicative combination of tuning for the two features can account for more than 90% of the variance in the joint tuning function for over 91% of MT neurons. These results suggest 1) that each feature can be read out independently from MT by simply averaging across the population without regard to the other feature and 2) that the inseparable representations seen in subsequent areas, such as MST, must be computed beyond MT. Intriguingly, we found that the remaining nonseparable component of the joint tuning function often manifested as small but systematic changes in the neurons' preferences for one feature as the other one was varied. We believe this reflects the local columnar organization of tuning for direction and binocular disparity in MT, indicating that joint tuning may provide a new tool with which to probe functional architecture.

Perceptual integration for qualitatively different 3-D cues in the human brain

The visual system's flexibility in estimating depth is remarkable: We readily perceive 3-D structure under diverse conditions from the seemingly random dots of a "magic eye" stereogram to the aesthetically beautiful, but obviously flat, canvasses of the Old Masters. Yet, 3-D perception is often enhanced when different cues specify the same depth. This perceptual process is understood as Bayesian inference that improves sensory estimates. Despite considerable behavioral support for this theory, insights into the cortical circuits involved are limited. Moreover, extant work tested quantitatively similar cues, reducing some of the challenges associated with integrating computationally and qualitatively different signals. Here we address this challenge by measuring fMRI responses to depth structures defined by shading, binocular disparity, and their combination. We quantified information about depth configurations (convex "bumps" vs. concave "dimples") in different visual cortical areas using pattern classification analysis. We found that fMRI responses in dorsal visual area V3B/KO were more discriminable when disparity and shading concurrently signaled depth, in line with the predictions of cue integration. Importantly, by relating fMRI and psychophysical tests of integration, we observed a close association between depth judgments and activity in this area. Finally, using a cross-cue transfer test, we found that fMRI responses evoked by one cue afford classification of responses evoked by the other. This reveals a generalized depth representation in dorsal visual cortex that combines qualitatively different information in line with 3-D perception.

Conjunctions between motion and disparity are encoded with the same spatial resolution as disparity alone

Neurons in cortical area MT respond well to transparent streaming motion in distinct depth planes, such as caused by observer self-motion, but do not contain subregions excited by opposite directions of motion. We therefore predicted that spatial resolution for transparent motion/disparity conjunctions would be limited by the size of MT receptive fields, just as spatial resolution for disparity is limited by the much smaller receptive fields found in primary visual cortex, V1. We measured this using a novel "joint motion/disparity grating," on which human observers detected motion/disparity conjunctions in transparent random-dot patterns containing dots streaming in opposite directions on two depth planes. Surprisingly, observers showed the same spatial resolution for these as for pure disparity gratings. We estimate the limiting receptive field diameter at 11 arcmin, similar to V1 and much smaller than MT. Higher internal noise for detecting joint motion/disparity produces a slightly lower high-frequency cutoff of 2.5 cycles per degree (cpd) versus 3.3 cpd for disparity. This suggests that information on motion/disparity conjunctions is available in the population activity of V1 and that this information can be decoded for perception even when it is invisible to neurons in MT.

Modulation of human visual evoked potentials in 3-dimensional perception after stimuli produced with an integral imaging method

We investigated the neurophysiological correlates of stereoscopic 3-dimensional (3-D) depth perception by studying human visual evoked potentials (VEPs) with an integral imaging method characterized by horizontal but not vertical disparity. The VEPs were recorded in 10 healthy men under 4 conditions. In condition I, stimuli A (flat, 2-dimensional [2-D] image) and B (concave 3-D image) were presented at random. In condition II, stimuli A and C (convex 3-D image) were presented at random. In condition III, stimuli B and C were presented at random. In condition IV, stimuli A, B, and C were presented at random. The data for flat VEPs to stimulus A were combined in conditions I and II. The data for concave VEPs to stimulus B were combined in conditions I and III. The data for convex VEPs to stimulus C were combined in conditions II and III. When 2-way analysis of variance (ANOVA) for 2 factors, stimulus conditions (flat VEPs, concave VEPs, and convex VEPs) and electrode positions, was applied for VEP data, the N1 and N2 peak amplitudes differed significantly among the 3 stimulus conditions. In condition IV, the N1 peak amplitudes differed significantly among the 3 stimuli. Multiple comparisons followed by Bonferroni adjustment did not detect differences in the N1 peak amplitude between stimuli A and B, between stimuli A and C, or between stimuli B and C. We concluded that VEPs to concave or convex 3-D stimuli were significantly different from VEPs to flat 2-D stimuli. This is the first report showing modulation of human VEPs in 3-D perception with an integral imaging method.

Adaptation state of the local-motion-pooling units determines the nature of the motion aftereffect to transparent motion

When observers adapt to a transparent-motion stimulus, the resulting motion aftereffect (MAE) is typically in the direction opposite to the vector average of the component directions. It has been proposed that the reason for this is that it is the adaptation state at the local-level (i.e. of the local-motion-pooling units) that determines the nature of the MAE (Vidnyanszky et al. Trends in Cognitive Sciences, 6(4), 157-161). The adapting stimuli used in these experiments typically consisted of random-dot kinematograms, with each dot being able to move over the entire viewing aperture. Here we used spatially-localised global-plaid stimuli which enabled us, over the course of adaptation, to present either one of both motion directions at each local region. A unidirectional MAE was perceived when two motion directions were presented at each location and a transparent MAE was perceived when a single direction was presented. These results support the notion that it is the adaptation state at the local-motion-pooling level that determines the nature of the MAE to transparent motion stimuli.

Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.

Decoding conjunctions of direction-of-motion and binocular disparity from human visual cortex

Motion and binocular disparity are two features in our environment that share a common correspondence problem. Decades of psychophysical research dedicated to understanding stereopsis suggest that these features interact early in human visual processing to disambiguate depth. Single-unit recordings in the monkey also provide evidence for the joint encoding of motion and disparity across much of the dorsal visual stream. Here, we used functional MRI and multivariate pattern analysis to examine where in the human brain conjunctions of motion and disparity are encoded. Subjects sequentially viewed two stimuli that could be distinguished only by their conjunctions of motion and disparity. Specifically, each stimulus contained the same feature information (leftward and rightward motion and crossed and uncrossed disparity) but differed exclusively in the way these features were paired. Our results revealed that a linear classifier could accurately decode which stimulus a subject was viewing based on voxel activation patterns throughout the dorsal visual areas and as early as V2. This decoding success was conditional on some voxels being individually sensitive to the unique conjunctions comprising each stimulus, thus a classifier could not rely on independent information about motion and binocular disparity to distinguish these conjunctions. This study expands on evidence that disparity and motion interact at many levels of human visual processing, particularly within the dorsal stream. It also lends support to the idea that stereopsis is subserved by early mechanisms also tuned to direction of motion.

Neurons in dorsal visual area V5/MT signal relative disparity

Judgments of visual depth rely crucially on the relative binocular disparity between two visual features. While areas of ventral visual cortex contain neurons that signal the relative disparity between spatially adjacent visual features, the same tests in dorsal visual areas yield little evidence for relative disparity selectivity. We investigated the sensitivity of neurons in dorsal visual area V5/MT of macaque monkeys to relative disparity, using two superimposed, transparent planes composed of dots moving in opposite directions. The separation of the planes in depth specifies their relative disparity, while absolute disparity can be altered independently by changing the binocular depth of the two planes with respect to the monkey's fixation point. Many V5/MT neurons were tuned to relative disparity, independent of the absolute disparities of the individual planes. For the two plane stimulus, neuronal responses were often linearly related to responses to the absolute disparity of each component plane presented individually, but some aspects of relative disparity tuning were not explained by linear combination. Selectivity for relative disparity could not predict whether neuronal firing was related to the monkeys' perceptual reports of the rotation direction of structure-from-motion figures centered on the plane of fixation. In sum, V5/MT neurons are not just selective for absolute disparity, but also code for relative disparity between visual features. This selectivity may be important for segmentation and depth order of moving visual features, particularly the processing of three-dimensional information in scenes viewed by an actively moving observer.

Spikes, BOLD, attention, and awareness: a comparison of electrophysiological and fMRI signals in V1

Early fMRI studies comparing results from fMRI and electrophysiological experiments support the notion that the blood oxygen level-dependent (BOLD) signal reliably follows the spiking activity of an underlying neuronal population averaged across a small region in space and a brief period in time. However, more recent studies focusing on higher level cognitive factors such as attention and visual awareness report striking discrepancies between the fMRI response in humans and electrophysiological signals in macaque early visual areas. Four hypotheses are discussed that can explain the discrepancies between the two methods: (1) the BOLD signal follows local field potential (LFP) signals closer than spikes, and only the LFP is modulated by top-down factors, (2) the BOLD signal is reflecting electrophysiological signals that are occurring later in time due to feedback delay, (3) the BOLD signal is more sensitive than traditional electrophysiological methods due to massive pooling by the hemodynamic coupling process, and finally (4) there is no real discrepancy, and instead, weak but reliable effects on firing rates may be obscured by differences in experimental design and interpretation of results across methods.

The Pulfrich effect in the clinic

First described in 1922 by Carl Pulfrich, the Pulfrich effect is a stereo-illusion thought to be caused by an inter-ocular signal latency difference stimulating neurons jointly tuned to disparity and motion. Clinically, this can be a spontaneous manifestation due to various ocular and central visual pathway pathologies, and cause symptoms independent of a range of routine visual parameters which may seem bizarre to both the patient and the clinician. Eliciting such symptoms of difficulties with motion and depth perception in a clinical history should direct the clinician to the possibility of the presence of the spontaneous Pulfrich effect, and to proceed to test for it. A spontaneous Pulfrich effect may be the first presentation of a previously undiagnosed condition, and warrants further investigation to identify the underlying cause. Alternatively, a spontaneous Pulfrich effect may be the unrecognized complication of a previously diagnosed condition. The symptoms associated with a spontaneous Pulfrich effect can be managed by treating the underlying cause in reversible conditions, or by prescribing appropriate monocular tinted lenses in front of the unaffected or less affected eye in persistent conditions.

On the inverse problem of binocular 3D motion perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy.

Humans and many other predators have two eyes that are set a short distance apart so that an extensive region of the world is seen simultaneously by both eyes from slightly different points of view. Although the images of the world are essentially two-dimensional, we vividly see the world as three-dimensional. This is true for static as well as dynamic images. Here we elaborate on how the visual system may establish 3D motion perception from local input in the left and right eye. Using tools from analytic geometry we show that existing 3D motion models offer no general solution to the inverse optics problem of 3D motion perception. We suggest a flexible framework of motion and depth processing and suggest default strategies for local 3D motion estimation. Our results on the aperture and inverse problem of 3D motion are likely to stimulate computational, behavioral, and neuroscientific studies because they address the fundamental issue of how 3D motion is represented in the visual system.

Binocular vision

This essay reviews major developments - empirical and theoretical - in the field of binocular vision during the last 25years. We limit our survey primarily to work on human stereopsis, binocular rivalry and binocular contrast summation, with discussion where relevant of single-unit neurophysiology and human brain imaging. We identify several key controversies that have stimulated important work on these problems. In the case of stereopsis those controversies include position vs. phase encoding of disparity, dependence of disparity limits on spatial scale, role of occlusion in binocular depth and surface perception, and motion in 3D. In the case of binocular rivalry, controversies include eye vs. stimulus rivalry, role of "top-down" influences on rivalry dynamics, and the interaction of binocular rivalry and stereopsis. Concerning binocular contrast summation, the essay focuses on two representative models that highlight the evolving complexity in this field of study.

Stereoscopic vision in the absence of the lateral occipital cortex

Both dorsal and ventral cortical visual streams contain neurons sensitive to binocular disparities, but the two streams may underlie different aspects of stereoscopic vision. Here we investigate stereopsis in the neurological patient D.F., whose ventral stream, specifically lateral occipital cortex, has been damaged bilaterally, causing profound visual form agnosia. Despite her severe damage to cortical visual areas, we report that DF's stereo vision is strikingly unimpaired. She is better than many control observers at using binocular disparity to judge whether an isolated object appears near or far, and to resolve ambiguous structure-from-motion. DF is, however, poor at using relative disparity between features at different locations across the visual field. This may stem from a difficulty in identifying the surface boundaries where relative disparity is available. We suggest that the ventral processing stream may play a critical role in enabling healthy observers to extract fine depth information from relative disparities within one surface or between surfaces located in different parts of the visual field.

Disparity- and velocity-based signals for three-dimensional motion perception in human MT+

How does the primate visual system encode three-dimensional motion? The macaque middle temporal area (MT) and the human MT complex (MT+) have well-established sensitivity to two-dimensional frontoparallel motion and static disparity. However, evidence for sensitivity to three-dimensional motion has remained elusive. We found that human MT+ encodes two binocular cues to three-dimensional motion: changing disparities over time and interocular comparisons of retinal velocities. By varying important properties of moving dot displays, we distinguished these three-dimensional motion signals from their constituents, instantaneous binocular disparity and monocular retinal motion. An adaptation experiment confirmed direction selectivity for three-dimensional motion. Our results indicate that MT+ carries critical binocular signals for three-dimensional motion processing, revealing an important and previously overlooked role for this well-studied brain area.

The motion/pursuit law for visual depth perception from motion parallax

One of vision's most important functions is specification of the layout of objects in the 3D world. While the static optical geometry of retinal disparity explains the perception of depth from binocular stereopsis, we propose a new formula to link the pertinent dynamic geometry to the computation of depth from motion parallax. Mathematically, the ratio of retinal image motion (motion) and smooth pursuit of the eye (pursuit) provides the necessary information for the computation of relative depth from motion parallax. We show that this could have been obtained with the approaches of Nakayama and Loomis [Nakayama, K., & Loomis, J. M. (1974). Optical velocity patterns, velocity-sensitive neurons, and space perception: A hypothesis. Perception, 3, 63-80] or Longuet-Higgens and Prazdny [Longuet-Higgens, H. C., & Prazdny, K. (1980). The interpretation of a moving retinal image. Proceedings of the Royal Society of London Series B, 208, 385-397] by adding pursuit to their treatments. Results of a psychophysical experiment show that changes in the motion/pursuit ratio have a much better relationship to changes in the perception of depth from motion parallax than do changes in motion or pursuit alone. The theoretical framework provided by the motion/pursuit law provides the quantitative foundation necessary to study this fundamental visual depth perception ability.

Pulfrich phenomena are coded effectively by a joint motion-disparity process

Pulfrich phenomena are a class of depth illusions generated by an interocular time delay. This may be demonstrated with continuously moving stimuli, stroboscopic displays undergoing apparent motion, or dynamic noise patterns. Previous studies suggest that neurons jointly tuned to motion and disparity may be responsible for the phenomena. Model cells with such joint coding can explain all Pulfrich phenomena in a unified way (N. Qian & R. A. Andersen, 1997). However, the joint-coding idea has been challenged by recent models (J. C. Read & B. G. Cumming, 2005a, 2005c) that focus on the S shaped functions of perceived disparity in stroboscopic Pulfrich effect (M. J. Morgan, 1979). Here we demonstrate fundamental problems with the recent models in terms of causality, physiological plausibility, and definitions for joint and separate coding, and we compare the two coding schemes under physiologically plausible assumptions. We show that joint coding of disparity and either unidirectional or bidirectional motion selectivity can account for the S curves, but unidirectional selectivity is required to explain direction-depth contingency in Pulfrich effects. In contrast, separate coding can explain neither the S curves nor the direction-depth contingency. We conclude that Pulfrich phenomena are logically accounted for by joint encoding of unidirectional-motion and disparity.

What a difference a parameter makes: a psychophysical comparison of random dot motion algorithms

Random dot motion (RDM) displays have emerged as one of the standard stimulus types employed in psychophysical and physiological studies of motion processing. RDMs are convenient because it is straightforward to manipulate the relative motion energy for a given motion direction in addition to stimulus parameters such as the speed, contrast, duration, density, aperture, etc. However, as widely as RDMs are employed so do they vary in their details of implementation. As a result, it is often difficult to make direct comparisons across studies employing different RDM algorithms and parameters. Here, we systematically measure the ability of human subjects to estimate motion direction for four commonly used RDM algorithms under a range of parameters in order to understand how these different algorithms compare in their perceptibility. We find that parametric and algorithmic differences can produce dramatically different performances. These effects, while surprising, can be understood in relationship to pertinent neurophysiological data regarding spatiotemporal displacement tuning properties of cells in area MT and how the tuning function changes with stimulus contrast and retinal eccentricity. These data help give a baseline by which different RDM algorithms can be compared, demonstrate a need for clearly reporting RDM details in the methods of papers, and also pose new constraints and challenges to models of motion direction processing.

Evidence for joint encoding of motion and disparity in human visual perception

Electrophysiological recordings have established that motion and disparity signals are jointly encoded by subpopulations of neurons in visual cortex. However, the question of whether these neurons play a perceptual role has proven challenging and remains open. To answer this question we combined two powerful psychophysical techniques: perceptual adaptation and reverse correlation. Our results provide a detailed picture of how visual information about motion and disparity is processed by human observers, and how this processing is modified by prolonged sensory stimulation. We were able to isolate two perceptual components: a separable component, supported by separate motion and disparity signals, and an inseparable joint component, supported by motion and disparity signals that are concurrently represented at the level of the same neural mechanism. Both components are involved in the perception of stimuli containing motion and disparity information in line with the known existence of corresponding neuronal subpopulations in visual cortex.

Bi-stable depth ordering of superimposed moving gratings

Ambiguous stimuli with two distinct interpretations give rise to perceptual alternations between them. During prolonged viewing of transparently moving gratings, observers report periods of perceiving one grating in front of the other, alternating with periods of the reverse depth ordering. We measured the percepts' dominance times to study the effect of depth cues (wavelength, duty cycle, and speed) on the perceived depth ordering. The grating with shorter wavelength, lower duty cycle, or higher speed was perceived as being behind the other for a fraction of time larger than one half. The fraction of time spent perceiving each grating as behind changed gradually as a function of the parameters. The fraction of dominance depended on the ratio between the gratings' wavelengths, not on their absolute sizes. The wavelength ratio had a stronger effect on perceived depth than that of duty cycle or speed and could override stereoscopic disparity cues. Similar results were obtained with superimposed moving surfaces of random dots. The findings are interpreted in terms of their relation to statistical properties of natural surfaces and provide evidence that the fraction of dominance of each percept represents the likelihood that it corresponds to the true interpretation of the underlying scene.

Spatial summation properties of the human ocular following response (OFR): evidence for nonlinearities due to local and global inhibitory interactions

Ocular following responses (OFRs) are the initial tracking eye movements that can be elicited at ultra-short latency by sudden motion of a textured pattern. A recent study used motion stimuli consisting of two large coextensive sine-wave gratings with the same orientation but different spatial frequency and moving in (1/4)-wavelength steps in the same or opposite directions: when the two gratings differed in contrast by more than about an octave then the one with the higher contrast completely dominated the OFR and the one with lower contrast lost its influence as though suppressed [Sheliga, B. M., Kodaka, Y., FitzGibbon, E. J., & Miles, F. A. (2006). Human ocular following initiated by competing image motions: Evidence for a winner-take-all mechanism. Vision Research, 46, 2041-2060]. This winner-take-all (WTA) outcome was attributed to nonlinear interactions in the form of mutual inhibition between the mechanisms sensing the competing motions. In the present study, we recorded the initial horizontal OFRs to the horizontal motion of two vertical sine-wave gratings that differed in spatial frequency and were each confined to horizontal strips that extended the full width of our display (45 degrees ) but were only 1-2 degrees high. The two gratings could be coextensive or separated by a vertical gap of up to 8 degrees , and each underwent motion consisting of successive (1/4)-wavelength steps. Initial OFRs showed strong dependence on the relative contrasts of the competing gratings and when these were coextensive this dependence was always highly nonlinear (WTA), regardless of whether the two gratings moved in the same or opposite direction. When the two gratings moved in opposite directions the nonlinear interactions were purely local: with a vertical gap of 1 degrees or more between the gratings OFRs approximated the linear sum of the responses to each grating alone. On the other hand, when the two gratings moved in the same direction the nonlinear interactions were more global: even with a gap of 8 degrees -the largest separation tried-OFRs were still substantially less than predicted by the linear sum. When the motions were in the same direction, we postulate two nonlinear interactions: local mutual inhibition (resulting in WTA) and global divisive inhibition (resulting in normalization). Motion stimuli whose responses were totally suppressed by coextensive opponent motion of higher contrast were rendered invisible to normalization, suggesting that the local interactions responsible for the WTA behavior here occur at an earlier stage of neural processing than the global interactions responsible for normalization.

Responses to direction and transparent motion stimuli in area FST of the macaque

Motion transparency occurs when multiple object velocities are present within a local region of retinotopic space. Transparent signals can carry information useful in the segmentation of moving objects and in the extraction of three-dimensional structure from relative motion cues. However, the physiological substrate underlying the detection of motion transparency is poorly understood. Direction tuned neurons in area MT are suppressed by transparent stimuli, suggesting that other motion sensitive areas may be needed to represent this signal robustly. Recent neuroimaging evidence implicated two such areas in the macaque superior temporal sulcus. We studied one of these, FST, with electrophysiological methods and found that a large fraction of the neurons responded well to two opposite directions of motion and to transparent stimuli containing those same directions. A linear combination of MT-like responses qualitatively reproduces this behavior and predicts that FST neurons can be tuned for transparent motion containing specific direction and depth components. We suggest that FST plays a role in motion segmentation based on transparent signals.

A neural representation of depth from motion parallax in macaque visual cortex

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal's movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.

Higher order visual processing in macaque extrastriate cortex

The extrastriate cortex of primates encompasses a substantial portion of the cerebral cortex and is devoted to the higher order processing of visual signals and their dispatch to other parts of the brain. A first step towards the understanding of the function of this cortical tissue is a description of the selectivities of the various neuronal populations for higher order aspects of the image. These selectivities present in the various extrastriate areas support many diverse representations of the scene before the subject. The list of the known selectivities includes that for pattern direction and speed gradients in middle temporal/V5 area; for heading in medial superior temporal visual area, dorsal part; for orientation of nonluminance contours in V2 and V4; for curved boundary fragments in V4 and shape parts in infero-temporal area (IT); and for curvature and orientation in depth from disparity in IT and CIP. The most common putative mechanism for generating such emergent selectivity is the pattern of excitatory and inhibitory linear inputs from the afferent area combined with nonlinear mechanisms in the afferent and receiving area.

Integrating motion and depth via parallel pathways

Processing of visual information is both parallel and hierarchical, with each visual area richly interconnected with other visual areas. An example of the parallel architecture of the primate visual system is the existence of two principal pathways providing input to the middle temporal visual area (MT): namely, a direct projection from striate cortex (V1), and a set of indirect projections that also originate in V1 but then relay through V2 and V3. Here we have reversibly inactivated the indirect pathways while recording from MT neurons and measuring eye movements in alert monkeys, a procedure that has enabled us to assess whether the two different input pathways are redundant or whether they carry different kinds of information. We find that this inactivation causes a disproportionate degradation of binocular disparity tuning relative to direction tuning in MT neurons, suggesting that the indirect pathways are important in the recovery of depth in three-dimensional scenes.