Conditions for motion flow in dynamic visual noise

A Framework for the Continuous Evaluation of 3D Motion Perception in Virtual Reality

OBJECTIVE

We present a novel framework for the detection and continuous evaluation of 3D motion perception by deploying a virtual reality environment with built-in eye tracking.

METHODS

We created a biologically-motivated virtual scene that involved a ball moving in a restricted Gaussian random walk against a background of 1/f noise. Sixteen visually healthy participants were asked to follow the moving ball while their eye movements were monitored binocularly using the eye tracker. We calculated the convergence positions of their gaze in 3D using their fronto-parallel coordinates and linear least-squares optimization. Subsequently, to quantify 3D pursuit performance, we employed a first-order linear kernel analysis known as the Eye Movement Correlogram technique to separately analyze the horizontal, vertical and depth components of the eye movements. Finally, we checked the robustness of our method by adding systematic and variable noise to the gaze directions and re-evaluating 3D pursuit performance.

RESULTS

We found that the pursuit performance in the motion-through depth component was reduced significantly compared to that for fronto-parallel motion components. We found that our technique was robust in evaluating 3D motion perception, even when systematic and variable noise was added to the gaze directions.

CONCLUSION

The proposed framework enables the assessment of 3D Motion perception by evaluating continuous pursuit performance through eye-tracking.

SIGNIFICANCE

Our framework paves the way for a rapid, standardized and intuitive assessment of 3D motion perception in patients with various eye disorders.

Mechanisms Underlying Directional Motion Processing and Form-Motion Integration Assessed with Visual Perceptual Learning

Dynamic Glass patterns (GPs) are visual stimuli commonly employed to study form–motion interactions. There is brain imaging evidence that non-directional motion induced by dynamic GPs and directional motion induced by random dot kinematograms (RDKs) depend on the activity of the human motion complex (hMT+). However, whether dynamic GPs and RDKs rely on the same processing mechanisms is still up for dispute. The current study uses a visual perceptual learning (VPL) paradigm to try to answer this question. Identical pre- and post-tests were given to two groups of participants, who had to discriminate random/noisy patterns from coherent form (dynamic GPs) and motion (RDKs). Subsequently, one group was trained on dynamic translational GPs, whereas the other group on RDKs. On the one hand, the generalization of learning to the non-trained stimulus would indicate that the same mechanisms are involved in the processing of both dynamic GPs and RDKs. On the other hand, learning specificity would indicate that the two stimuli are likely to be processed by separate mechanisms possibly in the same cortical network. The results showed that VPL is specific to the stimulus trained, suggesting that directional and non-directional motion may depend on different neural mechanisms.

Visual Short-Term Memory for Coherent and Sequential Motion: A rTMS Investigation

We investigated the role of the human medio-temporal complex (hMT+) in the memory encoding and storage of a sequence of four coherently moving random dot kinematograms (RDKs), by applying repetitive transcranial magnetic stimulation (rTMS) during an early or late phase of the retention interval. Moreover, in a second experiment, we also tested whether disrupting the functional integrity of hMT+ during the early phase impaired the precision of the encoded motion directions. Overall, results showed that both recognition accuracy and precision were worse in middle serial positions, suggesting the occurrence of primacy and recency effects. We found that rTMS delivered during the early (but not the late) phase of the retention interval was able to impair not only recognition of RDKs, but also the precision of the retained motion direction. However, such impairment occurred only for RDKs presented in middle positions along the presented sequence, where performance was already closer to chance level. Altogether these findings suggest an involvement of hMT+ in the memory encoding of visual motion direction. Given that both position sequence and rTMS modulated not only recognition but also the precision of the stored information, these findings are in support of a model of visual short-term memory with a variable resolution of each stored item, consistent with the assigned amount of memory resources, and that such item-specific memory resolution is supported by the functional integrity of area hMT+.

Overconfidence in visual perception in parkinson's disease

Increased dependence on visual cues in Parkinson's disease (PD) can unbalance the perception-action loop, impair multisensory integration, and affect everyday function of PD patients. It is currently unknown why PD patients seem to be more reliant on their visual cues. We hypothesized that PD patients may be overconfident in the reliability (precision) of their visual cues. In this study we tested coherent visual motion perception in PD, and probed subjective (self-reported) confidence in their visual motion perception. Twenty patients with idiopathic PD, 21 healthy aged-matched controls and 20 healthy young adult participants were presented with visual stimuli of moving dots (random dot kinematograms). They were asked to report: (1) whether the aggregate motion of dots was to the left or to the right, and (2) how confident they were that their perceptual discrimination was correct. Visual motion discrimination thresholds were similar (unimpaired) in PD compared to the other groups. By contrast, PD patients were significantly overconfident in their visual perceptual decisions (p = .002 and p < .001 vs. the age-matched and young adult groups, respectively). These results suggest intact visual motion perception, but overestimation of visual cue reliability, in PD. Overconfidence in visual (vs. other, e.g., somatosensory) cues could underlie increased visual dependence and impaired multisensory/sensorimotor integration in PD. It could thereby contribute to gait and balance impairments, and affect everyday activities, such as driving. Future work should investigate and compare PD confidence in somatosensory function. A better understanding of altered sensory reliance might open up new avenues to treat debilitating PD symptoms.

A response-locking protocol to boost sensitivity for fMRI-based neurochronometry

The timeline of brain‐wide neural activity relative to a behavioral event is crucial when decoding the neural implementation of a cognitive process. Yet, fMRI assesses neural processes indirectly via delayed and regionally variable hemodynamics. This method‐inherent temporal distortion impacts the interpretation of behavior‐linked neural timing. Here we describe a novel behavioral protocol that aims at disentangling the BOLD dynamics of the pre‐ and post‐response periods in response time tasks. We tested this response‐locking protocol in a perceptual decision‐making (random dot) task. Increasing perceptual difficulty produced expected activity increases over a broad network involving the lateral/medial prefrontal cortex and the anterior insula. However, response‐locking revealed a previously unreported functional dissociation within this network. preSMA and anterior premotor cortex (prePMV) showed post‐response activity modulations while anterior insula and anterior cingulate cortex did not. Furthermore, post‐response BOLD activity at preSMA showed a modulation in timing but not amplitude while this pattern was reversed at prePMV. These timeline dissociations with response‐locking thus revealed three functionally distinct sub‐networks in what was seemingly one shared distributed network modulated by perceptual difficulty. These findings suggest that our novel response‐locked protocol could boost the timing‐related sensitivity of fMRI.

Opto-locomotor reflexes of mice to reverse-phi stimuli

In a reverse-phi stimulus, the contrast luminance of moving dots is reversed each displacement step. Under those conditions, the direction of the moving dots is perceived in the direction opposite of the displacement direction of the dots. In this study, we investigate if mice respond oppositely to phi and reverse-phi stimuli. Mice ran head-fixed on a Styrofoam ball floating on pressurized air at the center of a large dome. We projected random dot patterns that were displaced rightward or leftward, using either a phi or a reverse-phi stimulus. For phi stimuli, changes in direction caused the mice to reflexively compensate and adjust their running direction in the direction of the displaced pattern. We show that for reverse-phi stimuli mice compensate in the direction opposite to the displacement direction of the dots, in accordance with the perceived direction of displacement in humans for reverse-phi stimuli.

Visual short-term memory for coherent motion in video game players: evidence from a memory-masking paradigm

In this study, we investigated visual short-term memory for coherent motion in action video game players (AVGPs), non-action video game players (NAVGPs), and non-gamers (control group: CONs). Participants performed a visual memory-masking paradigm previously used with macaque monkeys and humans. In particular, we tested whether video game players form a more robust visual short-term memory trace for coherent moving stimuli during the encoding phase, and whether such memory traces are less affected by an intervening masking stimulus presented 0.2 s after the offset of the to-be-remembered sample. The results showed that task performance of all groups was affected by the masking stimulus, but video game players were affected to a lesser extent than controls. Modelling of performance values and reaction times revealed that video game players have a lower guessing rate than CONs, and higher drift rates than CONs, indicative of more efficient perceptual decisions. These results suggest that video game players exhibit a more robust VSTM trace for moving objects and this trace is less prone to external interference.

Low-level mediation of directionally specific motion aftereffects: Motion perception is not necessary

Previous psychophysical experiments with normal human observers have shown that adaptation to a moving dot stream causes directionally specific repulsion in the perceived angle of a subsequently viewed moving probe. In this study, we used a two-alternative forced choice task with roving pedestals to determine the conditions that are necessary and sufficient for producing directionally specific repulsion with compound adaptors, each of which contains two oppositely moving, differently colored component streams. Experiment 1 provided a demonstration of repulsion between single-component adaptors and probes moving at approximately 90° or 270°. In Experiment 2, oppositely moving dots in the adaptor were paired to preclude the appearance of motion. Nonetheless, repulsion remained strong when the angle between each probe stream and one component was approximately 30°. In Experiment 3, adapting dot pairs were kept stationary during their limited lifetimes. Their orientation content alone proved insufficient for producing repulsion. In Experiments 4–6, the angle between the probe and both adapting components was approximately 90° or 270°. Directional repulsion was found when observers were asked to visually track one of the adapting components (Exp. 6), but not when they were asked to attentionally track it (Exp. 5), nor while they passively viewed the adaptor (Exp. 4). Our results are consistent with a low-level mechanism for motion adaptation. This mechanism is not selective for stimulus color and is not susceptible to attentional modulation. The most likely cortical locus of adaptation is area V1.

Binocular Mechanisms of 3D Motion Processing

The visual system must recover important properties of the external environment if its host is to survive. Because the retinae are effectively two-dimensional but the world is three-dimensional (3D), the patterns of stimulation both within and across the eyes must be used to infer the distal stimulus-the environment-in all three dimensions. Moreover, animals and elements in the environment move, which means the input contains rich temporal information. Here, in addition to reviewing the literature, we discuss how and why prior work has focused on purported isolated systems (e.g., stereopsis) or cues (e.g., horizontal disparity) that do not necessarily map elegantly on to the computations and complex patterns of stimulation that arise when visual systems operate within the real world. We thus also introduce the binoptic flow field (BFF) as a description of the 3D motion information available in realistic environments, which can foster the use of ecologically valid yet well-controlled stimuli. Further, it can help clarify how future studies can more directly focus on the computations and stimulus properties the visual system might use to support perception and behavior in a dynamic 3D world.

Linear Summation of Repulsive and Attractive Serial Dependencies: Orientation and Motion Dependencies Sum in Motion Perception

Recent work from several groups has shown that perception of various visual attributes in human observers at a given moment is biased toward what was recently seen. This positive serial dependency is a kind of temporal averaging that exploits short-term correlations in visual scenes to reduce noise and stabilize perception. To date, this stabilizing "continuity field" has been demonstrated on stable visual attributes such as orientation and face identity, yet it would be counterproductive to apply it to dynamic attributes in which change sensitivity is needed. Here, we tested this using motion direction discrimination and predict a negative perceptual dependency: a contrastive relationship that enhances sensitivity to change. Surprisingly, our data showed a cubic-like pattern of dependencies with positive and negative components. By interleaving various stimulus combinations, we separated the components and isolated a positive perceptual dependency for motion and a negative dependency for orientation. A weighted linear sum of the separate dependencies described the original cubic pattern well. The positive dependency for motion shows an integrative perceptual effect and was unexpected, although it is consistent with work on motion priming. These findings suggest that a perception-stabilizing continuity field occurs pervasively, occurring even when it obscures sensitivity to dynamic stimuli.SIGNIFICANCE STATEMENT Recent studies show that visual perception at a given moment is not entirely veridical, but rather biased toward recently seen stimuli: a positive serial dependency. This temporal smoothing process helps perceptual continuity by preserving stable aspects of the visual scene over time, yet, for dynamic stimuli, temporal smoothing would blur dynamics and reduce sensitivity to change. We tested whether this process is selective for stable attributes by examining dependencies in motion perception. We found a clear positive dependency for motion, suggesting that positive perceptual dependencies are pervasive. We also found a concurrent negative (contrastive) dependency for orientation. Both dependencies combined linearly to determine perception, showing that the brain can calculate contrastive and integrative dependencies simultaneously from recent stimulus history when making perceptual decisions.

Evidence and Counterevidence in Motion Perception

Sensory neurons gather evidence in favor of the specific stimuli to which they are tuned, but they could improve their sensitivity by also taking counterevidence into account. The Bours-Lankheet model for motion detection uses counterevidence that relies on a specific combination of the ON and OFF channels in the early visual system. Specifically, the model detects pairs of flashes that occur separated in space and time. If the flashes have the same contrast polarity, they are interpreted as evidence in favor of the corresponding motion. But if they have opposite contrasts, they are interpreted as evidence against it. This mechanism provides an explanation for reverse-phi (the perceived reversal of an apparent motion stimulus due to periodic contrast-inversions) that is a conceptual departure from the standard explanations of the effect. Here, we investigate this counterevidence mechanism by measuring directional tuning curves of neurons in the primary visual and middle temporal cortex areas of awake, behaving macaques using constant-contrast and inverting-contrast moving dot stimuli. Our electrophysiological data support the Bours-Lankheet model and suggest that the counterevidence computation occurs at an early stage of neural processing not captured by the standard models.

Comparing fixed and collapsing boundary versions of the diffusion model

Optimality studies and studies of decision-making in monkeys have been used to support a model in which the decision boundaries used to evaluate evidence collapse over time. This article investigates whether a diffusion model with collapsing boundaries provides a better account of human data than a model with fixed boundaries. We compared the models using data from four new numerosity discrimination experiments and two previously published motion discrimination experiments. When model selection was based on BIC values, the fixed boundary model was preferred over the collapsing boundary model for all of the experiments. When model selection was carried out using a parametric bootstrap cross-fitting method (PBCM), which takes into account the flexibility of the alternative models and the ability of one model to account for data from another model, data from 5 of 6 experiments favored either fixed boundaries or boundaries with only negligible collapse. We found that the collapsing boundary model produces response times distributions with the same shape as those produced by the fixed boundary model and that its parameters were not well-identified and were difficult to recover from data. Furthermore, the estimated boundaries of the best-fitting collapsing boundary model were relatively flat and very similar to those of the fixed-boundary model. Overall, a diffusion model with decision boundaries that converge over time does not provide an improvement over the standard diffusion model for our tasks with human data.

Transient—Sustained Input to Directionally Selective Motion Mechanisms

The characteristics of the sustained input to directionally selective motion sensors were examined in three human psychophysical studies on directional-motion discrimination. Apparent motion was produced by displaying a group of dots in two frames (F1 and F2), where F2 was a translated version of F1. All stimuli included parts that contained both F1 and F2 (combined images) and parts containing only F1 or F2 (single images). All displays began with a single image (F1), continued with the combined image, and ended with F2. Six durations of single and of combined images (10, 20, 40, 80, 160, or 320 ms) were crossed factorially. As the duration of the single image was increased, perception of directional motion first improved, and then declined at longer durations. This outcome contrasted with the monotonic increment obtained in earlier studies under low-luminance conditions. To account for the entire pattern of results, earlier models of the Reichardt motion sensor were modified so as to include a mixed transient – sustained input to one of the filters of the sensor. Predictions from the new model were tested and confirmed in two experiments carried out under both low-luminance and high-luminance viewing conditions.

Noisy decision thresholds can account for suboptimal detection of low coherence motion

Noise in sensory signals can vary over both space and time. Moving random dot stimuli are commonly used to quantify how the visual system accounts for spatial noise. In these stimuli, a fixed proportion of "signal" dots move in the same direction and the remaining "noise" dots are randomly replotted. The spatial coherence, or proportion of signal versus noise dots, is fixed across time; however, this means that little is known about how temporally-noisy signals are integrated. Here we use a stimulus with low temporal coherence; the signal direction is only presented on a fraction of frames. Human observers are able to reliably detect and discriminate the direction of a 200 ms motion pulse, even when just 25% of frames within the pulse move in the signal direction. Using psychophysical reverse-correlation analyses, we show that observers are strongly influenced by the number of near-target directions spread throughout the pulse, and that consecutive signal frames have only a small additional influence on perception. Finally, we develop a model inspired by the leaky integration of the responses of direction-selective neurons, which reliably represents motion direction, and which can account for observers' sub-optimal detection of motion pulses by incorporating a noisy decision threshold.

Sensitivity of a harbor seal (Phoca vitulina) to coherent visual motion in random dot displays

Motion vision is one of the fundamental properties of the visual system and is involved in numerous tasks. Previous work has shown that harbor seals are able to perceive visual motion. Tying in with this experimental finding, we assessed the sensitivity of harbor seals to visual motion using random dot displays. In these random dot displays, either all or a percentage of the dots plotted in the display area move into one direction which is referred to as percent coherence. Using random dot displays allows determining motion sensitivity free from form or position cues. Moreover, when reducing the lifetime of the dots, the experimental subjects need to rely on the global motion over the display area instead of on local motion events, such as the streaks of single dots. For marine mammals, the interpretation of global motion stimuli seems important in the context of locomotion, orientation and foraging. The first experiment required the seal to detect coherent motion directed upwards in one out of two stimulus displays and psychophysical motion coherence detection thresholds were obtained ranging from 5% to 35% coherence. At the beginning of the second experiment, which was conducted to reduce the differential flickering of the motion stimulus as secondary cue, the seal was directly able to transfer from coherent motion detection to a discrimination of coherent motion direction, leftward versus rightward. The seal performed well even when the duration of the local motion event was extremely short in the last experiment, in which noise was programmed as random position noise. Its coherence threshold was determined at 23% coherence in this experiment. This motion sensitivity compares well to the performance of most species tested so far excluding monkeys, humans and cats. To conclude, harbor seals possess an effective global motion processing system. For seals, the interpretation of global and coherent motion might e. g. play a role in the interpretation of optic flow information or when breaking the camouflage of cryptic prey items.

Testing neuronal accounts of anisotropic motion perception with computational modelling

There is an over-representation of neurons in early visual cortical areas that respond most strongly to cardinal (horizontal and vertical) orientations and directions of visual stimuli, and cardinal- and oblique-preferring neurons are reported to have different tuning curves. Collectively, these neuronal anisotropies can explain two commonly-reported phenomena of motion perception – the oblique effect and reference repulsion – but it remains unclear whether neuronal anisotropies can simultaneously account for both perceptual effects. We show in psychophysical experiments that reference repulsion and the oblique effect do not depend on the duration of a moving stimulus, and that brief adaptation to a single direction simultaneously causes a reference repulsion in the orientation domain, and the inverse of the oblique effect in the direction domain. We attempted to link these results to underlying neuronal anisotropies by implementing a large family of neuronal decoding models with parametrically varied levels of anisotropy in neuronal direction-tuning preferences, tuning bandwidths and spiking rates. Surprisingly, no model instantiation was able to satisfactorily explain our perceptual data. We argue that the oblique effect arises from the anisotropic distribution of preferred directions evident in V1 and MT, but that reference repulsion occurs separately, perhaps reflecting a process of categorisation occurring in higher-order cortical areas.

Two tales of how expectation of reward modulates behavior

Expectation of reward modulates many types of behaviors. Here we highlight two lines of research on reward-modulated perceptual decision making in primates and social context-modulated singing in songbirds, respectively. These two seemingly distinct behaviors are both known to involve cortico-basal ganglia-thalamic circuits. The underlying computations may be conceptualized using a simple, common framework. We summarize and compare our current knowledge of the two fields to motivate new experiments for each field, with the goal of finding general principles for how the brain implements reward-modulated behavior.

The speed and accuracy of perceptual decisions in a random-tone pitch task

Research in perceptual decision making is dominated by paradigms that tap the visual system, such as the random-dot motion (RDM) paradigm. In this study, we investigated whether the behavioral signature of perceptual decisions in the auditory domain is similar to those observed in the visual domain. We developed an auditory version of the RDM task, in which tones correspond to dots and pitch corresponds to motion (the random-tone pitch task, RTP). In this task, participants have to decide quickly whether the pitch of a “sound cloud” of tones is moving up or down. Stimulus strength and speed–accuracy trade-off were manipulated. To describe the relationship between stimulus strength and performance, we fitted the proportional-rate diffusion model to the data. The results showed a close coupling between stimulus strength and the speed and accuracy of perceptual decisions in both tasks. Additionally, we fitted the full drift diffusion model (DDM) to the data and showed that three of the four participants had similar speed–accuracy trade-offs in both tasks. However, for the RTP task, drift rates were larger and nondecision times slower, suggesting that some DDM parameters might be dependent on stimulus modality (drift rate and nondecision time), whereas others might not be (decision bound). The results illustrate that the RTP task is suitable for investigating the dynamics of auditory perceptual choices. Future studies using the task might help to investigate modality-specific effects on decision making at both the behavioral and neuronal levels.

The basal ganglia's contributions to perceptual decision making

Perceptual decision making is a computationally demanding process that requires the brain to interpret incoming sensory information in the context of goals, expectations, preferences, and other factors. These integrative processes engage much of cortex but also require contributions from subcortical structures to affect behavior. Here we summarize recent evidence supporting specific computational roles of the basal ganglia in perceptual decision making. These roles probably share common mechanisms with the basal ganglia's other, more well-established functions in motor control, learning, and other aspects of cognition and thus can provide insights into the general roles of this important subcortical network in higher brain function.

Spatial frequency information modulates response inhibition and decision-making processes

We interact with the world through the assessment of available, but sometimes imperfect, sensory information. However, little is known about how variance in the quality of sensory information affects the regulation of controlled actions. In a series of three experiments, comprising a total of seven behavioral studies, we examined how different types of spatial frequency information affect underlying processes of response inhibition and selection. Participants underwent a stop-signal task, a two choice speed/accuracy balance experiment, and a variant of both these tasks where prior information was given about the nature of stimuli. In all experiments, stimuli were either intact, or contained only high-, or low- spatial frequencies. Overall, drift diffusion model analysis showed a decreased rate of information processing when spatial frequencies were removed, whereas the criterion for information accumulation was lowered. When spatial frequency information was intact, the cost of response inhibition increased (longer SSRT), while a correct response was produced faster (shorter reaction times) and with more certainty (decreased errors). When we manipulated the motivation to respond with a deadline (i.e., be fast or accurate), removal of spatial frequency information slowed response times only when instructions emphasized accuracy. However, the slowing of response times did not improve error rates, when compared to fast instruction trials. These behavioral studies suggest that the removal of spatial frequency information differentially affects the speed of response initiation, inhibition, and the efficiency to balance fast or accurate responses. More generally, the present results indicate a task-independent influence of basic sensory information on strategic adjustments in action control.

Animal models and measures of perceptual processing in schizophrenia

This paper summarizes the discussions regarding animal paradigms for assessing perception at the seventh meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS). A breakout group at the meeting addressed candidate tests in animals that might best parallel the human paradigms selected previously in the CNTRICS program to assess two constructs in the domain of perception: gain control and visual integration. The perception breakout group evaluated the degree to which each of the nominated tasks met pre-specified criteria: comparability of tasks across multiple species; construct validity; neuroanatomical homology between species; and dynamic range across parametric variation.

Influence of correspondence noise and spatial scaling on the upper limit for spatial displacement in fully-coherent random-dot kinematogram stimuli

Correspondence noise is a major factor limiting direction discrimination performance in random-dot kinematograms [1]. In the current study we investigated the influence of correspondence noise on Dmax, which is the upper limit for the spatial displacement of the dots for which coherent motion is still perceived. Human direction discrimination performance was measured, using 2-frame kinematograms having leftward/rightward motion, over a 200-fold range of dot-densities and a four-fold range of dot displacements. From this data Dmax was estimated for the different dot densities tested. A model was proposed to evaluate the correspondence noise in the stimulus. This model summed the outputs of a set of elementary Reichardt-type local detectors that had receptive fields tiling the stimulus and were tuned to the two directions of motion in the stimulus. A key assumption of the model was that the local detectors would have the radius of their catchment areas scaled with the displacement that they were tuned to detect; the scaling factor k linking the radius to the displacement was the only free parameter in the model and a single value of k was used to fit all of the psychophysical data collected. This minimal, correspondence-noise based model was able to account for 91% of the variability in the human performance across all of the conditions tested. The results highlight the importance of correspondence noise in constraining the largest displacement that can be detected.

Reduction in receptive field size of macaque MT neurons in the presence of visual noise

The visual system faces a trade-off between increased spatial integration of disparate local signals and improved spatial resolution to filter out irrelevant noise. Increased spatial integration is beneficial when signals are weak, whereas increased spatial resolution is particularly beneficial when focusing on a small object in a cluttered natural scene. The receptive field (RF) size of visual cortical neurons can be modulated depending on various factors such as sensory context, allowing adaptive integration of sensory signals. In this study, we explored the spatial integration properties of neurons in macaque middle temporal visual area (MT). We hypothesized that spatial resolution would increase when high-contrast noise was presented simultaneously with a visual stimulus, enabling focus on a small object in a cluttered scene. To test this hypothesis, we mapped the RFs of MT neurons of two fixating monkeys in a 5 × 5 grid manner using a small patch of random-dot motion. To examine the effects of noise on RF profile, a dynamic noise (0% coherence dots) of varying diameter was concurrently presented at the RF center. We found that RF size decreased when noise diameter increased. Analyses based on the response normalization model and area summation provided evidence for the potential contribution of spatial summation properties within the RF and surround suppression to the apparent contraction of RF size. Our results suggest that MT neurons integrate smaller regions of motion signals when signals are embedded in noise, an efficient strategy to filter out surrounding noise.

Excitatory binocular interactions in two cases of alternating strabismus

PURPOSE

Individuals with alternating fixation due to strabismus have often been considered prime examples of monocular visual function. A growing body of evidence suggests, however, that, at least in the case of a fixed-angle strabismus, excitatory binocular function is possible in the strabismic visual cortex if interocular suppression is taken into account. We investigated whether excitatory binocular function might also be possible for patients with alternating strabismus.

METHODS

Suprathreshold binocular interaction was tested in two individuals with alternating fixation and no amblyopia using a dichoptic motion coherence paradigm that can measure and account for interocular suppression.

RESULTS

Both participants exhibited strong interocular suppression when stimuli of the same contrast were presented to each eye, whereas no such suppressive interactions were present for controls; however, in significantly reducing the contrast of the stimuli presented to the fixing eye, excitatory binocular interactions were demonstrated in both participants similar to those measured in controls without the contrast imbalance.

CONCLUSIONS

The cortical mechanisms necessary for combining information from the two eyes seem to have been present but suppressed in our 2 participants with alternating fixation, just as they have been shown to be present in patients with fixed-angle strabismus.

Stimulus-dependent modulation of suppressive influences in MT

Surround suppression contributes to important functions in visual processing, such as figure-ground segregation; however, this benefit comes at the cost of decreased neuronal sensitivity. Studies of receptive fields at several levels of the visual hierarchy have demonstrated that surround suppression is reduced for low contrast stimuli, thereby improving neuronal sensitivity. We investigated whether this reduction of surround suppression reflects a general processing strategy to boost sensitivity for weak signals by summing them over a larger region of the visual field (spatial integration) or if the reduction is limited to specialized stimulus conditions. To do this, we used stochastic motion stimuli to measure surround suppression in area MT of alert macaque monkeys. While varying stimulus size we also varied the strength of two other critical stimulus features: contrast and coherence (i.e., the proportion of dots moving in the preferred direction of the neuron). We found that reducing stimulus contrast weakened surround suppression, but reducing stimulus coherence had the opposite effect, indicating that diminished surround suppression is not a universal response to stimuli of low signal-to-noise. This can be partially explained by our other finding, which is that surrounds in MT are very broadly direction tuned. Instead of producing a reduction of surround suppression that would improve the ability of the neuron to integrate preferred direction motion, low coherence stimuli activated the broadly tuned surrounds relatively better than the centers, which are generally more direction selective. Our results are consistent with a normalization mechanism of surround suppression that pools broadly across multiple stimulus dimensions.

A game platform for treatment of amblyopia

We have developed a prototype device for take-home use that can be used in the treatment of amblyopia. The therapeutic scenario we envision involves patients first visiting a clinic, where their vision parameters are assessed and suitable parameters are determined for therapy. Patients then proceed with the actual therapeutic treatment on their own, using our device, which consists of an Apple iPod Touch running a specially modified game application. Our rationale for choosing to develop the prototype around a game stems from multiple requirements that such an application satisfies. First, system operation must be sufficiently straightforward that ease-of-use is not an obstacle. Second, the application itself should be compelling and motivate use more so than a traditional therapeutic task if it is to be used regularly outside of the clinic. This is particularly relevant for children, as compliance is a major issue for current treatments of childhood amblyopia. However, despite the traditional opinion that treatment of amblyopia is only effective in children, our initial results add to the growing body of evidence that improvements in visual function can be achieved in adults with amblyopia.

Does the noise matter? Effects of different kinematogram types on smooth pursuit eye movements and perception

We investigated how the human visual system and the pursuit system react to visual motion noise. We presented three different types of random-dot kinematograms at five different coherence levels. For transparent motion, the signal and noise labels on each dot were preserved throughout each trial, and noise dots moved with the same speed as the signal dots but in fixed random directions. For white noise motion, every 20 ms the signal and noise labels were randomly assigned to each dot and noise dots appeared at random positions. For Brownian motion, signal and noise labels were also randomly assigned, but the noise dots moved at the signal speed in a direction that varied randomly from moment to moment. Neither pursuit latency nor early eye acceleration differed among the different types of kinematograms. Late acceleration, pursuit gain, and perceived speed all depended on kinematogram type, with good agreement between pursuit gain and perceived speed. For transparent motion, pursuit gain and perceived speed were independent of coherence level. For white and Brownian motions, pursuit gain and perceived speed increased with coherence but were higher for white than for Brownian motion. This suggests that under our conditions, the pursuit system integrates across all directions of motion but not across all speeds.

Linear responses to stochastic motion signals in area MST

The medial superior temporal (MST) area contains neurons with tuning for complex motion patterns, but very little is known about the generation of such responses. To explore how neuronal responses varied across complex motion pattern coherence, we recorded from single units while varying the strength of the global motion pattern in random dot stimuli. Stimuli were a family of optic flow patterns, consisting of radial motion, rotary motion, or combinations thereof ("spiral space"). We controlled the strength of the motion in the stimuli by varying the coherence--the proportion of dots carrying the signal. This allows motion strength to be varied independently of stimulus size, speed, or contrast. Most neurons' responses were well described as a linear function of stimulus coherence. Although more than half the cells possessed significant nonlinearities, these typically accounted for little additional variance. Nonlinear coherence response functions could either be compressive (e.g., saturating) or expansive and occurred in both the preferred and null direction responses. The presence of nonlinearities was not related to neuronal response properties such as preferred spiral-space direction or tuning bandwidth; however, cells with compressive nonlinearities in both the preferred and null directions tended to have higher response amplitudes and were more sensitive to weak motion signals. These cells did not appear to form a distinct subpopulation within MST. Our results suggest that MST neurons predominantly linearly encode increasing pattern motion energy within their RFs.

Binocular influences on global motion processing in the human visual system

This study investigates four key issues concerning the binocular properties of the mechanisms that encode global motion in human vision: (1) the extent of any binocular advantage; (2) the possible site of this binocular summation; (3) whether or not purely monocular inputs exist for global motion perception; (4) the extent of any dichoptic interaction. Global motion coherence thresholds were measured using random-dot-kinematograms as a function of the dot modulation depth (contrast) for translational, radial and circular flow fields. We found a marked binocular advantage of approximately 1.7, comparable for all three types of motion and the performance benefit was due to a contrast rather than a global motion enhancement. In addition, we found no evidence for any purely monocular influences on global motion detection. The results suggest that the site of binocular combination for global motion perception occurs prior to the extra-striate cortex where motion integration occurs. All cells involved are binocular and exhibit dichoptic interactions, suggesting the existence of a neural mechanism that involves more than just simple summation of the two monocular inputs.

Second-order optic flow processing

Optic flow-large-field rotational and radial motion-is processed as efficiently as translational motion for first-order (luminance-defined) stimuli. However, it has been suggested recently that the same pattern does not hold for second-order (e.g. contrast-defined) stimuli. We used random dot kinematogram (RDK) stimuli to determine whether global processing of optic flow is as efficient as processing of global translational motion for both first- and second-order stimuli. For first-order stimuli, we found that coherence thresholds for radial and rotational motion were equivalent to thresholds for translational motion, supporting previous findings. For second-order stimuli we found, firstly, that given sufficient contrast, second-order optic flow can be processed as efficiently as first-order optic flow and, secondly, that rotational and translational second-order motion are processed with equal efficiency. This contradicts the suggestion that there is a loss of efficiency between integration of second-order global motion and second-order optic flow. The third interesting finding was that the processing of radial second-order motion appears to suffer from a deficit that is dependent upon both the contrast and spatial extent of the stimulus. Further experiments discounted the possibility that the observed deficit is caused by a centrifugal or centripetal bias, but demonstrated that a longer temporal integration period for radial second-order motion is responsible for the observed difference. For durations of approximately 850ms, all three types of motion are processed with equal efficiency.

Sensitivity to visual motion in amblyopic macaque monkeys

Amblyopia is usually considered to be a deficit in spatial vision. But there is evidence that amblyopes may also suffer specific deficits in motion sensitivity as opposed to losses that can be explained by the known deficits in spatial vision. We measured sensitivity to visual motion in random dot displays for strabismic and anisometropic amblyopic monkeys. We used a wide range of spatial and temporal offsets and compared the performance of the fellow and amblyopic eye for each monkey. The amblyopes were severely impaired at detecting motion at fine spatial and long temporal offsets, corresponding to fine spatial scale and slow speeds. This impairment was also evident for the untreated fellow eyes of strabismic but not anisometropic amblyopes. Motion sensitivity functions for amblyopic eyes were shifted toward large spatial scales for amblyopic compared to fellow eyes, to a degree that was correlated with the shift in scale of the spatial contrast sensitivity function. Amblyopic losses in motion sensitivity, however, were not correlated with losses in spatial contrast sensitivity. This, combined with the specific impairment for detecting long temporal offsets, reveals a deficit in spatiotemporal integration in amblyopia which cannot be explained by the lower spatial resolution of amblyopic vision.

Perception of visual motion coherence by rats and mice

The coherence thresholds to discriminate the direction of motion in random-dot kinematograms were measured in rats and mice. Performance was best in the rats when dot displacement from frame-to-frame was about 2 degrees, and frame duration was less than 100 ms. Mice had coherence thresholds similar to those of rats when tested at the same step size and frame duration. Although the lowest thresholds in the rats and mice occasionally reached human levels, average rodent values ( approximately 25%) were 2-3 times higher than those of humans. These data indicate that the rodent and primate visual systems are similar in that both have local motion detectors and a system for extracting global motion from a noisy signal.

Maturation of luminance- and motion-defined form perception beyond adolescence: a combined ERP and fMRI study

Abilities to discriminate forms defined by motion continue to develop throughout childhood. To investigate late development of the visual motion system, we measured brain activity with event-related EEG potentials (ERPs) and functional magnetic resonance imaging (fMRI) in groups of adolescents (15-17 years) and adults (20-30 years) during a visual form discrimination task--with forms being either defined by motion or luminance contrast. We further explored whether possible developmental changes varied with the degree of motion coherence reflecting maturation specific to global motion processing. Both the fMRI activation patterns and ERP topographies were very similar between adolescents and adults, suggesting that the basic visual networks for processing motion and form are established by the age of 15-17. The ERP response to luminance- and motion-defined forms was dominated by a posterior negativity (N1: 120-270 ms). The N1 of the motion contrast was delayed in adolescents, whereas the N1 of the static condition did not differ between groups. Since the motion-evoked N1 is thought to arise in the middle temporal area MT/V5, our results indicate that visual motion processing in MT continues to get faster, becoming still more efficient during late development. Neither the ERP nor the fMRI results revealed maturation effects specific to motion coherence. This indicates that the specific mechanisms to process global dot motion are already mature in adolescence. The present findings support the view that static perception matures earlier than dynamic perception, and that these visual systems have different developmental courses.

Local and global limitations on direction integration assessed using equivalent noise analysis

We used an equivalent noise (EN) paradigm to examine how the human visual system pools local estimates of direction across space in order to encode global direction. Observers estimated the mean direction (clockwise or counter-clockwise of vertical) of a field of moving band-pass elements whose directions were drawn from a wrapped normal distribution. By measuring discrimination thresholds for mean direction as a function of directional variance, we were able to infer both the precision of observers' representation of each element's direction (i.e., local noise) as well as how many of these estimates they were averaging (i.e., global pooling). We estimated EN for various numbers of moving elements occupying regions of various sizes. We report that both local and global limits on direction integration are determined by the number of elements present in the display (irrespective of their density or the size of region they occupy), and we go on to show how this dependence can be understood in terms of neural noise. Specifically, we use Monte Carlo simulations to show that a maximum-likelihood operator, operating on pooled directional signals from visual cortex corrupted by Poisson noise, accounts for psychophysical data across all conditions tested, as well as motion coherence thresholds (collected under similar experimental conditions). A population vector-averaging scheme (essentially a special case of ML estimation) produces similar predictions but out-performs subjects at high levels of directional variability and fails to predict motion coherence thresholds.

Development of sensitivity to visual motion in macaque monkeys

The development of spatial vision is relatively well documented in human and nonhuman primates. However, little is known about the development of sensitivity to motion. We measured the development of sensitivity to direction of motion, and the relationship between motion and contrast sensitivity in macaque monkeys as a function of age. Monkeys (Macaca nemestrina, aged between 10 days and 3 years) discriminated direction of motion in random-dot kinematograms. The youngest monkeys showed directionally selective orienting and the ability to integrate motion signals at large dot displacements and fast speeds. With age, coherence sensitivity improved for all spatial and temporal dot displacements tested. The temporal interval between the dots was far less important than the spatial offset in determining the animals' performance at all but the youngest ages. Motion sensitivity improved well beyond the end of the first postnatal year, when mid-spatial-frequency contrast sensitivity reached asymptote, and continued for at least 3 years. Sensitivity to contrast at high spatial frequencies also continued to develop beyond the end of the first year. We conclude that the development of motion sensitivity depends on mechanisms beyond the low-level filters presumed to limit acuity and contrast sensitivity, and most likely reflects the function of extrastriate visual areas.

Three-dimensional structure-from-motion selectivity in the anterior superior temporal polysensory area, STPa, of the behaving monkey

Human and non-human primates are able to perceive three-dimensional structure from motion displays. Three-dimensional structure-from-motion (object-motion) displays were used to test the hypothesis that neurons in the anterior division of the superior temporal polysensory area (STPa) of monkeys can selectively respond to three-dimensional structure-from-motion. Monkeys performed a reaction time task that required the detection of a change in the fraction of structure in three-dimensional transparent sphere displays. Neurons were able to distinguish structured and unstructured three-dimensional optic flow. These cells could differentiate the change in structure-from-motion at stimulus presentation and when the animal was detecting the amount of structure in the display. Some of these neurons were also tuned for characteristics of the sphere stimuli. Cells were also tested with navigational motion and many were found to respond both to three-dimensional structure-from-motion and navigational motion. These results suggest that STPa neurons represent specific aspects of three-dimensional surface structure and that neurons within STPa contribute to the perception of three-dimensional structure-from-motion.

Adaptive temporal integration of motion in direction-selective neurons in macaque visual cortex

Direction-selective neurons in the primary visual cortex (V1) and the extrastriate motion area MT/V5 constitute a critical channel that links early cortical mechanisms of spatiotemporal integration to downstream signals that underlie motion perception. We studied how temporal integration in direction-selective cells depends on speed, spatial frequency (SF), and contrast using randomly moving sinusoidal gratings and spike-triggered average (STA) analysis. The window of temporal integration revealed by the STAs varied substantially with stimulus parameters, extending farther back in time for slow motion, high SF, and low contrast. At low speeds and high SF, STA peaks were larger, indicating that a single spike often conveyed more information about the stimulus under conditions in which the mean firing rate was very low. The observed trends were similar in V1 and MT and offer a physiological correlate for a large body of psychophysical data on temporal integration. We applied the same visual stimuli to a model of motion detection based on oriented linear filters (a motion energy model) that incorporated an integrate-and-fire mechanism and found that it did not account for the neuronal data. Our results show that cortical motion processing in V1 and in MT is highly nonlinear and stimulus dependent. They cast considerable doubt on the ability of simple oriented filter models to account for the output of direction-selective neurons in a general manner. Finally, they suggest that spike rate tuning functions may miss important aspects of the neural coding of motion for stimulus conditions that evoke low firing rates.

Motion adaptation in area MT

In many sensory systems, exposure to a prolonged stimulus causes adaptation, which tends to reduce neural responses to subsequent stimuli. Such effects are usually stimulus-specific, making adaptation a powerful probe into information processing. We used dynamic random dot kinematograms to test the magnitude and selectivity of adaptation effects in the middle temporal area (MT) and to compare them to effects on human motion discrimination. After 3 s of adaptation to a random dot pattern moving in the preferred direction, MT neuronal responses to subsequent test patterns were reduced by 26% on average compared with adaptation to a static pattern. This reduction in response magnitude was largely independent of what test stimulus was presented. However, adaptation in the opposite direction changed responses less often and very inconsistently. Therefore motion adaptation systematically and profoundly affects the neurons in MT representing the adapted direction, but much less those representing the opposite direction. In human psychophysical experiments, such adapting stimuli affected direction discrimination, biasing choices away from the adaptation direction. The magnitude of this perceptual shift was consistent with the magnitude of the changes seen in area MT, if one assumes that a motion comparison step occurs after MT.

Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward

This article relates a theoretical framework developed by British codebreakers in World War II to the neural computations thought to be responsible for forming categorical decisions about sensory stimuli. In both, a weight of evidence is computed and accumulated to support or oppose the alternative interpretations. A decision is reached when the evidence reaches a threshold value. In the codebreaking scheme, the threshold determined the speed and accuracy of the decision process. Here we propose that in the brain, the threshold may be controlled by neural circuits that calculate the rate of reward.

Spatio-temporal tuning of motion coherence detection at different luminance levels

We studied effects of dark adaptation on spatial and temporal tuning for motion coherence detection. We compared tuning for step size and delay for moving random pixel arrays (RPAs) at two adaptation levels, one light adapted (50 cd/m(2)) and the other relatively dark adapted (0.05 cd/m(2)). To study coherence detection rather than contrast detection, RPAs were scaled for equal contrast detection at each luminance level, and a signal-to-noise ratio paradigm was used in which the RPA is always at a fixed, supra-threshold contrast level. The noise consists of a spatio-temporally incoherent RPA added to the moving RPA on a pixel-by-pixel basis. Spatial and temporal limits for coherence detection were measured using a single step pattern lifetime stimulus, in which patterns on alternate frames make a coherent step and are being refreshed. Therefore, the stimulus contains coherent motion at a single combination of step size and delay only. The main effect of dark adaptation is a large shift in step size, slightly less than the adjustment of spatial scale required for maintaining equal contrast sensitivity. A similar change of preferred step size occurs also for scaled stimuli at a light-adapted level, indicating that the spatial effect is not directly linked to dark adaptation, but more generally related to changes in the available low-level spatial information. Dark-adaptation shifts temporal tuning by about a factor of 2. Long delays are more effective at low luminance levels, whereas short delays no longer support motion coherence detection. Luminance-invariant velocity tuning curves, as reported previously [Lankheet, M.J.M., van Doorn, A.J., Bouman, M.A., & van de Grind, W.A. (2000) Motion coherence detection as a function of luminance in human central vision. Vision Research, 40, 3599-3611], result from recruitment of different sets of motion detectors, and an adjustment of their temporal properties.

Correlated firing in macaque visual area MT: time scales and relationship to behavior

We studied the simultaneous activity of pairs of neurons recorded with a single electrode in visual cortical area MT while monkeys performed a direction discrimination task. Previously, we reported the strength of interneuronal correlation of spike count on the time scale of the behavioral epoch (2 sec) and noted its potential impact on signal pooling (Zohary et al., 1994). We have now examined correlation at longer and shorter time scales and found that pair-wise cross-correlation was predominantly short term (10-100 msec). Narrow, central peaks in the spike train cross-correlograms were largely responsible for correlated spike counts on the time scale of the behavioral epoch. Longer-term (many seconds to minutes) changes in the responsiveness of single neurons were observed in auto-correlations; however, these slow changes in time were on average uncorrelated between neurons. Knowledge of the limited time scale of correlation allowed the derivation of a more efficient metric for spike count correlation based on spike timing information, and it also revealed a potential relative advantage of larger neuronal pools for shorter integration times. Finally, correlation did not depend on the presence of the visual stimulus or the behavioral choice of the animal. It varied little with stimulus condition but was stronger between neurons with similar direction tuning curves. Taken together, our results strengthen the view that common input, common stimulus selectivity, and common noise are tightly linked in functioning cortical circuits.

Magnetic response of human extrastriate cortex in the detection of coherent and incoherent motion

Although direction selectivity is a cardinal property of neurons in the visual motion detection system, movement of numerous elements without global direction (incoherent motion) has been shown to activate human and monkey visual systems, as does coherent motion which has global direction. We used magnetoencephalography to investigate the neural process underlying responses to these types of motions in the human extrastriate cortex. Both motions were created using a random dot kinematogram and four speeds (0, 0.6, 9.6 and 25 degrees /s). The visual stimuli were composed of two successive motions at different speeds; a coherent motion at a certain speed that changed to incoherent motion at another speed or vice versa. Magnetic responses to the change in motion consisted of a few components, the first of which was always largest. The peak latency of the first component was inversely related to the speed of the preceding motion, but for both motions it was not affected by the speed of the subsequent motion. For each subject, the estimated origin of the first component was always in the extrastriate cortex, and this changed with the speed of the preceding motion. For both motions, the location for the slower preceding motion was lateral to that for the faster preceding motion. Although the latency changes of the two motions differed, their overall response properties were markedly similar. These findings show that the speed of incoherent motion is represented in the human extrastriate cortex neurons to the same degree as coherent motion. We consider that the human visual system has a distinct neural mechanism to perceive random dots' motion even though they do not move in a specific direction as a whole.

Illusory spatial offset of a flash relative to a moving stimulus is caused by differential latencies for moving and flashed stimuli

A flash that is presented adjacent to a continuously moving bar is perceived to lag behind the bar. One explanation for this phenomenon is that there is a difference in the persistence of the flash and the bar. Another explanation is that the visual system compensates for the neural delays of processing visual motion information, such as the moving bar, by spatially extrapolating the bar's perceived location forward in space along its expected trajectory. Two experiments demonstrate that neither of these models is tenable. The first experiment masked the flash one video frame after its presentation. The flash was still perceived to lag behind the bar, suggesting that a difference in the persistence of the flash and bar, does not cause the apparent offset. The second experiment employed unpredictable changes in the velocity of the bar including an abrupt reversal, disappearance, acceleration, and deceleration. If the extrapolation model held, the bar would continue to be extrapolated in accordance with its initial velocity until the moment of an abrupt velocity change. The results were inconsistent with this prediction, suggesting that there is little or no spatial compensation for the neural delays of processing moving objects. The results support a new model of temporal facilitation for moving objects whereby the apparent flash lag is due to a latency advantage for moving over flashed stimuli.

Coherent global motion in the absence of coherent velocity signals

It is widely believed that form and motion are analysed separately in mammalian visual systems. Form is confined within a stream that projects ventrally from V1 to the inferotemporal cortex, and motion within a stream that projects more dorsally, to the posterior parietal cortex [1] [2] [3] [4] [5] [6] [7]. Current descriptions suggest that there is little contact between the two streams until the products of their separate analyses are bound together at a late (and still unidentified) stage in perception [3] [8] [9] [10]. There are, however, indications that form and motion signals may interact [11], and that form signals, streaks derived from motion, may assist in the analysis of its direction [12]. Lennie [13] proposes that all image attributes, form and motion included, remain intimately coupled within the same retinotopic map at all stages of visual analysis. Here we show that form, independent of motion, can give coherence to incoherent motion. Sequences of Glass patterns [14] built to a common global rule are devoid of coherent motion signals, but they produce motion consistent with the global rule for form, not with the random velocity components of the pattern sequence.

Stored representations of three-dimensional objects in the absence of two-dimensional cues

Object recognition was studied in human subjects to determine whether the storage of the visual objects was in a two-dimensional or a three-dimensional representation. Novel motion-based and disparity-based stimuli were generated in which three-dimensional and two-dimensional form cues could be manipulated independently. Subjects were required to generate internal representations from motion stimuli that lacked explicit two-dimensional cues. These stored internal representations were then matched against internal three-dimensional representations constructed from disparity stimuli. These new stimuli were used to confirm prior studies that indicated the primacy of two-dimensional cues for view-based object storage. However, under tightly controlled conditions for which only three-dimensional cues were available, human subjects were also able to match an internal representation derived from motion of that of disparity. This last finding suggests that there is an internal storage of an object's representations in three dimensions, a tenet that has been rejected by view-based theories. Thus, any complete theory of object recognition that is based on primate vision must incorporate three-dimensional stored representations.