Direction perception in complex dynamic displays: the integration of direction information

Segregation from direction differences in dynamic random-dot stimuli

Previous research has shown that a field of random dots in which each dot alternates between a slow and a fast speed, can give rise to the percept of two superimposed sheets of moving dots when the alternations are out of phase or asynchronous with each other [Vis. Res. 35 (1995) 1691]. Under those conditions, observers can discriminate changes in the slow speed independent of changes in the fast speed. The present study investigated whether such motion-based segregation could result when dots alternated between two different directions. Three observers viewed a variety of displays containing two directions of motion, one upward and one oblique, with the task of discriminating small trial-to-trial changes in the direction of the upward component. The oblique direction component also changed direction from trial-to-trial. The field of dots either alternated synchronously (all dots moved in the same direction and switched to the other direction simultaneously) or asynchronously. Results showed that when the dots alternated synchronously between the directions, observers' direction discrimination performance was generally poor. However, when dots switched directions asynchronously, direction discrimination was only slightly elevated in comparison to that produced by a field of dots all moving in a single direction. Additional experiments demonstrated that this performance was not due to judging the global direction of the random-dot display. Thus the visual system had to segregate the stimulus into its component directions before integrating to arrive at the motion signal to be discriminated. It is concluded that for displays comprising elements that alternate between different directions, local direction signals can be used by the human visual system to effectively segregate a display so long as both direction signals are present simultaneously.

An oscillatory correlation model of visual motion analysis

We describe and evaluate a model of motion perception based on the integration of information from two parallel pathways: a motion pathway and a luminance pathway. The motion pathway has two stages. The first stage measures and pools local motion across the input animation sequence and assigns reliability indices to these pooled measurements. The second stage groups locations on the basis of these measurements. In the luminance pathway, the input scene is segmented into regions on the basis of similarities in luminance. In a subsequent integration stage, motion and luminance segments are combined to obtain the final estimates of object motion. The neural network architecture we employ is based on LEGION (locally excitatory globally inhibitory oscillator networks), a scheme for feature binding and region labeling based on oscillatory correlation. Many aspects of the model are implemented at the neural network level, whereas others are implemented at a more abstract level. We apply this model to the computation of moving, uniformly illuminated, two-dimensional surfaces that are either opaque or transparent. Model performance replicates a number of distinctive features of human motion perception.

Greater plasticity in lower-level than higher-level visual motion processing in a passive perceptual learning task

Simple exposure is sufficient to sensitize the human visual system to a particular direction of motion, but the underlying mechanisms of this process are unclear. Here, in a passive perceptual learning task, we found that exposure to task-irrelevant motion improved sensitivity to the local motion directions within the stimulus, which are processed at low levels of the visual system. In contrast, task-irrelevant motion had no effect on sensitivity to the global motion direction, which is processed at higher levels. The improvement persisted for at least several months. These results indicate that when attentional influence is limited, lower-level motion processing is more receptive to long-term modification than higher-level motion processing in the visual cortex.

Failure of direction identification for briefly presented second-order motion stimuli: evidence for weak direction selectivity of the mechanisms encoding motion

We sought to investigate why the direction of second-order motion, unlike first-order motion, cannot be identified when the stimulus exposure duration is brief (<200 ms). In a series of experiments observers identified both the orientation (vertical or horizontal) and the direction (left, right, down or up) of a drifting sinusoidal modulation (0.93 c/ degrees ) in either the luminance (first order) or the contrast (second order) of a two-dimensional noise carrier. All motion stimuli were equated for visibility, and the duration was varied using the method of constant stimuli. Performance was measured for second-order motion over a range of drift temporal frequencies (0.63-5.04 Hz) and for first-order motion stimuli composed of two, opposite drifting modulations in luminance of unequal modulation depth. Orientation-identification performance was nearly 100% correct for both first-order and second-order motion stimuli, even at the briefest stimulus duration tested (26.49 ms). Direction identification for first-order motion was also typically good with brief presentations, but was poor for second-order motion when the exposure duration was < approximately 200 ms. Importantly increasing either the drift temporal frequency of second-order motion or the bidirectional nature of the first-order motion patterns produced comparable levels of performance for the two varieties of motion (i.e. the minimum duration required for reliable direction identification could be equated). As orientation-identification performance for the first-order and second-order motion stimuli was comparably good and minimally affected by duration, the marked differences on the direction-identification task must be specific to mechanisms that encode drift direction, rather than spatial structure. We propose that second-order motion detectors are much less selective for stimulus direction than first-order motion sensors, and thus are more susceptible to the deleterious effects of limiting stimulus duration (which introduces spurious motion in the opposite direction, particularly at low drift rates). Alternative explanations based on the delayed propagation of second-order motion signals or the temporal characteristics of the underlying motion mechanisms are not supported by our findings.

Directional performance in motion transparency

Motion transparency provides a challenging test case for our understanding of how visual motion, and other attributes, are computed and represented in the brain. However, previous studies of visual transparency have used subjective criteria which do not confirm the existence of independent representations of the superimposed motions. We have developed measures of performance in motion transparency that require observers to extract information about two motions jointly, and therefore test the information that is simultaneously represented for each motion. Observers judged whether two motions were at 90 degrees to one another; the base direction was randomized so that neither motion taken alone was informative. The precision of performance was determined by the standard deviations (S.D.s) of probit functions fitted to the data. Observers also made judgments of orthogonal directions between a single motion stream and a line, for one of two transparent motions against a line and for two spatially segregated motions. The data show that direction judgments with transparency can be made with comparable accuracy to segregated (non-transparent) conditions, supporting the idea that transparency involves the equivalent representation of two global motions in the same region. The precision of this joint direction judgment is, however, 2-3 times poorer than that for a single motion stream. The precision in directional judgment for a single stream is reduced only by a factor of about 1.5 by superimposing a second stream. The major effect in performance, therefore, appears to be associated with the need to compute and compare two global representations of motion, rather than with interference between the dot streams per se. Experiment 2 tested the transparency of motions separated by a range of angles from 5 degrees to 180 degrees by requiring subjects to set a line matching the perceived direction of each motion. The S.D.s of these settings demonstrated that directions of transparent motions were represented independently for separations over 20 degrees. Increasing dot speeds from 1 to 10 deg/s improved directional performance but had no effect on transparency perception. Transparency was also unaffected by variations of density between 0.1 and 19 dots/deg(2)

Time course of amodal completion revealed by a shape discrimination task

We measured the extent of amodal completion as a function of stimulus duration over the range of 15-210 msec, for both moving and stationary stimuli. Completion was assessed using a performance-based measure; a shape discrimination task that is easy if the stimulus is amodally completed and difficult if it is not. Specifically, participants judged whether an upright rectangle was longer horizontally or vertically, when the rectangle was unoccluded, occluded at its corners by four negative-contrast squares, or occluded at its corners by four zero-contrast squares. In the zero-contrast condition, amodal completion did not occur because there were no occlusion cues; in the unoccluded condition, the entire figure was present. Thus, comparing performance in the negative-contrast condition to these two extremes provided a quantitative measure of amodal completion. This measure revealed a rapid but measurable time course for amodal completion. Moving and stationary stimuli took the same amount of time to be completed (approximately 75 msec), but moving stimuli had slightly stronger completion at long durations.

Sensitivity to modulation of color distribution in multicolored textures

We evaluated the discriminability of color distributions in square-element textures. Each texture contained 225 colors, represented by a distribution of color vectors in color space defined by the L-M and S-(L+M) axes. Each color distribution was systematically manipulated by modulating the distribution of the vector lengths sinusoidally as a function of the direction in the color space. The results showed that it is difficult to resolve a color distribution modulated in more than three cycles per 360 degrees in the chromatic direction. The difference in components along the cardinal axes is not a critical factor in the discrimination of the color distribution. An analysis using a line-element model suggested that the discrimination of the color distribution is mediated by multiple chromatic channels that are tuned to a variety of directions in the color space with a half-height-half-bandwidth of about 40 degrees in the chromatic direction.

Motion information is spatially localized in a visual working-memory task

We asked if the information about stimulus motion used in a visual working-memory task is localized in space. Monkeys compared the directions of two moving random-dot stimuli, sample and test, separated by a temporal delay and reported whether the stimuli moved in the same or in different directions. By presenting the two comparison stimuli in separate locations in the visual field, we determined whether information about stimulus direction was spatially localized during the storage and retrieval/comparison components of the task. Two psychophysical measures of direction discrimination provided nearly identical estimates of the critical spatial separation between sample and test stimuli that lead to a loss in threshold. Direction range thresholds measured with dot stimuli consisting of a range of local directional vectors were affected by spatial separation when a random-motion mask was introduced during the delay into the location of the upcoming test. The selective masking at the test location suggests that the information about the remembered direction was localized and available at that location. Direction difference thresholds, measured with coherently moving random dots, were also affected by separation between the two comparison stimuli. The separation at which performance was affected in both tasks increased with retinal eccentricity in parallel with the increase in receptive-field size in neurons in cortical area MT. The loss with transfer of visual information between different spatial locations suggests a contribution of cortical areas with localized receptive fields to the performance of the memory task. The similarity in the spatial scale of the storage mechanism derived psychophysically and the receptive field size of neurons in area MT suggest that MT neurons are central to this task.

Temporal integration of optic flow, measured by contrast and coherence thresholds

We measured, as a function of exposure duration, contrast sensitivity and coherence sensitivity for discerning the direction of motion of random dot patterns moving in circular, radial or translational directions. Contrast sensitivity for these patterns increased linearly with exposure duration, up to about 200-300 ms, consistent with previous estimates of temporal summation of early motion units. Coherence sensitivity, however, showed much longer summation periods, about 3 s. When the stimulus was embedded within 10 s of noise, sensitivity increased with duration up to 2-3 s, approximately linearly, as expected from an ideal integrator. When presented without the noise period, sensitivity also increased, but in a different way. For radial and circular motion the increase tended towards the theoretically predicted square root relationship for the same duration as that found with the embedded noise (about 3 s). For translation, however, the curve was steeper than the theoretical prediction (nearly linear), and the summation estimates of around 1000 ms. When the duration of the target was constant at 200 ms, but that of the flanking noise varied, sensitivity decreased with total duration over a similar interval. We interpret our results to reflect at least two stages of analysis, a threshold-limited early stage of local-motion analysis, with a time constant of 200-300 ms, and a later global-motion integration stage with a much longer time constant, around 3000 ms. There may also exist an intermediate stage, with an integration time of around 1000 ms.

Global motion integration in the cat's lateral posterior-pulvinar complex

Our laboratory previously showed that thalamic neurons in an extrageniculate nucleus, the lateral posterior-pulvinar complex (LP-pulvinar) could perform higher-order neuronal operations that had until then only been attributed to higher-level cortical areas. To further assess the role of the thalamus in the analysis of complex percepts, we have investigated whether neurons in the LP-pulvinar complex can signal the direction of motion of random-dot kinematograms wherein the individual elements of the pattern do not provide coherent motion cues. Our results indicate that a subset of LP-pulvinar cells can integrate the displacement of individual elements into a global motion percept and that their large receptive fields permit the integration of motion for elements separated by large spatial intervals. We also found that almost all of the global motion-sensitive neurons were not systematically pattern-motion-selective when tested with plaid patterns. The results indicate that LP-pulvinar cells can perform the higher-level spatio-temporal integration required to detect the global displacement of objects in a complex visual scene, further supporting the notion that extrageniculate thalamic cells are involved in higher-order motion processing. Furthermore, these results provide some evidence that there may be specialized mechanisms for processing different types of complex motion within the LP-pulvinar complex.

Dependency of reaction times to motion onset on luminance and chromatic contrast

We measured reaction times for detecting the onset of motion of sinusoidal gratings of 1 c/deg, modulated in either luminance or chromatic contrast, caused to move abruptly at speeds ranging from 0.25 to 10 deg/s (0.25-10 Hz). At any given luminance or chromatic contrast, RTs varied linearly with temporal periodicity (r2 congruent with 0.97), yielding a Weber fraction of period. The value of the Weber fraction varied inversely with contrast, differently for luminance and chromatic contrast. The results were well simulated with a simple model that accumulated change in contrast over time until a critical threshold had been reached. Two crucial aspects of the model are a second-stage temporal integration mechanism, capable of accumulating information for periods of up to 2 s, and contrast gain control, different for luminance than for chromatic stimuli. The contrast response for luminance shows very low semi-saturating contrasts and high gain, similar to LGN M-cells and cells in MT; that for colour shows high semi-saturating contrasts and low gain, similar to LGN P-cells. The results suggest that motion onset for luminance and chromatic gratings are detected by different mechanisms, probably by the magno- and parvo-cellular systems.

Different populations of neurons contribute to the detection and discrimination of visual motion

The signal-to-noise ratio of a direction-selective neuron for 'detecting' visual motion is highest when the motion direction is close to the neuron's preferred direction. But because these neurons show a bell-shaped tuning for direction, they have the highest signal-to-noise ratio for 'discriminating' the direction of motion when their preferred direction is off the direction to be discriminated. In this paper, we demonstrate with an adaptation paradigm that the visual system shows a corresponding task-specific ability to select neurons depending on whether it is performing a detection or a discrimination task, relying preferentially on different neuronal populations in the two tasks. Detection is based on neuronal populations tuned to the test direction, while direction discrimination is based on neurons preferring directions 40-60 degrees off the test direction.

Temporal facilitation for moving stimuli is independent of changes in direction

A flash that is presented aligned with a moving stimulus appears to lag behind the position of the moving stimulus. This flash-lag phenomenon reflects a processing advantage for moving stimuli (Metzger, W. (1932) Psychologische Forschung 16, 176-200; MacKay, D. M. (1958) Nature 181, 507-508; Nijhawan, R. (1994) Nature 370, 256-257; Purushothaman, G., Patel, S.S., Bedell, H.E., & Ogmen, H. (1998) Nature 396, 424; Whitney, D. & Murakami, I. (1998) Nature Neuroscience 1, 656-657). The present study measures the sensitivity of the illusion to unpredictable changes in the direction of motion. A moving stimulus translated upwards and then made a 90 degrees turn leftward or rightward. The flash-lag illusion was measured and it was found that, although the change in direction was unpredictable, the flash was still perceived to lag behind the moving stimulus at all points along the trajectory, a finding that is at odds with the extrapolation hypothesis (Nijhawan, R. (1994) Nature 370, 256-257). The results suggest that there is a shorter latency of the neural response to motion even during unpredictable changes in direction. The latency facilitation therefore appears to be omnidirectional rather than specific to a predictable path of motion (Grzywacz, N. M. & Amthor, F. R. (1993) Journal of Neurophysiology 69, 2188-2199).

Direction repulsion in unfiltered and ring-filtered Julesz textures

Perceived directions of motion were measured for each of two superposed two-dimensional dynamic random patterns consisting of unfiltered or ring-filtered dense random-check (Julesz) textures. One pattern always moved in a cardinal direction (up, down, left, or right), and the other texture always moved in an oblique direction separated from the cardinal component by 20 degrees-80 degrees. Several cardinal/oblique speed ratios were tested. In Experiment 1, the textures were unfiltered. In Experiment 2, the textures were ring filtered and had the same center frequency (1, 2, or 4 cpd). In Experiment 3, a 1-cpd ring-filtered texture was paired with a 2-, 4-, or 8-cpd texture. Subjects consistently misperceived the directions of component motion in these experiments; the angular separation of movement of the two textures was perceptually exaggerated, a phenomenon referred to as direction repulsion (Marshak & Sekuler, 1979). The results show that (1) direction repulsion occurs across at least a fourfold range of spatial frequencies and a sixfold range of speed ratios, (2) direction repulsion varies systematically with speed ratio, and (3) across most conditions, direction repulsion is anisotropic--direction repulsion is more evident in the oblique directions than in the cardinal directions. These findings suggest that the spatiotemporal range of inhibitory interactions involved in motion transparency is much greater than previously appreciated.

Postcoincidence trajectory duration affects motion event perception

In a two-dimensional display, identical visual targets moving toward and across each other with equal, constant speed can be perceived either to reverse their motion directions at the coincidence point (bouncing percept) or to stream through one another (streaming percept). Although there is a strong tendency to perceive the streaming percept, various factors have been reported to induce the bouncing percept, such as a sound or a visual flash at the moment of the visual target coincidence. By changing duration of the postcoincidence trajectory (PCT), we investigated how long it would take for such bounce-inducing factors to be maximally effective after the visual coincidence. With bounce-inducing factors, the percentage of the bouncing percept did not reach its maximal level immediately after the coincidence but increased as a function of PCT duration up to 150-200 msec. The results clearly reject the possibility of the cognitive-bias hypothesis about the bounce-inducing effect and suggest rather that the bounce-inducing factors have to interact with the PCT for some period after the coincidence to be maximally effective.

Information processing with population codes

Information is encoded in the brain by populations or clusters of cells, rather than by single cells. This encoding strategy is known as population coding. Here we review the standard use of population codes for encoding and decoding information, and consider how population codes can be used to support neural computations such as noise removal and nonlinear mapping. More radical ideas about how population codes may directly represent information about stimulus uncertainty are also discussed.

Probabilistic motion estimation based on temporal coherence

We develop a theory for the temporal integration of visual motion motivated by psychophysical experiments. The theory proposes that input data are temporally grouped and used to predict and estimate the motion flows in the image sequence. This temporal grouping can be considered a generalization of the data association techniques that engineers use to study motion sequences. Our temporal grouping theory is expressed in terms of the Bayesian generalization of standard Kalman filtering. To implement the theory, we derive a parallel network that shares some properties of cortical networks. Computer simulations of this network demonstrate that our theory qualitatively accounts for psychophysical experiments on motion occlusion and motion outliers. In deriving our theory, we assumed spatial factorizability of the probability distributions and made the approximation of updating the marginal distributions of velocity at each point. This allowed us to perform local computations and simplified our implementation. We argue that these approximations are suitable for the stimuli we are considering (for which spatial coherence effects are negligible).

Larger effect of aging on the perception of higher-order stimuli

Widespread deficits are known to accompany normal aging. Contrast thresholds of older and younger observers were measured for static and drifting gratings defined by luminance (first-order) or by contrast (second-order), and for a temporally segmented second-order motion stimulus. Results showed that older individuals had a larger threshold elevation for the perception of second-order stimuli than for the perception of first-order stimuli. This suggests a dissociation between the mechanisms underlying the perception of first and second-order stimuli, and demonstrates that aging may affect the more numerous processing steps required for the analysis of higher level stimuli.

Motion and shape in common fate

We determined how much motion coherence was needed to detect a target group of four moving dots in a dynamic visual noise (DVN) background. The lifetimes of the trajectories of the target and that of the noise dots were the same. In addition to parallel trajectories and collinear dot arrangements, divergent, convergent, or crossing trajectories and non-collinear dot arrangements were also tested. Performance saturated at a lifetime of approximately 600 ms. It was best for parallel trajectories and collinear dots, and worse for crossed trajectories with non-collinear dots, where it approached performance in a no-motion, form-only control experiment. Results illustrate the importance of common fate in motion perception in DVN, when other factors are equated.

Illusory spatial offset of a flash relative to a moving stimulus is caused by differential latencies for moving and flashed stimuli

A flash that is presented adjacent to a continuously moving bar is perceived to lag behind the bar. One explanation for this phenomenon is that there is a difference in the persistence of the flash and the bar. Another explanation is that the visual system compensates for the neural delays of processing visual motion information, such as the moving bar, by spatially extrapolating the bar's perceived location forward in space along its expected trajectory. Two experiments demonstrate that neither of these models is tenable. The first experiment masked the flash one video frame after its presentation. The flash was still perceived to lag behind the bar, suggesting that a difference in the persistence of the flash and bar, does not cause the apparent offset. The second experiment employed unpredictable changes in the velocity of the bar including an abrupt reversal, disappearance, acceleration, and deceleration. If the extrapolation model held, the bar would continue to be extrapolated in accordance with its initial velocity until the moment of an abrupt velocity change. The results were inconsistent with this prediction, suggesting that there is little or no spatial compensation for the neural delays of processing moving objects. The results support a new model of temporal facilitation for moving objects whereby the apparent flash lag is due to a latency advantage for moving over flashed stimuli.

Collisions between moving visual targets: what controls alternative ways of seeing an ambiguous display?

When identical visual targets move directly toward and then past one another, they appear either to stream past one another or to bounce off each other. Bertenthal et al (1993 Perception 22 193-207) accounted for the relative strengths of these two percepts by invoking a directional bias, arising from cooperative interactions within a network of motion detectors. We tested this explanation by devising conditions that would enhance or diminish the strength of such a directional bias. In separate experiments we varied (i) the presence or absence of temporal transients (pausing, disappearance, occlusion); (ii) the distances travelled by the targets; and (iii) their acceleration or deceleration before and after collision. The tendency to see the objects stream past one another was not related to the strength of an hypothesized directional bias, suggesting that the perception of this ambiguous motion display was not mediated by directional recruitment. Instead, the results suggest that perceived direction reflects the operation of neural constraints that mirror the constraints operating upon moving objects in the three-dimensional natural world.

Discrimination of the speed and direction of global second-order motion in stochastic displays

The ability to integrate local second-order motion signals over space and time was examined using random-dot-kinematograms (RDKs) in which the dots were defined by spatial variation in the contrast, rather than luminance, of a random noise field. When either the speeds or the directions of the individual dots were selected at random from a range of possible values, globally the stimulus appeared to drift either in a single direction or at a single speed in a manner analogous to that reported previously for first-order (luminance-defined) RDKs. To quantify the precision with which observers could extract the global stimulus motion, speed- and direction-discrimination thresholds were measured using pairs of RDKs, one of which (the comparison) comprised dots whose speeds or directions were assigned stochastically and the other (the standard) comprised dots that all had the same drift direction and speed. Speed-discrimination thresholds were of the order of 8% and changed little as the range of dot speeds (bandwidth) of the comparison increased, in that performance was almost as good when the individual dot speeds were selected at random from a range spanning 3.84 deg/s as when all the dots moved at the same speed. There was a tendency for the perceived global speed of the comparison RDK to decrease as the speed bandwidth was increased and perceived speed tended to coincide with the geometric mean speed of the dots rather than the arithmetic mean speed. Direction-discrimination thresholds were lowest (approximately 4 degrees) when the range of dot directions was less than 90 degrees but increased markedly thereafter. Observers were able to perform both discrimination tasks when the lifetimes of the dots comprising the RDKs was reduced from 25 to 2 frames, a manipulation that prevented observers from determining the overall speed or direction of image motion from the extended trajectories of individual dots within the display. Thresholds under these conditions were somewhat higher but were otherwise comparable to those obtained with a dot lifetime of 25 frames. The similarities between the present results and those of previous studies that have employed first-order RDKs suggest that the extraction of the global speed and direction of each type of motion is likely to be based on computationally similar principles.

Infants' sensitivity to statistical distributions of motion direction and speed

Adults combine different local motions to form a global percept of motion. This study explores the origins of this process by testing how perturbations of local motion influence infants' sensitivity to global motion. Infants at 6-, 12-, and 18-weeks of age viewed random dots moving with a gaussian distribution of dot directions defined by a mean of 0 degree (rightward) or 180 degrees (leftward) and a standard deviation (SD) of 0, 34, or 68 degrees. A well-practiced observer used infants' optokinetic responses to judge the direction of stimulus motion. Infants were studied both cross-sectionally and longitudinally. Direction discrimination was relatively high at all ages when the SD was 0 degree. When the SD was 34 or 68 degrees, performance declined with age. Adult performance was nearly perfect at these SDs. A similar developmental pattern was found with distributions of dot speed. The decline in infant performance is consistent with the development of both neural tuning and receptive field size. The subsequent improvement by adulthood suggests the development of additional processes such as long-range interactions.

Predicting the present direction of heading

Humans perceive heading accurately when they rotate their eyes. This is remarkable, because (1) the pursuit eye movement makes the retinal flow more complicated; and (2) the eye rotation causes a continuous change of the heading direction on the retina. The first problem prevents a simple association of the centre of flow on the retina with the heading direction. To solve it, the brain needs to take into account the flow associated with the eye's rotation. But even if this is done correctly, the resulting estimate of the heading is retino-centric and changing over time. Thus, the processing time to retrieve the heading from the flow field will cause a lag with respect to the actual heading direction. We investigated the latency for heading perception. We presented step wise changes of the centre of expanding flow to stationary and moving eyes. This mimics the movement of the heading direction across the retina, but avoids the complicating effects of rotational flow. For a stationary eye, we found a bias in perceived heading that corresponds to a latency of 300 ms or more. Yet, errors in heading perception are marginal normally, because we found an opposite bias for the moving eye, which counters the errors due to latency and a changing retino-centric heading direction. This suggests that the current heading direction is predicted from the extra-retinal signal and the delayed visual signals.

Axis-of-motion affects direction discrimination, not speed discrimination

The motion of an object can be described by a single velocity vector, or equivalently, by direction and speed separately. Similarly, our ability to see subtle differences in the motion of two objects could be constrained by either a velocity-based sensory response, or separate sensory responses to direction and speed. To distinguish between these possibilities we investigated whether direction discrimination and speed discrimination were differentially affected by changes in the axis-of-motion. Psychophysical data from 12 naive observers indicated that direction discrimination depended on axis-of-motion, but speed discrimination did not. The difference suggests that a velocity-based sensory response is not the limiting factor on the two tasks. Instead, the results imply that the sensory response which constrains speed discrimination is at least partially independent from the sensory response which constrains direction discrimination.

Human heading judgments and object-based motion information

In four experiments, we explored observers' ability to make heading judgments from simulated linear and circular translations through sparse forests and with pursuit fixation on one tree. We assessed observers' performance and information use in both regression and factorial designs. In all experiments we found that observers used three sources of object-based information to make their judgments--the displacement direction of the nearest object seen (a heuristic), inward displacement towards the fovea (an invariant) and outward deceleration (a second invariant). We found no support for the idea that observers use motion information pooled over regions of the visual field.

Global motion cues and the chromatic system

The capacity of the isolated chromatic system to perceive global motion was tested in a 40-deg visual field by use of random-dot kinematograms. The method of equivalent cone contrasts was used to directly compare the chromatic and the achromatic systems. The minimum number of dots necessary to correctly identify the motion direction was on the order of 20% for the isochromatic conditions, whereas thresholds were rarely obtained in the chromatic conditions. For both the isochromatic and the chromatic conditions, the central visual field was the most sensitive area, whereas the periphery was slightly less sensitive. This study suggests that the chromatic system does not efficiently integrate local motion cues to generate a global motion percept.

Human smooth pursuit direction discrimination

The smooth pursuit system is usually studied using single moving objects as stimuli. However, the visual motion system can respond to stimuli that must be integrated spatially and temporally (Williams DG, Sekuler R. Vision Res 1984;24:55-62; Watamaniuk SNJ, Sekuler R, Williams DW. Vision Res 1989;29:47-59). For example, when each dot of a random-dot cinematogram (RDC) is assigned a new direction of motion each frame from a narrow distribution of directions, the whole field of dots appears to move in the average direction (Williams and Sekuler, 1984). We measured smooth pursuit eye movements generated in response to small (10 deg diameter) RDCs composed of 250 dynamic random dots. Smooth eye movements were assessed by analyzing only the first 130 ms of eye movements after pursuit initiation (open-loop period). Comparing smooth eye movements to RDCs and single spot targets, we find that both targets generate similar responses confirming that the signal supplied to the smooth pursuit system can result from a spatial integration of motion information. In addition, the change in directional precision of smooth eye movements to RDCs with different amounts of directional noise was similar to that found for psychophysical direction discrimination. These results imply that the motion processing system responsible for psychophysical performance may also provide input to the oculomotor system.

Spatial integration in human smooth pursuit

When viewing a moving object, details may appear blurred if the object's motion is not compensated for by the eyes. Smooth pursuit is a voluntary eye movement that is used to stabilize a moving object. Most studies of smooth pursuit have used small, foveal targets as stimuli (e.g. Lisberger SG and Westbrook LE. J Neurosci 1985;5:1662-1673.). However, in the laboratory, smooth pursuit is poorer when a small object is tracked across a background, presumably due to a conflict between the primitive optokinetic reflex and smooth pursuit. Functionally, this could occur if the motion signal arising from the target and its surroundings were averaged, resulting in a smaller net motion signal. We asked if the smooth pursuit system could spatially summate coherent motion, i.e. if its response would improve when motion in the peripheral retina was in the same direction as motion in the fovea. Observers tracked random-dot cinematograms (RDC) which were devoid of consistent position cues to isolate the motion response. Either the height or the density of the display was systematically varied. Eye speed at the end of the open-loop period was greater for cinematograms than for a single spot. In addition, eye acceleration increased and latency decreased as the size of the aperture increased. Changes in the density produced similar but smaller effects on both acceleration and latency. The improved pursuit for larger motion stimuli suggests that neuronal mechanisms subserving smooth pursuit spatially average motion information to obtain a stronger motion signal.

Anisotropies in visual motion perception: a fresh look

We measured motion-detection and motion-discrimination performance for different directions of motion, using stochastic motion sequences. Random-dot cinematograms containing 200 dots in a circular aperture were used as stimuli in a two-interval forced-choice procedure. In the motion-detection experiment, observers judged which of two intervals contained weak coherent motion, the other internal containing random motion only. In the direction-discrimination experiment, observers viewed a standard direction of motion followed by comparison motion in a slightly different direction. Observers indicated whether the comparison was clockwise or counterclockwise, relative to the standard. Twelve directions of motion were tested in the detection task and five standard directions (three cardinal directions and two oblique directions) in the discrimination task. Detection thresholds were invariant with direction of motion, but direction-discrimination thresholds were significantly higher for motion in oblique directions, even at low-coherence levels. Results from control conditions ruled out monitor artifacts and indicate that the oblique effect is relative to retinal coordinates. These results have broad implications for computational and physiological models of motion perception.

Independent speed-tuned global-motion systems

Several experiments were conducted to investigate the role of speed in global-motion processing; the extraction of the direction of motion of a small subset of coherently-moving (signal) dots in a stimulus in which the other (noise) dots move in random directions. The specific aim of the experiments was to determine whether multiple speed-tuned global-motion systems exist. The results of these experiments are: (1) when the signal dots were chosen from a group of dots moving at 1.2 degrees s-1, the speed of additional-noise dots had to be below 4.8 degrees s-1 for them to affect global-motion extraction; (2) the addition of static dots did not impair the extraction of a global-motion signal carried by dots moving at 1.2 degrees s-1; (3) noise dots moving at 1.2 degrees s-1 impaired the extraction of a global-motion signal from dots moving at 10.8 degrees s-1, though not to the same extent as dots moving at a higher speed; and (4) these results were dependent upon speed, not spatial-step size or luminance contrast. These results are interpreted as indicating that global-motion extraction occurs within at least two independent speed tuned systems. One of these systems is sensitive to high speeds and the other to low speeds.

Local and global factors affecting the coherent motion of gratings presented in multiple apertures

Using stimuli composed of two independent gratings viewed through multiple apertures, we investigate a number of parameters affecting the integration of locally ambiguous motions into globally coherent motion. In four experiments, we varied local factors (grating spatial frequency, speed, contrast, duty cycle, orientation) and global factors (degree of similarity and common fate between the gratings, and symmetry in the configuration of the grating pattern) and examined their effects on global motion coherence. Our results, confirming accounts offered by previous investigators, indicate that local competition between motion signals generated by contours (ambiguous) and their line terminations (unambiguous) is important in determining global motion coherence in multiple-aperture stimuli. Our results also indicate that global factors can affect perceived coherence independently of local motion signals, suggesting the involvement of higher-level motion areas and a role for non-motion processes such as those involved in pattern and form perception. Comparing motion coherence with other two-dimensional (2-D) stimuli (plaids) shows that 2-D multiple-aperture stimuli are not analogous and that coherence models derived from plaid stimuli do not account for the data.

Local and global representations of velocity: transparency, opponency, and global direction perception

Human subjects can perceive global motion or motions in displays containing diverse local motions, implying representation of velocity at multiple scales. The phenomena of flexible global direction judgments, and especially of motion transparency, also raise the issue of whether the representation of velocity at any one scale is single-valued or multi-valued. A new performance-based measure of transparency confirms that the visual system represents directional information for each component of a transparent display. However, results with the locally paired random-dot display introduced by Qian et al, show that representations of multiple velocities do not coexist at the finest spatial scale of motion analysis. Functionally distinct scales of motion processing may be associated with (i) local motion detectors which show a strong winner-take-all interaction; (ii) spatial integration of local signals to disambiguate velocity; (iii) selection of reliable velocity signals as proposed in the model of Nowlan and Sejnowski; (iv) object-based or surface-based representations that are not necessarily organised in a fixed spatial matrix. These possibilities are discussed in relation to the neurobiological organisation of the visual motion pathway.

Simultaneous encoding of direction at a local and global scale

Human observers can simultaneously encode direction information at two different scales, one local (an individual dot) and one global (the coherent motion of a field of dots distributed over a 10 degrees-diameter display). We assessed whether encoding global motion would preclude the encoding of a local trajectory component and vice versa. In the present experiments, a large number (100-150) of dots were randomly assigned directions in each frame from a uniform distribution of directions spanning a range of 160 degrees to create global motion in a single direction (Williams & Sekuler, 1984). Amidst these background dots, 1 dot moved in a consistent direction (trajectory) for the duration of the display. The direction of this "trajectory dot" was similar to the mean direction of the distribution of directions determining the movement of the background dots. Direction discrimination for both the global motion and the trajectory was measured, using the method of constant stimuli, under precued and postcued partial report conditions. A low- or high-frequency 85-msec tone signaled which motion the subject was to judge. In the precue condition, the tone was presented 200 msec before the onset of the stimulus, whereas in the postcue condition, the tone was presented immediately after the offset of the stimulus. Direction discrimination thresholds for both global and local motion in the postcued condition were not significantly different from those obtained in the precued condition. These results suggest that direction information for both global and local motion is encoded simultaneously and that the observer has access to either motion signal after the presentation of a stimulus.

Recruitment mechanisms in speed and fine-direction discrimination tasks

The minimum information necessary to specify motion requires a change in position across time. Previous studies have shown that human motion measurements improve with more than two frames of motion. This study clarifies how motion information is integrated to produce the best speed and direction discrimination. Using random-dot kinematograms, fine-direction discrimination thresholds and speed discrimination thresholds are assessed as a function of dot lifetime. Specifically, we ask if performance on both tasks depends on dot lifetime in the same manner. If both speed and direction discrimination performance improve the same way with increasing dot lifetime, this would indicate that both tasks have the same integration limit and both tasks may depend on the same underlying mechanisms. Experiment 1 shows that for both tasks a four-frame dot lifetime is necessary for observers to reach asymptotic threshold levels. The absolute level of performance improves with increasing stimulus duration or signal-to-noise ratio, but the integration limit itself does not vary. Experiment 2 examines whether this integration limit is constrained by the number of frames or by the temporal duration of the dot lifetime. The data in Experiment 2 suggest that both a minimum number of samples and a minimal temporal integration period determine the integration limit for recruitment mechanisms. The results suggest that speed and fine-direction discrimination depend upon the same underlying motion mechanisms. These results are discussed in relation to possible underlying physiological substrates and computational models of motion measurement.

Detection of spatial discontinuities in first-order optical flow fields

We investigated the extent to which the human visual system can detect discontinuities in first-order optical flow fields. We constructed two types of spatial discontinuities: a circular split field with a straight edge and a disk with annular surround. We used two different first-order optical flow components: an expansion and a rotation. We found an intriguing difference in the detection thresholds for straight and circular discontinuities. Whereas straight discontinuities yielded thresholds of 10%-50% difference in expansion or rotation, circular discontinuities could, at first, only be detected at extreme differences (>> 100%). After a learning period, thresholds for such stimuli decreased, but they remained significantly higher than thresholds for the straight edge. Thresholds rose for stimuli that formed a gradual transition between a circular and a straight edge, and they decreased with increasing eccentricity of the circular discontinuity. Results suggest that symmetry in this stimulus, defined by the coincidence of the center of expansion or rotation and the center of the circular discontinuity, was responsible for the difference in thresholds for circular and straight discontinuities.

Orthogonal motion after-effect illusion predicted by a model of cortical motion processing

The motion after-effect occurs after prolonged viewing of motion; a subsequent stationary scene is perceived as moving in the opposite direction. This illusion is thought to arise because motion is represented by the differential activities of populations of cortical neurons tuned to opposite directions; fatigue in one population leads to an imbalance that favours the opposite direction once the stimulus ceases. Following adaptation to multiple directions of motion, the after-effect is unidirectional, indicating that motion signals are integrated across all directions. Yet humans can perceive several directions of motion simultaneously. The question therefore arises as to how the visual system can perform both sharp segregation and global integration of motion signals. Here we show in computer simulations that this can occur if excitatory interactions between different directions are sharply tuned while inhibitory interactions are broadly tuned. Our model predicts that adaptation to simultaneous motion in opposite directions will lead to an orthogonal motion after-effect. This prediction was confirmed in psychophysical experiments. Thus, broadly tuned inhibitory interactions are likely to be important in the integration and segregation of motion signals. These interactions may occur in the cortical area MT, which contains motion-sensitive neurons with properties similar to those required by our model.

Integration across directions in dynamic random dot displays: vector summation or winner take all?

Recent studies have clearly demonstrated that the activity of directionally selective neuronal populations in the middle temporal (MT) and medial superior temporal (MST) cortical areas plays a direct role in the judgment of the direction of visual motion. However, the way in which the information is derived from a population of neurons remains unknown. Two principal models have been suggested in the past: the vector summation model suggests that the responses of neurons encoding all directions of motion are weighted and pooled to obtained an accurate estimate of the mean direction of motion; the winner-take-all model is based on a competition between different direction-specific channels, so that decisions are cast in favor of the channel generating the strongest directional signal. To discriminate between these two models we generated random dot stimuli that contained an asymmetric distribution of directions of motion. Human subjects were asked to adjust the global direction of motion to the upward vertical direction. When the directional signals were of similar strength, subjects tended to perceive global motion in the mean direction of motion (corresponding to vector summation), but as one directional signal became more prominent, most subjects' settings diverged from the mean towards the modal direction of motion. Some subjects could either match the mean or the modal direction of motion in the display, depending on the task instructions. These results suggest that the perceptual judgment of direction of motion is not based on any rigid algorithm generating a single valued output. Rather, human observers are able to judge different aspects of the distribution of activity in a cortical area depending on the task requirements.

What is noise for the motion system?

Motion coherence thresholds in random-dot patterns have been widely adopted as a measure of performance in visual motion processing. However, there has been diversity in the type of "noise" in which a coherent motion signal has to be detected. Here we compare coherence thresholds for three ways of creating motion noise: dots replotted in random positions in each new frame; dots with a set displacement but following a random walk from frame to frame; or dots moving in random directions which remain constant for a given dot over a sequence of displacements. In each case, the signal dots may either remain the same throughout the display sequence, or the signal dots may be re-selected afresh on each frame ("different"). With our display (3 deg square, 120 msec exposure, velocity = 5 or 10 deg sec-1), all these different noise conditions yielded similar thresholds around 5-8%. There were some small but systematic differences between conditions. Thresholds in random-direction displays were consistently higher than those in random-walk or random-position displays, especially at the lower velocity. However, this effect is much smaller than would be expected from the increased standard error of the noise mean in random direction, perhaps because the motion system integrates information most effectively over a local region of space and/or time. Subjects" performance could not be explained by a strategy of identifying individual signal dots with extended trajectories. The similarity between random-walk and random-position thresholds implies that subjects do not exploit the marked differences in speed distribution between signal and noise dots in the latter case. The practical message for the design and interpretation of experiments using coherence thresholds is that the results are not much affected by the choice of noise, at least within the range of stimuli tested here.

Temporal coherence theory for the detection and measurement of visual motion

A recent challenge to the completeness of some influential models of local-motion detection has come from experiments in which subjects had to detect a single dot moving along a trajectory amidst noise dots undergoing Brownian motion. We propose and test a new theory of the detection and measurement of visual motion, which can account for these signal-in-Brownian-noise experiments. The theory postulates that the signals from local-motion detectors are made coherent in space and time by a special purpose network, and that this coherence boosts signals of features moving along non-random trajectories over time. Two experiments were performed to estimate parameters and test the theory. These experiments showed that detection is impaired with increasing eccentricity, an effect that varies inversely with step size. They also showed that detection improves over durations extending to at least 600 msec. An implementation of the theory accounts for these psychophysical detection measurements.

Discrimination of speed distributions: sensitivity to statistical properties

Two experiments examining the ability of human observers to detect differences in the statistical properties underlying velocity distributions were conducted. A four-alternative forced-choice methodology, using four simultaneous velocity distributions, was used in both experiments. In the first experiment the value of one statistical moment (mean, variance, skewness, or kurtosis) was manipulated while the others were held constant. The subjects task was to determine which of four velocity distributions contained the dissimilar value. In the second experiment only the latter three moments were examined. A similar procedure was used, however feedback was given after each trial to maximize observer performance. The results from both experiments indicate that human observers can reliably detect differences in both mean and variance information underlying velocity distributions. The results of this research has important implications for image segmentation and the detection of heading from optic flow.

The effects of spatiotemporal integration on maximum displacement thresholds for the detection of coherent motion

In a series of nine experiments, observers were required to identify the shapes of moving targets, and to discriminate regions of motion from regions of uncorrelated noise. Maximum displacement thresholds (Dmax) for performing these tasks were obtained under a wide variety of conditions. The stimulus parameters manipulated included the number of distinct frames in the motion sequences, the stimulus onset asynchrony between each frame, the size of the moving dots, and the shape, area and eccentricity of the target regions. For two-frame displays presented in alternation, the area of the target region was the only one of these variables to have any significant effect on Dmax. For longer length sequences, in contrast, Dmax varied dramatically among the different conditions over a range of 10 min arc to 10 deg. In an effort to isolate the specific processes of spatiotemporal integration, we also examined how performance is affected by having overlapping transparent motions in opposite directions, or by the presence of dynamic noise or limited dot lifetimes within the moving target regions. The overall pattern of results suggest that Dmax is primarily determined by the ability of the visual system to isolate motion signals from the noise produced by spurious false target correlations. As a general rule, Dmax will increase as a result of any stimulus manipulation that increases the number of local signal correlations detected relative to those arising from noise, and vice versa.

Visual processing of motion boundaries

Psychometric functions for motion detection were measured for various spatial velocity profiles made of independently moving lines of random dots. In the first experiment, sensitivity was greater for square-wave velocity profiles than for sine waves of the same fundamental spatial frequency. Sensitivity for square waves depended on the phase of the waveform with respect to the fixation point, which precludes a characterization of the processes underlying the detection of shearing motion as a translation-invariant system. The second experiment, using velocity fields created by spatial super-position of sine waves, showed that motion boundaries facilitate detection of motion due to the steepness of the velocity gradient, and not simply because of added power at higher harmonics. In the third experiment, fluted velocity waveforms were created by subtracting the fundamental sinusoidal component from square waves, retaining sharp motion boundaries between opposing directions but removing the regions of uniform motion. Subtracting the fundamental from low-frequency square waves did not lower sensitivity to motion, indicating that sensitivity was largely determined by the presence of motion boundaries. In the final section of this article, a model is presented that can account for the data by using linear center-surround velocity mechanisms whose sizes increase with eccentricity while their sensitivity for shearing motion decreases.

Detecting a trajectory embedded in random-direction motion noise

Human observers can easily detect a signal dot moving, in apparent motion, on a trajectory embedded in a background of random-direction motion noise. A high detection rate is possible even though the spatial and temporal characteristics (step size and frame rate) of the signal are identical to the noise, making the signal indistinguishable from the noise on the basis of a single pair of frames. The success rate for detecting the signal dot was as high as 90% when the probability of mismatch from frame-to-frame, based on nearest-neighbor matching, was 0.3. Control experiments showed that trajectory detection is not based on detecting a "string" of collinear dots, i.e. a stationary position cue. Nor is a trajectory detected because it produces stronger signals in single independent motion detectors. For one thing, trajectory detection improves with increases in duration, up to 250-400 msec, a duration longer than the integration typically associated with a single motion detector. For another, the signal dot need not travel in a straight line to be detectable. The signal dot was as reliably detected when it changed its direction a small amount (about 30 deg or less) each frame. Consistent with this, circular paths of sufficiently low curvature were as detectable as straight trajectories. Our data suggest that trajectory motion is highly detectable in motion noise because the component local motion signals are enhanced when motion detectors with similar directional tuning are stimulated in a sequence along their preferred direction.

A reduction in the number of directionally selective neurons extends the spatial limit for global motion perception

Dynamic random-dot targets were used to study neural mechanisms underlying motion perception. Performance of cats with severely reduced numbers of cortical directionally selective neurons (reduced DS) was compared to that of normal animals. We assessed the spatial properties of the residual motion mechanism by measuring direction discriminations at various dot displacements. At small displacements, reduced DS cats' motion integration thresholds for opposite direction discrimination were nearly normal. At larger displacements, their thresholds surpassed those of normal cats and their upper displacement limit (dmax) was increased by 0.35 deg. The accuracy of direction discrimination was reduced at small displacements, but at larger displacements direction difference thresholds of reduced DS cats approached or surpassed those of normals. These data were compared to the performance of humans who showed an extension of dmax for peripherally viewed targets. The data support the hypothesis that expansion in spatial scale of the motion mechanism may contribute to extension of dmax. Additional support for this hypothesis is provided by a modified direction discriminating line-element model. The model also suggests that changes in sampling of motion mechanisms in the reduced DS system may play a role.

Is global motion really based on spatial integration of local motion signals?

Previous studies have shown that a random-dot kinematogram (RDK) comprising dots, each of which takes a random walk in direction or speed over time, can appear to flow in a single direction. This has been interpreted as evidence for the existence of a co-operative network linking neurons sensitive to different directions/speeds and different spatial locations. We have investigated the possibility that global motion perception in such patterns might simply reflect motion energy detection at a coarse spatial scale (such that many dots fall in the receptive field of one energy detector) without the need to encode local dot motions on a fine spatial scale and then integrate their motions over space. We created random-walk RDKs and then spatially high-pass filtered them to remove low spatial frequencies. Perception of global motion was unimpaired for both direction and speed random walks, showing that the phenomenon is not reliant on low spatial frequencies and must, therefore, involve integration of local motion signals across space, as originally postulated.

Perceived speed of moving lines depends on orientation, length, speed and luminance

In this study, the perceived speed of a tilted line translating horizontally (for a duration of 167 msec) is evaluated with respect to a vertical line undergoing the same translation. Perceived speed of the oblique line is shown to be underestimated when compared to the vertical line. This bias increases: (1) when the line is further tilted, (2) with greater line lengths, (3) with lower contrasts, and finally (4) with a speed of 2.1 deg/sec as compared to a higher speed of 4.2 deg/sec. These results may be accounted for by considering that two velocity signals are used by the visual system to estimate the speed of the line: the translation of this line (this signal does not depend on the line's orientation) and the motion component normal to the line (this signal depends on orientation). We suggest that these two signals are encoded by different types of units and that the translation signal is specifically extracted at the line endings. We further suggest that these signals are integrated by a weighted average process according to their perceptual salience. Other interpretations are considered at the light of current models dealing with the two-dimensional integration of different velocity signals.

Representational development of direction in motion perception: a fragile process

Response to a change in direction is more rapid if the target moves in a predictable direction before the change than if the pre-change direction is not predictable. However, if the target trajectory is viewed for approximately half a second before the change in direction, the effect of directional predictability disappears. Visual information gathered prior to change in direction is used to construct an increasingly more accurate representation of target trajectory. To study this process, we inject various temporal transients into the trajectory prior to the change in direction. We find that extraction of directional information is interrupted if: (i) motion continues along a constant trajectory, but the target disappears briefly behind an implicit or real occluder, (ii) the target pauses briefly, but remains visible, or (iii) the target changes speed briefly, while continuing to move in the same direction. The theoretical implications for motion perception are discussed. These implications include a framework for understanding interactions between stimulus-derived information and a priori information.

Dependence of speed and direction perception on cinematogram dot density

In the present experiments, we find that with abrupt decreases in dot density of random-dot cinematograms, perceived speed decreases, while with abrupt increases in dot density, perceived speed increases. Further, in steady-state conditions, perceived speed is also affected in the same way, but to a lesser degree, by the dot density of cinematograms. Direction discrimination of random-dot cinematograms is enhanced when dot density increases abruptly from one stimulus to the next, but is degraded when dot density decreases abruptly. Finally, speed discrimination remains constant even when density changes abruptly. The perceived-speed and direction-discrimination data are consistent with the Motion Coherence theory which motivated this study, and with models that include a smoothing stage similar to this theory. Of the other models that we consider, most predict that increasing dot density reduces perceived speed. The speed-discrimination data could not distinguish between the different theories.